Search Results for author:
Found 67 papers, 23 papers with code
Beyond Standard MoE: Mixture of Latent Experts for Resource-Efficient Language Models
In this paper, we introduce Mixture of Latent Experts (MoLE), a novel parameterization methodology that facilitates the mapping of specific experts into a shared latent space.
Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
Model merging, on the other hand, offers a cost-effective and robust alternative by integrating the quick-thinking capabilities of System 1 models with the methodical reasoning of System 2 models.
Open3DBench: Open-Source Benchmark for 3D-IC Backend Implementation and PPA Evaluation
This work introduces Open3DBench, an open-source 3D-IC backend implementation benchmark built upon the OpenROAD-flow-scripts framework, enabling comprehensive evaluation of power, performance, area, and thermal metrics.
Timing-Driven Global Placement by Efficient Critical Path Extraction
Timing optimization during the global placement of integrated circuits has been a significant focus for decades, yet it remains a complex, unresolved issue.
ARS: Automatic Routing Solver with Large Language Models
Real-world Vehicle Routing Problems (VRPs) are characterized by a variety of practical constraints, making manual solver design both knowledge-intensive and time-consuming.
Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models
Notably, when combined with existing merging techniques, our method enables merged models to outperform specialized fine-tuned models, particularly in code generation tasks.
1bit-Merging: Dynamic Quantized Merging for Large Language Models
Recent advances in large language models have led to specialized models excelling in specific domains, creating a need for efficient model merging techniques.
LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
While most current approaches rely on further training techniques, such as fine-tuning or reinforcement learning, to enhance model capacities, model merging stands out for its ability of improving models without requiring any additional training.
Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks
The widespread deployment of pre-trained language models (PLMs) has exposed them to textual backdoor attacks, particularly those planted during the pre-training stage.
CMoE: Fast Carving of Mixture-of-Experts for Efficient LLM Inference
Large language models (LLMs) achieve impressive performance by scaling model parameters, but this comes with significant inference overhead.
AttentionPredictor: Temporal Pattern Matters for Efficient LLM Inference
With the development of large language models (LLMs), efficient inference through Key-Value (KV) cache compression has attracted considerable attention, especially for long-context generation.
KVTuner: Sensitivity-Aware Layer-wise Mixed Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
KV cache quantization can improve Large Language Models (LLMs) inference throughput and latency in long contexts and large batch-size scenarios while preserving LLMs effectiveness.
E2ESlack: An End-to-End Graph-Based Framework for Pre-Routing Slack Prediction
The framework includes a TimingParser that supports DEF, SDF and LIB files for feature extraction and graph construction, an arrival time prediction model and a fast RAT estimation module.
MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
Transformer-based large language models (LLMs) have achieved remarkable success as model sizes continue to grow, yet their deployment remains challenging due to significant computational and memory demands.
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformers
In this paper, we propose FuseGPT, a novel methodology to recycle the pruned transformer blocks to further recover the model performance.
The Graph's Apprentice: Teaching an LLM Low Level Knowledge for Circuit Quality Estimation
Logic synthesis is a crucial phase in the circuit design process, responsible for transforming hardware description language (HDL) designs into optimized netlists.
SeaDAG: Semi-autoregressive Diffusion for Conditional Directed Acyclic Graph Generation
We explicitly train the model to learn graph conditioning with a condition loss, which enhances the diffusion model's capacity to generate graphs that are both realistic and aligned with specified properties.
A Systematic Survey on Large Language Models for Algorithm Design
Algorithm Design (AD) is crucial for effective problem-solving across various domains.
Logic Synthesis Optimization with Predictive Self-Supervision via Causal Transformers
Contemporary hardware design benefits from the abstraction provided by high-level logic gates, streamlining the implementation of logic circuits.
MTLSO: A Multi-Task Learning Approach for Logic Synthesis Optimization
On the other hand, we employ a hierarchical graph representation learning strategy to improve the model's capacity for learning expressive graph-level representations of large AIGs, surpassing traditional plain GNNs.
RTLRewriter: Methodologies for Large Models aided RTL Code Optimization
Our comparative analysis with established compilers such as Yosys and E-graph demonstrates significant improvements, highlighting the benefits of integrating large models into the early stages of circuit design.
ShortCircuit: AlphaZero-Driven Circuit Design
Chip design relies heavily on generating Boolean circuits, such as AND-Inverter Graphs (AIGs), from functional descriptions like truth tables.
Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms
We executed six state-of-the-art AI-based chip placement algorithms on these designs and plugged the results of each single-point algorithm into the physical implementation workflow to obtain the final PPA results.
UDC: A Unified Neural Divide-and-Conquer Framework for Large-Scale Combinatorial Optimization Problems
Employing a high-efficiency Graph Neural Network (GNN) for global instance dividing and a fixed-length sub-path solver for conquering divided sub-problems, the proposed UDC framework demonstrates extensive applicability, achieving superior performance in 10 representative large-scale CO problems.
Logic Synthesis with Generative Deep Neural Networks
While deep learning has achieved significant success in various domains, its application to logic circuit design has been limited due to complex constraints and strict feasibility requirement.
SmoothGNN: Smoothing-aware GNN for Unsupervised Node Anomaly Detection
We introduce Individual Smoothing Patterns (ISP) and Neighborhood Smoothing Patterns (NSP), which indicate that the representations of anomalous nodes are harder to smooth than those of normal ones.
DPN: Decoupling Partition and Navigation for Neural Solvers of Min-max Vehicle Routing Problems
The min-max vehicle routing problem (min-max VRP) traverses all given customers by assigning several routes and aims to minimize the length of the longest route.
Prompt Learning for Generalized Vehicle Routing
To be concrete, we propose a novel prompt learning method to facilitate fast zero-shot adaptation of a pre-trained model to solve routing problem instances from different distributions.
GraSS: Combining Graph Neural Networks with Expert Knowledge for SAT Solver Selection
In this paper we present GraSS, a novel approach for automatic SAT solver selection based on tripartite graph representations of instances and a heterogeneous graph neural network (GNN) model.
Instance-Conditioned Adaptation for Large-scale Generalization of Neural Combinatorial Optimization
The neural combinatorial optimization (NCO) approach has shown great potential for solving routing problems without the requirement of expert knowledge.
Learning to Cut via Hierarchical Sequence/Set Model for Efficient Mixed-Integer Programming
Moreover, we observe that (P3) what order of selected cuts to prefer significantly impacts the efficiency of MILP solvers as well.
Self-Improved Learning for Scalable Neural Combinatorial Optimization
The end-to-end neural combinatorial optimization (NCO) method shows promising performance in solving complex combinatorial optimization problems without the need for expert design.
HDLdebugger: Streamlining HDL debugging with Large Language Models
In the domain of chip design, Hardware Description Languages (HDLs) play a pivotal role.
Circuit Transformer: A Transformer That Preserves Logical Equivalence
In this study, we introduce a generative neural model, the "Circuit Transformer", which eliminates such wrong predictions and produces logic circuits strictly equivalent to given Boolean functions.
IB-Net: Initial Branch Network for Variable Decision in Boolean Satisfiability
Boolean Satisfiability problems are vital components in Electronic Design Automation, particularly within the Logic Equivalence Checking process.
PreRoutGNN for Timing Prediction with Order Preserving Partition: Global Circuit Pre-training, Local Delay Learning and Attentional Cell Modeling
First, we propose global circuit training to pre-train a graph auto-encoder that learns the global graph embedding from circuit netlist.
Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalization
The results show that the unified model demonstrates superior performance in the eleven VRPs, reducing the average gap to around 5% from over 20% in the existing approach and achieving a significant performance boost on benchmark datasets as well as a real-world logistics application.
DiLA: Enhancing LLM Tool Learning with Differential Logic Layer
In this paper, we propose a novel differential logic layer-aided language modeling (DiLA) approach, where logical constraints are integrated into the forward and backward passes of a network layer, to provide another option for LLM tool learning.
BetterV: Controlled Verilog Generation with Discriminative Guidance
In this work, we propose a Verilog generation framework, BetterV, which fine-tunes the large language models (LLMs) on processed domain-specific datasets and incorporates generative discriminators for guidance on particular design demands.
Machine Learning Insides OptVerse AI Solver: Design Principles and Applications
To this end, we present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI Solver, which aims to mitigate the scarcity of real-world mathematical programming instances, and to surpass the capabilities of traditional optimization techniques.
Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model
EoH represents the ideas of heuristics in natural language, termed thoughts.
LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation
Since circuit can be represented with HDL in a textual format, it is reasonable to question whether LLMs can be leveraged in the EDA field to achieve fully automated chip design and generate circuits with improved power, performance, and area (PPA).
Algorithm Evolution Using Large Language Model
In this paper, we propose a novel approach called Algorithm Evolution using Large Language Model (AEL).
Large Language Model for Multi-objective Evolutionary Optimization
It is also promising to see the operator only learned from a few instances can have robust generalization performance on unseen problems with quite different patterns and settings.
A Circuit Domain Generalization Framework for Efficient Logic Synthesis in Chip Design
In particular, we notice that the runtime of the Resub and Mfs2 operators often dominates the overall runtime of LS optimization processes.
DeepGate2: Functionality-Aware Circuit Representation Learning
Circuit representation learning aims to obtain neural representations of circuit elements and has emerged as a promising research direction that can be applied to various EDA and logic reasoning tasks.
SGDP: A Stream-Graph Neural Network Based Data Prefetcher
This paper proposes a novel Stream-Graph neural network-based Data Prefetcher (SGDP).
Conflict-driven Structural Learning Towards Higher Coverage Rate in ATPG
(2) Conflict-driven implication and justification have been applied to increase decision accuracy and solving efficiency.
Heuristics for Vehicle Routing Problem: A Survey and Recent Advances
Vehicle routing is a well-known optimization research topic with significant practical importance.
HardSATGEN: Understanding the Difficulty of Hard SAT Formula Generation and A Strong Structure-Hardness-Aware Baseline
On top of the observations that industrial formulae exhibit clear community structure and oversplit substructures lead to the difficulty in semantic formation of logical structures, we propose HardSATGEN, which introduces a fine-grained control mechanism to the neural split-merge paradigm for SAT formula generation to better recover the structural and computational properties of the industrial benchmarks.
Learning Cut Selection for Mixed-Integer Linear Programming via Hierarchical Sequence Model
Cut selection -- which aims to select a proper subset of the candidate cuts to improve the efficiency of solving MILPs -- heavily depends on (P1) which cuts should be preferred, and (P2) how many cuts should be selected.
SATformer: Transformer-Based UNSAT Core Learning
This paper introduces SATformer, a novel Transformer-based approach for the Boolean Satisfiability (SAT) problem.
Branch Ranking for Efficient Mixed-Integer Programming via Offline Ranking-based Policy Learning
Deriving a good variable selection strategy in branch-and-bound is essential for the efficiency of modern mixed-integer programming (MIP) solvers.
LQoCo: Learning to Optimize Cache Capacity Overloading in Storage Systems
Cache plays an important role to maintain high and stable performance (i. e. high throughput, low tail latency and throughput jitter) in storage systems.
A Survey for Solving Mixed Integer Programming via Machine Learning
This paper surveys the trend of leveraging machine learning to solve mixed integer programming (MIP) problems.
Machine Learning Methods in Solving the Boolean Satisfiability Problem
This paper reviews the recent literature on solving the Boolean satisfiability problem (SAT), an archetypal NP-complete problem, with the help of machine learning techniques.
A Data-Driven Column Generation Algorithm For Bin Packing Problem in Manufacturing Industry
The bin packing problem exists widely in real logistic scenarios (e. g., packing pipeline, express delivery), with its goal to improve the packing efficiency and reduce the transportation cost.
Introduction to The Dynamic Pickup and Delivery Problem Benchmark -- ICAPS 2021 Competition
The Dynamic Pickup and Delivery Problem (DPDP) is an essential problem within the logistics domain.
An Improved Reinforcement Learning Algorithm for Learning to Branch
Similar to offline reinforcement learning, we initially train on the demonstration data to accelerate learning massively.
Learning to Reformulate for Linear Programming
In the past decades, a serial of traditional operation research algorithms have been proposed to obtain the optimum of a given LP in a fewer solving time.
A Hierarchical Reinforcement Learning Based Optimization Framework for Large-scale Dynamic Pickup and Delivery Problems
To address this problem, existing methods partition the overall DPDP into fixed-size sub-problems by caching online generated orders and solve each sub-problem, or on this basis to utilize the predicted future orders to optimize each sub-problem further.
Learning-Aided Heuristics Design for Storage System
Computer systems such as storage systems normally require transparent white-box algorithms that are interpretable for human experts.
Learning to Select Cuts for Efficient Mixed-Integer Programming
In the online A/B testing of the product planning problems with more than $10^7$ variables and constraints daily, Cut Ranking has achieved the average speedup ratio of 12. 42% over the production solver without any accuracy loss of solution.
Learning to Optimize Industry-Scale Dynamic Pickup and Delivery Problems
Our method is entirely data driven and thus adaptive, i. e., the relational representation of adjacent vehicles can be learned and corrected by ST-DDGN from data periodically.
Bilevel Learning Model Towards Industrial Scheduling
In this paper, we propose a Bilevel Deep reinforcement learning Scheduler, \textit{BDS}, in which the higher level is responsible for exploring an initial global sequence, whereas the lower level is aiming at exploitation for partial sequence refinements, and the two levels are connected by a sliding-window sampling mechanism.
Block Hankel Tensor ARIMA for Multiple Short Time Series Forecasting
This work proposes a novel approach for multiple time series forecasting.
Transfer Learning-Based Outdoor Position Recovery with Telco Data
In this paper, by leveraging the recently developed transfer learning techniques, we design a novel Telco position recovery framework, called TLoc, to transfer good models in the carefully selected source domains (those fine-grained small subareas) to a target one which originally suffers from poor localization accuracy.
