Search Results for author:
Found 20 papers, 6 papers with code
Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference
We identify specific attention heads in transformer-based LLMs, which we designate as evaluator heads, that are capable of selecting tokens in long inputs that are most significant for inference.
Towards Faster Graph Partitioning via Pre-training and Inductive Inference
Graph partitioning (GP) is a classic problem that divides the node set of a graph into densely-connected blocks.
A Mean Field Ansatz for Zero-Shot Weight Transfer
Specifically, we propose the row-column (RC) ansatz under the mean field point of view, which describes the measure structure of the weights in the neural network (NN) and admits a close measure dynamic.
Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understanding
Current Large Language Models (LLMs) face inherent limitations due to their pre-defined context lengths, which impede their capacity for multi-hop reasoning within extensive textual contexts.
Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory
In particular, the energy function in modern continuous Hopfield networks serves as an explanation for the attention mechanism, which we approximate with a distance-based energy function.
Extreme Video Compression with Pre-trained Diffusion Models
The results showcase the potential of exploiting the temporal relations in video data using generative models.
Extending Context Window of Large Language Models via Semantic Compression
Transformer-based Large Language Models (LLMs) often impose limitations on the length of the text input to ensure the generation of fluent and relevant responses.
High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models
We consider the image transmission problem over a noisy wireless channel via deep learning-based joint source-channel coding (DeepJSCC) along with a denoising diffusion probabilistic model (DDPM) at the receiver.
A Constrained BA Algorithm for Rate-Distortion and Distortion-Rate Functions
The Blahut-Arimoto (BA) algorithm has played a fundamental role in the numerical computation of rate-distortion (RD) functions.
Trading off Quality for Efficiency of Community Detection: An Inductive Method across Graphs
The trained model is then directly generalized to new unseen graphs for online CD without additional optimization, where a better trade-off between quality and efficiency can be achieved.
GPPF: A General Perception Pre-training Framework via Sparsely Activated Multi-Task Learning
SIMT lays the foundation of pre-training with large-scale multi-task multi-domain datasets and is proved essential for stable training in our GPPF experiments.
A Tensor-BTD-based Modulation for Massive Unsourced Random Access
In this letter, we propose a novel tensor-based modulation scheme for massive unsourced random access.
PartialFed: Cross-Domain Personalized Federated Learning via Partial Initialization
The superiority of our algorithm is proved by demonstrating the new state-of-the-art results on cross-domain federated classification and detection.
Harmonic Retrieval with $L_1$-Tucker Tensor Decomposition
We also develop a novel subcarrier recovery method for the proposed model.
Branch and Bound in Mixed Integer Linear Programming Problems: A Survey of Techniques and Trends
In order to improve the speed of B&B algorithms, learning techniques have been introduced in this algorithm recently.
Trading Quality for Efficiency of Graph Partitioning: An Inductive Method across Graphs
IGP is also a generic framework that can capture the permutation invariant partitioning ground-truth of historical snapshots in the offline training and tackle the online GP on graphs with non-fixed number of nodes and clusters.
A Convergent Semi-Proximal Alternating Direction Method of Multipliers for Recovering Internet Traffics from Link Measurements
Our model can be divided into a series of subproblems, which only relate to the traffics in a certain individual time interval.
Optimization and Control
Lower Bound on the Optimal Access Bandwidth of ($K+2,K,2$)-MDS Array Code with Degraded Read Friendly
Accessing the data in the failed disk (degraded read) with low latency is crucial for an erasure-coded storage system.
Information Theory Information Theory
Asymmetric Gained Deep Image Compression With Continuous Rate Adaptation
With the development of deep learning techniques, the combination of deep learning with image compression has drawn lots of attention.
GCN-GAN: A Non-linear Temporal Link Prediction Model for Weighted Dynamic Networks
Different from conventional techniques of temporal link prediction that ignore the potential non-linear characteristics and the informative link weights in the dynamic network, we introduce a novel non-linear model GCN-GAN to tackle the challenging temporal link prediction task of weighted dynamic networks.
