Search Results for author:
Found 59 papers, 21 papers with code
Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers
Data-free quantization (DFQ), which facilitates model quantization without real data to address increasing concerns about data security, has garnered significant attention within the model compression community.
On Shaping Gain of Multidimensional Constellation in Linear and Nonlinear Optical Fiber Channel
In this paper, we introduce an analytical nonlinear interference (NLI) power model-based shaping gain estimation method to enable a fast performance evaluation of various MD modulation formats in coherent dual-polarization (DP) optical transmission system.
Chanel-Orderer: A Channel-Ordering Predictor for Tri-Channel Natural Images
This paper shows a proof-of-concept that, given a typical 3-channel images but in a randomly permuted channel order, a model (termed as Chanel-Orderer) with ad-hoc inductive biases in terms of both architecture and loss functions can accurately predict the channel ordering and knows how to make it right.
GraphCL: Graph-based Clustering for Semi-Supervised Medical Image Segmentation
Semi-supervised learning (SSL) has made notable advancements in medical image segmentation (MIS), particularly in scenarios with limited labeled data and significantly enhancing data utilization efficiency.
ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization
Quantization stands as a pivotal technique for large language model (LLM) serving, yet it poses significant challenges particularly in achieving effective low-bit quantization.
DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization
To address the above issue, we propose a novel distribution-driven delta compression framework DeltaDQ, which utilizes Group-wise Dropout and Separate Quantization to achieve ultra-high compression for the delta weight.
Multidimensional Voronoi Constellations vs. Short Blocklength Probabilistic Shaping: A Comparison for Multilevel Coding Approach
Performance of concatenated multilevel coding with probabilistic shaping (PS) and Voronoi constellations (VCs) is analysed over AWGN channel.
ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition
This equivalence motivates an ID-preserving sampling algorithm, which operates over an adjusted gradient vector field, enabling the generation of fake face recognition datasets that approximate the distribution of real-world faces.
Enhancing Preference-based Linear Bandits via Human Response Time
Theoretical and empirical analyses show that for queries with strong preferences, response times complement choices by providing extra information about preference strength, leading to significantly improved utility estimation.
Safety Layers in Aligned Large Language Models: The Key to LLM Security
Aligned LLMs are secure, capable of recognizing and refusing to answer malicious questions.
DiffPoGAN: Diffusion Policies with Generative Adversarial Networks for Offline Reinforcement Learning
Offline reinforcement learning (RL) can learn optimal policies from pre-collected offline datasets without interacting with the environment, but the sampled actions of the agent cannot often cover the action distribution under a given state, resulting in the extrapolation error issue.
Effectively Compress KV Heads for LLM
The advent of pre-trained large language models (LLMs) has revolutionized various natural language processing tasks.
Wukong: Towards a Scaling Law for Large-Scale Recommendation
Scaling laws play an instrumental role in the sustainable improvement in model quality.
Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation
We study a mismatch between the deep learning recommendation models' flat architecture, common distributed training paradigm and hierarchical data center topology.
Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
To support various applications, a prevalent and efficient approach for business owners is leveraging their valuable datasets to fine-tune a pre-trained LLM through the API provided by LLM owners or cloud servers.
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
We propose a unified protein language model, xTrimoPGLM, to address these two types of tasks simultaneously through an innovative pre-training framework.
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection
Despite considerable advances in automated fake news detection, due to the timely nature of news, it remains a critical open question how to effectively predict the veracity of news articles based on limited fact-checks.
On the Performance of Multidimensional Constellation Shaping for Linear and Nonlinear Optical Fiber Channel
Multidimensional constellation shaping of up to 32 dimensions with different spectral efficiencies are compared through AWGN and fiber-optic simulations.
A Robust Integrated Multi-Strategy Bus Control System via Deep Reinforcement Learning
An efficient urban bus control system has the potential to significantly reduce travel delays and streamline the allocation of transportation resources, thereby offering enhanced and user-friendly transit services to passengers.
Proximity-Informed Calibration for Deep Neural Networks
We examine the problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples.
How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence
In general, a large scale of testing in simulation environment is conducted and then the learned driving knowledge is transferred to the real world, so how to adapt driving knowledge learned in simulation to reality becomes a critical issue.
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains.
Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness
Trustworthy machine learning is of primary importance to the practical deployment of deep learning models.
Probabilistic Knowledge Distillation of Face Ensembles
Mean ensemble (i. e. averaging predictions from multiple models) is a commonly-used technique in machine learning that improves the performance of each individual model.
Birds of a Feather Trust Together: Knowing When to Trust a Classifier via Adaptive Neighborhood Aggregation
How do we know when the predictions made by a classifier can be trusted?
Coordinating CAV Swarms at Intersections with a Deep Learning Model
Connected and automated vehicles (CAVs) are viewed as a special kind of robots that have the potential to significantly improve the safety and efficiency of traffic.
lo-fi: distributed fine-tuning without communication
When fine-tuning DeiT-base and DeiT-large on ImageNet, this procedure matches accuracy in-distribution and improves accuracy under distribution shift compared to the baseline, which observes the same amount of data but communicates gradients at each step.
Neural PCA for Flow-Based Representation Learning
Without exploiting any label information, the principal components recovered store the most informative elements in their \emph{leading} dimensions and leave the negligible in the \emph{trailing} ones, allowing for clear performance improvements of $5\%$-$10\%$ in downstream tasks.
Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations
In this work, we identify the roots of this challenge as the failure of a learned continuous policy to satisfy the discrete plan implicit in the demonstration.
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning
With the dramatically increased number of parameters in language models, sparsity methods have received ever-increasing research focus to compress and accelerate the models.
Neighborhood Attention Transformer
We present Neighborhood Attention (NA), the first efficient and scalable sliding-window attention mechanism for vision.
Ranked #123 on
Semantic Segmentation
on ADE20K
A General Framework for Debiasing in CTR Prediction
Most of the existing methods for debaising in click-through rate (CTR) prediction depend on an oversimplified assumption, i. e., the click probability is the product of observation probability and relevance probability.
Probabilistic Contrastive Loss for Self-Supervised Learning
This paper proposes a probabilistic contrastive loss function for self-supervised learning.
Enhancing Top-N Item Recommendations by Peer Collaboration
In this paper, we plan to exploit such redundancy phenomena to improve the performance of RS.
R4: A Framework for Route Representation and Route Recommendation
For user representation, we utilize a series of historical navigation to extract user preference.
Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty
Consistent state estimation is challenging, especially under the epistemic uncertainties arising from learned (nonlinear) dynamic and observation models.
Supervised Bayesian Specification Inference from Demonstrations
When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.
Spherical Confidence Learning for Face Recognition
Probabilistic Face Embeddings (PFE) is the first attempt to address this dilemma.
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.
1xN Pattern for Pruning Convolutional Neural Networks
We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.
Tackling Variabilities in Autonomous Driving
In this work, we aim to extensively explore the above system design challenges and these challenges motivate us to propose a comprehensive framework that synergistically handles the heterogeneous hardware accelerator design principles, system design criteria, and task scheduling mechanism.
Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking
This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN), for visual object tracking.
PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers
PipeTransformer automatically adjusts the pipelining and data parallelism by identifying and freezing some layers during the training, and instead allocates resources for training of the remaining active layers.
Hypersphere Face Uncertainty Learning
To address these issues, in this paper, we propose a novel framework for face uncertainty learning in hyperspherical space.
PyTorch Distributed: Experiences on Accelerating Data Parallel Training
This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module.
Quantum Inspired Word Representation and Computation
Word meaning has different aspects, while the existing word representation "compresses" these aspects into a single vector, and it needs further analysis to recover the information in different dimensions.
Identifying through Flows for Recovering Latent Representations
Yet, most deep generative models do not address the question of identifiability, and thus fail to deliver on the promise of the recovery of the true latent sources that generate the observations.
Self-Balanced Dropout
Dropout is known as an effective way to reduce overfitting via preventing co-adaptations of units.
A Prism Module for Semantic Disentanglement in Name Entity Recognition
To solve this problem, we propose a prism module to disentangle the semantic aspects of words and reduce noise at the input layer of a model.
Ranked #53 on
Named Entity Recognition (NER)
on CoNLL 2003 (English)
Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View
Diachronic word embeddings have been widely used in detecting temporal changes.
Bayesian Inference of Temporal Task Specifications from Demonstrations
When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.
Generalize Symbolic Knowledge With Neural Rule Engine
As neural networks have dominated the state-of-the-art results in a wide range of NLP tasks, it attracts considerable attention to improve the performance of neural models by integrating symbolic knowledge.
From Random to Supervised: A Novel Dropout Mechanism Integrated with Global Information
Dropout is used to avoid overfitting by randomly dropping units from the neural networks during training.
Analogical Reasoning on Chinese Morphological and Semantic Relations
Analogical reasoning is effective in capturing linguistic regularities.
Achievable Information Rates for Nonlinear Fiber Communication via End-to-end Autoencoder Learning
Machine learning is used to compute achievable information rates (AIRs) for a simplified fiber channel.
RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
Recent advances in deep learning have led various applications to unprecedented achievements, which could potentially bring higher intelligence to a broad spectrum of mobile and ubiquitous applications.
Initializing Convolutional Filters with Semantic Features for Text Classification
Convolutional Neural Networks (CNNs) are widely used in NLP tasks.
Ngram2vec: Learning Improved Word Representations from Ngram Co-occurrence Statistics
The existing word representation methods mostly limit their information source to word co-occurrence statistics.
