Search Results for author:
Found 33 papers, 16 papers with code
An Overview of Uncertainty Calibration for Text Classification and the Role of Distillation
Many recalibration methods have been proposed in the literature for quantifying predictive uncertainty and calibrating model outputs, with varying degrees of complexity.
Pushing the Limits of Large Language Model Quantization via the Linearity Theorem
Quantizing large language models has become a standard way to reduce their memory and computational costs.
The Surprising Effectiveness of Test-Time Training for Abstract Reasoning
TTT significantly improves performance on ARC tasks, achieving up to 6x improvement in accuracy compared to base fine-tuned models; applying TTT to an 8B-parameter language model, we achieve 53% accuracy on the ARC's public validation set, improving the state-of-the-art by nearly 25% for public and purely neural approaches.
Training-Free Activation Sparsity in Large Language Models
Activation sparsity can enable practical inference speedups in large language models (LLMs) by reducing the compute and memory-movement required for matrix multiplications during the forward pass.
Fast Matrix Multiplications for Lookup Table-Quantized LLMs
The deployment of large language models (LLMs) is often constrained by memory bandwidth, where the primary bottleneck is the cost of transferring model parameters from the GPU's global memory to its registers.
A Late-Stage Bitemporal Feature Fusion Network for Semantic Change Detection
Recently some multi-task learning based semantic change detection methods have been proposed to decompose the task into semantic segmentation and binary change detection subtasks.
Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization
Masked Autoencoder (MAE) is a notable method for self-supervised pretraining in visual representation learning.
Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning
The learned preference information is used to progressively guide policy optimization towards policies of interest.
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
Our approach uses an iterative algorithm to decompose each pretrained matrix into a high-precision low-rank component and a memory-efficient quantized component.
Recouple Event Field via Probabilistic Bias for Event Extraction
Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs).
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
A recent alternative formulation instead treats federated learning as a distributed inference problem, where the goal is to infer a global posterior from partitioned client data (Al-Shedivat et al., 2021).
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.
MPCFormer: fast, performant and private Transformer inference with MPC
Through extensive evaluations, we show that MPCFORMER significantly speeds up Transformer inference in MPC settings while achieving similar ML performance to the input model.
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
RLPrompt formulates a parameter-efficient policy network that generates the desired discrete prompt after training with reward.
Ligandformer: A Graph Neural Network for Predicting Compound Property with Robust Interpretation
Robust and efficient interpretation of QSAR methods is quite useful to validate AI prediction rationales with subjective opinion (chemist or biologist expertise), understand sophisticated chemical or biological process mechanisms, and provide heuristic ideas for structure optimization in pharmaceutical industry.
Text Generation with Efficient (Soft) $Q$-Learning
We apply the approach to a wide range of text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
With increasing deployment of machine learning systems in various real-world tasks, there is a greater need for accurate quantification of predictive uncertainty.
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data
We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning
To the best of our knowledge, this is the first work that improves data efficiency of image captioning by utilizing LM pretrained on unimodal data.
CPT: Efficient Deep Neural Network Training via Cyclic Precision
In this paper, we attempt to explore low-precision training from a new perspective as inspired by recent findings in understanding DNN training: we conjecture that DNNs' precision might have a similar effect as the learning rate during DNN training, and advocate dynamic precision along the training trajectory for further boosting the time/energy efficiency of DNN training.
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
With the availability of the fast influence functions, we demonstrate their usefulness in four applications.
DORB: Dynamically Optimizing Multiple Rewards with Bandits
Further, it is important to consider using a dynamic combination and curriculum of metric rewards that flexibly changes over time.
Franck-Condon tuning of optical cycling centers by organic functionalization
Laser induced electronic excitations that spontaneously emit photons and decay directly to the initial ground state ("optical cycling transitions") are used in quantum information and precision measurement for state initialization and readout.
Atomic Physics
Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits
Next, we develop a DistanceNet model which uses these distance measures, or a mixture of these distance measures, as an additional loss function to be minimized jointly with the task's loss function, so as to achieve better unsupervised domain adaptation.
AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning
To address these issues, we present AutoSeM, a two-stage MTL pipeline, where the first stage automatically selects the most useful auxiliary tasks via a Beta-Bernoulli multi-armed bandit with Thompson Sampling, and the second stage learns the training mixing ratio of these selected auxiliary tasks via a Gaussian Process based Bayesian optimization framework.
Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
In this work, we first present a strong pointer-copy mechanism based sequence-to-sequence sentence simplification model, and then improve its entailment and paraphrasing capabilities via multi-task learning with related auxiliary tasks of entailment and paraphrase generation.
Ranked #2 on
Text Simplification
on Newsela
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
An accurate abstractive summary of a document should contain all its salient information and should be logically entailed by the input document.
Ranked #33 on
Text Summarization
on GigaWord
Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs
In this paper, we propose a novel Recurrent Neural Network with an at- tention mechanism (att-RNN) to fuse multimodal features for e ective rumor detection.
Video Denoising and Enhancement via Dynamic Video Layering
Video denoising refers to the problem of removing "noise" from a video sequence.
Towards Improving Abstractive Summarization via Entailment Generation
Abstractive summarization, the task of rewriting and compressing a document into a short summary, has achieved considerable success with neural sequence-to-sequence models.
Spice up Your Chat: The Intentions and Sentiment Effects of Using Emoji
Second, from a perspective of message recipients, we further study the sentiment effects of emojis, as well as their duplications, on verbal messages.
Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated
We obtain a correctness result for the standard eigenvalue decomposition (EVD) based solution to PCA under simple assumptions on the data-noise correlation.
Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated
We obtain a correctness result for the standard eigenvalue decomposition (EVD) based solution to PCA under simple assumptions on the data-noise correlation.
