Search Results for author:
Found 45 papers, 25 papers with code
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads
We introduce Orthus, an autoregressive (AR) transformer that excels in generating images given textual prompts, answering questions based on visual inputs, and even crafting lengthy image-text interleaved contents.
Exploring Aleatoric Uncertainty in Object Detection via Vision Foundation Models
Datasets collected from the open world unavoidably suffer from various forms of randomness or noiseness, leading to the ubiquity of aleatoric (data) uncertainty.
Aligning CodeLLMs with Direct Preference Optimization
We also contribute a pipeline for collecting preference pairs for DPO on CodeLLMs.
MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
We empirically witness the high data efficiency of our training procedure and find that our method can sustain over 90% performance with an average KV cache compression rate of 60% (and up to 75% in certain extreme scenarios) for popular LLMs like LLaMA2-7B-base and Mistral-7B-v0. 3-base.
Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts
With the rapid progress of diffusion-based content generation, significant efforts are being made to unlearn harmful or copyrighted concepts from pretrained diffusion models (DMs) to prevent potential model misuse.
In-context KV-Cache Eviction for LLMs via Attention-Gate
In supervised fine-tuning, it not only evicts many tokens but also outperforms LoRA-finetuned LLMs on some datasets, such as RTE, where it improves accuracy by 13. 9% while evicting 62. 8% of tokens, showing that effective eviction of redundant tokens can even enhance performance.
Lost in Translation: Latent Concept Misalignment in Text-to-Image Diffusion Models
The root of such misalignment is attributed to the confusion in the latent semantic space of text-to-image diffusion models, and hence we refer to the "a tea cup of iced coke" phenomenon as Latent Concept Misalignment (LC-Mis).
FaceScore: Benchmarking and Enhancing Face Quality in Human Generation
Targeting addressing such an issue, we first assess the face quality of generations from popular pre-trained DMs with the aid of human annotators and then evaluate the alignment between existing metrics with human judgments.
Optimizing Speculative Decoding for Serving Large Language Models Using Goodput
SmartSpec dynamically determines the best speculation length for each request (from 0, i. e., no speculation, to many tokens) -- hence the associated speculative execution costs -- based on a new metric called goodput, which characterizes the current observed load of the entire system and the speculation accuracy.
AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
In this sense, we introduce AdaMoE to realize token-adaptive routing for MoE, where different tokens are permitted to select a various number of experts.
3D-Properties: Identifying Challenges in DPO and Charting a Path Forward
Aligning large language models (LLMs) with human preference has recently gained tremendous attention, with the canonical yet costly RLHF-PPO and the simple and straightforward Direct Preference Optimization (DPO) as two examples.
TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Steps
Distilling latent diffusion models (LDMs) into ones that are fast to sample from is attracting growing research interest.
SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN
Spiking neural network (SNN) has attracted great attention due to its characteristic of high efficiency and accuracy.
SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation
In contrast to vanilla consistency distillation (CD) which distills the ordinary differential equation solvers-based sampling process of a pretrained teacher model into a student, SCott explores the possibility and validates the efficacy of integrating stochastic differential equation (SDE) solvers into CD to fully unleash the potential of the teacher.
CLLMs: Consistency Large Language Models
Parallel decoding methods such as Jacobi decoding show promise for more efficient LLM inference as it breaks the sequential nature of the LLM decoding process and transforms it into parallelizable computation.
Bayesian Exploration of Pre-trained Models for Low-shot Image Classification
Low-shot image classification is a fundamental task in computer vision and the emergence of large-scale vision-language models such as CLIP has greatly advanced the forefront of research in this field.
Improved Operator Learning by Orthogonal Attention
Neural operators, as an efficient surrogate model for learning the solutions of PDEs, have received extensive attention in the field of scientific machine learning.
BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference
In particular, we derive a novel uncertainty iteration principle to characterize the uncertainty dynamics in diffusion, and leverage the last-layer Laplace approximation for efficient Bayesian inference.
LOVECon: Text-driven Training-Free Long Video Editing with ControlNet
Leveraging pre-trained conditional diffusion models for video editing without further tuning has gained increasing attention due to its promise in film production, advertising, etc.
Online Speculative Decoding
Adapting to query distribution mitigates the shifts between the training distribution of the draft model and the query distribution, enabling the draft model to more accurately predict the target model's outputs.
Heterogeneous Multi-Task Gaussian Cox Processes
This paper presents a novel extension of multi-task Gaussian Cox processes for modeling multiple heterogeneous correlated tasks jointly, e. g., classification and regression, via multi-output Gaussian processes (MOGP).
Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks
Text-to-image (T2I) diffusion models (DMs) have shown promise in generating high-quality images from textual descriptions.
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model
The detection of machine-generated text, especially from large language models (LLMs), is crucial in preventing serious social problems resulting from their misuse.
Learning Neural Eigenfunctions for Unsupervised Semantic Segmentation
Unsupervised semantic segmentation is a long-standing challenge in computer vision with great significance.
On Calibrating Diffusion Probabilistic Models
In this work, we observe that the stochastic reverse process of data scores is a martingale, from which concentration bounds and the optional stopping theorem for data scores can be derived.
Confidence-based Reliable Learning under Dual Noises
Deep neural networks (DNNs) have achieved remarkable success in a variety of computer vision tasks, where massive labeled images are routinely required for model optimization.
Neural Eigenfunctions Are Structured Representation Learners
Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network.
Accelerated Linearized Laplace Approximation for Bayesian Deep Learning
Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks.
Deep Ensemble as a Gaussian Process Approximate Posterior
In this way, we relate DE to Bayesian inference to enjoy reliable Bayesian uncertainty.
NeuralEF: Deconstructing Kernels by Deep Neural Networks
Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems.
Deep Ensemble as a Gaussian Process Posterior
Deep Ensemble (DE) is a flexible, feasible, and effective alternative to Bayesian neural networks (BNNs) for uncertainty estimation in deep learning.
Exploring Memorization in Adversarial Training
In this paper, we explore the memorization effect in adversarial training (AT) for promoting a deeper understanding of model capacity, convergence, generalization, and especially robust overfitting of the adversarially trained models.
Accurate and Reliable Forecasting using Stochastic Differential Equations
It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments.
LiBRe: A Practical Bayesian Approach to Adversarial Detection
Despite their appealing flexibility, deep neural networks (DNNs) are vulnerable against adversarial examples.
Black-box Detection of Backdoor Attacks with Limited Information and Data
Although deep neural networks (DNNs) have made rapid progress in recent years, they are vulnerable in adversarial environments.
AutoSync: Learning to Synchronize for Data-Parallel Distributed Deep Learning
Synchronization is a key step in data-parallel distributed machine learning (ML).
Understanding and Exploring the Network with Stochastic Architectures
In this work, we decouple the training of a network with stochastic architectures (NSA) from NAS and provide a first systematical investigation on it as a stand-alone problem.
BayesAdapter: Being Bayesian, Inexpensively and Reliably, via Bayesian Fine-tuning
Despite their theoretical appealingness, Bayesian neural networks (BNNs) are left behind in real-world adoption, mainly due to persistent concerns on their scalability, accessibility, and reliability.
BayesAdapter: Being Bayesian, Inexpensively and Robustly, via Bayesian Fine-tuning
Despite their theoretical appealingness, Bayesian neural networks (BNNs) are falling far behind in terms of adoption in real-world applications compared with normal NNs, mainly due to their limited scalability in training, and low fidelity in their uncertainty estimates.
Adversarial Distributional Training for Robust Deep Learning
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.
Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure
Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights.
Deep Bayesian Structure Networks
Bayesian neural networks (BNNs) introduce uncertainty estimation to deep networks by performing Bayesian inference on network weights.
Cluster Alignment with a Teacher for Unsupervised Domain Adaptation
Deep learning methods have shown promise in unsupervised domain adaptation, which aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution.
Ranked #3 on
Domain Adaptation
on SVNH-to-MNIST
Batch Virtual Adversarial Training for Graph Convolutional Networks
We present batch virtual adversarial training (BVAT), a novel regularization method for graph convolutional networks (GCNs).
Structured Generative Adversarial Networks
We study the problem of conditional generative modeling based on designated semantics or structures.
