Search Results for author:
Found 71 papers, 38 papers with code
ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
Mixture-of-Experts (MoE) Transformer, the backbone architecture of multiple phenomenal language models, leverages sparsity by activating only a fraction of model parameters for each input token.
Large model enhanced computational ghost imaging
Ghost imaging (GI) achieves 2D image reconstruction through high-order correlation of 1D bucket signals and 2D light field information, particularly demonstrating enhanced detection sensitivity and high-quality image reconstruction via efficient photon collection in scattering media.
LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversation
To fill this gap, we propose LexRAG, the first benchmark to evaluate RAG systems for multi-turn legal consultations.
CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation
To the best of our knowledge, CaseGen is the first benchmark designed to evaluate LLMs in the context of legal case document generation.
SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
To address this gap, we present SuperGPQA, a comprehensive benchmark that evaluates graduate-level knowledge and reasoning capabilities across 285 disciplines.
Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems
This paper is concerned with the approximation of probability distributions known up to normalization constants, with a focus on Bayesian inference for large-scale inverse problems in scientific computing.
LeetDecoding: A PyTorch Library for Exponentially Decaying Causal Linear Attention with CUDA Implementations
The machine learning and data science community has made significant while dispersive progress in accelerating transformer-based large language models (LLMs), and one promising approach is to replace the original causal attention in a generative pre-trained transformer (GPT) with \emph{exponentially decaying causal linear attention}.
Remodeling Peptide-MHC-TCR Triad Binding as Sequence Fusion for Immunogenicity Prediction
The complex nature of tripartite peptide-MHC-TCR interactions is a critical yet underexplored area in immunogenicity prediction.
Catch Causal Signals from Edges for Label Imbalance in Graph Classification
Despite significant advancements in causal research on graphs and its application to cracking label imbalance, the role of edge features in detecting the causal effects within graphs has been largely overlooked, leaving existing methods with untapped potential for further performance gains.
Optimized Gradient Clipping for Noisy Label Learning
Previous research has shown that constraining the gradient of loss function with respect to model-predicted probabilities can enhance the model robustness against noisy labels.
Revisiting Energy-Based Model for Out-of-Distribution Detection
We recognize the "energy barrier" in OOD detection, which characterizes the energy difference between in-distribution (ID) and OOD samples and eases detection.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Active Negative Loss: A Robust Framework for Learning with Noisy Labels
By replacing MAE in APL with our proposed NNLFs, we enhance APL and present a new framework called Active Negative Loss (ANL).
When Spatial meets Temporal in Action Recognition
Capturing the rich temporal evolution of video frames, while preserving their spatial details, is crucial for improving accuracy.
A Survey of Foundation Models for Music Understanding
Music is essential in daily life, fulfilling emotional and entertainment needs, and connecting us personally, socially, and culturally.
LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorch
Multiobjective optimization problems (MOPs) are prevalent in machine learning, with applications in multi-task learning, learning under fairness or robustness constraints, etc.
Convergence of Unadjusted Langevin in High Dimensions: Delocalization of Bias
The unadjusted Langevin algorithm is commonly used to sample probability distributions in extremely high-dimensional settings.
ComKD-CLIP: Comprehensive Knowledge Distillation for Contrastive Language-Image Pre-traning Model
In this paper, we propose a novel approach, ComKD-CLIP: Comprehensive Knowledge Distillation for Contrastive Language-Image Pre-traning Model, which aims to comprehensively distill the knowledge from a large teacher CLIP model into a smaller student model, ensuring comparable performance with significantly reduced parameters.
SentenceVAE: Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
Current large language models (LLMs) primarily utilize next-token prediction method for inference, which significantly impedes their processing speed.
LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformer
To address these challenges, we introduce LayoutDiT, an effective framework that balances content and graphic features to generate high-quality, visually appealing layouts.
Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows
The proposed methodology results in an efficient derivative-free sampler flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI).
Common and Rare Fundus Diseases Identification Using Vision-Language Foundation Model with Knowledge of Over 400 Diseases
Here we introduce RetiZero, a vision-language foundation model that leverages knowledge from over 400 fundus diseases.
Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models
Although Retrieval-Augmented Large Language Models (RALMs) demonstrate their superiority in terms of factuality, they do not consistently outperform the original retrieval-free Language Models (LMs).
Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors
Diffusion models (DMs) have recently shown outstanding capabilities in modeling complex image distributions, making them expressive image priors for solving Bayesian inverse problems.
Gaussian Measures Conditioned on Nonlinear Observations: Consistency, MAP Estimators, and Simulation
The article presents a systematic study of the problem of conditioning a Gaussian random variable $\xi$ on nonlinear observations of the form $F \circ \phi(\xi)$ where $\phi: \mathcal{X} \to \mathbb{R}^N$ is a bounded linear operator and $F$ is nonlinear.
LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives
Large garages are ubiquitous yet intricate scenes that present unique challenges due to their monotonous colors, repetitive patterns, reflective surfaces, and transparent vehicle glass.
Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations
Sequential-in-time methods solve a sequence of training problems to fit nonlinear parametrizations such as neural networks to approximate solution trajectories of partial differential equations over time.
Probabilistic Forecasting with Stochastic Interpolants and Föllmer Processes
We propose a framework for probabilistic forecasting of dynamical systems based on generative modeling.
UMOEA/D: A Multiobjective Evolutionary Algorithm for Uniform Pareto Objectives based on Decomposition
Multiobjective optimization (MOO) is prevalent in numerous applications, in which a Pareto front (PF) is constructed to display optima under various preferences.
LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-frame 3D Human Pose Estimation
Several methods have been proposed to estimate 3D human pose from multi-view images, achieving satisfactory performance on public datasets collected under relatively simple conditions.
HumanReg: Self-supervised Non-rigid Registration of Human Point Cloud
In this paper, we present a novel registration framework, HumanReg, that learns a non-rigid transformation between two human point clouds end-to-end.
Conversational Recommender System and Large Language Model Are Made for Each Other in E-commerce Pre-sales Dialogue
This paper investigates the effectiveness of combining LLM and CRS in E-commerce pre-sales dialogues, proposing two collaboration methods: CRS assisting LLM and LLM assisting CRS.
Provable Probabilistic Imaging using Score-Based Generative Priors
PMC is able to incorporate expressive score-based generative priors for high-quality image reconstruction while also performing uncertainty quantification via posterior sampling.
Sampling via Gradient Flows in the Space of Probability Measures
Our third contribution is to study, and develop efficient algorithms based on Gaussian approximations of the gradient flows; this leads to an alternative to particle methods.
Pixel Adapter: A Graph-Based Post-Processing Approach for Scene Text Image Super-Resolution
Additionally, we introduce the MLP-based Sequential Residual Block (MSRB) for robust feature extraction from text images, and a Local Contour Awareness loss ($\mathcal{L}_{lca}$) to enhance the model's perception of details.
MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language Models
Fine-tuning a pre-trained language model (PLM) emerges as the predominant strategy in many natural language processing applications.
A Gromov--Wasserstein Geometric View of Spectrum-Preserving Graph Coarsening
The study includes a set of experiments to support the theory and method, including approximating the GW distance, preserving the graph spectrum, classifying graphs using spectral information, and performing regression using graph convolutional networks.
Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs
We introduce a priori Sobolev-space error estimates for the solution of nonlinear, and possibly parametric, PDEs using Gaussian process and kernel based methods.
U-NEED: A Fine-grained Dataset for User Needs-Centric E-commerce Conversational Recommendation
In this paper, we construct a user needs-centric E-commerce conversational recommendation dataset (U-NEED) from real-world E-commerce scenarios.
Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes
The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices.
A Flexible Multi-view Multi-modal Imaging System for Outdoor Scenes
The experimental results show that multi-view point clouds greatly improve 3D object detection and tracking accuracy regardless of complex and various outdoor environments.
Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance
The flow in the Gaussian space may be understood as a Gaussian approximation of the flow.
Robust convex biclustering with a tuning-free method
Therefore, we propose a tuning-free method for automatically selecting the optimal robustification parameter with high efficiency.
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning
Prefix-tuning, or more generally continuous prompt tuning, has become an essential paradigm of parameter-efficient transfer learning.
The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines
In the metric aspect, we design the new conversational DER (CDER) evaluation metric, which calculates the SD accuracy at the utterance level.
Combining deep learning and crowdsourcing geo-images to predict housing quality in rural China
Housing quality is an essential proxy for regional wealth, security and health.
Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations
The randomly pivoted partial Cholesky algorithm (RPCholesky) computes a factorized rank-k approximation of an N x N positive-semidefinite (psd) matrix.
Microwave Chirality Imaging for the Early Diagnosis of Neurological Degenerative Diseases
We propose a system to visualize the chirality of the protein in brains, which would be helpful to diagnose early neurological degenerative diseases in vivo.
Interrelate Training and Searching: A Unified Online Clustering Framework for Speaker Diarization
For online speaker diarization, samples arrive incrementally, and the overall distribution of the samples is invisible.
Calibrate and Debias Layer-wise Sampling for Graph Convolutional Networks
This paper revisits the approach from a matrix approximation perspective, and identifies two issues in the existing layer-wise sampling methods: suboptimal sampling probabilities and estimation biases induced by sampling without replacement.
Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention
The massive amount of trainable parameters in the pre-trained language models (PLMs) makes them hard to be deployed to multiple downstream tasks.
Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset
As a Mandarin speech dataset designed for dialog scenarios with high quality and rich annotations, MagicData-RAMC enriches the data diversity in the Mandarin speech community and allows extensive research on a series of speech-related tasks, including automatic speech recognition, speaker diarization, topic detection, keyword search, text-to-speech, etc.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Spatio-temporal Gait Feature with Global Distance Alignment
In this paper, we try to enhance the discrimination of spatio-temporal gait features from two aspects: effective extraction of spatio-temporal gait features and reasonable refinement of extracted features.
Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences
Transformer-based models are not efficient in processing long sequences due to the quadratic space and time complexity of the self-attention modules.
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method
Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism.
Revisiting Layer-wise Sampling in Fast Training for Graph Convolutional Networks
To accelerate the training of graph convolutional networks (GCN), many sampling-based methods have been developed for approximating the embedding aggregation.
A FAIR and AI-ready Higgs boson decay dataset
To enable the reusability of massive scientific datasets by humans and machines, researchers aim to adhere to the principles of findability, accessibility, interoperability, and reusability (FAIR) for data and artificial intelligence (AI) models.
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method
Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
Region-Adaptive Deformable Network for Image Quality Assessment
Image quality assessment (IQA) aims to assess the perceptual quality of images.
Solving and Learning Nonlinear PDEs with Gaussian Processes
The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points.
Fast Statistical Leverage Score Approximation in Kernel Ridge Regression
Nystr\"om approximation is a fast randomized method that rapidly solves kernel ridge regression (KRR) problems through sub-sampling the n-by-n empirical kernel matrix appearing in the objective function.
Accumulations of Projections--A Unified Framework for Random Sketches in Kernel Ridge Regression
Building a sketch of an n-by-n empirical kernel matrix is a common approach to accelerate the computation of many kernel methods.
A Corpus of Very Short Scientific Summaries
We present a new summarisation task, taking scientific articles and producing journal table-of-contents entries in the chemistry domain.
Pre-Trained Models for Heterogeneous Information Networks
We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network.
Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation
The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature.
Deep Attention Aware Feature Learning for Person Re-Identification
Visual attention has proven to be effective in improving the performance of person re-identification.
Ranked #11 on
Person Re-Identification
on DukeMTMC-reID
Towards Learning a Self-inverse Network for Bidirectional Image-to-image Translation
A self-inverse network shares several distinct advantages: only one network instead of two, better generalization and more restricted parameter space.
Regularized Ensembles and Transferability in Adversarial Learning
Despite the considerable success of convolutional neural networks in a broad array of domains, recent research has shown these to be vulnerable to small adversarial perturbations, commonly known as adversarial examples.
Run-and-Inspect Method for Nonconvex Optimization and Global Optimality Bounds for R-Local Minimizers
If no sufficient decrease is found, the current point is called an approximate $R$-local minimizer.
Leveraging High-Dimensional Side Information for Top-N Recommendation
Top-$N$ recommender systems typically utilize side information to address the problem of data sparsity.
Information Retrieval
Content-Based Top-N Recommendation using Heterogeneous Relations
In this paper, we propose a content-based top-$N$ recommender system by learning the global term weights in profiles.
