Search Results for author:
Found 37 papers, 18 papers with code
Optimized Gradient Clipping for Noisy Label Learning
Previous research has shown that constraining the gradient of loss function with respect to model-predicted probabilities can enhance the model robustness against noisy labels.
Revisiting Energy-Based Model for Out-of-Distribution Detection
We recognize the "energy barrier" in OOD detection, which characterizes the energy difference between in-distribution (ID) and OOD samples and eases detection.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Active Negative Loss: A Robust Framework for Learning with Noisy Labels
By replacing MAE in APL with our proposed NNLFs, we enhance APL and present a new framework called Active Negative Loss (ANL).
MagicFace: Training-free Universal-Style Human Image Customized Synthesis
In the first stage, RSA enables the latent image to query features from all reference concepts simultaneously, extracting the overall semantic understanding to facilitate the initial semantic layout establishment.
TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Data
Achieving this goal poses non-trivial challenges: 1) data selection requires accurate data representations that reflect the training samples' quality, 2) considering the diverse nature of instruction datasets, and 3) ensuring the efficiency of the coreset selection algorithm for large models.
Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient
We instead propose a novel optimization-based structural pruning that learns the pruning masks in a probabilistic space directly by optimizing the loss of the pruned model.
DreamText: High Fidelity Scene Text Synthesis
Our key idea is to reconstruct the diffusion training process, introducing more refined guidance tailored to this task, to expose and rectify the model's attention at the character level and strengthen its learning of text regions.
Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost
We aim at exploiting additional auxiliary labels from an independent (auxiliary) task to boost the primary task performance which we focus on, while preserving a single task inference cost of the primary task.
ELiTe: Efficient Image-to-LiDAR Knowledge Transfer for Semantic Segmentation
Cross-modal knowledge transfer enhances point cloud representation learning in LiDAR semantic segmentation.
Robust Fine-tuning for Pre-trained 3D Point Cloud Models
We apply this robust fine-tuning method to mainstream 3D point cloud pre-trained models and evaluate the quality of model parameters and the degradation of downstream task performance.
PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steering
This prior information is encoded into the attention weights, which are then integrated into the self-attention layers of the generator to guide the synthesis process.
Out-of-Distribution Detection using Neural Activation Prior
Our neural activation prior is based on a key observation that, for a channel before the global pooling layer of a fully trained neural network, the probability of a few neurons being activated with a large response by an in-distribution (ID) sample is significantly higher than that by an OOD sample.
PoseIRM: Enhance 3D Human Pose Estimation on Unseen Camera Settings via Invariant Risk Minimization
A natural solution is to artificially synthesize some samples i. e. 2D-3D pose pairs under massive new camera settings.
Point Cloud Part Editing: Segmentation, Generation, Assembly, and Selection
Based on this process, we introduce SGAS, a model for part editing that employs two strategies: feature disentanglement and constraint.
High-fidelity Person-centric Subject-to-Image Synthesis
Specifically, we first develop two specialized pre-trained diffusion models, i. e., Text-driven Diffusion Model (TDM) and Subject-augmented Diffusion Model (SDM), for scene and person generation, respectively.
Spurious Feature Diversification Improves Out-of-distribution Generalization
Contrary to the conventional wisdom that focuses on learning invariant features for better OOD performance, our findings suggest that incorporating a large number of diverse spurious features weakens their individual contributions, leading to improved overall OOD generalization performance.
Test-Time Compensated Representation Learning for Extreme Traffic Forecasting
In this paper, we identify an underexplored problem in multivariate traffic series prediction: extreme events.
Probabilistic Bilevel Coreset Selection
The goal of coreset selection in supervised learning is to produce a weighted subset of data, so that training only on the subset achieves similar performance as training on the entire dataset.
Model Agnostic Sample Reweighting for Out-of-Distribution Learning
The overfitting issue is addressed by considering a bilevel formulation to search for the sample reweighting, in which the generalization complexity depends on the search space of sample weights instead of the model size.
Normalizing Flow with Variational Latent Representation
The resulting method is significantly more powerful than the standard normalization flow approach for generating data distributions with multiple modes.
DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics
Specifically, we first reserve a short trajectory of global model snapshots on the server.
Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization
Since at each round, the number of tunable parameters optimized on the server side equals the number of participating clients (thus independent of the model size), we are able to train a global model with massive parameters using only a small amount of proxy data (e. g., around one hundred samples).
Fast Adversarial Training with Adaptive Step Size
While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet.
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
In this paradigm, the synthesized data from the PLM acts as the carrier of knowledge, which is used to train a task-specific model with orders of magnitude fewer parameters than the PLM, achieving both higher performance and efficiency than prompt-based zero-shot learning methods on PLMs.
Finding Dynamics Preserving Adversarial Winning Tickets
Modern deep neural networks (DNNs) are vulnerable to adversarial attacks and adversarial training has been shown to be a promising method for improving the adversarial robustness of DNNs.
Safe Screening for Sparse Conditional Random Fields
To the best of our knowledge, this is the first screening method which introduces the dual optimum estimation technique -- by carefully exploring and exploiting the strong convexity and the complex structure of the dual problem -- in static screening methods to dynamic screening.
Efficient Neural Network Training via Forward and Backward Propagation Sparsification
For the latter step, instead of using the chain rule based gradient estimators as in existing methods, we propose a variance reduced policy gradient estimator, which only requires two forward passes without backward propagation, thus achieving completely sparse training.
A Structure Feature Algorithm for Multi-modal Forearm Registration
Augmented reality technology based on image registration is becoming increasingly popular for the convenience of pre-surgery preparation and medical education.
Effective Sparsification of Neural Networks with Global Sparsity Constraint
Weight pruning is an effective technique to reduce the model size and inference time for deep neural networks in real-world deployments.
Effective Training of Sparse Neural Networks under Global Sparsity Constraint
An appealing feature of ProbMask is that the amounts of weight redundancy can be learned automatically via our constraint and thus we avoid the problem of tuning pruning rates individually for different layers in a network.
How to Characterize The Landscape of Overparameterized Convolutional Neural Networks
With the help of a new technique called {\it neural network grafting}, we demonstrate that even during the entire training process, feature distributions of differently initialized networks remain similar at each layer.
Parsimonious Quantile Regression of Financial Asset Tail Dynamics via Sequential Learning
We propose a parsimonious quantile regression framework to learn the dynamic tail behaviors of financial asset returns.
Convex Formulation of Overparameterized Deep Neural Networks
This new analysis is consistent with empirical observations that deep neural networks are capable of learning efficient feature representations.
A Sufficient Condition for Convergences of Adam and RMSProp
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex setting via a few simple counterexamples.
Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction
Time series prediction has been studied in a variety of domains.
Safe Element Screening for Submodular Function Minimization
Relying on this study, we subsequently propose a novel safe screening method to quickly identify the elements guaranteed to be included (we refer to them as active) or excluded (inactive) in the final optimal solution of SFM during the optimization process.
Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction
By noting that sparse SVMs induce sparsities in both feature and sample spaces, we propose a novel approach, which is based on accurate estimations of the primal and dual optima of sparse SVMs, to simultaneously identify the inactive features and samples that are guaranteed to be irrelevant to the outputs.
