Search Results for author:
Found 18 papers, 8 papers with code
“Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models
In this paper, we present an empirical property of these representations—”average” approximates “first principal component”.
Towards Adaptive Residual Network Training: A Neural-ODE Perspective
Serving as a crucial factor, the depth of residual networks balances model capacity, performance, and training efficiency.
Physics-Informed Data Denoising for Real-Life Sensing Systems
By obviating the need for ground truth clean data, our method offers a practical denoising solution for real-world applications.
Fast-ELECTRA for Efficient Pre-training
Although ELECTRA offers a significant boost in efficiency, its potential is constrained by the training cost brought by the auxiliary model.
Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models
Medical image classification is a critical problem for healthcare, with the potential to alleviate the workload of doctors and facilitate diagnoses of patients.
Learning Concise and Descriptive Attributes for Visual Recognition
Recent advances in foundation models present new opportunities for interpretable visual recognition -- one can first query Large Language Models (LLMs) to obtain a set of attributes that describe each class, then apply vision-language models to classify images via these attributes.
SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank
To address the annotation bottleneck, we introduce SELFOOD, a self-supervised OOD detection method that requires only in-distribution samples as supervision.
Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification
We show that the limited performance of seed matching is largely due to the label bias injected by the simple seed-match rule, which prevents the classifier from learning reliable confidence for selecting high-quality pseudo-labels.
Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications
Deep neural networks have seen enormous success in various real-world applications.
SoTeacher: A Student-oriented Teacher Network Training Framework for Knowledge Distillation
To fill this gap, we propose a novel student-oriented teacher network training framework SoTeacher, inspired by recent findings that student performance hinges on teacher's capability to approximate the true label distribution of training samples.
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification
Weakly supervised text classification methods typically train a deep neural classifier based on pseudo-labels.
A Heterogeneous Graph Learning Model for Cyber-Attack Detection
To effective and efficient detect cyber-attacks from a huge number of system events in the provenance data, we firstly model the provenance data by a heterogeneous graph to capture the rich context information of each system entities (e. g., process, file, socket, etc.
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
We show that label noise exists in adversarial training.
Perturbation Deterioration: The Other Side of Catastrophic Overfitting
While this phenomenon is commonly explained as overfitting, we observe that it is a twin process: not only does the model catastrophic overfits to one type of perturbation, but also the perturbation deteriorates into random noise.
BFClass: A Backdoor-free Text Classification Framework
To recognize the poisoned subset, we examine the training samples with these identified triggers as the most suspicious token, and check if removing the trigger will change the poisoned model's prediction.
"Average" Approximates "First Principal Component"? An Empirical Analysis on Representations from Neural Language Models
In this paper, we present an empirical property of these representations -- "average" approximates "first principal component".
Data Quality Matters For Adversarial Training: An Empirical Study
Specifically, we first propose a strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental to the adversarial robustness.
Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training
Our goal is to understand why the robustness drops after conducting adversarial training for too long.
