Search Results for author:
Found 118 papers, 54 papers with code
Analysis and Improvement of Policy Gradient Estimation
We also theoretically show that PGPE with the optimal baseline is more preferable than REINFORCE with the optimal baseline in terms of the variance of gradient estimates.
Semi-Supervised Information-Maximization Clustering
Semi-supervised clustering aims to introduce prior knowledge in the decision process of a clustering algorithm.
Transductive Learning with Multi-class Volume Approximation
Given a hypothesis space, the large volume principle by Vladimir Vapnik prioritizes equivalence classes according to their volume in the hypothesis space.
Analysis of Learning from Positive and Unlabeled Data
We next analyze the excess risk when the class prior is estimated from data, and show that the classification accuracy is not sensitive to class prior estimation if the unlabeled data is dominated by the positive data (this is naturally satisfied in inlier-based outlier detection because inliers are dominant in the unlabeled dataset).
Non-Gaussian Component Analysis with Log-Density Gradient Estimation
Non-Gaussian component analysis (NGCA) is aimed at identifying a linear subspace such that the projected data follows a non-Gaussian distribution.
Whitening-Free Least-Squares Non-Gaussian Component Analysis
Non-Gaussian component analysis (NGCA) is an unsupervised linear dimension reduction method that extracts low-dimensional non-Gaussian "signals" from high-dimensional data contaminated with Gaussian noise.
Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data.
Semi-Supervised Classification Based on Classification from Positive and Unlabeled Data
Most of the semi-supervised classification methods developed so far use unlabeled data for regularization purposes under particular distributional assumptions such as the cluster assumption.
Class-prior Estimation for Learning from Positive and Unlabeled Data
Under the assumption that an additional labeled dataset is available, the class prior can be estimated by fitting a mixture of class-wise data distributions to the unlabeled data distribution.
Does Distributionally Robust Supervised Learning Give Robust Classifiers?
Since the DRSL is explicitly formulated for a distribution shift scenario, we naturally expect it to give a robust classifier that can aggressively handle shifted distributions.
Positive-Unlabeled Learning with Non-Negative Risk Estimator
From only positive (P) and unlabeled (U) data, a binary classifier could be trained with PU learning, in which the state of the art is unbiased PU learning.
Semi-Supervised AUC Optimization based on Positive-Unlabeled Learning
Maximizing the area under the receiver operating characteristic curve (AUC) is a standard approach to imbalanced classification.
Learning from Complementary Labels
Collecting complementary labels would be less laborious than collecting ordinary labels, since users do not have to carefully choose the correct class from a long list of candidate classes.
Mode-Seeking Clustering and Density Ridge Estimation via Direct Estimation of Density-Derivative-Ratios
Based on the proposed estimator, novel methods both for mode-seeking clustering and density ridge estimation are developed, and the respective convergence rates to the mode and ridge of the underlying density are also established.
Information-Theoretic Representation Learning for Positive-Unlabeled Classification
Recent advances in weakly supervised classification allow us to train a classifier only from positive and unlabeled (PU) data.
Binary Classification from Positive-Confidence Data
Can we learn a binary classifier from only positive data, without any negative data or unlabeled data?
Classification from Pairwise Similarity and Unlabeled Data
Supervised learning needs a huge amount of labeled data, which can be a big bottleneck under the situation where there is a privacy concern or labeling cost is high.
Active Feature Acquisition with Supervised Matrix Completion
Feature missing is a serious problem in many applications, which may lead to low quality of training data and further significantly degrade the learning performance.
Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
Deep learning with noisy labels is practically challenging, as the capacity of deep models is so high that they can totally memorize these noisy labels sooner or later during training.
Ranked #8 on
Learning with noisy labels
on CIFAR-10N-Random3
Masking: A New Perspective of Noisy Supervision
It is important to learn various types of classifiers given training data with noisy labels.
Ranked #42 on
Image Classification
on Clothing1M
(using extra training data)
Beyond Unfolding: Exact Recovery of Latent Convex Tensor Decomposition under Reshuffling
Exact recovery of tensor decomposition (TD) methods is a desirable property in both unsupervised learning and scientific data analysis.
Matrix Co-completion for Multi-label Classification with Missing Features and Labels
We consider a challenging multi-label classification problem where both feature matrix $\X$ and label matrix $\Y$ have missing entries.
On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
In this paper, we study training arbitrary (from linear to deep) binary classifier from only unlabeled (U) data by ERM.
Alternate Estimation of a Classifier and the Class-Prior from Positive and Unlabeled Data
In this paper, we propose a novel unified approach to estimating the class-prior and training a classifier alternately.
Pumpout: A Meta Approach for Robustly Training Deep Neural Networks with Noisy Labels
To handle these issues, by using the memorization effects of deep neural networks, we may train deep neural networks on the whole dataset only the first few iterations.
SIGUA: Forgetting May Make Learning with Noisy Labels More Robust
Given data with noisy labels, over-parameterized deep networks can gradually memorize the data, and fit everything in the end.
Classification from Positive, Unlabeled and Biased Negative Data
In binary classification, there are situations where negative (N) data are too diverse to be fully labeled and we often resort to positive-unlabeled (PU) learning in these scenarios.
Complementary-Label Learning for Arbitrary Losses and Models
In contrast to the standard classification paradigm where the true class is given to each training pattern, complementary-label learning only uses training patterns each equipped with a complementary label, which only specifies one of the classes that the pattern does not belong to.
Ranked #22 on
Image Classification
on Kuzushiji-MNIST
How does Disagreement Help Generalization against Label Corruption?
Learning with noisy labels is one of the hottest problems in weakly-supervised learning.
Ranked #13 on
Learning with noisy labels
on CIFAR-100N
Revisiting Sample Selection Approach to Positive-Unlabeled Learning: Turning Unlabeled Data into Positive rather than Negative
May there be a new sample selection method that can outperform the latest importance reweighting method in the deep learning age?
A Pseudo-Label Method for Coarse-to-Fine Multi-Label Learning with Limited Supervision
To address the need, we propose a special weakly supervised MLL problem that not only focuses on the situation of limited fine-grained supervision but also leverages the hierarchical relationship between the coarse concepts and the fine-grained ones.
Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
Hence, we consider a new, more realistic and more challenging problem setting, where classifiers have to be trained with noisy labeled data from SD and unlabeled data from TD -- we name it wildly UDA (WUDA).
Unsupervised Domain Adaptation
Wildly Unsupervised Domain Adaptation
Fast and Robust Rank Aggregation against Model Misspecification
Specifically, the properties of our CoarsenRank are summarized as follows: (1) CoarsenRank is designed for mild model misspecification, which assumes there exist the ideal preferences (consistent with model assumption) that locates in a neighborhood of the actual preferences.
Uncoupled Regression from Pairwise Comparison Data
We propose two practical methods for uncoupled regression from pairwise comparison data and show that the learned regression model converges to the optimal model with the optimal parametric convergence rate when the target variable distributes uniformly.
Are Anchor Points Really Indispensable in Label-Noise Learning?
Existing theories have shown that the transition matrix can be learned by exploiting \textit{anchor points} (i. e., data points that belong to a specific class almost surely).
Ranked #18 on
Learning with noisy labels
on CIFAR-10N-Random3
Direction Matters: On Influence-Preserving Graph Summarization and Max-cut Principle for Directed Graphs
On the other hand, compressing the vertices while preserving the directed edge information provides a way to learn the small-scale representation of a directed graph.
Wildly Unsupervised Domain Adaptation and Its Powerful and Efficient Solution
Hence, we consider a new, more realistic and more challenging problem setting, where classifiers have to be trained with noisy labeled data from SD and unlabeled data from TD---we name it wildly UDA (WUDA).
Unsupervised Domain Adaptation
Wildly Unsupervised Domain Adaptation
Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
The recently proposed unlabeled-unlabeled (UU) classification method allows us to train a binary classifier only from two unlabeled datasets with different class priors.
Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning
We consider the situation in which a user has collected a small set of documents on a cohesive topic, and they want to retrieve additional documents on this topic from a large collection.
Searching to Exploit Memorization Effect in Learning from Corrupted Labels
Sample selection approaches are popular in robust learning from noisy labels.
Where is the Bottleneck of Adversarial Learning with Unlabeled Data?
Deep neural networks (DNNs) are incredibly brittle due to adversarial examples.
Learning with Multiple Complementary Labels
In this paper, we propose a novel problem setting to allow MCLs for each example and two ways for learning with MCLs.
Confidence Scores Make Instance-dependent Label-noise Learning Possible
We find with the help of confidence scores, the transition distribution of each instance can be approximately estimated.
Rethinking Class-Prior Estimation for Positive-Unlabeled Learning
Hitherto, the distributional-assumption-free CPE methods rely on a critical assumption that the support of the positive data distribution cannot be contained in the support of the negative data distribution.
Multi-Class Classification from Noisy-Similarity-Labeled Data
We further estimate the transition matrix from only noisy data and build a novel learning system to learn a classifier which can assign noise-free class labels for instances.
Progressive Identification of True Labels for Partial-Label Learning
Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label.
Do We Need Zero Training Loss After Achieving Zero Training Error?
We experimentally show that flooding improves performance and, as a byproduct, induces a double descent curve of the test loss.
Attacks Which Do Not Kill Training Make Adversarial Learning Stronger
Adversarial training based on the minimax formulation is necessary for obtaining adversarial robustness of trained models.
Rethinking Importance Weighting for Deep Learning under Distribution Shift
Under distribution shift (DS) where the training data distribution differs from the test one, a powerful technique is importance weighting (IW) which handles DS in two separate steps: weight estimation (WE) estimates the test-over-training density ratio and weighted classification (WC) trains the classifier from weighted training data.
Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
By this intermediate class, the original transition matrix can then be factorized into the product of two easy-to-estimate transition matrices.
Part-dependent Label Noise: Towards Instance-dependent Label Noise
Learning with the \textit{instance-dependent} label noise is challenging, because it is hard to model such real-world noise.
Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels
To give an affirmative answer, in this paper, we propose a framework called Class2Simi: it transforms data points with noisy class labels to data pairs with noisy similarity labels, where a similarity label denotes whether a pair shares the class label or not.
Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels
In weakly supervised learning, unbiased risk estimator(URE) is a powerful tool for training classifiers when training and test data are drawn from different distributions.
Provably Consistent Partial-Label Learning
Partial-label learning (PLL) is a multi-class classification problem, where each training example is associated with a set of candidate labels.
Class2Simi: A New Perspective on Learning with Label Noise
It is worthwhile to perform the transformation: We prove that the noise rate for the noisy similarity labels is lower than that of the noisy class labels, because similarity labels themselves are robust to noise.
Geometry-aware Instance-reweighted Adversarial Training
The belief was challenged by recent studies where we can maintain the robustness and improve the accuracy.
Pointwise Binary Classification with Pairwise Confidence Comparisons
To alleviate the data requirement for training effective binary classifiers in binary classification, many weakly supervised learning settings have been proposed.
Maximum Mean Discrepancy Test is Aware of Adversarial Attacks
However, it has been shown that the MMD test is unaware of adversarial attacks -- the MMD test failed to detect the discrepancy between natural and adversarial data.
A Survey of Label-noise Representation Learning: Past, Present and Future
Classical machine learning implicitly assumes that labels of the training data are sampled from a clean distribution, which can be too restrictive for real-world scenarios.
SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning
We also instantiate our framework with different combinations, which set the new state of the art on benchmark-simulated and real-world datasets with noisy labels.
On the Role of Pre-training for Meta Few-Shot Learning
One is to (pre-)train a classifier with examples from known classes, and then transfer the pre-trained classifier to unknown classes using the new examples.
Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model
The drastic increase of data quantity often brings the severe decrease of data quality, such as incorrect label annotations, which poses a great challenge for robustly training Deep Neural Networks (DNNs).
Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification
SSC can be solved by a standard (multi-class) classification method, and we use the SSC solution to obtain the final binary classifier through a certain linear-fractional transformation.
Learning Diverse-Structured Networks for Adversarial Robustness
In adversarial training (AT), the main focus has been the objective and optimizer while the model has been less studied, so that the models being used are still those classic ones in standard training (ST).
Provably End-to-end Label-Noise Learning without Anchor Points
In label-noise learning, the transition matrix plays a key role in building statistically consistent classifiers.
Ranked #14 on
Learning with noisy labels
on CIFAR-100N
Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
To estimate the transition matrix from noisy data, existing methods often need to estimate the noisy class-posterior, which could be unreliable due to the overconfidence of neural networks.
Understanding the Interaction of Adversarial Training with Noisy Labels
A recent adversarial training (AT) study showed that the number of projected gradient descent (PGD) steps to successfully attack a point (i. e., find an adversarial example in its proximity) is an effective measure of the robustness of this point.
Meta Discovery: Learning to Discover Novel Classes given Very Limited Data
In this paper, we demystify assumptions behind NCD and find that high-level semantic features should be shared among the seen and unseen classes.
CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
By comparing \textit{non-robust} (normally trained) and \textit{robustified} (adversarially trained) models, we observe that adversarial training (AT) robustifies CNNs by aligning the channel-wise activations of adversarial data with those of their natural counterparts.
Learning from Similarity-Confidence Data
Weakly supervised learning has drawn considerable attention recently to reduce the expensive time and labor consumption of labeling massive data.
Guided Interpolation for Adversarial Training
To enhance adversarial robustness, adversarial training learns deep neural networks on the adversarial variants generated by their natural data.
Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network
Motivated by that classifiers mostly output Bayes optimal labels for prediction, in this paper, we study to directly model the transition from Bayes optimal labels to noisy labels (i. e., Bayes-label transition matrix (BLTM)) and learn a classifier to predict Bayes optimal labels.
NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels
First, we thoroughly investigate noisy labels (NLs) injection into AT's inner maximization and outer minimization, respectively and obtain the observations on when NL injection benefits AT.
Instance Correction for Learning with Open-set Noisy Labels
Lots of approaches, e. g., loss correction and label correction, cannot handle such open-set noisy labels well, since they need training data and test data to share the same label space, which does not hold for learning with open-set noisy labels.
Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
In this way, we also give large-loss but less selected data a try; then, we can better distinguish between the cases (a) and (b) by seeing if the losses effectively decrease with the uncertainty after the try.
Ranked #26 on
Image Classification
on mini WebVision 1.0
To Smooth or Not? When Label Smoothing Meets Noisy Labels
We provide understandings for the properties of LS and NLS when learning with noisy labels.
Ranked #9 on
Learning with noisy labels
on CIFAR-10N-Random3
Reliable Adversarial Distillation with Unreliable Teachers
However, when considering adversarial robustness, teachers may become unreliable and adversarial distillation may not work: teachers are pretrained on their own adversarial data, and it is too demanding to require that teachers are also good at every adversarial data queried by students.
On the Robustness of Average Losses for Partial-Label Learning
Partial-label learning (PLL) utilizes instances with PLs, where a PL includes several candidate labels but only one is the true label (TL).
CausalAdv: Adversarial Robustness through the Lens of Causality
The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning.
Probabilistic Margins for Instance Reweighting in Adversarial Training
Reweighting adversarial data during training has been recently shown to improve adversarial robustness, where data closer to the current decision boundaries are regarded as more critical and given larger weights.
Multi-Class Classification from Single-Class Data with Confidences
We show that without any assumptions on the loss functions, models, and optimizers, we can successfully learn a multi-class classifier from only data of a single class with a rigorous consistency guarantee when confidences (i. e., the class-posterior probabilities for all the classes) are available.
Local Reweighting for Adversarial Training
However, when tested on attacks different from the given attack simulated in training, the robustness may drop significantly (e. g., even worse than no reweighting).
Understanding and Improving Early Stopping for Learning with Noisy Labels
Instead of the early stopping, which trains a whole DNN all at once, we initially train former DNN layers by optimizing the DNN with a relatively large number of epochs.
Ranked #8 on
Learning with noisy labels
on CIFAR-10N-Aggregate
Instance-dependent Label-noise Learning under a Structural Causal Model
In particular, we show that properly modeling the instances will contribute to the identifiability of the label noise transition matrix and thus lead to a better classifier.
Does Adversarial Robustness Really Imply Backdoor Vulnerability?
Based on thorough experiments, we find that such trade-off ignores the interactions between the perturbation budget of adversarial training and the magnitude of the backdoor trigger.
Understanding Generalized Label Smoothing when Learning with Noisy Labels
It was shown that LS serves as a regularizer for training data with hard labels and therefore improves the generalization of the model.
Ranked #15 on
Learning with noisy labels
on CIFAR-10N-Worst
Active Refinement for Multi-Label Learning: A Pseudo-Label Approach
The goal of multi-label learning (MLL) is to associate a given instance with its relevant labels from a set of concepts.
Contrastive Label Disambiguation for Partial Label Learning
Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity.
Exploiting Class Activation Value for Partial-Label Learning
As the first contribution, we empirically show that the class activation map (CAM), a simple technique for discriminating the learning patterns of each class in images, is surprisingly better at making accurate predictions than the model itself on selecting the true label from candidate labels.
Can Label-Noise Transition Matrix Help to Improve Sample Selection and Label Correction?
Existing methods for learning with noisy labels can be generally divided into two categories: (1) sample selection and label correction based on the memorization effect of neural networks; (2) loss correction with the transition matrix.
Unsupervised Federated Learning is Possible
We propose federation of unsupervised learning (FedUL), where the unlabeled data are transformed into surrogate labeled data for each of the clients, a modified model is trained by supervised FL, and the wanted model is recovered from the modified model.
Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
These observations require us to rethink the treatment of noisy labels, and we hope the availability of these two datasets would facilitate the development and evaluation of future learning with noisy label solutions.
PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning
Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity.
Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification
In contrast to others, our method is model-free and even instance-free.
On the Effectiveness of Adversarial Training against Backdoor Attacks
To explore whether adversarial training could defend against backdoor attacks or not, we conduct extensive experiments across different threat models and perturbation budgets, and find the threat model in adversarial training matters.
Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clients
We propose federation of unsupervised learning (FedUL), where the unlabeled data are transformed into surrogate labeled data for each of the clients, a modified model is trained by supervised FL, and the wanted model is recovered from the modified model.
Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation
In label-noise learning, estimating the transition matrix has attracted more and more attention as the matrix plays an important role in building statistically consistent classifiers.
Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack
The AutoAttack (AA) has been the most reliable method to evaluate adversarial robustness when considerable computational resources are available.
FedMT: Federated Learning with Mixed-type Labels
In federated learning (FL), classifiers (e. g., deep networks) are trained on datasets from multiple data centers without exchanging data across them, which improves the sample efficiency.
Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attacks
Adversarial training (AT) with imperfect supervision is significant but receives limited attention.
Generalized Consistent Multi-Class Classification with Rejection to be Compatible with Arbitrary Losses
\emph{Classification with rejection} (CwR) refrains from making a prediction to avoid critical misclassification when encountering test samples that are difficult to classify.
Mitigating Memorization of Noisy Labels by Clipping the Model Prediction
In the presence of noisy labels, designing robust loss functions is critical for securing the generalization performance of deep neural networks.
Fairness Improves Learning from Noisily Labeled Long-Tailed Data
Both long-tailed and noisily labeled data frequently appear in real-world applications and impose significant challenges for learning.
Investigating and Mitigating the Side Effects of Noisy Views for Self-Supervised Clustering Algorithms in Practical Multi-View Scenarios
Multi-view clustering (MVC) aims at exploring category structures among multi-view data in self-supervised manners.
Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning
Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data.
Towards Effective Visual Representations for Partial-Label Learning
Under partial-label learning (PLL) where, for each training instance, only a set of ambiguous candidate labels containing the unknown true label is accessible, contrastive learning has recently boosted the performance of PLL on vision tasks, attributed to representations learned by contrasting the same/different classes of entities.
Enhancing Label Sharing Efficiency in Complementary-Label Learning with Label Augmentation
Our analysis reveals that the efficiency of implicit label sharing is closely related to the performance of existing CLL models.
Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision
Training a classifier exploiting a huge amount of supervised data is expensive or even prohibited in a situation, where the labeling cost is high.
A Universal Unbiased Method for Classification from Aggregate Observations
This motivates the study on classification from aggregate observations (CFAO), where the supervision is provided to groups of instances, instead of individual instances.
Diversity-enhancing Generative Network for Few-shot Hypothesis Adaptation
In this paper, we propose a diversity-enhancing generative network (DEG-Net) for the FHA problem, which can generate diverse unlabeled data with the help of a kernel independence measure: the Hilbert-Schmidt independence criterion (HSIC).
Distribution Shift Matters for Knowledge Distillation with Webly Collected Images
Knowledge distillation aims to learn a lightweight student network from a pre-trained teacher network.
Multi-Label Knowledge Distillation
Existing knowledge distillation methods typically work by imparting the knowledge of output logits or intermediate feature maps from the teacher network to the student network, which is very successful in multi-class single-label learning.
Atom-Motif Contrastive Transformer for Molecular Property Prediction
As motifs in a molecule are significant patterns that are of great importance for determining molecular properties (e. g., toxicity and solubility), overlooking motif interactions inevitably hinders the effectiveness of MPP.
The Selected-completely-at-random Complementary Label is a Practical Weak Supervision for Multi-class Classification
Complementary-label learning is a weakly supervised learning problem in which each training example is associated with one or multiple complementary labels indicating the classes to which it does not belong.
Direct Distillation between Different Domains
Then, we build a fusion-activation mechanism to transfer the valuable domain-invariant knowledge to the student network, while simultaneously encouraging the adapter within the teacher network to learn the domain-specific knowledge of the target data.
Generating Chain-of-Thoughts with a Direct Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought
In this paper, motivated by Vapnik's principle, we propose a novel comparison-based CoT generation algorithm that directly identifies the most promising thoughts with the noisy feedback from the LLM.
Counterfactual Reasoning for Multi-Label Image Classification via Patching-Based Training
In this paper, we provide a causal inference framework to show that the correlative features caused by the target object and its co-occurring objects can be regarded as a mediator, which has both positive and negative impacts on model predictions.
Searching to Exploit Memorization Effect in Learning with Noisy Labels
Sample selection approaches are popular in robust learning from noisy labels.
