Search Results for author:
Found 39 papers, 26 papers with code
ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data
Despite powering sensitive systems like autonomous vehicles, object detection remains fairly brittle in part due to annotation errors that plague most real-world training datasets.
Quantifying Uncertainty in Answers from any Language Model via Intrinsic and Extrinsic Confidence Assessment
We introduce BSDetector, a method for detecting bad and speculative answers from a pretrained Large Language Model by estimating a numeric confidence score for any output it generated.
Estimating label quality and errors in semantic segmentation data via any model
We study algorithms to automatically detect such annotation errors, in particular methods to score label quality, such that the images with the lowest scores are least likely to be correctly labeled.
Detecting Errors in Numerical Data via any Regression Model
Noise plagues many numerical datasets, where the recorded values in the data may fail to match the true underlying values due to reasons including: erroneous sensors, data entry/processing mistakes, or imperfect human estimates.
Detecting Dataset Drift and Non-IID Sampling via k-Nearest Neighbors
We present a straightforward statistical test to detect certain violations of the assumption that the data are Independent and Identically Distributed (IID).
ActiveLab: Active Learning with Re-Labeling by Multiple Annotators
It is thus common to employ multiple annotators to label data with some overlap between their examples.
Identifying Incorrect Annotations in Multi-Label Classification Data
In multi-label classification, each example in a dataset may be annotated as belonging to one or more classes (or none of the classes).
CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators
For analyzing such data, we introduce CROWDLAB, a straightforward approach to utilize any trained classifier to estimate: (1) A consensus label for each example that aggregates the available annotations; (2) A confidence score for how likely each consensus label is correct; (3) A rating for each annotator quantifying the overall correctness of their labels.
Detecting Label Errors in Token Classification Data
Mislabeled examples are a common issue in real-world data, particularly for tasks like token classification where many labels must be chosen on a fine-grained basis.
Learning under Data Drift with Time-Varying Importance Weights
Real-world deployment of machine learning models is challenging because data evolves over time.
DataPerf: Benchmarks for Data-Centric AI Development
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems.
Back to the Basics: Revisiting Out-of-Distribution Detection Baselines
We study simple methods for out-of-distribution (OOD) image detection that are compatible with any already trained classifier, relying on only its predictions or learned representations.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
A Robust Stacking Framework for Training Deep Graph Models with Multifaceted Node Features
Graph Neural Networks (GNNs) with numerical node features and graph structure as inputs have demonstrated superior performance on various supervised learning tasks with graph data.
Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challenges
We pose two hypotheses: (1) task-agnostic methods might provide advantages in settings with limited data, computation, or high dimensionality, and (2) faster adaptation may be particularly beneficial in continual learning settings, helping to mitigate the effects of catastrophic forgetting.
Benchmarking Multimodal AutoML for Tabular Data with Text Fields
We consider the use of automated supervised learning systems for data tables that not only contain numeric/categorical columns, but one or more text fields as well.
Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features
For supervised learning with tabular data, decision tree ensembles produced via boosting techniques generally dominate real-world applications involving iid training/test sets.
Convergent Boosted Smoothing for Modeling GraphData with Tabular Node Features
Many practical modeling tasks require making predictions using tabular data composed of heterogeneous feature types (e. g., text-based, categorical, continuous, etc.).
Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
We aim to identify how different components in the KD pipeline affect the resulting performance and how much the optimal KD pipeline varies across different datasets/tasks, such as the data augmentation policy, the loss function, and the intermediate representation for transferring the knowledge between teacher and student.
Deep Learning for Functional Data Analysis with Adaptive Basis Layers
Despite their widespread success, the application of deep neural networks to functional data remains scarce today.
Multimodal AutoML on Structured Tables with Text Fields
We design automated supervised learning systems for data tables that not only contain numeric/categorical columns, but text fields as well.
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
Errors in test sets are numerous and widespread: we estimate an average of at least 3. 3% errors across the 10 datasets, where for example label errors comprise at least 6% of the ImageNet validation set.
Flexible Model Aggregation for Quantile Regression
Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions, or to model a diverse population without being overly reductive.
Continuous Doubly Constrained Batch Reinforcement Learning
Reliant on too many experiments to learn good actions, current Reinforcement Learning (RL) algorithms have limited applicability in real-world settings, which can be too expensive to allow exploration.
TraDE: A Simple Self-Attention-Based Density Estimator
We present TraDE, a self-attention-based architecture for auto-regressive density estimation with continuous and discrete valued data.
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation
Automated machine learning (AutoML) can produce complex model ensembles by stacking, bagging, and boosting many individual models like trees, deep networks, and nearest neighbor estimators.
ResNeSt: Split-Attention Networks
It is well known that featuremap attention and multi-path representation are important for visual recognition.
Ranked #8 on
Instance Segmentation
on COCO test-dev
(APM metric)
TraDE: Transformers for Density Estimation
We present TraDE, a self-attention-based architecture for auto-regressive density estimation with continuous and discrete valued data.
Overinterpretation reveals image classification model pathologies
Here, we demonstrate that neural networks trained on CIFAR-10 and ImageNet suffer from overinterpretation, and we find models on CIFAR-10 make confident predictions even when 95% of input images are masked and humans cannot discern salient features in the remaining pixel-subsets.
AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models on an unprocessed tabular dataset such as a CSV file.
Denoising Improves Latent Space Geometry in Text Autoencoders
Neural language models have recently shown impressive gains in unconditional text generation, but controllable generation and manipulation of text remain challenging.
Recognizing Variables from their Data via Deep Embeddings of Distributions
A key obstacle in automated analytics and meta-learning is the inability to recognize when different datasets contain measurements of the same variable.
Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles
The inaccuracy of neural network models on inputs that do not stem from the training data distribution is both problematic and at times unrecognized.
Educating Text Autoencoders: Latent Representation Guidance via Denoising
We prove that this simple modification guides the latent space geometry of the resulting model by encouraging the encoder to map similar texts to similar latent representations.
IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation
Text attribute transfer aims to automatically rewrite sentences such that they possess certain linguistic attributes, while simultaneously preserving their semantic content.
What made you do this? Understanding black-box decisions with sufficient input subsets
Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model.
Low-Rank Bandit Methods for High-Dimensional Dynamic Pricing
We consider dynamic pricing with many products under an evolving but low-dimensional demand model.
Sequence to Better Sequence: Continuous Revision of Combinatorial Structures
Under this model, gradient methods can be used to efficiently optimize the continuous latent factors with respect to inferred outcomes.
Learning Optimal Interventions
Assuming the underlying relationship remains invariant under intervention, we develop efficient algorithms to identify the optimal intervention policy from limited data and provide theoretical guarantees for our approach in a Gaussian Process setting.
Principal Differences Analysis: Interpretable Characterization of Differences between Distributions
We introduce principal differences analysis (PDA) for analyzing differences between high-dimensional distributions.
