Search Results for author: Jonas Mueller

Found 39 papers, 26 papers with code

ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data

1 code implementation2 Sep 2023 Ulyana Tkachenko, Aditya Thyagarajan, Jonas Mueller

Despite powering sensitive systems like autonomous vehicles, object detection remains fairly brittle in part due to annotation errors that plague most real-world training datasets.

Autonomous Vehicles object-detection +1

Quantifying Uncertainty in Answers from any Language Model via Intrinsic and Extrinsic Confidence Assessment

no code implementations30 Aug 2023 Jiuhai Chen, Jonas Mueller

We introduce BSDetector, a method for detecting bad and speculative answers from a pretrained Large Language Model by estimating a numeric confidence score for any output it generated.

Language Modelling Large Language Model

Estimating label quality and errors in semantic segmentation data via any model

1 code implementation11 Jul 2023 Vedang Lad, Jonas Mueller

We study algorithms to automatically detect such annotation errors, in particular methods to score label quality, such that the images with the lowest scores are least likely to be correctly labeled.

Autonomous Vehicles Semantic Segmentation

Detecting Errors in Numerical Data via any Regression Model

2 code implementations26 May 2023 Hang Zhou, Jonas Mueller, Mayank Kumar, Jane-Ling Wang, Jing Lei

Noise plagues many numerical datasets, where the recorded values in the data may fail to match the true underlying values due to reasons including: erroneous sensors, data entry/processing mistakes, or imperfect human estimates.


Detecting Dataset Drift and Non-IID Sampling via k-Nearest Neighbors

1 code implementation25 May 2023 Jesse Cummings, Elías Snorrason, Jonas Mueller

We present a straightforward statistical test to detect certain violations of the assumption that the data are Independent and Identically Distributed (IID).

ActiveLab: Active Learning with Re-Labeling by Multiple Annotators

1 code implementation27 Jan 2023 Hui Wen Goh, Jonas Mueller

It is thus common to employ multiple annotators to label data with some overlap between their examples.

Active Learning

Identifying Incorrect Annotations in Multi-Label Classification Data

2 code implementations25 Nov 2022 Aditya Thyagarajan, Elías Snorrason, Curtis Northcutt, Jonas Mueller

In multi-label classification, each example in a dataset may be annotated as belonging to one or more classes (or none of the classes).

Classification Label Error Detection +2

CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators

2 code implementations13 Oct 2022 Hui Wen Goh, Ulyana Tkachenko, Jonas Mueller

For analyzing such data, we introduce CROWDLAB, a straightforward approach to utilize any trained classifier to estimate: (1) A consensus label for each example that aggregates the available annotations; (2) A confidence score for how likely each consensus label is correct; (3) A rating for each annotator quantifying the overall correctness of their labels.

Detecting Label Errors in Token Classification Data

2 code implementations8 Oct 2022 Wei-Chen Wang, Jonas Mueller

Mislabeled examples are a common issue in real-world data, particularly for tasks like token classification where many labels must be chosen on a fine-grained basis.

General Classification Token Classification

Back to the Basics: Revisiting Out-of-Distribution Detection Baselines

2 code implementations7 Jul 2022 Johnson Kuan, Jonas Mueller

We study simple methods for out-of-distribution (OOD) image detection that are compatible with any already trained classifier, relying on only its predictions or learned representations.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

A Robust Stacking Framework for Training Deep Graph Models with Multifaceted Node Features

no code implementations16 Jun 2022 Jiuhai Chen, Jonas Mueller, Vassilis N. Ioannidis, Tom Goldstein, David Wipf

Graph Neural Networks (GNNs) with numerical node features and graph structure as inputs have demonstrated superior performance on various supervised learning tasks with graph data.

Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challenges

2 code implementations28 May 2022 Massimo Caccia, Jonas Mueller, Taesup Kim, Laurent Charlin, Rasool Fakoor

We pose two hypotheses: (1) task-agnostic methods might provide advantages in settings with limited data, computation, or high dimensionality, and (2) faster adaptation may be particularly beneficial in continual learning settings, helping to mitigate the effects of catastrophic forgetting.

Continual Learning Continuous Control +3

Benchmarking Multimodal AutoML for Tabular Data with Text Fields

1 code implementation4 Nov 2021 Xingjian Shi, Jonas Mueller, Nick Erickson, Mu Li, Alexander J. Smola

We consider the use of automated supervised learning systems for data tables that not only contain numeric/categorical columns, but one or more text fields as well.

AutoML Benchmarking

Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features

1 code implementation26 Oct 2021 Jiuhai Chen, Jonas Mueller, Vassilis N. Ioannidis, Soji Adeshina, Yangkun Wang, Tom Goldstein, David Wipf

For supervised learning with tabular data, decision tree ensembles produced via boosting techniques generally dominate real-world applications involving iid training/test sets.

Convergent Boosted Smoothing for Modeling GraphData with Tabular Node Features

no code implementations ICLR 2022 Jiuhai Chen, Jonas Mueller, Vassilis N. Ioannidis, Soji Adeshina, Yangkun Wang, Tom Goldstein, David Wipf

Many practical modeling tasks require making predictions using tabular data composed of heterogeneous feature types (e. g., text-based, categorical, continuous, etc.).

Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing

no code implementations EMNLP (sustainlp) 2021 Haoyu He, Xingjian Shi, Jonas Mueller, Zha Sheng, Mu Li, George Karypis

We aim to identify how different components in the KD pipeline affect the resulting performance and how much the optimal KD pipeline varies across different datasets/tasks, such as the data augmentation policy, the loss function, and the intermediate representation for transferring the knowledge between teacher and student.

Data Augmentation Hyperparameter Optimization

Deep Learning for Functional Data Analysis with Adaptive Basis Layers

2 code implementations19 Jun 2021 Junwen Yao, Jonas Mueller, Jane-Ling Wang

Despite their widespread success, the application of deep neural networks to functional data remains scarce today.

Dimensionality Reduction

Multimodal AutoML on Structured Tables with Text Fields

2 code implementations ICML Workshop AutoML 2021 Xingjian Shi, Jonas Mueller, Nick Erickson, Mu Li, Alex Smola

We design automated supervised learning systems for data tables that not only contain numeric/categorical columns, but text fields as well.


Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks

2 code implementations26 Mar 2021 Curtis G. Northcutt, Anish Athalye, Jonas Mueller

Errors in test sets are numerous and widespread: we estimate an average of at least 3. 3% errors across the 10 datasets, where for example label errors comprise at least 6% of the ImageNet validation set.

BIG-bench Machine Learning

Flexible Model Aggregation for Quantile Regression

1 code implementation26 Feb 2021 Rasool Fakoor, Taesup Kim, Jonas Mueller, Alexander J. Smola, Ryan J. Tibshirani

Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions, or to model a diverse population without being overly reductive.

Econometrics Prediction Intervals +1

Continuous Doubly Constrained Batch Reinforcement Learning

1 code implementation NeurIPS 2021 Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Pratik Chaudhari, Alexander J. Smola

Reliant on too many experiments to learn good actions, current Reinforcement Learning (RL) algorithms have limited applicability in real-world settings, which can be too expensive to allow exploration.

reinforcement-learning Reinforcement Learning (RL)

TraDE: A Simple Self-Attention-Based Density Estimator

no code implementations1 Jan 2021 Rasool Fakoor, Pratik Anil Chaudhari, Jonas Mueller, Alex Smola

We present TraDE, a self-attention-based architecture for auto-regressive density estimation with continuous and discrete valued data.

Density Estimation Out-of-Distribution Detection

Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation

1 code implementation NeurIPS 2020 Rasool Fakoor, Jonas Mueller, Nick Erickson, Pratik Chaudhari, Alexander J. Smola

Automated machine learning (AutoML) can produce complex model ensembles by stacking, bagging, and boosting many individual models like trees, deep networks, and nearest neighbor estimators.

AutoML Data Augmentation

TraDE: Transformers for Density Estimation

no code implementations6 Apr 2020 Rasool Fakoor, Pratik Chaudhari, Jonas Mueller, Alexander J. Smola

We present TraDE, a self-attention-based architecture for auto-regressive density estimation with continuous and discrete valued data.

Density Estimation Out-of-Distribution Detection

Overinterpretation reveals image classification model pathologies

2 code implementations NeurIPS 2021 Brandon Carter, Siddhartha Jain, Jonas Mueller, David Gifford

Here, we demonstrate that neural networks trained on CIFAR-10 and ImageNet suffer from overinterpretation, and we find models on CIFAR-10 make confident predictions even when 95% of input images are masked and humans cannot discern salient features in the remaining pixel-subsets.

Classification General Classification +1

AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data

8 code implementations13 Mar 2020 Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, Alexander Smola

We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models on an unprocessed tabular dataset such as a CSV file.

Neural Architecture Search

Denoising Improves Latent Space Geometry in Text Autoencoders

no code implementations25 Sep 2019 Tianxiao Shen, Jonas Mueller, Regina Barzilay, Tommi Jaakkola

Neural language models have recently shown impressive gains in unconditional text generation, but controllable generation and manipulation of text remain challenging.

Denoising Text Generation

Recognizing Variables from their Data via Deep Embeddings of Distributions

no code implementations11 Sep 2019 Jonas Mueller, Alex Smola

A key obstacle in automated analytics and meta-learning is the inability to recognize when different datasets contain measurements of the same variable.


Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles

no code implementations18 Jun 2019 Siddhartha Jain, Ge Liu, Jonas Mueller, David Gifford

The inaccuracy of neural network models on inputs that do not stem from the training data distribution is both problematic and at times unrecognized.

Bayesian Optimization

Educating Text Autoencoders: Latent Representation Guidance via Denoising

3 code implementations ICML 2020 Tianxiao Shen, Jonas Mueller, Regina Barzilay, Tommi Jaakkola

We prove that this simple modification guides the latent space geometry of the resulting model by encouraging the encoder to map similar texts to similar latent representations.

Denoising Style Transfer +1

IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation

3 code implementations IJCNLP 2019 Zhijing Jin, Di Jin, Jonas Mueller, Nicholas Matthews, Enrico Santus

Text attribute transfer aims to automatically rewrite sentences such that they possess certain linguistic attributes, while simultaneously preserving their semantic content.

Style Transfer Text Attribute Transfer +2

What made you do this? Understanding black-box decisions with sufficient input subsets

1 code implementation9 Oct 2018 Brandon Carter, Jonas Mueller, Siddhartha Jain, David Gifford

Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model.

Decision Making

Sequence to Better Sequence: Continuous Revision of Combinatorial Structures

no code implementations ICML 2017 Jonas Mueller, David Gifford, Tommi Jaakkola

Under this model, gradient methods can be used to efficiently optimize the continuous latent factors with respect to inferred outcomes.

Learning Optimal Interventions

no code implementations16 Jun 2016 Jonas Mueller, David N. Reshef, George Du, Tommi Jaakkola

Assuming the underlying relationship remains invariant under intervention, we develop efficient algorithms to identify the optimal intervention policy from limited data and provide theoretical guarantees for our approach in a Gaussian Process setting.

Cannot find the paper you are looking for? You can Submit a new open access paper.