Search Results for author:
Found 64 papers, 31 papers with code
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we present the latest model of the Gemini family, Gemini 1. 5 Pro, a highly compute-efficient multimodal mixture-of-experts model capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #19 on
Code Generation
on HumanEval
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Self-Evaluation Improves Selective Generation in Large Language Models
Safe deployment of large language models (LLMs) may benefit from a reliable method for assessing their generated content to determine when to abstain or to selectively generate.
Morse Neural Networks for Uncertainty Quantification
We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points.
Building One-class Detector for Anything: Open-vocabulary Zero-shot OOD Detection Using Text-image Models
Despite considerable effort, the problem remains significantly challenging in deep learning models due to their propensity to output over-confident predictions for OOD inputs.
What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel
A wide breadth of research has devised data augmentation approaches that can improve both accuracy and generalization performance for neural networks.
A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models
In particular, we ask "Given a large pool of prompts, can we automatically score the prompts and ensemble those that are most suitable for a particular downstream dataset, without needing access to labeled validation data?".
Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play
Standard empirical risk minimization (ERM) training can produce deep neural network (DNN) models that are accurate on average but under-perform in under-represented population subgroups, especially when there are imbalanced group distributions in the long-tailed training data.
Improving the Robustness of Summarization Models by Detecting and Removing Input Noise
We present a large empirical study quantifying the sometimes severe loss in performance (up to 12 ROUGE-1 points) from different types of input noise for a range of datasets and model sizes.
Improving Zero-shot Generalization and Robustness of Multi-modal Models
We also show that our method improves across ImageNet shifted datasets, four other datasets, and other model architectures such as LiT.
Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Furthermore, the space of potential low-quality outputs is larger as arbitrary text can be generated and it is important to know when to trust the generated output.
Abstractive Text Summarization
Out-of-Distribution Detection
+1
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness
The most popular approaches to estimate predictive uncertainty in deep learning are methods that combine predictions from multiple neural networks, such as Bayesian neural networks (BNNs) and deep ensembles.
Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift
The concern of overconfident mis-predictions under distributional shift demands extensive reliability research on Graph Neural Networks used in critical tasks in drug discovery.
Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation
We show that patch-based negative augmentation consistently improves robustness of ViTs across a wide set of ImageNet based robustness benchmarks.
Sparse MoEs meet Efficient Ensembles
Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models.
Deep Classifiers with Label Noise Modeling and Distance Awareness
Uncertainty estimation in deep learning has recently emerged as a crucial area of interest to advance reliability and robustness in safety-critical applications.
Soft Calibration Objectives for Neural Networks
When incorporated into training, these soft calibration losses achieve state-of-the-art single-model ECE across multiple datasets with less than 1% decrease in accuracy.
An Instance-Dependent Simulation Framework for Learning with Label Noise
We propose a simulation framework for generating instance-dependent noisy labels via a pseudo-labeling paradigm.
BEDS-Bench: Behavior of EHR-models under Distributional Shift--A Benchmark
Most ML approaches focus on generalization performance on unseen data that are similar to the training data (In-Distribution, or IND).
Task-agnostic Continual Learning with Hybrid Probabilistic Models
Learning new tasks continuously without forgetting on a constantly changing data distribution is essential for real-world problems but extremely challenging for modern deep learning.
A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks.
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in representation space.
Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
In this paper we introduce Uncertainty Baselines: high-quality implementations of standard and state-of-the-art deep learning methods on a variety of tasks.
Exploring the Limits of Out-of-Distribution Detection
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks.
Ranked #2 on
Out-of-Distribution Detection
on CIFAR-10 vs CIFAR-100
(using extra training data)
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier.
What are effective labels for augmented data? Improving robustness with AutoLabel
Despite this, most existing work simply reuses the original label from the clean data, and the choice of label accompanying the augmented data is relatively less explored.
Why Are Bootstrapped Deep Ensembles Not Better?
Ensemble methods have consistently reached state of the art across predictive, uncertainty, and out-of-distribution robustness benchmarks.
Combining Ensembles and Data Augmentation can Harm your Calibration
Ensemble methods which average over multiple neural network predictions are a simple approach to improve a model's calibration and robustness.
Training independent subnetworks for robust prediction
Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.
Bayesian Deep Ensembles via the Neural Tangent Kernel
We explore the link between deep ensembles and Gaussian processes (GPs) through the lens of the Neural Tangent Kernel (NTK): a recent development in understanding the training dynamics of wide neural networks (NNs).
Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
Accurate estimation of predictive uncertainty in modern neural networks is critical to achieve well calibrated predictions and detect out-of-distribution (OOD) inputs.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
Using this one line code change, we achieve state-of-the-art on recent covariate shift benchmarks and an mCE of 60. 28\% on the challenging ImageNet-C dataset; to our knowledge, this is the best result for any model that does not incorporate additional data augmentation or modification of the training pipeline.
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
Bayesian neural networks (BNN) and deep ensembles are principled approaches to estimate the predictive uncertainty of a deep learning model.
Density of States Estimation for Out-of-Distribution Detection
Drawing on the statistical physics notion of ``density of states,'' the DoSE decision rule avoids direct comparison of model probabilities, and instead utilizes the ``probability of the model probability,'' or indeed the frequency of any reasonable statistic.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern deep learning.
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
We propose AugMix, a data processing technique that is simple to implement, adds limited computational overhead, and helps models withstand unforeseen corruptions.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Deep Ensembles: A Loss Landscape Perspective
One possible explanation for this gap between theory and practice is that popular scalable variational Bayesian methods tend to focus on a single mode, whereas deep ensembles tend to explore diverse modes in function space.
Normalizing Flows for Probabilistic Modeling and Inference
In this review, we attempt to provide such a perspective by describing flows through the lens of probabilistic modeling and inference.
Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
To determine whether or not inputs reside in the typical set, we propose a statistically principled, easy-to-implement test using the empirical distribution of model likelihoods.
Likelihood Ratios for Out-of-Distribution Detection
We propose a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}.
Hybrid Models with Deep and Invertible Features
We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i. e. a normalizing flow).
Adapting Auxiliary Losses Using Gradient Similarity
One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations.
Do Deep Generative Models Know What They Don't Know?
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data.
Learning from Delayed Outcomes via Proxies with Applications to Recommender Systems
Predicting delayed outcomes is an important problem in recommender systems (e. g., if customers will finish reading an ebook).
Distribution Matching in Variational Inference
With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations.
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
Variational Approaches for Auto-Encoding Generative Adversarial Networks
In this paper, we develop a principle upon which auto-encoders can be combined with generative adversarial networks by exploiting the hierarchical structure of the generative model.
The Cramer Distance as a Solution to Biased Wasserstein Gradients
We show that the Cram\'er distance possesses all three desired properties, combining the best of the Wasserstein and Kullback-Leibler divergences.
Comparison of Maximum Likelihood and GAN-based training of Real NVPs
We train a generator by maximum likelihood and we also train the same generator architecture by Wasserstein GAN.
Learning Deep Nearest Neighbor Representations Using Differentiable Boundary Trees
We introduce a new method called differentiable boundary tree which allows for learning deep kNN representations.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks.
Learning in Implicit Generative Models
We frame GANs within the wider landscape of algorithms for learning in implicit generative models--models that only specify a stochastic procedure with which to generate data--and relate these ideas to modelling problems in related fields, such as econometrics and approximate Bayesian computation.
The Mondrian Kernel
We introduce the Mondrian kernel, a fast random feature approximation to the Laplace kernel.
Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server
The posterior server allows scalable and robust Bayesian learning in cases where a data set is stored in a distributed manner across a cluster, with each compute node containing a disjoint subset of data.
Approximate Inference with the Variational Holder Bound
We introduce the Variational Holder (VH) bound as an alternative to Variational Bayes (VB) for approximate Bayesian inference.
Mondrian Forests for Large-Scale Regression when Uncertainty Matters
Many real-world regression problems demand a measure of the uncertainty associated with each prediction.
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output.
Particle Gibbs for Bayesian Additive Regression Trees
Additive regression trees are flexible non-parametric models and popular off-the-shelf tools for real-world non-linear regression.
Distributed Bayesian Posterior Sampling via Moment Sharing
We propose a distributed Markov chain Monte Carlo (MCMC) inference algorithm for large scale Bayesian posterior simulation.
Mondrian Forests: Efficient Online Random Forests
Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics.
Inferring ground truth from multi-annotator ordinal data: a probabilistic approach
A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators.
Top-down particle filtering for Bayesian decision trees
Unlike classic decision tree learning algorithms like ID3, C4. 5 and CART, which work in a top-down manner, existing Bayesian algorithms produce an approximation to the posterior distribution by evolving a complete tree (or collection thereof) iteratively via local Monte Carlo modifications to the structure of the tree, e. g., using Markov chain Monte Carlo (MCMC).
