Search Results for author:
Found 26 papers, 22 papers with code
OpenAI o1 System Card
The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought.
Can a Confident Prior Replace a Cold Posterior?
First, we introduce a "DirClip" prior that is practical to sample and nearly matches the performance of a cold posterior.
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of humans to supervise model behavior - for example, to evaluate whether a model faithfully followed instructions or generated safe outputs.
Simple and Fast Group Robustness by Automatic Feature Reweighting
A major challenge to out-of-distribution generalization is reliance on spurious features -- patterns that are predictive of the class label in the training data distribution, but not causally related to the target.
FlexiViT: One Model for All Patch Sizes
Vision Transformers convert images to sequences by slicing them into patches.
On Feature Learning in the Presence of Spurious Correlations
Deep classifiers are known to rely on spurious features $\unicode{x2013}$ patterns which are correlated with the target on the training data but not inherently relevant to the learning problem, such as the image backgrounds when classifying the foregrounds.
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations
Neural network classifiers can largely rely on simple spurious features, such as backgrounds, to make predictions.
Ranked #1 on
Out-of-Distribution Generalization
on UrbanCars
On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
In Bayesian regression, we often use a Gaussian observation model, where we control the level of aleatoric uncertainty with a noise variance parameter.
Bayesian Model Selection, the Marginal Likelihood, and Generalization
We provide a partial remedy through a conditional marginal likelihood, which we show is more aligned with generalization, and practically valuable for large-scale hyperparameter learning, such as in deep kernel learning.
Dangers of Bayesian Model Averaging under Covariate Shift
Approximate Bayesian inference for neural networks is considered a robust alternative to standard training, often providing good performance on out-of-distribution data.
Does Knowledge Distillation Really Work?
Knowledge distillation is a popular technique for training a small student network to emulate a larger teacher model, such as an ensemble of networks.
What Are Bayesian Neural Network Posteriors Really Like?
The posterior over Bayesian neural network (BNN) parameters is extremely high-dimensional and non-convex.
Learning Invariances in Neural Networks from Training Data
Invariances to translations have imbued convolutional neural networks with powerful generalization properties.
Learning Invariances in Neural Networks
Invariances to translations have imbued convolutional neural networks with powerful generalization properties.
Why Normalizing Flows Fail to Detect Out-of-Distribution Data
Detecting out-of-distribution (OOD) data is crucial for robust machine learning systems.
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
The translation equivariance of convolutional layers enables convolutional neural networks to generalize well on image problems.
Bayesian Deep Learning and a Probabilistic Perspective of Generalization
The key distinguishing property of a Bayesian approach is marginalization, rather than using a single setting of weights.
Semi-Supervised Learning with Normalizing Flows
Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood.
Semi-Supervised Image Classification
Semi-Supervised Text Classification
Subspace Inference for Bayesian Deep Learning
Bayesian inference was once a gold standard for learning with neural networks, providing accurate full predictive distributions and well calibrated uncertainty.
A Simple Baseline for Bayesian Uncertainty in Deep Learning
We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning.
There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
Presently the most successful approaches to semi-supervised learning are based on consistency regularization, whereby a model is trained to be robust to small perturbations of its inputs and parameters.
Averaging Weights Leads to Wider Optima and Better Generalization
Deep neural networks are typically trained by optimizing a loss function with an SGD variant, in conjunction with a decaying learning rate, until convergence.
Ranked #77 on
Image Classification
on CIFAR-100
(using extra training data)
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
The loss functions of deep neural networks are complex and their geometric properties are not well understood.
Tensor Train decomposition on TensorFlow (T3F)
Tensor Train decomposition is used across many branches of machine learning.
Mathematical Software Numerical Analysis
Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition
We propose a method (TT-GP) for approximate inference in Gaussian Process (GP) models.
Faster variational inducing input Gaussian process classification
However, the new lower bound depends on $O(m^2)$ variational parameter, which makes optimization challenging in case of big m. In this work we develop a new approach for training inducing input GP models for classification problems.
