Search Results for author:
Found 299 papers, 181 papers with code
Bridging the Gap Between f-GANs and Wasserstein GANs
Generative adversarial networks (GANs) variants approximately minimize divergences between the model and the data distribution using a discriminator.
Exploring Diffusion Transformer Designs via Grafting
Designing model architectures requires decisions such as selecting operators (e. g., attention, convolution) and configurations (e. g., depth, width).
Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding
In arbitrary-order language models, it is an open question how to sample tokens in parallel from the correct joint distribution.
Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF
In this work, we propose an active learning approach to efficiently select prompt and preference pairs using a risk assessment strategy based on the Sharpe Ratio.
Preference-Guided Diffusion for Multi-Objective Offline Optimization
Offline multi-objective optimization aims to identify Pareto-optimal solutions given a dataset of designs and their objective values.
Inductive Moment Matching
Diffusion models and Flow Matching generate high-quality samples but are slow at inference, and distilling them into few-step models often leads to instability and extensive tuning.
Data Unlearning in Diffusion Models
Existing concept unlearning techniques require an anchor prompt/class/distribution to guide unlearning, which is not available in the data unlearning setting.
FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users
Overall, FSPO achieves an 87% Alpaca Eval winrate on average in generating responses that are personalized to synthetic users and a 72% winrate with real human users in open-ended question answering.
Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale
We introduce convolutional multi-hybrid architectures, with a design grounded on two simple observations.
Training-Free Safe Denoisers for Safe Use of Diffusion Models
There is growing concern over the safety of powerful diffusion models (DMs), as they are often misused to produce inappropriate, not-safe-for-work (NSFW) content or generate copyrighted material or data of individuals who wish to be forgotten.
TFG-Flow: Training-free Guidance in Multimodal Generative Flow
Given an unconditional generative model and a predictor for a target property (e. g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training.
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
Personalized Preference Fine-tuning of Diffusion Models
With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way, enabling generalization to unseen users.
Efficient Scaling of Diffusion Transformers for Text-to-Image Generation
We empirically study the scaling properties of various Diffusion Transformers (DiTs) for text-to-image generation by performing extensive and rigorous ablations, including training scaled DiTs ranging from 0. 3B upto 8B parameters on datasets up to 600M images.
Non-Myopic Multi-Objective Bayesian Optimization
We address this challenge by using hypervolume improvement (HVI) as our scalarization approach, which allows us to use a lower-bound on the Bellman equation to approximate the finite-horizon using a batch expected hypervolume improvement (EHVI) acquisition function (AF) for MOO.
Self-Refining Diffusion Samplers: Enabling Parallelization via Parareal Iterations
In diffusion models, samples are generated through an iterative refinement process, requiring hundreds of sequential model evaluations.
Convolutional Differentiable Logic Gate Networks
With the increasing inference cost of machine learning models, there is a growing interest in models with fast and efficient inference.
TrAct: Making First-layer Pre-Activations Trainable
Thus, we propose the conceptual procedure of (i) a gradient descent step on first layer activations to construct an activation proposal, and (ii) finding the optimal weights of the first layer, i. e., those weights which minimize the squared distance to the activation proposal.
$f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization
Preference optimization has made significant progress recently, with numerous methods developed to align language models with human preferences.
Energy-Based Diffusion Language Models for Text Generation
Unfortunately, these models still underperform the autoregressive counterparts, with the performance gap increasing when reducing the number of sampling steps.
Ranked #3 on
Language Modelling
on OpenWebText
TabDiff: a Multi-Modal Diffusion Model for Tabular Data Generation
In this paper, we introduce TabDiff, a joint diffusion framework that models all multi-modal distributions of tabular data in one model.
Newton Losses: Using Curvature Information for Learning with Differentiable Algorithms
Instead of training the neural network with second-order techniques, we only utilize the loss function's second-order information to replace it by a Newton Loss, while training the network with gradient descent.
Improving Vector-Quantized Image Modeling with Latent Consistency-Matching Diffusion
By embedding discrete representations into a continuous latent space, we can leverage continuous-space latent diffusion models to handle generative modeling of discrete data.
Geometric Trajectory Diffusion Models
In this work, we propose geometric trajectory diffusion models (GeoTDM), the first diffusion model for modeling the temporal distribution of 3D geometric trajectories.
Generalizing Stochastic Smoothing for Differentiation and Gradient Estimation
We deal with the problem of gradient estimation for stochastic differentiable relaxations of algorithms, operators, simulators, and other non-differentiable functions.
Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis
Then we propose a geometry-aware 4D transition network to realize a complex scene-level 4D transition based on the plan, which involves expressive geometrical object deformation.
G2D2: Gradient-guided Discrete Diffusion for image inverse problem solving
Recent literature has effectively utilized diffusion models trained on continuous variables as priors for solving inverse problems.
TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data
Large vision and language assistants have enabled new capabilities for interpreting natural images.
Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation
We further demonstrate that 50-75% of samples can be pruned early in generation with minimal degradation in performance.
Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space
To address this, we introduce Bellman Diffusion, a novel DGM framework that maintains linearity in MDPs through gradient and scalar field modeling.
Calibrated Probabilistic Forecasts for Arbitrary Sequences
Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors, which challenges the validity of forecasts.
TFG: Unified Training-Free Guidance for Diffusion Models
Given an unconditional diffusion model and a predictor for a target property of interest (e. g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training.
CPSample: Classifier Protected Sampling for Guarding Training Data During Diffusion
Diffusion models have a tendency to exactly replicate their training data, especially when trained on small datasets.
Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes
We consider the problem of active learning for global sensitivity analysis of expensive black-box functions.
Unsupervised Anomaly Detection Using Diffusion Trend Analysis
Conventional anomaly detection techniques based on reconstruction via denoising diffusion model are widely used due to their ability to identify anomaly locations and shapes with high performance.
Consistency Flow Matching: Defining Straight Flows with Velocity Consistency
Additionally, we propose a multi-segment training approach for Consistency-FM to enhance expressiveness, achieving a better trade-off between sampling quality and speed.
Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization
AliDiff shifts the target-conditioned chemical distribution towards regions with higher binding affinity and structural rationality, specified by user-defined reward functions, via the preference optimization approach.
Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model
Changen2 is a generative change foundation model that can be trained at scale via self-supervision, and can produce change supervisory signals from unlabeled single-temporal images.
TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning
To fill this gap, we propose TorchSpatial, a learning framework and benchmark for location (point) encoding, which is one of the most fundamental data types of spatial representation learning.
ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts
An under-explored question of PEFT is in extending the pre-training phase without supervised labels; that is, can we adapt a pre-trained foundation model to a new domain via efficient self-supervised pre-training on this new domain?
PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher
The diffusion model performs remarkable in generating high-dimensional content but is computationally intensive, especially during training.
Ranked #1 on
Image Generation
on ImageNet 32x32
State-Free Inference of State-Space Models: The Transfer Function Approach
We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel inference algorithm that is state-free: unlike other proposed algorithms, state-free inference does not incur any significant memory or computational cost with an increase in state size.
Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data
Our main finding is that, in general, approaches that use on-policy sampling or attempt to push down the likelihood on certain responses (i. e., employ a "negative gradient") outperform offline and maximum likelihood objectives.
Disentangling Length from Quality in Direct Preference Optimization
A number of approaches have been developed to control those biases in the classical RLHF literature, but the problem remains relatively under-explored for Direct Alignment Algorithms such as Direct Preference Optimization (DPO).
Mechanistic Design and Scaling of Hybrid Architectures
The development of deep learning architectures is a resource-demanding process, due to a vast design space, long prototyping times, and high compute costs associated with at-scale model training and evaluation.
Contextualized Diffusion Models for Text-Guided Image and Video Generation
To address this issue, we propose a novel and general contextualized diffusion model (ContextDiff) by incorporating the cross-modal context encompassing interactions and alignments between text condition and visual sample into forward and reverse processes.
Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution
Deep learning-based surrogate models have demonstrated remarkable advantages over classical solvers in terms of speed, often achieving speedups of 10 to 1000 times over traditional partial differential equation (PDE) solvers.
Large Language Models are Geographically Biased
Initially, we demonstrate that LLMs are capable of making accurate zero-shot geospatial predictions in the form of ratings that show strong monotonic correlation with ground truth (Spearman's $\rho$ of up to 0. 89).
Segment Any Change
Visual foundation models have achieved remarkable results in zero-shot image classification and segmentation, but zero-shot change detection remains an open problem.
Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
In this paper, we propose a brand new training-free text-to-image generation/editing framework, namely Recaption, Plan and Generate (RPG), harnessing the powerful chain-of-thought reasoning ability of multimodal LLMs to enhance the compositionality of text-to-image diffusion models.
Diffusion Personalization Tuning Free
Large Language Model
+1
Equivariant Graph Neural Operator for Modeling 3D Dynamics
Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods, thanks to the equivariant temporal modeling.
Equivariant Flow Matching with Hybrid Probability Transport
The generation of 3D molecules requires simultaneously deciding the categorical features~(atom types) and continuous features~(atom coordinates).
DiffusionSat: A Generative Foundation Model for Satellite Imagery
Our method outperforms previous state-of-the-art methods for satellite image generation and is the first large-scale generative foundation model for satellite imagery.
Sample Efficient Preference Alignment in LLMs via Active Exploration
For many applications of preference alignment, the cost of acquiring human feedback can be substantial.
Manifold Preserving Guided Diffusion
Despite the recent advancements, conditional image generation still faces challenges of cost, generalizability, and the need for task-specific training.
DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling
Recent methods such as Score Distillation Sampling (SDS) and Variational Score Distillation (VSD) using 2D diffusion models for text-to-3D generation have demonstrated impressive generation quality.
Diffusion Model Alignment Using Direct Preference Optimization
Large language models (LLMs) are fine-tuned using human comparison data with Reinforcement Learning from Human Feedback (RLHF) methods to make them better aligned with users' preferences.
Ranked #2 on
Text-to-Image Generation
on DrawBench
Holistic Evaluation of Text-To-Image Models
The stunning qualitative improvement of recent text-to-image models has led to their widespread attention and adoption.
Calibration by Distribution Matching: Trainable Kernel Calibration Metrics
Calibration ensures that probabilistic forecasts meaningfully capture uncertainty by requiring that predicted probabilities align with empirical frequencies.
Generative Fractional Diffusion Models
To ensure tractable inference and learning, we employ a recently popularized Markov approximation of fBM (MA-fBM) and derive its reverse-time model, resulting in generative fractional diffusion models (GFDM).
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
Experimentally, we test our Score Entropy Discrete Diffusion models (SEDD) on standard language modeling tasks.
Ranked #12 on
Language Modelling
on OpenWebText
GeoLLM: Extracting Geospatial Knowledge from Large Language Models
With GeoLLM, we observe that GPT-3. 5 outperforms Llama 2 and RoBERTa by 19% and 51% respectively, suggesting that the performance of our method scales well with the size of the model and its pretraining dataset.
The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models
Compositionality is a common property in many modalities including natural languages and images, but the compositional generalization of multi-modal models is not well-understood.
Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion
Consistency Models (CM) (Song et al., 2023) accelerate score-based diffusion model sampling at the cost of sample quality but lack a natural way to trade-off quality for speed.
Ranked #1 on
Image Generation
on ImageNet 64x64
(NFE metric)
SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution
Existing digital sensors capture images at fixed spatial and spectral resolutions (e. g., RGB, multispectral, and hyperspectral images), and each combination requires bespoke machine learning models.
Denoising Diffusion Bridge Models
However, for many applications such as image editing, the model input comes from a distribution that is not random noise.
HarvestNet: A Dataset for Detecting Smallholder Farming Activity Using Harvest Piles and Remote Sensing
Small farms contribute to a large share of the productive land in developing countries.
Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
Advances in artificial intelligence (AI) are fueling a new paradigm of discoveries in natural sciences.
Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions
So when applied to large-scale real-world GPS coordinate datasets, which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D).
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Leveraging Hyena's new long-range capabilities, we present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level - an up to 500x increase over previous dense attention-based models.
SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking
This allows us to minimize a variety of divergences between the distribution of sequences generated by an autoregressive model and sequences from a dataset, including divergences with weight on OOD generated sequences.
GEO-Bench: Toward Foundation Models for Earth Monitoring
Recent progress in self-supervision has shown that pre-training large neural networks on vast amounts of unsupervised data can lead to substantial increases in generalization to downstream tasks.
On the Equivalence of Consistency-Type Models: Consistency Models, Consistent Diffusion Models, and Fokker-Planck Regularization
The emergence of various notions of ``consistency'' in diffusion models has garnered considerable attention and helped achieve improved sample quality, likelihood estimation, and accelerated sampling.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Existing methods for gaining such steerability collect human labels of the relative quality of model generations and fine-tune the unsupervised LM to align with these preferences, often with reinforcement learning from human feedback (RLHF).
Ranked #3 on
Preference Mapping
on Reddit TLDR (Unseen)
MADiff: Offline Multi-agent Learning with Diffusion Models
Offline reinforcement learning (RL) aims to learn policies from pre-existing datasets without further interactions, making it a challenging task.
Parallel Sampling of Diffusion Models
Instead of reducing the number of denoising steps (trading quality for speed), in this paper we explore an orthogonal approach: can we run the denoising steps in parallel (trading compute for speed)?
UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild
Visual generative foundation models such as Stable Diffusion show promise in navigating these goals, especially when prompted with arbitrary languages.
Geometric Latent Diffusion Models for 3D Molecule Generation
Generative models, especially diffusion models (DMs), have achieved promising results for generating feature-rich geometries and advancing foundational science problems such as molecule design.
Ranked #2 on
Unconditional Molecule Generation
on QM9
CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations
To directly leverage the abundant geospatial information associated with images in pre-training, fine-tuning, and inference stages, we present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images.
MUDiff: Unified Diffusion for Complete Molecule Generation
Our model is a promising approach for designing stable and diverse molecules and can be applied to a wide range of tasks in molecular modeling.
MERMAIDE: Learning to Align Learners using Model-Based Meta-Learning
We then show that our model-based meta-learning approach is cost-effective in intervening on bandit agents with unseen explore-exploit strategies.
Reflected Diffusion Models
To incorporate data constraints in a principled manner, we present Reflected Diffusion Models, which instead reverse a reflected stochastic differential equation evolving on the support of the data.
Ranked #8 on
Image Generation
on ImageNet 32x32
(bpd metric)
Ideal Abstractions for Decision-Focused Learning
We present a methodology for formulating simplifying abstractions in machine learning systems by identifying and harnessing the utility structure of decisions.
End-to-End Diffusion Latent Optimization Improves Classifier Guidance
Classifier guidance -- using the gradients of an image classifier to steer the generations of a diffusion model -- has the potential to dramatically expand the creative control over image generation and editing.
GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation
Empirical results show that GlueNet can be trained efficiently and enables various capabilities beyond previous state-of-the-art models: 1) multilingual language models such as XLM-Roberta can be aligned with existing T2I models, allowing for the generation of high-quality images from captions beyond English; 2) GlueNet can align multi-modal encoders such as AudioCLIP with the Stable Diffusion model, enabling sound-to-image generation; 3) it can also upgrade the current text encoder of the latent diffusion model for challenging case generation.
HIVE: Harnessing Human Feedback for Instructional Visual Editing
Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences.
Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching
In this case, a well-known issue is the distribution shift between the learned policy and the behavior policy that collects the offline data.
Hyena Hierarchy: Towards Larger Convolutional Language Models
Recent advances in deep learning have relied heavily on the use of large Transformers due to their ability to learn at scale.
Ranked #37 on
Language Modelling
on WikiText-103
Long Horizon Temperature Scaling
LHTS is compatible with all likelihood-based models, and optimizes for the long horizon likelihood of samples.
GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration
Pre-trained diffusion models have been successfully used as priors in a variety of linear inverse problems, where the goal is to reconstruct a signal from noisy linear measurements.
Extreme Q-Learning: MaxEnt RL without Entropy
Using EVT, we derive our \emph{Extreme Q-Learning} framework and consequently online and, for the first time, offline MaxEnt Q-learning algorithms, that do not explicitly require access to a policy or its entropy.
Building Coverage Estimation with Low-resolution Remote Sensing Imagery
In this paper, we propose a method for estimating building coverage using only publicly available low-resolution satellite imagery that is more frequently updated.
Deep Latent State Space Models for Time-Series Generation
Methods based on ordinary differential equations (ODEs) are widely used to build generative models of time-series.
Transform Once: Efficient Operator Learning in Frequency Domain
Instead, this work introduces a blueprint for frequency domain learning through a single transform: transform once (T1).
Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models
With about $1\%$-area edits, SIGE accelerates DDPM by $3. 0\times$ on NVIDIA RTX 3090 and $4. 6\times$ on Apple M1 Pro GPU, Stable Diffusion by $7. 2\times$ on 3090, and GauGAN by $5. 6\times$ on 3090 and $5. 2\times$ on M1 Pro GPU.
Concrete Score Matching: Generalized Score Matching for Discrete Data
To this end, we propose an analogous score function called the "Concrete score", a generalization of the (Stein) score for discrete settings.
LMPriors: Pre-Trained Language Models as Task-Specific Priors
Particularly in low-data regimes, an outstanding challenge in machine learning is developing principled techniques for augmenting our models with suitable priors.
FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation
Score-based generative models (SGMs) learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise.
Exploration via Planning for Information about the Optimal Trajectory
In this work, we develop a method that allows us to plan for exploration while taking both the task and the current knowledge about the dynamics into account.
On Distillation of Guided Diffusion Models
For standard diffusion models trained on the pixel-space, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps on ImageNet 64x64 and CIFAR-10, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from.
Generalizing Bayesian Optimization with Decision-theoretic Entropies
Bayesian optimization (BO) is a popular method for efficiently inferring optima of an expensive black-box function via a sequence of queries.
Towards General-Purpose Representation Learning of Polygonal Geometries
For the spatial domain approach, we propose ResNet1D, a 1D CNN-based polygon encoder, which uses circular padding to achieve loop origin invariance on simple polygons.
ButterflyFlow: Building Invertible Layers with Butterfly Matrices
Normalizing flows model complex probability distributions using maps obtained by composing invertible layers.
JPEG Artifact Correction using Denoising Diffusion Restoration Models
Diffusion models can be used as learned priors for solving various inverse problems.
Improving Self-Supervised Learning by Characterizing Idealized Representations
For non-contrastive learning, we use our framework to derive a simple and novel objective.
Multipoint-BAX: A New Approach for Efficiently Tuning Particle Accelerator Emittance via Virtual Objectives
Traditional black-box optimizers such as Bayesian optimization are slow and inefficient when dealing with such objectives as they must acquire the full series of measurements, but return only the emittance, with each query.
SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
Unsupervised pre-training methods for large vision models have shown to enhance performance on downstream supervised tasks.
A General Recipe for Likelihood-free Bayesian Optimization
To extend BO to a broader class of models and utilities, we propose likelihood-free BO (LFBO), an approach based on likelihood-free inference.
Modular Conformal Calibration
We introduce a versatile class of algorithms for recalibration in regression that we call Modular Conformal Calibration (MCC).
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Training and Inference on Any-Order Autoregressive Models the Right Way
Conditional inference on arbitrary subsets of variables is a core problem in probabilistic inference with important applications such as masked language modeling and image inpainting.
Self-Similarity Priors: Neural Collages as Differentiable Fractal Representations
We investigate how to leverage the representations produced by Neural Collages in various tasks, including data compression and generation.
Tracking Urbanization in Developing Regions with Remote Sensing Spatial-Temporal Super-Resolution
Automated tracking of urban development in areas where construction information is not available became possible with recent advancements in machine learning and remote sensing.
Generative Modeling Helps Weak Supervision (and Vice Versa)
The model outperforms baseline weak supervision label models on a number of multiclass image classification datasets, improves the quality of generated images, and further improves end-model performance through data augmentation with synthetic samples.
Dual Diffusion Implicit Bridges for Image-to-Image Translation
Image translation with DDIBs relies on two diffusion models trained independently on each domain, and is a two-step process: DDIBs first obtain latent encodings for source images with the source diffusion model, and then decode such encodings using the target model to construct target images.
GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation
GeoDiff treats each atom as a particle and learns to directly reverse the diffusion process (i. e., transforming from a noise distribution to stable conformations) as a Markov chain.
LISA: Learning Interpretable Skill Abstractions from Language
Learning policies that effectively utilize language instructions in complex, multi-task environments is an important problem in sequential decision-making.
Imitation Learning by Estimating Expertise of Demonstrators
In this work, we show that unsupervised learning over demonstrator expertise can lead to a consistent boost in the performance of imitation learning algorithms.
Denoising Diffusion Restoration Models
Many interesting tasks in image restoration can be cast as linear inverse problems.
Conditional Imitation Learning for Multi-Agent Games
In this work, we study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time, and we must interact with and adapt to new partners at test time.
IS-COUNT: Large-scale Object Counting from Satellite Images with Covariate-based Importance Sampling
We show empirically that the proposed framework achieves strong performance on estimating the number of buildings in the United States and Africa, cars in Kenya, brick kilns in Bangladesh, and swimming pools in the U. S., while requiring as few as 0. 01% of satellite images compared to an exhaustive approach.
Quantifying and Understanding Adversarial Examples in Discrete Input Spaces
In this work, we seek to understand and extend adversarial examples across domains in which inputs are discrete, particularly across new domains, such as computational biology.
An Experimental Design Perspective on Model-Based Reinforcement Learning
In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.
A Unified Framework for Multi-distribution Density Ratio Estimation
Binary density ratio estimation (DRE), the problem of estimating the ratio $p_1/p_2$ given their empirical samples, provides the foundation for many state-of-the-art machine learning algorithms such as contrastive representation learning and covariate shift adaptation.
BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery
We propose Bayesian Causal Discovery Nets (BCD Nets), a variational inference framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM.
HyperSPNs: Compact and Expressive Probabilistic Circuits
Probabilistic circuits (PCs) are a family of generative models which allows for the computation of exact likelihoods and marginals of its probability distributions.
D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation
Conditional generative models of high-dimensional images have many applications, but supervision signals from conditions to images can be expensive to acquire.
Reliable Decisions with Threshold Calibration
We propose a stronger notion of calibration called threshold calibration, which is exactly the condition required to ensure that decision loss is predicted accurately for threshold decisions.
Density Ratio Estimation via Infinitesimal Classification
We then estimate the instantaneous rate of change of the bridge distributions indexed by time (the "time score") -- a quantity defined analogously to data (Stein) scores -- with a novel time score matching objective.
Solving Inverse Problems in Medical Imaging with Score-Based Generative Models
These measurements are typically synthesized from images using a fixed physical model of the measurement process, which hinders the generalization capability of models to unknown measurement processes.
Estimating High Order Gradients of the Data Distribution by Denoising
By leveraging Tweedie's formula on higher order moments, we generalize denoising score matching to estimate higher order derivatives.
SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning
Our goals for SustainBench are to (1) lower the barriers to entry for the machine learning community to contribute to measuring and achieving the SDGs; (2) provide standard benchmarks for evaluating machine learning models on tasks across a variety of SDGs; and (3) encourage the development of novel machine learning methods where improved model performance facilitates progress towards the SDGs.
Pseudo-Spherical Contrastive Divergence
Energy-based models (EBMs) offer flexible distribution parametrization.
Equivariant Neural Network for Factor Graphs
Several indices used in a factor graph data structure can be permuted without changing the underlying probability distribution.
H-Entropy Search: Generalizing Bayesian Optimization with a Decision-theoretic Uncertainty Measure
For special cases of the loss and design space, we develop gradient-based methods to efficiently optimize our proposed family of acquisition functions, and demonstrate that the resulting BO procedure shows strong empirical performance on a diverse set of optimization tasks.
An Experimental Design Perspective on Exploration in Reinforcement Learning
In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.
Comparing Distributions by Measuring Differences that Affect Decision Making
Measuring the discrepancy between two probability distributions is a fundamental problem in machine learning and statistics.
Sphere2Vec: Self-Supervised Location Representation Learning on Spherical Surfaces
Location encoding is valuable for a multitude of tasks where both the absolute positions and local contexts (image, text, and other types of metadata) of spatial objects are needed for accurate predictions.
Provably Calibrated Regression Under Distribution Drift
Accurate uncertainty quantification is a key building block of trustworthy machine learning systems.
Mind Your Bits and Errors: Prioritizing the Bits that Matter in Variational Autoencoders
In this work, we consider the task of image generative modeling with variational autoencoders and posit that the nature of high-dimensional image data distributions poses an intrinsic challenge.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
The key challenge is balancing faithfulness to the user input (e. g., hand-drawn colored strokes) and realism of the synthesized image.
Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration
In this work, we introduce a new notion -- \emph{decision calibration} -- that requires the predicted distribution and true distribution to be ``indistinguishable'' to a set of downstream decision-makers.
Multi-Agent Imitation Learning with Copulas
Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions, which is essential for understanding physical, social, and team-play systems.
CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation
In this paper, we propose Conditional Score-based Diffusion models for Imputation (CSDI), a novel time series imputation method that utilizes score-based diffusion models conditioned on observed data.
Featurized Density Ratio Estimation
Density ratio estimation serves as an important technique in the unsupervised machine learning toolbox.
IQ-Learn: Inverse soft-Q Learning for Imitation
In many sequential decision-making problems (e. g., robotics control, game playing, sequential prediction), human or expert data is available containing useful information about the task.
Ranked #1 on
MuJoCo Games
on Walker2d
Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis
High-resolution satellite imagery has proven useful for a broad range of tasks, including measurement of global human population, local economic livelihoods, and biodiversity, among many others.
Temporal Predictive Coding For Model-Based Planning In Latent Space
High-dimensional observations are a major challenge in the application of model-based reinforcement learning (MBRL) to real-world environments.
D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
Conditional generative models of high-dimensional images have many applications, but supervision signals from conditions to images can be expensive to acquire.
Improving Compositionality of Neural Networks by Decoding Representations to Inputs
In traditional software programs, it is easy to trace program logic from variables back to input, apply assertion statements to block erroneous behavior, and compose programs together.
Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information
Given such an $\mathcal{A}$, and a prior distribution over $f$, we refer to the problem of inferring the output of $\mathcal{A}$ using $T$ evaluations as Bayesian Algorithm Execution (BAX).
On the Critical Role of Conventions in Adaptive Human-AI Collaboration
Humans can quickly adapt to new partners in collaborative tasks (e. g. playing basketball), because they understand which fundamental skills of the task (e. g. how to dribble, how to shoot) carry over across new partners.
Improved Autoregressive Modeling with Distribution Smoothing
While autoregressive models excel at image compression, their sample quality is often lacking.
Hybrid Mutual Information Lower-bound Estimators for Representation Learning
We illustrate that with one set of representations, the hybrid approach is able to achieve good performance on multiple downstream tasks such as classification, reconstruction, and generation.
Anytime Sampling for Autoregressive Models via Ordered Autoencoding
Experimentally, we demonstrate in several image and audio generation tasks that sample quality degrades gracefully as we reduce the computational budget for sampling.
Local Calibration: Metrics and Recalibration
In this work, we propose the local calibration error (LCE) to span the gap between average and individual reliability.
Neural Network Compression for Noisy Storage Devices
Compression and efficient storage of neural network (NN) parameters is critical for applications that run on resource-constrained devices.
Negative Data Augmentation
Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities.
Ranked #8 on
Image Generation
on CIFAR-100
Maximum Likelihood Training of Score-Based Diffusion Models
Score-based diffusion models synthesize samples by reversing a stochastic process that diffuses data to noise, and are trained by minimizing a weighted combination of score matching losses.
Ranked #10 on
Image Generation
on ImageNet 32x32
(bpd metric)
H-divergence: A Decision-Theoretic Discrepancy Measure for Two Sample Tests
Based on ideas from decision theory, we investigate a new class of discrepancies that are based on the optimal decision loss.
Non-Markovian Predictive Coding For Planning In Latent Space
High-dimensional observations are a major challenge in the application of model-based reinforcement learning (MBRL) to real-world environments.
Understanding Classifiers with Generative Models
Although deep neural networks are effective on supervised learning tasks, they have been shown to be brittle.
Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients
We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions.
PiRank: Scalable Learning To Rank via Differentiable Sorting
A key challenge with machine learning approaches for ranking is the gap between the performance metrics of interest and the surrogate loss functions that can be optimized with gradient-based methods.
Score-Based Generative Modeling through Stochastic Differential Equations
Combined with multiple architectural improvements, we achieve record-breaking performance for unconditional image generation on CIFAR-10 with an Inception score of 9. 89 and FID of 2. 20, a competitive likelihood of 2. 99 bits/dim, and demonstrate high fidelity generation of 1024 x 1024 images for the first time from a score-based generative model.
Ranked #8 on
Density Estimation
on CIFAR-10
Efficient Conditional Pre-training for Transfer Learning
Finally, we improve standard ImageNet pre-training by 1-3% by tuning available models on our subsets and pre-training on a dataset filtered from a larger scale dataset.
Geography-Aware Self-Supervised Learning
Contrastive learning methods have significantly narrowed the gap between supervised and unsupervised learning on computer vision tasks.
Ranked #6 on
Semantic Segmentation
on SpaceNet 1
(using extra training data)
Right Decisions from Wrong Predictions: A Mechanism Design Alternative to Individual Calibration
Decision makers often need to rely on imperfect probabilistic forecasts.
Autoregressive Score Matching
To increase flexibility, we propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariate log-conditionals (scores), which need not be normalized.
Probabilistic Circuits for Variational Inference in Discrete Graphical Models
Inference in discrete graphical models with variational methods is difficult because of the inability to re-parameterize gradients of the Evidence Lower Bound (ELBO).
Noisy Neural Network Compression for Analog Storage Devices
Efficient compression and storage of neural network (NN) parameters is critical for resource-constrained, downstream machine learning applications.
Imitation with Neural Density Models
We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Maximum Occupancy Entropy Reinforcement Learning (RL) using the density as a reward.
Denoising Diffusion Implicit Models
Denoising diffusion probabilistic models (DDPMs) have achieved high quality image generation without adversarial training, yet they require simulating a Markov chain for many steps to produce a sample.
Understanding Classifier Mistakes with Generative Models
From this observation, we develop a detection criteria for samples on which a classifier is likely to fail at test time.
Privacy Preserving Recalibration under Domain Shift
In an extensive empirical study, we find that our algorithm improves calibration on domain-shift benchmarks under the constraints of differential privacy.
HiPPO: Recurrent Memory with Optimal Polynomial Projections
A central problem in learning from sequential data is representing cumulative history in an incremental fashion as more data is processed.
Ranked #8 on
Sequential Image Classification
on Sequential MNIST
Multi-label Contrastive Predictive Coding
We demonstrate that the proposed approach is able to lead to better mutual information estimation, gain empirical improvements in unsupervised representation learning, and beat a current state-of-the-art knowledge distillation method over 10 out of 13 tasks.
Efficient Learning of Generative Models via Finite-Difference Score Matching
Several machine learning applications involve the optimization of higher-order derivatives (e. g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation.
Belief Propagation Neural Networks
Learned neural solvers have successfully been used to solve combinatorial optimization and decision problems.
Unsupervised Calibration under Covariate Shift
A probabilistic model is said to be calibrated if its predicted probabilities match the corresponding empirical frequencies.
Experience Replay with Likelihood-free Importance Weights
The use of past experiences to accelerate temporal difference (TD) learning of value functions, or experience replay, is a key component in deep reinforcement learning.
A Framework for Sample Efficient Interval Estimation with Control Variates
We consider the problem of estimating confidence intervals for the mean of a random variable, where the goal is to produce the smallest possible interval for a given number of samples.
Individual Calibration with Randomized Forecasting
We show that calibration for individual samples is possible in the regression setup if the predictions are randomized, i. e. outputting randomized credible intervals.
Improved Techniques for Training Score-Based Generative Models
Score-based generative models can produce high quality image samples comparable to GANs, without requiring adversarial optimization.
Ranked #46 on
Image Generation
on CIFAR-10
Predicting Livelihood Indicators from Community-Generated Street-Level Imagery
Major decisions from governments and other large organizations rely on measurements of the populace's well-being, but making such measurements at a broad scale is expensive and thus infrequent in much of the developing world.
Efficient Poverty Mapping using Deep Reinforcement Learning
The combination of high-resolution satellite imagery and machine learning have proven useful in many sustainability-related tasks, including poverty prediction, infrastructure measurement, and forest monitoring.
Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
Learning disentangled representations is regarded as a fundamental task for improving the generalization, robustness, and interpretability of generative models.
MOPO: Model-based Offline Policy Optimization
We also characterize the trade-off between the gain and risk of leaving the support of the batch data.
Farmland Parcel Delineation Using Spatio-temporal Convolutional Networks
Using satellite imaging can be a scalable and cost effective manner to perform the task of farm parcel delineation to collect this valuable data.
Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
Adversarial attacks often involve random perturbations of the inputs drawn from uniform or Gaussian distributions, e. g., to initialize optimization-based white-box attacks or generate update directions in black-box attacks.
Training Deep Energy-Based Models with f-Divergence Minimization
Experimental results demonstrate the superiority of f-EBM over contrastive divergence, as well as the benefits of training EBMs using f-divergences other than KL.
Gaussianization Flows
Iterative Gaussianization is a fixed-point iteration procedure that can transform any continuous random vector into a Gaussian one.
Predictive Coding for Locally-Linear Control
High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks.
Permutation Invariant Graph Generation via Score-Based Generative Modeling
In particular, we design a permutation equivariant, multi-channel graph neural network to model the gradient of the data distribution at the input graph (a. k. a., the score function).
Learning When and Where to Zoom with Deep Reinforcement Learning
While high resolution images contain semantically more useful information than their lower resolution counterparts, processing them is computationally more expensive, and in some applications, e. g. remote sensing, they can be much more expensive to acquire.
A Theory of Usable Information Under Computational Constraints
We propose a new framework for reasoning about information in complex systems.
Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving
Feedforward computation, such as evaluating a neural network or sampling from an autoregressive model, is ubiquitous in machine learning.
