Search Results for author:
Found 114 papers, 72 papers with code
Contextually Supervised Source Separation with Application to Energy Disaggregation
We propose a new framework for single-channel source separation that lies between the fully supervised and unsupervised setting.
Probabilistic Segmentation via Total Variation Regularization
We present a convex approach to probabilistic segmentation and modeling of time series data.
Input Convex Neural Networks
We show that many existing neural network architectures can be made input-convex with a minor modification, and develop specialized optimization algorithms tailored to this setting.
OptNet: Differentiable Optimization as a Layer in Neural Networks
This paper presents OptNet, a network architecture that integrates optimization problems (here, specifically in the form of quadratic programs) as individual layers in larger end-to-end trainable deep networks.
Task-based End-to-end Model Learning in Stochastic Optimization
With the increasing popularity of machine learning techniques, it has become common to see prediction algorithms operating within some larger process.
The Mixing method: low-rank coordinate descent for semidefinite programming with diagonal constraints
In this paper, we propose a low-rank coordinate descent approach to structured semidefinite programming with diagonal constraints.
Gradient descent GAN optimization is locally stable
Despite the growing prominence of generative adversarial networks (GANs), optimization in GANs is still a poorly understood topic.
Provable defenses against adversarial examples via the convex outer adversarial polytope
We propose a method to learn deep ReLU-based classifiers that are provably robust against norm-bounded adversarial perturbations on the training data.
Convolutional Sequence Modeling Revisited
This paper revisits the problem of sequence modeling using convolutional architectures.
Ranked #84 on
Language Modelling
on WikiText-103
Realtime query completion via deep language models
Search engine users nowadays heavily depend on query completion and correction to shape their queries.
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory.
Ranked #4 on
Music Modeling
on Nottingham
What game are we playing? End-to-end learning in normal and extensive form games
Although recent work in AI has made great progress in solving large, zero-sum, extensive-form games, the underlying assumption in most past work is that the parameters of the game itself are known to the agents.
Scaling provable adversarial defenses
Recent work has developed methods for learning deep network classifiers that are provably robust to norm-bounded adversarial perturbation; however, these methods are currently only possible for relatively small feedforward networks.
Trellis Networks for Sequence Modeling
On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices.
A Continuous-Time View of Early Stopping for Least Squares
Our primary focus is to compare the risk of gradient flow to that of ridge regression.
Differentiable MPC for End-to-end Planning and Control
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces.
End-to-End Differentiable Physics for Learning and Control
We present a differentiable physics engine that can be integrated as a module in deep neural networks for end-to-end learning.
Low-rank semidefinite programming for the MAX2SAT problem
This paper proposes a new algorithm for solving MAX2SAT problems based on combining search methods with semidefinite programming approaches.
Generalization in Deep Networks: The Role of Distance from Initialization
Why does training deep neural networks using stochastic gradient descent (SGD) result in a generalization error that does not worsen with the number of parameters in the network?
Certified Adversarial Robustness via Randomized Smoothing
We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the $\ell_2$ norm.
Ranked #3 on
Robust classification
on CIFAR-10
Uniform convergence may be unable to explain generalization in deep learning
Aimed at explaining the surprisingly good generalization behavior of overparameterized deep networks, recent works have developed a variety of generalization bounds for deep learning, all based on the fundamental learning-theoretic technique of uniform convergence.
Wasserstein Adversarial Examples via Projected Sinkhorn Iterations
In this paper, we propose a new threat model for adversarial attacks based on the Wasserstein distance.
Large Scale Learning of Agent Rationality in Two-Player Zero-Sum Games
With the recent advances in solving large, zero-sum extensive form games, there is a growing interest in the inverse problem of inferring underlying game parameters given only access to agent actions.
Adversarial camera stickers: A physical camera-based attack on deep learning systems
In this work, we consider an alternative question: is it possible to fool deep classifiers, over all perceived objects of a certain type, by physically manipulating the camera itself?
Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience
The ability of overparameterized deep networks to generalize well has been linked to the fact that stochastic gradient descent (SGD) finds solutions that lie in flat, wide minima in the training loss -- minima where the output of the network is resilient to small random noise added to its parameters.
Multimodal Transformer for Unaligned Multimodal Language Sequences
Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors.
Ranked #5 on
Multimodal Sentiment Analysis
on MOSI
Perceptual Based Adversarial Audio Attacks
Recent work has shown the possibility of adversarial attacks on automatic speechrecognition (ASR) systems.
Audio and Speech Processing Sound
The Limited Multi-Label Projection Layer
We propose the Limited Multi-Label (LML) projection layer as a new primitive operation for end-to-end learning systems.
Deep Equilibrium Models
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ).
Ranked #29 on
Language Modelling
on Penn Treebank (Word Level)
Adversarial Robustness Against the Union of Multiple Perturbation Models
Owing to the susceptibility of deep learning systems to adversarial attacks, there has been a great deal of work in developing (both empirically and certifiably) robust classifiers.
Certified Robustness to Adversarial Label-Flipping Attacks via Randomized Smoothing
This paper considers label-flipping attacks, a type of data poisoning attack where an adversary relabels a small number of examples in a training set in order to degrade the performance of the resulting classifier.
Black-box Adversarial Attacks with Bayesian Optimization
We focus on the problem of black-box adversarial attacks, where the aim is to generate adversarial examples using information limited to loss function evaluations of input-output pairs.
Adversarial Music: Real World Audio Adversary Against Wake-word Detection System
In this work, we target our attack on the wake-word detection system, jamming the model with some inconspicuous background music to deactivate the VAs while our audio adversary is present.
Dynamic Modeling and Equilibria in Fair Decision Making
Recent studies on fairness in automated decision making systems have both investigated the potential future impact of these decisions on the population at large, and emphasized that imposing ''typical'' fairness constraints such as demographic parity or equality of opportunity does not guarantee a benefit to disadvantaged groups.
AP-Perf: Incorporating Generic Performance Metrics in Differentiable Learning
We propose a method that enables practitioners to conveniently incorporate custom non-decomposable performance metrics into differentiable learning pipelines, notably those based upon neural network architectures.
Fast is better than free: Revisiting adversarial training
Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with $\epsilon=8/255$ in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at $\epsilon=2/255$ in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds.
Learning Stable Deep Dynamics Models
Deep networks are commonly used to model dynamical systems, predicting how the state of a system will evolve over time (either autonomously or in response to control inputs).
Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
Machine learning algorithms are known to be susceptible to data poisoning attacks, where an adversary manipulates the training data to degrade performance of the resulting classifier.
Overfitting in adversarially robust deep learning
Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping.
Denoised Smoothing: A Provable Defense for Pretrained Classifiers
We present a method for provably defending any pretrained image classifier against $\ell_p$ adversarial attacks.
Differentiable learning of numerical rules in knowledge graphs
Rules over a knowledge graph (KG) capture interpretable patterns in data and can be used for KG cleaning and completion.
Multiscale Deep Equilibrium Models
These simultaneously-learned multi-resolution features allow us to train a single model on a diverse set of tasks and loss functions, such as using a single MDEQ to perform both image classification and semantic segmentation.
Ranked #46 on
Semantic Segmentation
on Cityscapes val
Monotone operator equilibrium networks
We then develop a parameterization of the network which ensures that all operators remain monotone, which guarantees the existence of a unique equilibrium point.
Neural Network Virtual Sensors for Fuel Injection Quantities with Provable Performance Specifications
Additionally, we show how specific intervals of fuel injection quantities can be targeted to maximize robustness for certain ranges, allowing us to train a virtual sensor for fuel injection which is provably guaranteed to have at most 10. 69% relative error under noise while maintaining 3% relative error on non-adversarial data within normalized fuel injection ranges of 0. 6 to 1. 0.
Provably Safe PAC-MDP Exploration Using Analogies
A key challenge in applying reinforcement learning to safety-critical domains is understanding how to balance exploration (needed to attain good performance on the task) with safety (needed to avoid catastrophic failure).
Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction
Solving large complex partial differential equations (PDEs), such as those that arise in computational fluid dynamics (CFD), is a computationally expensive process.
Simple and Efficient Hard Label Black-box Adversarial Attacks in Low Query Budget Regimes
We focus on the problem of black-box adversarial attacks, where the aim is to generate adversarial examples for deep learning models solely based on information limited to output label~(hard label) to a queried data input.
Learning perturbation sets for robust machine learning
In this paper, we aim to bridge this gap by learning perturbation sets from data, in order to characterize real-world effects for robust training and evaluation.
Gaussian MRF Covariance Modeling for Efficient Black-Box Adversarial Attacks
We study the problem of generating adversarial examples in a black-box setting, where we only have access to a zeroth order oracle, providing us with loss function evaluations.
Poisoned classifiers are not only backdoored, they are fundamentally broken
Under this threat model, we propose a test-time, human-in-the-loop attack method to generate multiple effective alternative triggers without access to the initial backdoor and the training data.
Enforcing robust control guarantees within neural network policies
When designing controllers for safety-critical systems, practitioners often face a challenging tradeoff between robustness and performance.
Challenging common interpretability assumptions in feature attribution explanations
As machine learning and algorithmic decision making systems are increasingly being leveraged in high-stakes human-in-the-loop settings, there is a pressing need to understand the rationale of their predictions.
Decision Making
Explainable Artificial Intelligence (XAI)
+1
Community detection using fast low-cardinality semidefinite programming
Modularity maximization has been a fundamental tool for understanding the community structure of a network, but the underlying optimization problem is nonconvex and NP-hard to solve.
Efficient semidefinite-programming-based inference for binary and multi-class MRFs
Probabilistic inference in pairwise Markov Random Fields (MRFs), i. e. computing the partition function or computing a MAP estimate of the variables, is a foundational problem in probabilistic graphical models.
Deep Archimedean Copulas
A central problem in machine learning and statistics is to model joint densities of random variables from data.
On Proximal Policy Optimization's Heavy-tailed Gradients
In this paper, we present a detailed empirical study to characterize the heavy-tailed nature of the gradients of the PPO surrogate reward function.
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
We empirically demonstrate that full-batch gradient descent on neural network training objectives typically operates in a regime we call the Edge of Stability.
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verification
Compared to the typically tightest but very costly semidefinite programming (SDP) based incomplete verifiers, we obtain higher verified accuracy with three orders of magnitudes less verification time.
Orthogonalizing Convolutional Layers with the Cayley Transform
Recent work has highlighted several advantages of enforcing orthogonality in the weight layers of deep networks, such as maintaining the stability of activations, preserving gradient norms, and enhancing adversarial robustness by enforcing low Lipschitz constants.
DC3: A learning method for optimization with hard constraints
Large optimization problems with hard constraints arise in many settings, yet classical solvers are often prohibitively slow, motivating the use of deep networks as cheap "approximate solvers."
RATT: Leveraging Unlabeled Data to Guarantee Generalization
To assess generalization, machine learning scientists typically either (i) bound the generalization gap and then (after training) plug in the empirical risk to obtain a bound on the true risk; or (ii) validate empirically on holdout data.
Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization
Specifically, we incorporate a differentiable projection layer within a neural network-based policy to enforce that all learned actions are feasible.
DORO: Distributional and Outlier Robust Optimization
Many machine learning tasks involve subpopulation shift where the testing data distribution is a subpopulation of the training distribution.
DeepSplit: Scalable Verification of Deep Neural Networks via Operator Splitting
Analyzing the worst-case performance of deep neural networks against input perturbations amounts to solving a large-scale non-convex optimization problem, for which several past works have proposed convex relaxations as a promising alternative.
Assessing Generalization of SGD via Disagreement
We empirically show that the test error of deep networks can be estimated by simply training the same architecture on the same training set but with a different run of Stochastic Gradient Descent (SGD), and measuring the disagreement rate between the two networks on unlabeled test data.
Stabilizing Equilibrium Models by Jacobian Regularization
Deep equilibrium networks (DEQs) are a new class of models that eschews traditional depth in favor of finding the fixed point of a single nonlinear layer.
Communicating via Markov Decision Processes
We contribute a theoretically grounded approach to MCGs based on maximum entropy reinforcement learning and minimum entropy coupling that we call MEME.
Adversarially Robust Learning for Security-Constrained Optimal Power Flow
In recent years, the ML community has seen surges of interest in both adversarially robust learning and implicit layers, but connections between these two areas have seldom been explored.
Joint inference and input optimization in equilibrium networks
Many tasks in deep learning involve optimizing over the \emph{inputs} to a network to minimize or maximize some objective; examples include optimization over latent spaces in a generative model to match a target image, or adversarially perturbing an input to worsen classifier performance.
$(\textrm{Implicit})^2$: Implicit Layers for Implicit Representations
Recent research in deep learning has investigated two very different forms of ''implicitness'': implicit representations model high-frequency data such as images or 3D shapes directly via a low-dimensional neural network (often using e. g., sinusoidal bases or nonlinearities); implicit layers, in contrast, refer to techniques where the forward pass of a network is computed via non-linear dynamical systems, such as fixed-point or differential equation solutions, with the backward pass computed via the implicit function theorem.
Monte Carlo Tree Search With Iteratively Refining State Abstractions
In this work, we present a method, called abstraction refining, for extending MCTS to stochastic environments which, unlike progressive widening, leverages the geometry of the state space.
Robustness between the worst and average case
Several recent works in machine learning have focused on evaluating the test-time robustness of a classifier: how well the classifier performs not just on the target domain it was trained upon, but upon perturbed examples.
Patches Are All You Need?
Despite its simplicity, we show that the ConvMixer outperforms the ViT, MLP-Mixer, and some of their variants for similar parameter counts and data set sizes, in addition to outperforming classical vision models such as the ResNet.
Ranked #96 on
Image Classification
on CIFAR-10
Deep Equilibrium Optical Flow Estimation
Many recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms by encouraging iterative refinements toward a stable flow estimation.
Ranked #1 on
Optical Flow Estimation
on KITTI 2015 (train)
Smooth-Reduce: Leveraging Patches for Improved Certified Robustness
Randomized smoothing (RS) has been shown to be a fast, scalable technique for certifying the robustness of deep neural network classifiers.
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
This work studies an algorithm, which we call magnetic mirror descent, that is inspired by mirror descent and the non-Euclidean proximal gradient algorithm.
(Certified!!) Adversarial Robustness for Free!
In this paper we show how to achieve state-of-the-art certified adversarial robustness to 2-norm bounded perturbations by relying exclusively on off-the-shelf pretrained models.
General Cutting Planes for Bound-Propagation-Based Neural Network Verification
Our generalized bound propagation method, GCP-CROWN, opens up the opportunity to apply general cutting plane methods for neural network verification while benefiting from the efficiency and GPU acceleration of bound propagation methods.
Understanding the Covariance Structure of Convolutional Filters
In this work, we first observe that such learned filters have highly-structured covariance matrices, and moreover, we find that covariances calculated from small networks may be used to effectively initialize a variety of larger networks of different depths, widths, patch sizes, and kernel sizes, indicating a degree of model-independence to the covariance structure.
Perfectly Secure Steganography Using Minimum Entropy Coupling
Steganography is the practice of encoding secret information into innocuous content in such a manner that an adversarial third party would not realize that there is hidden meaning.
Characterizing Datapoints via Second-Split Forgetting
Popular metrics derived from these dynamics include (i) the epoch at which examples are first correctly classified; (ii) the number of times their predictions flip during training; and (iii) whether their prediction flips if they are held out.
Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth
Neural networks with sinusoidal activations have been proposed as an alternative to networks with traditional activation functions.
Losses over Labels: Weakly Supervised Learning via Direct Loss Construction
Owing to the prohibitive costs of generating large amounts of labeled data, programmatic weak supervision is a growing paradigm within machine learning.
Function Approximation for Solving Stackelberg Equilibrium in Large Perfect Information Games
Function approximation (FA) has been a critical component in solving large zero-sum games.
Abstracting Imperfect Information Away from Two-Player Zero-Sum Games
Because these regularized equilibria can be made arbitrarily close to Nash equilibria, our result opens the door to a new perspective to solving two-player zero-sum games and yields a simplified framework for decision-time planning in two-player zero-sum games, void of the unappealing properties that plague existing decision-time planning approaches.
Single Image Backdoor Inversion via Robust Smoothed Classifiers
Insipired by recent advances in adversarial robustness, our method SmoothInv starts from a single clean image, and then performs projected gradient descent towards the target class on a robust smoothed version of the original backdoored classifier.
Model-tuning Via Prompts Makes NLP Models Adversarially Robust
Across 5 NLP datasets, 4 adversarial attacks, and 3 different models, MVP improves performance against adversarial substitutions by an average of 8% over standard methods and even outperforms adversarial training-based state-of-art defenses by 3. 5%.
Sinkhorn-Flow: Predicting Probability Mass Flow in Dynamical Systems Using Optimal Transport
Predicting how distributions over discrete variables vary over time is a common task in time series forecasting.
Learning with Explanation Constraints
In this paper, we formalize this notion as learning from explanation constraints and provide a learning theoretic framework to analyze how such explanations can improve the learning of our models.
The Update-Equivalence Framework for Decision-Time Planning
Using this framework, we derive a provably sound search algorithm for fully cooperative games based on mirror descent and a search algorithm for adversarial games based on magnetic mirror descent.
Mimetic Initialization of Self-Attention Layers
It is notoriously difficult to train Transformers on small datasets; typically, large pre-trained models are instead used as the starting point.
On the Joint Interaction of Models, Data, and Features
Thus, we believe this work provides valuable new insight into our understanding of feature learning.
A Simple and Effective Pruning Approach for Large Language Models
Motivated by the recent observation of emergent large magnitude features in LLMs, our approach prunes weights with the smallest magnitudes multiplied by the corresponding input activations, on a per-output basis.
Localized Text-to-Image Generation for Free via Cross Attention Control
Despite the tremendous success in text-to-image generative models, localized text-to-image generation (that is, generating objects or features at specific locations in an image while maintaining a consistent overall generation) still requires either explicit training or substantial additional inference time.
T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
However, naively removing all such data could also be wasteful, as it throws away images that contain visual features (in addition to overlapping text).
Text Descriptions are Compressive and Invariant Representations for Visual Learning
This method first automatically generates multiple visual descriptions of each class via a large language model (LLM), then uses a VLM to translate these descriptions to a set of visual feature embeddings of each image, and finally uses sparse logistic regression to select a relevant subset of these features to classify each image.
Monotone deep Boltzmann machines
Deep Boltzmann machines (DBMs), one of the first ``deep'' learning methods ever studied, are multi-layered probabilistic models governed by a pairwise energy function that describes the likelihood of all variables/nodes in the network.
Can Neural Network Memorization Be Localized?
Recent efforts at explaining the interplay of memorization and generalization in deep overparametrized networks have posited that neural networks $\textit{memorize}$ "hard" examples in the final few layers of the model.
Universal and Transferable Adversarial Attacks on Aligned Language Models
Specifically, our approach finds a suffix that, when attached to a wide range of queries for an LLM to produce objectionable content, aims to maximize the probability that the model produces an affirmative response (rather than refusing to answer).
Representation Engineering: A Top-Down Approach to AI Transparency
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience.
Ranked #3 on
Question Answering
on TruthfulQA
Understanding prompt engineering may not require rethinking generalization
Zero-shot learning in prompted vision-language models, the practice of crafting prompts to build classifiers without an explicit training process, has achieved impressive performance in many settings.
On the Neural Tangent Kernel of Equilibrium Models
This work studies the neural tangent kernel (NTK) of the deep equilibrium (DEQ) model, a practical ``infinite-depth'' architecture which directly computes the infinite-depth limit of a weight-tied network via root-finding.
TorchDEQ: A Library for Deep Equilibrium Models
Deep Equilibrium (DEQ) Models, an emerging class of implicit models that maps inputs to fixed points of neural networks, are of growing interest in the deep learning community.
Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning
A key problem in off-policy Reinforcement Learning (RL) is the mismatch, or distribution shift, between the dataset and the distribution over states and actions visited by the learned policy.
Manifold Preserving Guided Diffusion
Despite the recent advancements, conditional image generation still faces challenges of cost, generalizability, and the need for task-specific training.
Deep Equilibrium Based Neural Operators for Steady-State PDEs
Motivated by this observation, we propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE as the infinite-depth fixed point of an implicit operator layer using a black-box root solver and differentiates analytically through this fixed point resulting in $\mathcal{O}(1)$ training memory.
One-Step Diffusion Distillation via Deep Equilibrium Models
We demonstrate that the DEQ architecture is crucial to this capability, as GET matches a $5\times$ larger ViT in terms of FID scores while striking a critical balance of computational cost and image quality.
TOFU: A Task of Fictitious Unlearning for LLMs
Large language models trained on massive corpora of data from the web can memorize and reproduce sensitive or private data raising both legal and ethical concerns.
An Axiomatic Approach to Model-Agnostic Concept Explanations
Concept explanation is a popular approach for examining how human-interpretable concepts impact the predictions of a model.
Bayesian Neural Networks with Domain Knowledge Priors
Bayesian neural networks (BNNs) have recently gained popularity due to their ability to quantify model uncertainty.
Massive Activations in Large Language Models
We observe an empirical phenomenon in Large Language Models (LLMs) -- very few activations exhibit significantly larger values than others (e. g., 100, 000 times larger).
AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs
Existing causal discovery methods based on combinatorial optimization or search are slow, prohibiting their application on large-scale datasets.
Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation
Prompt engineering is effective for controlling the output of text-to-image (T2I) generative models, but it is also laborious due to the need for manually crafted prompts.
Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic
Vision-language models (VLMs) are trained for thousands of GPU hours on carefully curated web datasets.
