Search Results for author:
Found 230 papers, 102 papers with code
Tensor-GaLore: Memory-Efficient Training via Gradient Tensor Decomposition
We present Tensor-GaLore, a novel method for efficient training of neural networks with higher-order tensor weights.
Ultrasound Lung Aeration Map via Physics-Aware Neural Operators
Lung ultrasound is a growing modality in clinics for diagnosing and monitoring acute and chronic lung diseases due to its low cost and accessibility.
A Library for Learning Neural Operators
We present NeuralOperator, an open-source Python library for operator learning.
Sequential Controlled Langevin Diffusions
In this work, we present a principled framework for combining SMC with diffusion-based samplers by viewing both methods in continuous time and considering measures on path space.
Automating Feedback Analysis in Surgical Training: Detection, Categorization, and Assessment
This work introduces the first framework for reconstructing surgical dialogue from unstructured real-world recordings, which is crucial for characterizing teaching tasks.
Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment
Creating such an automated system poses challenges, as it requires an understanding of both the verbal feedback delivered by the trainer and the visual context of the real-time surgical scene.
BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery
We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs.
LeanAgent: Lifelong Learning for Formal Theorem Proving
We present LeanAgent, a novel lifelong learning framework for formal theorem proving that continuously generalizes to and improves on ever-expanding mathematical knowledge without forgetting previously learned knowledge.
A Unified Model for Compressed Sensing MRI Across Undersampling Patterns
We propose a unified model robust to different measurement undersampling patterns and image resolutions in compressed sensing MRI.
Diffusion State-Guided Projected Gradient for Inverse Problems
We highlight that DiffStateGrad improves the robustness of diffusion models in terms of the choice of measurement guidance step size and noise while improving the worst-case performance.
Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization
While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of complexity are still blank.
Manifold-Constrained Nucleus-Level Denoising Diffusion Model for Structure-Based Drug Design
Artificial intelligence models have shown great potential in structure-based drug design, generating ligands with high binding affinities.
Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems
In this manuscript, we use FNOs to model the evolution of random quantum spin systems, so chosen due to their representative quantum dynamics and minimal symmetry.
Beyond Closure Models: Learning Chaotic-Systems via Physics-Informed Neural Operators
We propose an alternative end-to-end learning approach using a physics-informed neural operator (PINO) that overcomes this limitation by not using a closure model or a coarse-grid solver.
Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training
We introduce Mini-Sequence Transformer (MsT), a simple and effective methodology for highly efficient and accurate LLM training with extremely long sequences.
Dynamical Measure Transport and Neural PDE Solvers for Sampling
The task of sampling from a probability density can be approached as transporting a tractable density function to the target, known as dynamical measure transport.
Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing
Diffusion models have recently achieved success in solving Bayesian inverse problems with learned data priors.
CARE: a Benchmark Suite for the Classification and Retrieval of Enzymes
We introduce CARE, a benchmark and dataset suite for the Classification And Retrieval of Enzymes (CARE).
ARDuP: Active Region Video Diffusion for Universal Policies
Sequential decision-making can be formulated as a text-conditioned video generation problem, where a video planner, guided by a text-defined goal, generates future frames visualizing planned actions, from which control actions are subsequently derived.
Solving Poisson Equations using Neural Walk-on-Spheres
Leveraging stochastic representations and Walk-on-Spheres methods, we develop novel losses for neural networks based on the recursive solution of Poisson equations on spheres inside the domain.
Autoformalizing Euclidean Geometry
One challenge in Euclidean geometry is that informal proofs rely on diagrams, leaving gaps in texts that are hard to formalize.
Towards Large Language Models as Copilots for Theorem Proving in Lean
In this paper, we explore LLMs as copilots that assist humans in proving theorems.
What is Point Supervision Worth in Video Instance Segmentation?
Video instance segmentation (VIS) is a challenging vision task that aims to detect, segment, and track objects in videos.
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
To tackle this issue, we propose content-motion latent diffusion model (CMD), a novel efficient extension of pretrained image diffusion models for video generation.
Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data.
Improving Distant 3D Object Detection Using 2D Box Supervision
This mapping allows the depth estimation of distant objects conditioned on their 2D boxes, making long-range 3D detection with 2D supervision feasible.
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Our approach reduces memory usage by up to 65. 5% in optimizer states while maintaining both efficiency and performance for pre-training on LLaMA 1B and 7B architectures with C4 dataset with up to 19. 7B tokens, and on fine-tuning RoBERTa on GLUE tasks.
DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
Pre-training has been investigated to improve the efficiency and performance of training neural operators in data-scarce settings.
Neural Operators with Localized Integral and Differential Kernels
In this work, we present a principled approach to operator learning that can capture local features under two frameworks by learning differential operators and integral operators with locally supported kernels.
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model.
ChatGPT Based Data Augmentation for Improved Parameter-Efficient Debiasing of LLMs
Our results reveal that: (1) ChatGPT can efficiently produce high-quality training data for debiasing other LLMs; (2) data produced via our approach surpasses existing datasets in debiasing performance while also preserving internal knowledge of a pre-trained LLM; and (3) synthetic data exhibits generalizability across categories, effectively mitigating various biases, including intersectional ones.
Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction
Operator learning has been increasingly adopted in scientific and engineering applications, many of which require calibrated uncertainty quantification.
A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics
We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$\times$ speedup compared to standard numerical MD simulations.
Equivariant Graph Neural Operator for Modeling 3D Dynamics
Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods, thanks to the equivariant temporal modeling.
Fully Attentional Networks with Self-emerging Token Labeling
With the proposed STL framework, our best model based on FAN-L-Hybrid (77. 3M parameters) achieves 84. 8% Top-1 accuracy and 42. 1% mCE on ImageNet-1K and ImageNet-C, and sets a new state-of-the-art for ImageNet-A (46. 1%) and ImageNet-R (56. 6%) without using extra data, outperforming the original FAN counterpart by significant margins.
Ranked #16 on
Domain Generalization
on ImageNet-C
Perspectives on the State and Future of Deep Learning -- 2023
The goal of this series is to chronicle opinions and issues in the field of machine learning as they stand today and as they change over time.
Deep Multimodal Fusion for Surgical Feedback Classification
This work offers an important first look at the feasibility of automated classification of real-world live surgical feedback based on text, audio, and video modalities.
Exploring Social Bias in Downstream Applications of Text-to-Image Foundation Models
Text-to-image diffusion models have been adopted into key commercial workflows, such as art generation and image editing.
Plasma Surrogate Modelling using Fourier Neural Operators
Predicting plasma evolution within a Tokamak reactor is crucial to realizing the goal of sustainable fusion.
EKGNet: A 10.96μW Fully Analog Neural Network for Intra-Patient Arrhythmia Classification
We present an integrated approach by combining analog computing and deep learning for electrocardiogram (ECG) arrhythmia classification.
Eureka: Human-Level Reward Design via Coding Large Language Models
The generality of Eureka also enables a new gradient-free in-context learning approach to reinforcement learning from human feedback (RLHF), readily incorporating human inputs to improve the quality and the safety of the generated rewards without model updating.
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences.
Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs
Our contributions are threefold: i) we enable parallelization over input samples with a novel multi-grid-based domain decomposition, ii) we represent the parameters of the model in a high-order latent subspace of the Fourier domain, through a global tensor factorization, resulting in an extreme reduction in the number of parameters and improved generalization, and iii) we propose architectural improvements to the backbone FNO.
Neural Operators for Accelerating Scientific Simulations and Design
Scientific discovery and engineering design are currently limited by the time and cost of physical experiments, selected mostly through trial-and-error and intuition that require deep domain expertise.
Tipping Point Forecasting in Non-Stationary Dynamics on Function Spaces
Tipping points are abrupt, drastic, and often irreversible changes in the evolution of non-stationary and chaotic dynamical systems.
FB-BEV: BEV Representation from Forward-Backward View Transformations
Currently, the two most prominent VTM paradigms are forward projection and backward projection.
Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs
Deep learning often faces the challenge of efficiently processing dynamic inputs, such as sensor data or user inputs.
Guaranteed Approximation Bounds for Mixed-Precision Neural Operators
Neural operators, such as Fourier Neural Operators (FNO), form a principled approach for learning solution operators for PDEs and other mappings between function spaces.
Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
Advances in artificial intelligence (AI) are fueling a new paradigm of discoveries in natural sciences.
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models
Using this data, we develop ReProver (Retrieval-Augmented Prover): an LLM-based prover augmented with retrieval for selecting premises from a vast math library.
InRank: Incremental Low-Rank Learning
To remedy this, we design a new training algorithm Incremental Low-Rank Learning (InRank), which explicitly expresses cumulative weight updates as low-rank matrices while incrementally augmenting their ranks during training.
Fast Training of Diffusion Models with Masked Transformers
For masked training, we introduce an asymmetric encoder-decoder architecture consisting of a transformer encoder that operates only on unmasked patches and a lightweight transformer decoder on full patches.
Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials
Artificial intelligence for scientific discovery has recently generated significant interest within the machine learning and scientific communities, particularly in the domains of chemistry, biology, and material discovery.
Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere
Fourier Neural Operators (FNOs) have proven to be an efficient and effective method for resolution-independent operator learning in a broad variety of application areas across scientific machine learning.
Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo
One of the key shortcomings of existing Thompson sampling algorithms is the need to perform a Gaussian approximation of the posterior distribution, which is not a good surrogate in most practical settings.
Voyager: An Open-Ended Embodied Agent with Large Language Models
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.
Fast Monocular Scene Reconstruction with Global-Sparse Local-Dense Grids
To apply this representation to monocular scene reconstruction, we develop a scale calibration algorithm for fast geometric initialization from monocular depth priors.
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Thus, it is still an open question: shall we pretrain large autoregressive LMs with retrieval?
Prismer: A Vision-Language Model with Multi-Task Experts
Recent vision-language models have shown impressive multi-modal generation capabilities.
Ranked #1 on
Image Captioning
on nocaps val
VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion
To enable such capability in AI systems, we propose VoxFormer, a Transformer-based semantic scene completion framework that can output complete 3D volumetric semantics from only 2D images.
3D geometry
3D Semantic Scene Completion from a single RGB image
+1
Score-based Diffusion Models in Function Space
They consist of a forward process that perturbs input data with Gaussian white noise and a reverse process that learns a score function to generate samples by denoising.
BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models
We thus enable seamless open-ended social bias testing of PLMs by domain experts through an automatic large-scale generation of diverse test sentences for any combination of social categories and attributes.
PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees
In this paper, we propose PerAda, a parameter-efficient pFL framework that reduces communication and computational costs and exhibits superior generalization performance, especially under test-time distribution shifts.
I$^2$SB: Image-to-Image Schrödinger Bridge
We propose Image-to-Image Schr\"odinger Bridge (I$^2$SB), a new class of conditional diffusion models that directly learn the nonlinear diffusion processes between two given distributions.
A Text-guided Protein Design Framework
Current AI-assisted protein design mainly utilizes protein sequential and structural information.
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
Augmenting pretrained language models (LMs) with a vision encoder (e. g., Flamingo) has obtained the state-of-the-art results in image-to-text generation.
Forecasting subcritical cylinder wakes with Fourier Neural Operators
The learned FNO solution operator can be evaluated in milliseconds, potentially enabling faster-than-real-time modeling for predictive flow control in physical systems.
Vision Transformers Are Good Mask Auto-Labelers
We propose Mask Auto-Labeler (MAL), a high-quality Transformer-based mask auto-labeling framework for instance segmentation using only box annotations.
FocalFormer3D: Focusing on Hard Instance for 3D Object Detection
For 3D object detection, we instantiate this method as FocalFormer3D, a simple yet effective detector that excels at excavating difficult objects and improving prediction recall.
Spacetime Surface Regularization for Neural Dynamic Scene Reconstruction
We propose an algorithm, 4DRegSDF, for the spacetime surface regularization to improve the fidelity of neural rendering and reconstruction in dynamic scenes.
End-to-end 3D Tracking with Decoupled Queries
In this work, we present an end-to-end framework for camera-based 3D multi-object tracking, called DQTrack.
Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing
Here we present a multi-modal molecule structure-text model, MoleculeSTM, by jointly learning molecules' chemical structures and textual descriptions via a contrastive learning strategy.
Towards Neural Variational Monte Carlo That Scales Linearly with System Size
Quantum many-body problems are some of the most challenging problems in science and are central to demystifying some exotic quantum phenomena, e. g., high-temperature superconductors.
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression
Transformers have attained superior performance in natural language processing and computer vision.
Fourier Continuation for Exact Derivative Computation in Physics-Informed Neural Operators
The physics-informed neural operator (PINO) is a machine learning architecture that has shown promising empirical results for learning partial differential equations.
Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEs
Fourier Neural Operators (FNO) offer a principled approach to solving challenging partial differential equations (PDE) such as turbulent flows.
Machine Learning Accelerated PDE Backstepping Observers
State estimation is important for a variety of tasks, from forecasting to substituting for unmeasured states in feedback controllers.
Fast Sampling of Diffusion Models via Operator Learning
Diffusion models have found widespread adoption in various areas.
Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions
Finally, we find that not all transfer scenarios yield a positive gain, which seems related to the PLMs initial performance on the target-domain task.
DensePure: Understanding Diffusion Models towards Adversarial Robustness
By using the highest density point in the conditional distribution as the reversed sample, we identify the robust region of a given instance under the diffusion model's reverse process.
Real-time high-resolution CO$_2$ geological storage prediction using nested Fourier neural operators
Carbon capture and storage (CCS) plays an essential role in global decarbonization.
An Adversarial Active Sampling-based Data Augmentation Framework for Manufacturable Chip Design
These synthetic mask images will augment the original limited training dataset used to finetune the lithography model for improved performance.
1st Place Solution of The Robust Vision Challenge 2022 Semantic Segmentation Track
The model is trained on a composite dataset consisting of images from 9 datasets (ADE20K, Cityscapes, Mapillary Vistas, ScanNet, VIPER, WildDash 2, IDD, BDD, and COCO) with a simple dataset balancing strategy.
Context Generation Improves Open Domain Question Answering
Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge.
VIMA: General Robot Manipulation with Multimodal Prompts
We show that a wide spectrum of robot manipulation tasks can be expressed with multimodal prompts, interleaving textual and visual tokens.
State-specific protein-ligand complex structure prediction with a multi-scale deep generative model
The binding complexes formed by proteins and small molecule ligands are ubiquitous and critical to life.
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors.
Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control
In this paper, we propose a stability-constrained reinforcement learning (RL) method for real-time voltage control, that guarantees system stability both during policy learning and deployment of the learned policy.
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models
In evaluating cross-dataset generalization with unseen categories, TPT performs on par with the state-of-the-art approaches that use additional training data.
Retrieval-based Controllable Molecule Generation
On various tasks ranging from simple design criteria to a challenging real-world scenario for designing lead compounds that bind to the SARS-CoV-2 main protease, we demonstrate our approach extrapolates well beyond the retrieval database, and achieves better performance and wider applicability than previous methods.
MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Training
By only training a query-based image instance segmentation model, MinVIS outperforms the previous best result on the challenging Occluded VIS dataset by over 10% AP.
Ranked #15 on
Video Instance Segmentation
on YouTube-VIS validation
Robust Trajectory Prediction against Adversarial Attacks
We demonstrate that our method is able to improve the performance by 46% on adversarial data and at the cost of only 3% performance degradation on clean data, compared to the model trained with clean data.
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries
The resulting geo-FNO model has both the computation efficiency of FFT and the flexibility of handling arbitrary geometries.
Large Scale Mask Optimization Via Convolutional Fourier Neural Operator and Litho-Guided Self Training
Machine learning techniques have been extensively studied for mask optimization problems, aiming at better mask printability, shorter turnaround time, better mask manufacturability, and so on.
Langevin Monte Carlo for Contextual Bandits
Existing Thompson sampling-based algorithms need to construct a Laplace approximation (i. e., a Gaussian distribution) of the posterior distribution, which is inefficient to sample in high dimensional applications for general covariance matrices.
Thompson Sampling Achieves $\tilde O(\sqrt{T})$ Regret in Linear Quadratic Control
By carefully prescribing an early exploration strategy and a policy update rule, we show that TS achieves order-optimal regret in adaptive control of multidimensional stabilizable LQRs.
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Autonomous agents have made great strides in specialist domains like Atari games and Go.
Finite-Time Regret of Thompson Sampling Algorithms for Exponential Family Multi-Armed Bandits
We study the regret of Thompson sampling (TS) algorithms for exponential family bandits, where the reward distribution is from a one-dimensional exponential family, which covers many common reward distributions including Bernoulli, Gaussian, Gamma, Exponential, etc.
KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems
However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems.
Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions
A significant gap remains between today's visual pattern recognition models and human-level visual cognition especially when it comes to few-shot learning and compositional reasoning of novel concepts.
Ranked #1 on
Few-Shot Image Classification
on Bongard-HOI
Diffusion Models for Adversarial Purification
Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model.
Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds
Last, our control design extrapolates to unseen wind conditions, is shown to be effective for outdoor flights with only onboard sensors, and can transfer across drones with minimal performance degradation.
Generative Adversarial Neural Operators
The inputs to the generator are samples of functions from a user-specified probability measure, e. g., Gaussian random field (GRF), and the generator outputs are synthetic data functions.
Understanding The Robustness in Vision Transformers
Our study is motivated by the intriguing properties of the emerging visual grouping in Vision Transformers, which indicates that self-attention may promote robustness through improved mid-level representations.
Ranked #4 on
Domain Generalization
on ImageNet-R
(using extra training data)
RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning
This task remains challenging for current deep learning algorithms since it requires addressing three key technical problems jointly: 1) identifying object entities and their properties, 2) inferring semantic relations between pairs of entities, and 3) generalizing to novel object-relation combinations, i. e., systematic generalization.
Ranked #1 on
Zero-Shot Human-Object Interaction Detection
on HICO
M$^2$BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation
In this paper, we propose M$^2$BEV, a unified framework that jointly performs 3D object detection and map segmentation in the Birds Eye View~(BEV) space with multi-camera image inputs.
ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation
Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, bring substantial challenges due to infinite shape variations, non-rigid motions, and partial observability.
Generic Lithography Modeling with Dual-band Optics-Inspired Neural Networks
Lithography simulation is a critical step in VLSI design and optimization for manufacturability.
FreeSOLO: Learning to Segment Objects without Annotations
FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by +9. 8% AP when fine-tuning instance segmentation with only 5% COCO masks.
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models.
Pre-Trained Language Models for Interactive Decision-Making
Together, these results suggest that language modeling induces representations that are useful for modeling not just language, but also goals and plans; these representations can aid learning and generalization even outside of language processing.
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
We then provide the LM with instruction that consists of this subset of labeled exemplars, the query text to be classified, a definition of bias, and prompt it to make a decision.
CEM-GD: Cross-Entropy Method with Gradient Descent Planner for Model-Based Reinforcement Learning
It then uses the top trajectories as initialization for gradient descent and applies gradient updates to each of these trajectories to find the optimal action sequence.
Simulation Intelligence: Towards a New Generation of Scientific Methods
We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence.
Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions
In this paper, we systematically study the impact of various self-supervised learning proxy tasks on different architectures and threat models for 3D point clouds with adversarial training.
Coupled Segmentation and Edge Learning via Dynamic Graph Propagation
It is therefore interesting to study how these two tasks can be coupled to benefit each other.
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution.
Polymatrix Competitive Gradient Descent
In this work we propose polymatrix competitive gradient descent (PCGD) as a method for solving general sum competitive optimization involving arbitrary numbers of agents.
Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization
However, these approaches require larger state distributions to be covered as more policies are sequenced, and thus are limited to short skill sequences.
Physics-Informed Neural Operator for Learning Partial Differential Equations
Specifically, in PINO, we combine coarse-resolution training data with PDE constraints imposed at a higher resolution.
Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds
Certified robustness is a desirable property for deep neural networks in safety-critical applications, and popular training algorithms can certify robustness of a neural network by computing a global bound on its Lipschitz constant.
AugMax: Adversarial Composition of Random Augmentations for Robust Training
Diversity and hardness are two complementary dimensions of data augmentation to achieve robustness.
ZerO Initialization: Initializing Neural Networks with only Zeros and Ones
Deep neural networks are usually initialized with random weights, with adequately selected initial variance to ensure stable signal propagation during training.
Controllable and Compositional Generation with Latent-Space Energy-Based Models
In compositional generation, our method excels at zero-shot generation of unseen attribute combinations.
OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation
Operational Space Control (OSC) has been used as an effective task-space controller for manipulation.
Stability Constrained Reinforcement Learning for Real-Time Voltage Control
Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems.
Cost-Sensitive Hierarchical Classification through Layer-wise Abstentions
For each layer, we also achieve higher accuracy when the overall accuracy is kept fixed across different methods.
Efficient Token Mixing for Transformers via Adaptive Fourier Neural Operators
AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution.
Auditing AI models for Verified Deployment under Semantic Specifications
We enable such unit tests through variations in a semantically-interpretable latent space of a generative model.
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers
Specifically, we supervise the attention modules in the mask decoder in a layer-wise manner.
Ranked #4 on
Panoptic Segmentation
on COCO test-dev
U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow
Here we present U-FNO, a novel neural network architecture for solving multiphase flow problems with superior accuracy, speed, and data efficiency.
Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems
Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or convex quadratic regulating costs.
Neural Operator: Learning Maps Between Function Spaces
The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets.
Tensor Methods in Computer Vision and Deep Learning
Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions.
Long-Short Transformer: Efficient Transformers for Language and Vision
For instance, Transformer-LS achieves 0. 97 test BPC on enwik8 using half the number of parameters than previous method, while being faster and is able to handle 3x as long sequences compared to its full-attention version on the same hardware.
Ranked #1 on
Language Modelling
on enwik8 dev
LNS-Madam: Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update
Representing deep neural networks (DNNs) in low-precision is a promising approach to enable efficient acceleration and memory reduction.