no code implementations • 18 Jun 2025 • Shufan Li, Aditya Grover
Large Language Models (LLMs) are widely used in real-time voice chat applications, typically in combination with text-to-speech (TTS) systems to generate audio responses.
1 code implementation • 4 Jun 2025 • Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, Ludwig Schmidt
Yet, there are still many open questions about the best training recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available.
no code implementations • 31 May 2025 • Daniel Israel, Guy Van Den Broeck, Aditya Grover
The generation speed of LLMs are bottlenecked by autoregressive decoding, where tokens are predicted sequentially one by one.
1 code implementation • 22 May 2025 • Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, Aditya Grover
We build LaViDa by equipping DMs with a vision encoder and jointly fine-tune the combined parts for multimodal instruction following.
no code implementations • 16 Apr 2025 • Siyan Zhao, Devaansh Gupta, Qinqing Zheng, Aditya Grover
Recent large language models (LLMs) have demonstrated strong reasoning capabilities that benefits from online reinforcement learning (RL).
no code implementations • 15 Apr 2025 • Vinay Shukla, Prachee Sharma, Ryan Rossi, Sungchul Kim, Tong Yu, Aditya Grover
In this work, we propose WaterFlow (WF), a fast and extremely robust approach for high fidelity visual watermarking based on a learned latent-dependent watermark.
1 code implementation • 15 Mar 2025 • Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Arsh Koneru, Yusuke Kato, Kazuki Kozuka, Aditya Grover
Currently, inference-time scaling for text-to-image diffusion models is largely limited to best-of-N sampling, where multiple images are generated per prompt and a selection model chooses the best output.
1 code implementation • 9 Mar 2025 • Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, Kai-Wei Chang
We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos.
no code implementations • 9 Feb 2025 • Daniel Israel, Aditya Grover, Guy Van Den Broeck
Historically, LLMs have been trained using either autoregressive (AR) or masked language modeling (MLM) objectives, with AR models gaining dominance in recent years.
1 code implementation • 17 Dec 2024 • Hritik Bansal, Daniel Israel, Siyan Zhao, Shufan Li, Tung Nguyen, Aditya Grover
To address these gaps, we present MedMax, the first large-scale multimodal biomedical instruction-tuning dataset for mixed-modal foundation models.
1 code implementation • CVPR 2025 • Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Zichun Liao, Yusuke Kato, Kazuki Kozuka, Aditya Grover
We introduce OmniFlow, a novel generative model designed for any-to-any generation tasks such as text-to-image, text-to-audio, and audio-to-image synthesis.
1 code implementation • 30 Oct 2024 • Qinqing Zheng, Mikael Henaff, Amy Zhang, Aditya Grover, Brandon Amos
Our approach annotates the agent's collected experience via an asynchronous LLM server, which is then distilled into an intrinsic reward model.
1 code implementation • 28 Jun 2024 • Shufan Li, Harkanwar Singh, Aditya Grover
To address this limitation, we introduce PopAlign, a novel approach for population-level preference optimization, while standard optimization would prefer entire sets of samples over others.
no code implementations • 27 Jun 2024 • Tung Nguyen, Aditya Grover
However, directly prompting a pretrained language model to produce predictions is not feasible in many scientific domains due to the scarcity of domain-specific data in the pretraining corpora and the challenges of articulating complex problems in natural language.
1 code implementation • 17 Jun 2024 • Siyan Zhao, Tung Nguyen, Aditya Grover
In-context learning is a key paradigm in large language models (LLMs) that enables them to generalize to new tasks and domains by simply prompting these models with a few exemplars without explicit parameter updates.
no code implementations • 5 Jun 2024 • Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, Aditya Grover
Recent advances in internet-scale video data pretraining have led to the development of text-to-video generative models that can create high-quality videos across a broad range of visual concepts, synthesize realistic motions and render complex objects.
1 code implementation • 7 May 2024 • Hritik Bansal, Yonatan Bitton, Michal Yarom, Idan Szpektor, Aditya Grover, Kai-Wei Chang
For instance, we condition the visual features of the earlier and later scenes of the generated video with the representations of the first scene description (e. g., 'a red panda climbing a tree') and second scene description (e. g., 'the red panda sleeps on the top of the tree'), respectively.
1 code implementation • 15 Apr 2024 • Siyan Zhao, Daniel Israel, Guy Van Den Broeck, Aditya Grover
In this work, we highlight the following pitfall of prefilling: for batches containing high-varying prompt lengths, significant computation is wasted by the standard practice of padding sequences to the maximum length.
1 code implementation • 31 Mar 2024 • Hritik Bansal, Ashima Suvarna, Gantavya Bhatt, Nanyun Peng, Kai-Wei Chang, Aditya Grover
A common technique for aligning large language models (LLMs) relies on acquiring human preferences by comparing multiple generations conditioned on a fixed context.
no code implementations • 27 Mar 2024 • Valay Bundele, Mahesh Bhupati, Biplab Banerjee, Aditya Grover
The study of vision-and-language navigation (VLN) has typically relied on expert trajectories, which may not always be available in real-world situations due to the significant effort required to collect them.
1 code implementation • 8 Feb 2024 • Shufan Li, Harkanwar Singh, Aditya Grover
A recent architecture, Mamba, based on state space models has been shown to achieve comparable performance for modeling text sequences, while scaling linearly with the sequence length.
1 code implementation • 1 Feb 2024 • Juan Nathaniel, Yongquan Qu, Tung Nguyen, Sungduk Yu, Julius Busecke, Aditya Grover, Pierre Gentine
Thus, we propose ChaosBench, a challenging benchmark to extend the predictability range of data-driven weather emulators to S2S timescale.
1 code implementation • 11 Dec 2023 • Shufan Li, Harkanwar Singh, Aditya Grover
We demonstrate that our system can perform a series of novel instruction-guided editing tasks.
1 code implementation • 6 Dec 2023 • Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Romit Maulik, Veerabhadra Kotamarthi, Ian Foster, Sandeep Madireddy, Aditya Grover
At the core of Stormer is a randomized forecasting objective that trains the model to forecast the weather dynamics over varying time intervals.
no code implementations • 22 Nov 2023 • Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, Ricky T. Q. Chen
Classifier-free guidance is a key component for enhancing the performance of conditional generative models across diverse tasks.
1 code implementation • CVPR 2024 • Hritik Bansal, Yonatan Bitton, Idan Szpektor, Kai-Wei Chang, Aditya Grover
Despite being (pre)trained on a massive amount of data, state-of-the-art video-language alignment models are not robust to semantically-plausible contrastive changes in the video captions.
1 code implementation • NeurIPS 2023 • Tung Nguyen, Sudhanshu Agrawal, Aditya Grover
In this work, we address the more challenging yet realistic setting of few-shot experimental design, where only a few labeled data points of input designs and their corresponding values are available.
1 code implementation • 17 Oct 2023 • Siyan Zhao, John Dang, Aditya Grover
We introduce Group Preference Optimization (GPO), an alignment framework that steers language models to preferences of individual groups in a few-shot manner.
1 code implementation • 9 Oct 2023 • Daniel Israel, Aditya Grover, Guy Van Den Broeck
For example, in medical settings, backdoor adjustment can be used to control for confounding and estimate the effectiveness of a treatment.
1 code implementation • 30 Aug 2023 • Hritik Bansal, John Dang, Aditya Grover
In particular, we find that LLMs that leverage rankings data for alignment (say model X) are preferred over those that leverage ratings data (say model Y), with a rank-based evaluation protocol (is X/Y's response better than reference response?)
1 code implementation • NeurIPS 2023 • Tung Nguyen, Jason Jewik, Hritik Bansal, Prakhar Sharma, Aditya Grover
Modeling weather and climate is an essential endeavor to understand the near- and long-term impacts of climate change, as well as inform technology and policymaking for adaptation and mitigation efforts.
1 code implementation • 12 Jun 2023 • Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover
The goal of offline black-box optimization (BBO) is to optimize an expensive black-box function using a fixed dataset of function evaluations.
1 code implementation • 30 Apr 2023 • Baiting Zhu, Meihua Dang, Aditya Grover
In this work, we propose a new data-driven setup for offline MORL, where we wish to learn a preference-agnostic policy agent using only a finite dataset of offline demonstrations of other agents and their preferences.
1 code implementation • ICCV 2023 • Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin, Aditya Grover, Kai-Wei Chang
Multimodal contrastive pretraining has been used to train multimodal representation models, such as CLIP, on large amounts of paired image-text data.
1 code implementation • 5 Feb 2023 • Hritik Bansal, Aditya Grover
Recent research on robustness has revealed significant performance gaps between neural image classifiers trained on datasets that are similar to the test set, and those that are from a naturally shifted distribution, such as sketches, paintings, and animations of the object categories observed during training.
1 code implementation • 24 Jan 2023 • Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, Aditya Grover
We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings.
1 code implementation • 23 Nov 2022 • Fangchen Liu, Hao liu, Aditya Grover, Pieter Abbeel
We are interested in learning scalable agents for reinforcement learning that can learn from large-scale, diverse sequential data similar to current large vision and language models.
1 code implementation • 12 Oct 2022 • Qinqing Zheng, Mikael Henaff, Brandon Amos, Aditya Grover
For this setting, we develop and study a simple meta-algorithmic pipeline that learns an inverse dynamics model on the labelled data to obtain proxy-labels for the unlabelled data, followed by the use of any offline RL algorithm on the true and proxy-labelled trajectories.
1 code implementation • 11 Oct 2022 • Tung Nguyen, Qinqing Zheng, Aditya Grover
We study CWBC in the context of RvS (Emmons et al., 2021) and Decision Transformers (Chen et al., 2021), and show that CWBC significantly boosts their performance on various benchmarks.
no code implementations • 11 Jul 2022 • Heli Ben-Hamu, samuel cohen, Joey Bose, Brandon Amos, Aditya Grover, Maximilian Nickel, Ricky T. Q. Chen, Yaron Lipman
Continuous Normalizing Flows (CNFs) are a class of generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE).
1 code implementation • 9 Jul 2022 • Tung Nguyen, Aditya Grover
We propose Transformer Neural Processes (TNPs), a new member of the NP family that casts uncertainty-aware meta learning as a sequence modeling problem.
no code implementations • 23 Jun 2022 • Manmeet Singh, Vaisakh S B, Nachiketa Acharya, Aditya Grover, Suryachandra A Rao, Bipin Kumar, Zong-Liang Yang, Dev Niyogi
We augment the output of the well-known NWP model CFSv2 with deep learning to create a hybrid model that improves short-range global precipitation at 1-, 2-, and 3-day lead times.
1 code implementation • 22 Jun 2022 • Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover
Many problems in science and engineering involve optimizing an expensive black-box function over a high-dimensional space.
1 code implementation • 28 May 2022 • Shashank Goel, Hritik Bansal, Sumit Bhatia, Ryan A. Rossi, Vishwa Vinay, Aditya Grover
Recent advances in contrastive representation learning over paired image-text data have led to models such as CLIP that achieve state-of-the-art performance for zero-shot classification and distributional robustness.
no code implementations • 7 Apr 2022 • Carl Qi, Pieter Abbeel, Aditya Grover
The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal.
no code implementations • 22 Feb 2022 • Yuqing Du, Pieter Abbeel, Aditya Grover
Training such agents efficiently requires automatic generation of a goal curriculum.
2 code implementations • 11 Feb 2022 • Qinqing Zheng, Amy Zhang, Aditya Grover
Recent work has shown that offline reinforcement learning (RL) can be formulated as a sequence modeling problem (Chen et al., 2021; Janner et al., 2021) and solved via approaches similar to large-scale language modeling.
no code implementations • 31 Dec 2021 • Nimit S. Sohoni, Maziar Sanjabi, Nicolas Ballas, Aditya Grover, Shaoliang Nie, Hamed Firooz, Christopher Ré
Theoretically, we provide generalization bounds for our approach in terms of the worst-group performance, which scale with respect to both the total number of training points and the number of training points with group labels.
1 code implementation • NeurIPS 2021 • Chris Cundy, Aditya Grover, Stefano Ermon
We propose Bayesian Causal Discovery Nets (BCD Nets), a variational inference framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM.
1 code implementation • ICLR 2022 • Omri Puny, Matan Atzmon, Heli Ben-Hamu, Ishan Misra, Aditya Grover, Edward J. Smith, Yaron Lipman
For example, Euclidean motion invariant/equivariant graph or point cloud neural networks.
no code implementations • 29 Sep 2021 • Joey Bose, Ricardo Pio Monti, Aditya Grover
Deep generative models excel at generating complex, high-dimensional data, often exhibiting impressive generalization beyond the training distribution.
1 code implementation • ICLR 2022 • Yuqing Du, Pieter Abbeel, Aditya Grover
We are interested in training general-purpose reinforcement learning agents that can solve a wide variety of goals.
1 code implementation • NeurIPS 2021 • Noam Rozen, Aditya Grover, Maximilian Nickel, Yaron Lipman
MF also produces a CNF via a solution to the change-of-variable formula, however differently from other CNF methods, its model (learned) density is parameterized as the source (prior) density minus the divergence of a neural network (NN).
1 code implementation • 28 Jun 2021 • Wenshuo Guo, Kumar Krishna Agrawal, Aditya Grover, Vidya Muthukumar, Ashwin Pananjady
We introduce the "inverse bandit" problem of estimating the rewards of a multi-armed bandit instance from observing the learning process of a low-regret demonstrator.
1 code implementation • 17 Jun 2021 • Muhammed Shuaibi, Adeesh Kolluru, Abhishek Das, Aditya Grover, Anuroop Sriram, Zachary Ulissi, C. Lawrence Zitnick
We introduce a novel approach to modeling angular information between sets of neighboring atoms in a graph neural network.
Ranked #3 on
Initial Structure to Relaxed Energy (IS2RE)
on OC20
Graph Neural Network
Initial Structure to Relaxed Energy (IS2RE)
20 code implementations • NeurIPS 2021 • Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch
In particular, we present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling.
Ranked #3 on
Offline RL
on D4RL
1 code implementation • 2 Jun 2021 • Kourosh Hakhamaneshi, Pieter Abbeel, Vladimir Stojanovic, Aditya Grover
Such a decomposition can dynamically control the reliability of information derived from the online and offline data and the use of pretrained neural networks permits scalability to large offline datasets.
4 code implementations • 9 Mar 2021 • Kevin Lu, Aditya Grover, Pieter Abbeel, Igor Mordatch
We investigate the capability of a transformer pretrained on natural language to generalize to other modalities with minimal finetuning -- in particular, without finetuning of the self-attention and feedforward layers of the residual blocks.
1 code implementation • ICLR 2021 • Yilun Xu, Yang song, Sahaj Garg, Linyuan Gong, Rui Shu, Aditya Grover, Stefano Ermon
Experimentally, we demonstrate in several image and audio generation tasks that sample quality degrades gracefully as we reduce the computational budget for sampling.
no code implementations • 1 Jan 2021 • Carl Qi, Pieter Abbeel, Aditya Grover
The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal.
1 code implementation • NeurIPS 2021 • Robin Swezey, Aditya Grover, Bruno Charron, Stefano Ermon
A key challenge with machine learning approaches for ranking is the gap between the performance metrics of interest and the surrogate loss functions that can be optimized with gradient-based methods.
1 code implementation • ICLR 2021 • Kevin Lu, Aditya Grover, Pieter Abbeel, Igor Mordatch
We propose Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills.
1 code implementation • 2 Mar 2020 • Chenhao Niu, Yang song, Jiaming Song, Shengjia Zhao, Aditya Grover, Stefano Ermon
In particular, we design a permutation equivariant, multi-channel graph neural network to model the gradient of the data distribution at the input graph (a. k. a., the score function).
1 code implementation • ICML 2020 • Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, Stefano Ermon
Real-world datasets are often biased with respect to key demographic factors such as race and gender.
2 code implementations • NeurIPS 2019 • Aditya Grover, Jiaming Song, Alekh Agarwal, Kenneth Tran, Ashish Kapoor, Eric Horvitz, Stefano Ermon
A standard technique to correct this bias is importance sampling, where samples from the model are weighted by the likelihood ratio under model and true distributions.
2 code implementations • ICLR Workshop DeepGenStruct 2019 • Aditya Grover, Christopher Chute, Rui Shu, Zhangjie Cao, Stefano Ermon
Given datasets from multiple domains, a key challenge is to efficiently exploit these data sources for modeling a target domain.
no code implementations • ICLR Workshop DeepGenStruct 2019 • Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric Horvitz, Stefano Ermon
A standard technique to correct this bias is by importance weighting samples from the model by the likelihood ratio under the model and true distributions.
1 code implementation • ICLR 2019 • Aditya Grover, Eric Wang, Aaron Zweig, Stefano Ermon
Sorting input objects is an important step in many machine learning pipelines.
no code implementations • 26 Dec 2018 • Aditya Grover, Stefano Ermon
We treat the low-dimensional projections as noisy latent representations of an autoencoder and directly learn both the acquisition (i. e., encoding) and amortized recovery (i. e., decoding) procedures.
3 code implementations • 11 Dec 2018 • Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, Stefano Ermon
Learning data representations that are transferable and are fair with respect to certain protected attributes is crucial to reducing unfair decisions while preserving the utility of the data.
1 code implementation • NeurIPS 2018 • Aditya Grover, Tudor Achim, Stefano Ermon
Several algorithms for solving constraint satisfaction problems are based on survey propagation, a variational inference scheme used to obtain approximate marginal probability estimates for variable assignments.
1 code implementation • 19 Nov 2018 • Kristy Choi, Kedar Tatwawadi, Aditya Grover, Tsachy Weissman, Stefano Ermon
For reliable transmission across a noisy communication channel, classical results from information theory show that it is asymptotically optimal to separate out the source and channel coding processes.
2 code implementations • ICML 2018 • Manik Dhar, Aditya Grover, Stefano Ermon
In compressed sensing, a small number of linear measurements can be used to reconstruct an unknown signal.
no code implementations • ICML 2018 • Aditya Grover, Maruan Al-Shedivat, Jayesh K. Gupta, Yura Burda, Harrison Edwards
Modeling agent behavior is central to understanding the emergence of complex phenomena in multiagent systems.
no code implementations • 5 Apr 2018 • Aditya Grover, Ramki Gummadi, Miguel Lazaro-Gredilla, Dale Schuurmans, Stefano Ermon
Learning latent variable models with stochastic variational inference is challenging when the approximate posterior is far from the true posterior, due to high variance in the gradient estimates.
no code implementations • 29 Mar 2018 • Aditya Grover, Todor Markov, Peter Attia, Norman Jin, Nicholas Perkins, Bryan Cheong, Michael Chen, Zi Yang, Stephen Harris, William Chueh, Stefano Ermon
We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback.
1 code implementation • 28 Mar 2018 • Aditya Grover, Aaron Zweig, Stefano Ermon
Graphs are a fundamental abstraction for modeling relational data.
Ranked #8 on
Link Prediction
on Citeseer
3 code implementations • 24 May 2017 • Aditya Grover, Manik Dhar, Stefano Ermon
Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood.
1 code implementation • 27 Feb 2017 • Aditya Grover, Stefano Ermon
We propose a novel approach for using unsupervised boosting to create an ensemble of generative models, where models are trained in sequence to correct earlier mistakes.
no code implementations • NeurIPS 2016 • Aditya Grover, Stefano Ermon
We provide a new approach for learning latent variable models based on optimizing our new bounds on the log-likelihood.
20 code implementations • 3 Jul 2016 • Aditya Grover, Jure Leskovec
Taken together, our work represents a new way for efficiently learning state-of-the-art task-independent representations in complex networks.
Ranked #1 on
Node Classification
on PPI
(Macro-F1 metric)
no code implementations • 30 Jun 2016 • Ankit Anand, Aditya Grover, Mausam, Parag Singla
We extend previous work on exploiting symmetries in the MCMC framework to the case of contextual symmetries.