Search Results for author:
Found 33 papers, 19 papers with code
Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching
We introduce Adjoint Sampling, a highly scalable and efficient algorithm for learning diffusion processes that sample from unnormalized densities, or energy functions.
All-atom Diffusion Transformers: Unified generative modelling of molecules and materials
We introduce the All-atom Diffusion Transformer (ADiT), a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems using the same model: (1) An autoencoder maps a unified, all-atom representations of molecules and materials to a shared latent embedding space; and (2) A diffusion model is trained to generate new latent embeddings that the autoencoder can decode to sample new molecules or materials.
Ranked #1 on
Unconditional Molecule Generation
on QM9
Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspective
Through the lens of optimizing the symmetric kinetic energy, we propose velocity formulas that can be applied to any given probability path, completely decoupling the probability and velocity, and giving the user the freedom to specify any desirable probability path based on expert knowledge specific to the data domain.
FlowLLM: Flow Matching for Material Generation with Large Language Models as Base Distributions
Material discovery is a critical area of research with the potential to revolutionize various fields, including carbon capture, renewable energy, and electronics.
Ranked #2 on
Unconditional Crystal Generation
on MP20
Distribution Learning for Molecular Regression
Using "soft" targets to improve model performance has been shown to be effective in classification settings, but the usage of soft targets for regression is a much less studied topic in machine learning.
FlowMM: Generating Materials with Riemannian Flow Matching
Crystalline materials are a fundamental component in next-generation technologies, yet modeling their distribution presents unique computational challenges.
Ranked #3 on
Unconditional Crystal Generation
on MP20
Fine-Tuned Language Models Generate Stable Inorganic Materials as Text
We propose fine-tuning large language models for generation of stable materials.
The Open DAC 2023 Dataset and Challenges for Sorbent Discovery in Direct Air Capture
We also trained state-of-the-art ML models on this dataset to approximate calculations at the DFT level.
Spherical Channels for Modeling Atomic Interactions
We propose the Spherical Channel Network (SCN) to model atomic energies and forces.
Wav2Vec-Aug: Improved self-supervised training with limited data
Self-supervised learning (SSL) of speech representations has received much attention over the last few years but most work has focused on languages and domains with an abundance of unlabeled data.
The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts
The development of machine learning models for electrocatalysts requires a broad set of training data to enable their use across a wide variety of materials.
GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets
This work investigates this question by first developing the GemNet-OC model based on the large Open Catalyst 2020 (OC20) dataset.
Ranked #1 on
Initial Structure to Relaxed Energy (IS2RE)
on OC20
Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations
Recent progress in Graph Neural Networks (GNNs) for modeling atomic simulations has the potential to revolutionize catalyst discovery, which is a key step in making progress towards the energy breakthroughs needed to combat climate change.
Ranked #2 on
Initial Structure to Relaxed Energy (IS2RE)
on OC20
Rotation Invariant Graph Neural Networks using Spin Convolutions
We introduce a novel approach to modeling angular information between sets of neighboring atoms in a graph neural network.
Ranked #3 on
Initial Structure to Relaxed Energy (IS2RE)
on OC20
Graph Neural Network
Initial Structure to Relaxed Energy (IS2RE)
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training
On a large-scale competitive setup, we show that pre-training on unlabeled in-domain data reduces the gap between models trained on in-domain and out-of-domain labeled data by 66%-73%.
ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
By not imposing explicit physical constraints, we can flexibly design expressive models while maintaining their computational efficiency.
COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction
The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0. 742 for predicting an adverse event within 96 hours (compared to 0. 703 with supervised pretraining) and an AUC of 0. 765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0. 749 with supervised pretraining).
ForceNet: A Graph Neural Network for Large-Scale Quantum Chemistry Simulation
We use ForceNet to perform quantum chemistry simulations, where ForceNet is able to achieve 4x higher success rate than existing ML models.
Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction
Accelerating MRI scans is one of the principal outstanding problems in the MRI research community.
MLS: A Large-Scale Multilingual Dataset for Speech Research
This paper introduces Multilingual LibriSpeech (MLS) dataset, a large multilingual corpus suitable for speech research.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
The Open Catalyst 2020 (OC20) Dataset and Community Challenges
Catalyst discovery and optimization is key to solving many societal and energy challenges including solar fuels synthesis, long-term energy storage, and renewable fertilizer production.
An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage
As we increase our reliance on renewable energy sources such as wind and solar, which produce intermittent power, storage is needed to transfer power from times of peak generation to peak demand.
Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters
We study training a single acoustic model for multiple languages with the aim of improving automatic speech recognition (ASR) performance on low-resource languages, and over-all simplifying deployment of ASR systems that support diverse languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
End-to-End Variational Networks for Accelerated MRI Reconstruction
The slow acquisition speed of magnetic resonance imaging (MRI) has led to the development of two complementary methods: acquiring multiple views of the anatomy simultaneously (parallel imaging) and acquiring fewer samples than necessary for traditional signal processing methods (compressed sensing).
Ranked #1 on
MRI Reconstruction
on fastMRI Knee 4x
Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge
Conclusion: The challenge led to new developments in machine learning for image reconstruction, provided insight into the current state of the art in the field, and highlighted remaining hurdles for clinical adoption.
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
We study pseudo-labeling for the semi-supervised training of ResNet, Time-Depth Separable ConvNets, and Transformers for speech recognition, with either CTC or Seq2Seq loss functions.
Ranked #19 on
Speech Recognition
on LibriSpeech test-other
(using extra training data)
RNN-T For Latency Controlled ASR With Improved Beam Search
Neural transducer-based systems such as RNN Transducers (RNN-T) for automatic speech recognition (ASR) blend the individual components of a traditional hybrid ASR systems (acoustic model, language model, punctuation model, inverse text normalization) into one single model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction
In this paper, we present a novel method to integrate traditional parallel imaging methods into deep neural networks that is able to generate high quality reconstructions even for high acceleration factors.
fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measurements has the potential to reduce medical costs, minimize stress to patients and make MRI possible in applications where it is currently prohibitively slow or expensive.
Robust Speech Recognition Using Generative Adversarial Networks
This paper describes a general, scalable, end-to-end framework that uses the generative adversarial network (GAN) objective to enable robust speech recognition.
Cold Fusion: Training Seq2Seq Models Together with Language Models
Sequence-to-sequence (Seq2Seq) models with attention have excelled at tasks which involve generating natural language sentences such as machine translation, image captioning and speech recognition.
Exploring Neural Transducers for End-to-End Speech Recognition
In this work, we perform an empirical comparison among the CTC, RNN-Transducer, and attention-based Seq2Seq models for end-to-end speech recognition.
Reducing Bias in Production Speech Models
Replacing hand-engineered pipelines with end-to-end deep learning systems has enabled strong results in applications like speech and object recognition.
