Search Results for author:
Found 81 papers, 46 papers with code
ChipNeMo: Domain-Adapted LLMs for Chip Design
ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design.
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
Our results highlights the promising direction to obtain a better GPT decoder for QA through continued pretraining with retrieval before instruction tuning.
Retrieval meets Long Context Large Language Models
Perhaps surprisingly, we find that LLM with 4K context window using simple retrieval-augmentation at generation can achieve comparable performance to finetuned LLM with 16K context window via positional interpolation on long context tasks, while taking much less computation.
P-Flow: A Fast and Data-Efficient Zero-Shot TTS through Speech Prompting
P-Flow comprises a speech-prompted text encoder for speaker adaptation and a flow matching generative decoder for high-quality and fast speech synthesis.
CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram
In this work, we present CleanUNet 2, a speech denoising model that combines the advantages of waveform denoiser and spectrogram denoiser and achieves the best of both worlds.
RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models
In this paper, we investigate the in-context learning ability of retrieval-augmented encoder-decoder language models.
GraPhSyM: Graph Physical Synthesis Model
Given a gate-level netlist of a circuit represented as a graph, GraPhSyM utilizes graph structure, connectivity, and electrical property features to predict the impact of physical synthesis transformations such as buffer insertion and gate sizing.
Progressive Learning of 3D Reconstruction Network from 2D GAN Data
In this work, to overcome these limitations of generated datasets, we have two main contributions which lead us to achieve state-of-the-art results on challenging objects: 1) A robust multi-stage learning scheme that gradually relies more on the models own predictions when calculating losses, 2) A novel adversarial learning pipeline with online pseudo-ground truth generations to achieve fine details.
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Despite tremendous progress in generating high-quality images using diffusion models, synthesizing a sequence of animated frames that are both photorealistic and temporally coherent is still in its infancy.
Ranked #3 on
Text-to-Video Generation
on UCF-101
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
To answer it, we perform a comprehensive study on a scalable pre-trained retrieval-augmented LM (i. e., RETRO) compared with standard GPT and retrieval-augmented GPT incorporated at fine-tuning or inference stages.
VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation
We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system.
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models
Pretrained large language models have become indispensable for solving various natural language processing (NLP) tasks.
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
Augmenting pretrained language models (LMs) with a vision encoder (e. g., Flamingo) has obtained the state-of-the-art results in image-to-text generation.
Multilingual Multiaccented Multispeaker TTS with RADTTS
We work to create a multilingual speech synthesis system which can generate speech with the proper accent while retaining the characteristics of an individual voice.
eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
Therefore, in contrast to existing works, we propose to train an ensemble of text-to-image diffusion models specialized for different synthesis stages.
Ranked #13 on
Text-to-Image Generation
on COCO
Evaluating Parameter Efficient Learning for Generation
For cross-domain and cross-dataset cases, we show that (a) Adapter (Houlsby et al., 2019) performs the best amongst all the PERMs studied here, and (b) it outperforms finetuning if the task dataset is below a certain size.
Context Generation Improves Open Domain Question Answering
Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge.
Factuality Enhanced Language Models for Open-Ended Text Generation
In this work, we measure and improve the factual accuracy of large-scale LMs for open-ended text generation.
BigVGAN: A Universal Neural Vocoder with Large-Scale Training
Despite recent progress in generative adversarial network (GAN)-based vocoders, where the model generates raw waveform conditioned on acoustic features, it is challenging to synthesize high-fidelity audio for numerous speakers across various recording environments.
Ranked #4 on
Speech Synthesis
on LibriTTS
PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning
Deep Convolutional RL agents trained on this environment produce prefix adder circuits that Pareto-dominate existing baselines with up to 16. 0% and 30. 2% lower area for the same delay in the 32b and 64b settings respectively.
Reducing Activation Recomputation in Large Transformer Models
In this paper, we show how to significantly accelerate training of large transformer models by reducing activation recomputation.
Fine Detailed Texture Learning for 3D Meshes with Generative Models
The reconstruction is posed as an adaptation problem and is done progressively where in the first stage, we focus on learning accurate geometry, whereas in the second stage, we focus on learning the texture with a generative adversarial network.
Multi-Stage Prompting for Knowledgeable Dialogue Generation
We propose a multi-stage prompting approach to generate knowledgeable responses from a single pretrained LM.
Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows
Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2.
Speech Denoising in the Waveform Domain with Self-Attention
In this work, we present CleanUNet, a causal speech denoising model on the raw waveform.
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models.
Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement
Video compression is a central feature of the modern internet powering technologies from social media to video conferencing.
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Next, we detail the training process, the design of our training corpus, and our data curation techniques, which we believe is a key ingredient to the success of the model.
Ranked #2 on
Language Modelling
on LAMBADA
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
We then provide the LM with instruction that consists of this subset of labeled exemplars, the query text to be classified, a definition of bias, and prompt it to make a decision.
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution.
Efficient Token Mixing for Transformers via Adaptive Fourier Neural Operators
AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution.
Guiding Global Placement With Reinforcement Learning
In this work we augment state-of-the-art, force-based global placement solvers with a reinforcement learning agent trained to improve the final detail placed Half Perimeter Wire Length (HPWL).
One TTS Alignment To Rule Them All
However, these alignments tend to be brittle and often fail to generalize to long utterances and out-of-domain text, leading to missing or repeating words.
Long-Short Transformer: Efficient Transformers for Language and Vision
For instance, Transformer-LS achieves 0. 97 test BPC on enwik8 using half the number of parameters than previous method, while being faster and is able to handle 3x as long sequences compared to its full-attention version on the same hardware.
Ranked #1 on
Language Modelling
on enwik8 dev
View Generalization for Single Image Textured 3D Models
We describe a cycle consistency loss that encourages model textures to be aligned, so as to encourage sharing.
RAD-TTS: Parallel Flow-Based TTS with Robust Alignment Learning and Diverse Synthesis
This work introduces a predominantly parallel, end-to-end TTS model based on normalizing flows.
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
In this paper, we show how different types of parallelism methods (tensor, pipeline, and data parallelism) can be composed to scale to thousands of GPUs and models with trillions of parameters.
Dual Contrastive Loss and Attention for GANs
Lastly, we study different attention architectures in the discriminator, and propose a reference attention mechanism.
End-to-End Training of Neural Retrievers for Open-Domain Question Answering
We also explore two approaches for end-to-end supervised training of the reader and retriever components in OpenQA models.
Can Q-Learning with Graph Networks Learn a Generalizable Branching Heuristic for a SAT Solver?
While more work is needed to apply Graph-Q-SAT to reduce wall clock time in modern SAT solving settings, it is a compelling proof-of-concept showing that RL equipped with Graph Neural Networks can learn a generalizable branching heuristic for SAT search.
Neural FFTs for Universal Texture Image Synthesis
The conventional CNNs, recently adopted for synthesis, require to train and test on the same set of images and fail to generalize to unseen images.
Local Knowledge Powered Conversational Agents
State-of-the-art conversational agents have advanced significantly in conjunction with the use of large transformer-based language models.
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models
We showcase the controllability of our model by replacing the keywords used to generate stories and re-running the generation process.
DiffWave: A Versatile Diffusion Model for Audio Synthesis
In this work, we propose DiffWave, a versatile diffusion probabilistic model for conditional and unconditional waveform generation.
Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
Specifically, we directly treat the whole encoded feature map of the input texture as transposed convolution filters and the features' self-similarity map, which captures the auto-correlation information, as input to the transposed convolution.
Hierarchical Multi-Scale Attention for Semantic Segmentation
Multi-scale inference is commonly used to improve the results of semantic segmentation.
Ranked #4 on
Semantic Segmentation
on Cityscapes val
(using extra training data)
Large Scale Multi-Actor Generative Dialog Modeling
This work introduces the Generative Conversation Control model, an augmented and fine-tuned GPT-2 language model that conditions on past reference conversations to probabilistically model multi-turn conversations in the actor's persona.
Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
In this paper we propose Flowtron: an autoregressive flow-based generative network for text-to-speech synthesis with control over speech variation and style transfer.
Ranked #1 on
Text-To-Speech Synthesis
on LJSpeech
(Pleasantness MOS metric, using extra
training data)
Panoptic-based Image Synthesis
Conditional image synthesis for generating photorealistic images serves various applications for content editing to content generation.
Genome Variant Calling with a Deep Averaging Network
Variant calling, the problem of estimating whether a position in a DNA sequence differs from a reference sequence, given noisy, redundant, overlapping short sequences that cover that position, is fundamental to genomics.
Training Question Answering Models From Synthetic Data
On the SQuAD1. 1 question answering task, we achieve higher accuracy using solely synthetic questions and answers than when using the SQuAD1. 1 training set questions alone.
Unsupervised Disentanglement of Pose, Appearance and Background from Images and Videos
However, the reconstruction task of the entire image forces the model to allocate landmarks to model the background.
Neural ODEs for Image Segmentation with Level Sets
We propose a novel approach for image segmentation that combines Neural Ordinary Differential Equations (NODEs) and the Level Set method.
Zero-shot Text Classification With Generative Language Models
This work investigates the use of natural language to enable zero-shot model adaptation to new tasks.
Few-shot Video-to-Video Synthesis
To address the limitations, we propose a few-shot vid2vid framework, which learns to synthesize videos of previously unseen subjects or scenes by leveraging few example images of the target at test time.
Ranked #1 on
Video-to-Video Synthesis
on YouTube Dancing
Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens
Mellotron is a multispeaker voice synthesis model based on Tacotron 2 GST that can make a voice emote and sing without emotive or singing training data.
Can $Q$-Learning with Graph Networks Learn a Generalizable Branching Heuristic for a SAT Solver?
While more work is needed to apply Graph-$Q$-SAT to reduce wall clock time in modern SAT solving settings, it is a compelling proof-of-concept showing that RL equipped with Graph Neural Networks can learn a generalizable branching heuristic for SAT search.
Improving SAT Solver Heuristics with Graph Networks and Reinforcement Learning
We present GQSAT, a branching heuristic in a Boolean SAT solver trained with value-based reinforcement learning (RL) using Graph Neural Networks for function approximation.
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
To demonstrate that large language models can further advance the state of the art (SOTA), we train an 8. 3 billion parameter transformer language model similar to GPT-2 and a 3. 9 billion parameter model similar to BERT.
Ranked #2 on
Reading Comprehension
on RACE
Video Interpolation and Prediction with Unsupervised Landmarks
Prediction and interpolation for long-range video data involves the complex task of modeling motion trajectories for each visible object, occlusions and dis-occlusions, as well as appearance changes due to viewpoint and lighting.
Unsupervised Video Interpolation Using Cycle Consistency
We further introduce a pseudo supervised loss term that enforces the interpolated frames to be consistent with predictions of a pre-trained interpolation model.
Ranked #1 on
Video Frame Interpolation
on UCF101
(PSNR (sRGB) metric)
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Graphical Contrastive Losses for Scene Graph Parsing
The first, Entity Instance Confusion, occurs when the model confuses multiple instances of the same type of entity (e. g. multiple cups).
Practical Text Classification With Large Pre-Trained Language Models
Multi-emotion sentiment classification is a natural language processing (NLP) problem with valuable use cases on real-world data.
Ranked #3 on
Emotion Classification
on SemEval 2018 Task 1E-c
(Macro-F1 metric)
Improving Semantic Segmentation via Video Propagation and Label Relaxation
In this paper, we present a video prediction-based methodology to scale up training sets by synthesizing new training samples in order to improve the accuracy of semantic segmentation networks.
Ranked #2 on
Semantic Segmentation
on KITTI Semantic Segmentation
(using extra training data)
Partial Convolution based Padding
In this paper, we present a simple yet effective padding scheme that can be used as a drop-in module for existing convolutional neural networks.
An Interpretable Model for Scene Graph Generation
We propose an efficient and interpretable scene graph generator.
SDCNet: Video Prediction Using Spatially-Displaced Convolution
We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.
Introduction to the 1st Place Winning Model of OpenImages Relationship Detection Challenge
This article describes the model we built that achieved 1st place in the OpenImage Visual Relationship Detection Challenge on Kaggle.
WaveGlow: A Flow-based Generative Network for Speech Synthesis
In this paper we propose WaveGlow: a flow-based network capable of generating high quality speech from mel-spectrograms.
Ranked #7 on
Speech Synthesis
on LibriTTS
SDC-Net: Video prediction using spatially-displaced convolution
We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.
Ranked #1 on
Video Prediction
on YouTube-8M
Video-to-Video Synthesis
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
Large Scale Language Modeling: Converging on 40GB of Text in Four Hours
We provide a learning rate schedule that allows our model to converge with a 32k batch size.
Image Inpainting for Irregular Holes Using Partial Convolutions
Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value).
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs).
Ranked #2 on
Sketch-to-Image Translation
on COCO-Stuff
Conditional Image Generation
Fundus to Angiography Generation
+4
Malware Detection by Eating a Whole EXE
In this work we introduce malware detection from raw byte sequences as a fruitful research area to the larger machine learning community.
DSD: Dense-Sparse-Dense Training for Deep Neural Networks
We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance.
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages.
Deep Speech: Scaling up end-to-end speech recognition
We present a state-of-the-art speech recognition system developed using end-to-end deep learning.
cuDNN: Efficient Primitives for Deep Learning
To address this problem, we have created a library similar in intent to BLAS, with optimized routines for deep learning workloads.
PyCUDA and PyOpenCL: A Scripting-Based Approach to GPU Run-Time Code Generation
In introducing PyCUDA and PyOpenCL, this article proposes the combination of a dynamic, high-level scripting language with the massive performance of a GPU as a compelling two-tiered computing platform, potentially offering significant performance and productivity advantages over conventional single-tier, static systems.
Distributed, Parallel, and Cluster Computing Software Engineering D.1.2
