Search Results for author:
Found 23 papers, 16 papers with code
Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
We introduce Nemotron-H, a family of 8B and 56B/47B hybrid Mamba-Transformer models designed to reduce inference cost for a given accuracy level.
IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark for LLMs
The benchmark also consists of a curated mixture of different mention types and corresponding entities, allowing for a fine-grained analysis of model performance.
OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
However, most of the cutting-edge progress in mathematical reasoning with LLMs has become \emph{closed-source} due to lack of access to training data.
Ranked #5 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Major Entity Identification: A Generalizable Alternative to Coreference Resolution
Rather than relying on this additional annotation, we propose an alternative referential task, Major Entity Identification (MEI), where we: (a) assume the target entities to be specified in the input, and (b) limit the task to only the frequent entities.
Nemotron-4 340B Technical Report
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward.
Code Pretraining Improves Entity Tracking Abilities of Language Models
Recent work has provided indirect evidence that pretraining language models on code improves the ability of models to track state changes of discourse entities expressed in natural language.
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
Building on the recent progress in open-source LLMs, our proposed prompting novelty, and some brute-force scaling, we construct OpenMathInstruct-1, a math instruction tuning dataset with 1. 8M problem-solution pairs.
Ranked #1 on
Math Word Problem Solving
on MAWPS
(using extra training data)
Adapting Pretrained Text-to-Text Models for Long Text Sequences
We present an empirical study of adapting an existing pretrained text-to-text model for long-sequence inputs.
Ranked #1 on
Text Summarization
on QMSum
Efficient and Interpretable Neural Models for Entity Tracking
We also argue for the integration of entity tracking into language models as it will allow for: (i) wider application given the current ubiquitous use of pretrained language models in NLP applications, and (ii) easier adoption since it is much easier to swap in a new pretrained language model than to integrate a separate standalone entity tracking model.
Ranked #7 on
Coreference Resolution
on OntoNotes
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
On Generalization in Coreference Resolution
While coreference resolution is defined independently of dataset domain, most models for performing coreference resolution do not transfer well to unseen domains.
Ranked #1 on
Coreference Resolution
on Quizbowl
Chess as a Testbed for Language Model State Tracking
Motivated by this issue, we consider the task of language modeling for the game of chess.
Learning Chess Blindfolded
Motivated by this issue, we consider the task of language modeling for the game of chess.
Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks
Long document coreference resolution remains a challenging task due to the large memory and runtime requirements of current models.
Ranked #10 on
Coreference Resolution
on CoNLL 2012
A Cross-Task Analysis of Text Span Representations
Many natural language processing (NLP) tasks involve reasoning with textual spans, including question answering, entity recognition, and coreference resolution.
PeTra: A Sparsely Supervised Memory Model for People Tracking
We propose PeTra, a memory-augmented neural network designed to track entities in its memory slots.
Ranked #1 on
Coreference Resolution
on GAP
(F1 metric)
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition
In this paper, we compare a suite of past methods and some of our own proposed methods for using unpaired text data to improve encoder-decoder models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Hierarchical Multitask Learning for CTC-based Speech Recognition
Previous work has shown that neural encoder-decoder speech recognition can be improved with hierarchical multitask learning, where auxiliary tasks are added at intermediate layers of a deep encoder.
Multilingual Speech Recognition With A Single End-To-End Model
Training a conventional automatic speech recognition (ASR) system to support multiple languages is challenging because the sub-word unit, lexicon and word inventories are typically language specific.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Parsing Speech: A Neural Approach to Integrating Lexical and Acoustic-Prosodic Information
In conversational speech, the acoustic signal provides cues that help listeners disambiguate difficult parses.
Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognition
We hypothesize that using intermediate representations as auxiliary supervision at lower levels of deep networks may be a good way of combining the advantages of end-to-end training and more traditional pipeline approaches.
Jointly Learning to Align and Convert Graphemes to Phonemes with Neural Attention Models
We propose an attention-enabled encoder-decoder model for the problem of grapheme-to-phoneme conversion.
