Search Results for author:
Found 36 papers, 13 papers with code
Data, Data Everywhere: A Guide for Pretraining Dataset Construction
The impressive capabilities of recent language models can be largely attributed to the multi-trillion token pretraining datasets that they are trained on.
Nemotron-4 340B Technical Report
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward.
AgentKit: Structured LLM Reasoning with Dynamic Graphs
The chains of nodes can be designed to explicitly enforce a naturally structured "thought process".
Nemotron-4 15B Technical Report
We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens.
SPRING: Studying the Paper and Reasoning to Play Games
Finally, we show the potential of games as a test bed for LLMs.
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
We propose the Plan, Eliminate, and Track (PET) framework.
Self-Refine: Iterative Refinement with Self-Feedback
Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement.
BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models
We thus enable seamless open-ended social bias testing of PLMs by domain experts through an automatic large-scale generation of diverse test sentences for any combination of social categories and attributes.
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models
Pretrained large language models have become indispensable for solving various natural language processing (NLP) tasks.
Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions
Finally, we find that not all transfer scenarios yield a positive gain, which seems related to the PLMs initial performance on the target-domain task.
Evaluating Parameter Efficient Learning for Generation
For cross-domain and cross-dataset cases, we show that (a) Adapter (Houlsby et al., 2019) performs the best amongst all the PERMs studied here, and (b) it outperforms finetuning if the task dataset is below a certain size.
Context Generation Improves Open Domain Question Answering
Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge.
Multi-Stage Prompting for Knowledgeable Dialogue Generation
We propose a multi-stage prompting approach to generate knowledgeable responses from a single pretrained LM.
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Next, we detail the training process, the design of our training corpus, and our data curation techniques, which we believe is a key ingredient to the success of the model.
Ranked #35 on
Language Modelling
on LAMBADA
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
We then provide the LM with instruction that consists of this subset of labeled exemplars, the query text to be classified, a definition of bias, and prompt it to make a decision.
Focused Attention Improves Document-Grounded Generation
Document grounded generation is the task of using the information provided in a document to improve text generation.
CURIE: An Iterative Querying Approach for Reasoning About Situations
Recently, models have been shown to predict the effects of unexpected situations, e. g., would cloudy skies help or hinder plant growth?
EIGEN: Event Influence GENeration using Pre-trained Language Models
Reasoning about events and tracking their influences is fundamental to understanding processes.
Case Study: Deontological Ethics in NLP
Recent work in natural language processing (NLP) has focused on ethical challenges such as understanding and mitigating bias in data and algorithms; identifying objectionable content like hate speech, stereotypes and offensive language; and building frameworks for better system design and data handling practices.
Exploring Controllable Text Generation Techniques
In this work, we provide a new schema of the pipeline of the generation process by classifying it into five modules.
Topological Sort for Sentence Ordering
Sentence ordering is the task of arranging the sentences of a given text in the correct order.
Politeness Transfer: A Tag and Generate Approach
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.
I love your chain mail! Making knights smile in a fantasy game world: Open-domain goal-oriented dialogue agents
Dialogue research tends to distinguish between chit-chat and goal-oriented tasks.
Modeling Product Search Relevance in e-Commerce
With the rapid growth of e-Commerce, online product search has emerged as a popular and effective paradigm for customers to find desired products and engage in online shopping.
Generating Interactive Worlds with Text
We show that the game environments created with our approach are cohesive, diverse, and preferred by human evaluators compared to other machine learning based world construction algorithms.
Equity Beyond Bias in Language Technologies for Education
There is a long record of research on equity in schools.
``My Way of Telling a Story'': Persona based Grounded Story Generation
To this end, we propose five models which are incremental extensions to the baseline model to perform the task at hand.
Principled Frameworks for Evaluating Ethics in NLP Systems
We critique recent work on ethics in natural language processing.
"My Way of Telling a Story": Persona based Grounded Story Generation
To this end, we propose five models which are incremental extensions to the baseline model to perform the task at hand.
Towards Content Transfer through Grounded Text Generation
Recent work in neural generation has attracted significant interest in controlling the form of text, such as style, persona, and politeness.
The Second Conversational Intelligence Challenge (ConvAI2)
We describe the setting and results of the ConvAI2 NeurIPS competition that aims to further the state-of-the-art in open-domain chatbots.
A Dataset for Document Grounded Conversations
We define "Document Grounded Conversations" as conversations that are about the contents of a specified document.
Style Transfer Through Back-Translation
We first learn a latent representation of the input sentence which is grounded in a language translation model in order to better preserve the meaning of the sentence while reducing stylistic properties.
Ranked #10 on
Unsupervised Text Style Transfer
on Yelp
Linguistic Markers of Influence in Informal Interactions
There has been a long standing interest in understanding `Social Influence' both in Social Sciences and in Computational Linguistics.
