Search Results for author:
Found 133 papers, 58 papers with code
Contextualized Role Interaction for Neural Machine Translation
Word inputs tend to be represented as single continuous vectors in deep neural networks.
Talk The Walk: Navigating Grids in New York City through Grounded Dialogue
We introduce `"Talk The Walk", the first large-scale dialogue dataset grounded in action and perception.
LLM Pretraining with Continuous Concepts
We propose Continuous Concept Mixing (CoCoMix), a novel pretraining framework that combines discrete next token prediction with continuous concepts.
R.I.P.: Better Models by Survival of the Fittest Prompts
Using Llama 3. 1-8B-Instruct, RIP improves AlpacaEval2 LC Win Rate by 9. 4%, Arena-Hard by 8. 7%, and WildBench by 9. 9%.
Diverse Preference Optimization
In this work we introduce Diverse Preference Optimization (DivPO), an optimization method which learns to generate much more diverse responses than standard pipelines, while maintaining the quality of the generations.
Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
LLM-as-a-Judge models generate chain-of-thought (CoT) sequences intended to capture the step-bystep reasoning process that underlies the final evaluation of a response.
Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
Large language models (LLMs) have recently demonstrated remarkable success in mathematical reasoning.
Byte Latent Transformer: Patches Scale Better Than Tokens
We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency and robustness.
Training Large Language Models to Reason in a Continuous Latent Space
For example, most word tokens are primarily for textual coherence and not essential for reasoning, while some critical tokens require complex planning and pose huge challenges to LLMs.
ALMA: Alignment with Minimal Annotation
Recent approaches to large language model (LLM) alignment typically require millions of human annotations or rely on external aligned models for synthetic data generation.
Adaptive Decoding via Latent Preference Optimization
During language model decoding, it is known that using higher temperature sampling gives more creative responses, while lower temperatures are more factually accurate.
Self-Consistency Preference Optimization
Self-alignment, whereby models learn to improve themselves without human annotation, is a rapidly growing research area.
Thinking LLMs: General Instruction Following with Thought Generation
LLMs are typically trained to answer user questions or follow instructions similarly to how human experts respond.
Backtracking Improves Generation Safety
In the context of language model safety, when a partial unsafe generation is produced, language models by their nature tend to happily keep on generating similarly unsafe additional text.
Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
Our method improves performance by 25. 51% for TQA on WikiSQL and 22. 57% for MHQA on HotPotQA compared to the fine-tuned baselines.
Better Alignment with Instruction Back-and-Forth Translation
We propose a new method, instruction back-and-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs).
Self-Taught Evaluators
Model-based evaluation is at the heart of successful model development -- as a reward model for training, and as a replacement for human evaluation.
Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
Large Language Models (LLMs) are rapidly surpassing human knowledge in many domains.
Distilling System 2 into System 1
Large language models (LLMs) can spend extra compute during inference to generate intermediate thoughts, which helps to produce better final responses.
Following Length Constraints in Instructions
Aligned instruction following models can better fulfill user requests than their unaligned counterparts.
Contextual Position Encoding: Learning to Count What's Important
Incorporating position encoding (PE) makes it possible to address by position, such as attending to the i-th token.
Reverse Training to Nurse the Reversal Curse
Large language models (LLMs) have a surprising failure: when trained on "A has a feature B", they do not generalize to "B is a feature of A", which is termed the Reversal Curse.
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.
Ranked #32 on
Multi-task Language Understanding
on MMLU
TOOLVERIFIER: Generalization to New Tools via Self-Verification
Teaching language models to use tools is an important milestone towards building general assistants, but remains an open problem.
Self-Rewarding Language Models
We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal.
Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss
Practitioners commonly align large language models using pairwise preferences, i. e., given labels of the type response A is preferred to response B for a given input.
System 2 Attention (is something you might need too)
Soft attention in Transformer-based Large Language Models (LLMs) is susceptible to incorporating irrelevant information from the context into its latent representations, which adversely affects next token generations.
The ART of LLM Refinement: Ask, Refine, and Trust
In recent years, Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations?
Ranked #55 on
Arithmetic Reasoning
on GSM8K
Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
Large Language Models (LLMs) are frequently used for multi-faceted language generation and evaluation tasks that involve satisfying intricate user constraints or taking into account multiple aspects and criteria.
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
Large language models (LLMs) have advanced in large strides due to the effectiveness of the self-attention mechanism that processes and compares all tokens at once.
Chain-of-Verification Reduces Hallucination in Large Language Models
Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models.
Self-Alignment with Instruction Backtranslation
We present a scalable method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions.
Leveraging Implicit Feedback from Deployment Data in Dialogue
We study improving social conversational agents by learning from natural dialogue between users and a deployed model, without extra annotations.
System-Level Natural Language Feedback
Natural language (NL) feedback offers rich insights into user experience.
Improving Open Language Models by Learning from Organic Interactions
We present BlenderBot 3x, an update on the conversational model BlenderBot 3, which is now trained using organic conversation and feedback data from participating users of the system in order to improve both its skills and safety.
The HCI Aspects of Public Deployment of Research Chatbots: A User Study, Design Recommendations, and Open Challenges
Publicly deploying research chatbots is a nuanced topic involving necessary risk-benefit analyses.
Large Language Model Programs
In recent years, large pre-trained language models (LLMs) have demonstrated the ability to follow instructions and perform novel tasks from a few examples.
Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models
We compare models trained on our new dataset to existing pairwise-trained dialogue models, as well as large language models with few-shot prompting.
The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation
State-of-the-art language generation models can degenerate when applied to open-ended generation problems such as text completion, story generation, or dialog modeling.
Infusing Commonsense World Models with Graph Knowledge
While language models have become more capable of producing compelling language, we find there are still gaps in maintaining consistency, especially when describing events in a dynamically changing world.
The CRINGE Loss: Learning what language not to model
Standard language model training employs gold human documents or human-human interaction data, and treats all training data as positive examples.
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels
Deployed dialogue agents have the potential to integrate human feedback to continuously improve themselves.
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback
We then study various algorithms for improving from such feedback, including standard supervised learning, rejection sampling, model-guiding and reward-based learning, in order to make recommendations on which type of feedback and algorithms work best.
Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls
The promise of interaction between intelligent conversational agents and humans is that models can learn from such feedback in order to improve.
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large number of user defined tasks.
DIRECTOR: Generator-Classifiers For Supervised Language Modeling
Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions.
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion
We show that, when using SeeKeR as a dialogue model, it outperforms the state-of-the-art model BlenderBot 2 (Chen et al., 2021) on open-domain knowledge-grounded conversations for the same number of parameters, in terms of consistency, knowledge and per-turn engagingness.
Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents
At the heart of improving conversational AI is the open problem of how to evaluate conversations.
Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity
State-of-the-art dialogue models still often stumble with regards to factual accuracy and self-contradiction.
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue
Large language models can produce fluent dialogue but often hallucinate factual inaccuracies.
NormFormer: Improved Transformer Pretraining with Extra Normalization
The extra operations incur negligible compute cost (+0. 4% parameter increase), but improve pretraining perplexity and downstream task performance for both causal and masked language models ranging from 125 Million to 2. 7 Billion parameters.
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Despite recent improvements in open-domain dialogue models, state of the art models are trained and evaluated on short conversations with little context.
Internet-Augmented Dialogue Generation
The largest store of continually updating knowledge on our planet can be accessed via internet search.
Hash Layers For Large Sparse Models
We investigate the training of sparse layers that use different parameters for different inputs based on hashing in large Transformer models.
Staircase Attention for Recurrent Processing of Sequences
Attention mechanisms have become a standard tool for sequence modeling tasks, in particular by stacking self-attention layers over the entire input sequence as in the Transformer architecture.
Bot-Adversarial Dialogue for Safe Conversational Agents
Conversational agents trained on large unlabeled corpora of human interactions will learn patterns and mimic behaviors therein, which include offensive or otherwise toxic behavior.
Not All Memories are Created Equal: Learning to Forget by Expiring
We demonstrate that Expire-Span can help models identify and retain critical information and show it can achieve strong performance on reinforcement learning tasks specifically designed to challenge this functionality.
Ranked #4 on
Language Modelling
on enwik8
Retrieval Augmentation Reduces Hallucination in Conversation
Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2020).
I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling
To quantify how well natural language understanding models can capture consistency in a general conversation, we introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues.
Recipes for Safety in Open-domain Chatbots
Models trained on large unlabeled corpora of human interactions will learn patterns and mimic behaviors therein, which include offensive or otherwise toxic behavior and unwanted biases.
Multi-Modal Open-Domain Dialogue
Recent work in open-domain conversational agents has demonstrated that significant improvements in model engagingness and humanness metrics can be achieved via massive scaling in both pre-training data and model size (Adiwardana et al., 2020; Roller et al., 2020).
Ranked #1 on
Visual Dialog
on Wizard of Wikipedia
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
We seek to create agents that both act and communicate with other agents in pursuit of a goal.
Deploying Lifelong Open-Domain Dialogue Learning
As argued in de Vries et al. (2020), crowdsourced data has the issues of lack of naturalness and relevance to real-world use cases, while the static dataset paradigm does not allow for a model to learn from its experiences of using language (Silver et al., 2013).
Image-Chat: Engaging Grounded Conversations
To test such models, we collect a dataset of grounded human-human conversations, where speakers are asked to play roles given a provided emotional mood or style, as the use of such traits is also a key factor in engagingness (Guo et al., 2019).
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
We present our view of what is necessary to build an engaging open-domain conversational agent: covering the qualities of such an agent, the pieces of the puzzle that have been built so far, and the gaping holes we have not filled yet.
Multi-Dimensional Gender Bias Classification
We show our classifiers prove valuable for a variety of important applications, such as controlling for gender bias in generative models, detecting gender bias in arbitrary text, and shed light on offensive language in terms of genderedness.
Recipes for building an open-domain chatbot
Building open-domain chatbots is a challenging area for machine learning research.
Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skills
Being engaging, knowledgeable, and empathetic are all desirable general qualities in a conversational agent.
I love your chain mail! Making knights smile in a fantasy game world: Open-domain goal-oriented dialogue agents
Dialogue research tends to distinguish between chit-chat and goal-oriented tasks.
Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
The use of deep pre-trained transformers has led to remarkable progress in a number of applications (Devlin et al., 2018).
All-in-One Image-Grounded Conversational Agents
As single-task accuracy on individual language and image tasks has improved substantially in the last few years, the long-term goal of a generally skilled agent that can both see and talk becomes more feasible to explore.
Improving Conditioning in Context-Aware Sequence to Sequence Models
Neural sequence to sequence models are well established for applications which can be cast as mapping a single input sequence into a single output sequence.
Ranked #6 on
Open-Domain Question Answering
on ELI5
Generating Interactive Worlds with Text
We show that the game environments created with our approach are cohesive, diverse, and preferred by human evaluators compared to other machine learning based world construction algorithms.
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
Models often easily learn biases present in the training data, and their predictions directly reflect this bias.
Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training
Generative dialogue models currently suffer from a number of problems which standard maximum likelihood training does not address.
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents
We introduce dodecaDialogue: a set of 12 tasks that measures if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, discuss topics and situations, and perceive and converse about images.
Finding Generalizable Evidence by Learning to Convince Q\&A Models
We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed.
Adversarial NLI: A New Benchmark for Natural Language Understanding
We introduce a new large-scale NLI benchmark dataset, collected via an iterative, adversarial human-and-model-in-the-loop procedure.
Finding Generalizable Evidence by Learning to Convince Q&A Models
We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed.
Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue
These issues can be alleviated by treating recommendation as an interactive dialogue task instead, where an expert recommender can sequentially ask about someone's preferences, react to their requests, and recommend more appropriate items.
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
While dialogue remains an important end-goal of natural language research, the difficulty of evaluation is an oft-quoted reason why it remains troublesome to make real progress towards its solution.
Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack
The detection of offensive language in the context of a dialogue has become an increasingly important application of natural language processing.
Neural Text Generation with Unlikelihood Training
Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core.
ELI5: Long Form Question Answering
We introduce the first large-scale corpus for long-form question answering, a task requiring elaborate and in-depth answers to open-ended questions.
Why Build an Assistant in Minecraft?
In this document we describe a rationale for a research program aimed at building an open "assistant" in the game Minecraft, in order to make progress on the problems of natural language understanding and learning from dialogue.
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
The use of deep pre-trained bidirectional transformers has led to remarkable progress in a number of applications (Devlin et al., 2018).
Learning to Speak and Act in a Fantasy Text Adventure Game
We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.
What makes a good conversation? How controllable attributes affect human judgments
A good conversation requires balance -- between simplicity and detail; staying on topic and changing it; asking questions and answering them.
The Second Conversational Intelligence Challenge (ConvAI2)
We describe the setting and results of the ConvAI2 NeurIPS competition that aims to further the state-of-the-art in open-domain chatbots.
Learning from Dialogue after Deployment: Feed Yourself, Chatbot!
As our agent engages in conversation, it also estimates user satisfaction in its responses.
Wizard of Wikipedia: Knowledge-Powered Conversational agents
In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date.
Importance of Search and Evaluation Strategies in Neural Dialogue Modeling
We investigate the impact of search strategies in neural dialogue modeling.
Image Chat: Engaging Grounded Conversations
To test such models, we collect a dataset of grounded human-human conversations, where speakers are asked to play roles given a provided emotional mood or style, as the use of such traits is also a key factor in engagingness (Guo et al., 2019).
Ranked #2 on
Text Retrieval
on Image-Chat
Engaging Image Captioning Via Personality
While such tasks are useful to verify that a machine understands the content of an image, they are not engaging to humans as captions.
Jump to better conclusions: SCAN both left and right
Lake and Baroni (2018) recently introduced the SCAN data set, which consists of simple commands paired with action sequences and is intended to test the strong generalization abilities of recurrent sequence-to-sequence models.
Retrieve and Refine: Improved Sequence Generation Models For Dialogue
Sequence generation models for dialogue are known to have several problems: they tend to produce short, generic sentences that are uninformative and unengaging.
Talk the Walk: Navigating New York City through Grounded Dialogue
We introduce "Talk The Walk", the first large-scale dialogue dataset grounded in action and perception.
Vehicle Communication Strategies for Simulated Highway Driving
Interest in emergent communication has recently surged in Machine Learning.
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Chit-chat models are known to have several problems: they lack specificity, do not display a consistent personality and are often not very captivating.
Ranked #5 on
Dialogue Generation
on Persona-Chat
(using extra training data)
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent
Contrary to most natural language processing research, which makes use of static datasets, humans learn language interactively, grounded in an environment.
Emergent Translation in Multi-Agent Communication
While most machine translation systems to date are trained on large parallel corpora, humans learn language in a different way: by being grounded in an environment and interacting with other humans.
StarSpace: Embed All The Things!
A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
ParlAI: A Dialog Research Software Platform
We introduce ParlAI (pronounced "par-lay"), an open-source software platform for dialog research implemented in Python, available at http://parl. ai.
Reading Wikipedia to Answer Open-Domain Questions
This paper proposes to tackle open- domain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia article.
Ranked #1 on
Open-Domain Question Answering
on SQuAD1.1
Learning through Dialogue Interactions by Asking Questions
A good dialogue agent should have the ability to interact with users by both responding to questions and by asking questions, and importantly to learn from both types of interaction.
Tracking the World State with Recurrent Entity Networks
The EntNet sets a new state-of-the-art on the bAbI tasks, and is the first method to solve all the tasks in the 10k training examples setting.
Ranked #5 on
Procedural Text Understanding
on ProPara
Dialogue Learning With Human-In-The-Loop
An important aspect of developing conversational agents is to give a bot the ability to improve through communicating with humans and to learn from the mistakes that it makes.
Key-Value Memory Networks for Directly Reading Documents
Directly reading documents and being able to answer questions from them is an unsolved challenge.
Ranked #12 on
Question Answering
on WikiQA
Learning End-to-End Goal-Oriented Dialog
We show similar result patterns on data extracted from an online concierge service.
Dialog-based Language Learning
A long-term goal of machine learning research is to build an intelligent dialog agent.
Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
A long-term goal of machine learning is to build intelligent conversational agents.
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
We introduce a new test of how well language models capture meaning in children's books.
A Neural Attention Model for Abstractive Sentence Summarization
Summarization based on text extraction is inherently limited, but generation-style abstractive methods have proven challenging to build.
Ranked #1 on
Extractive Text Summarization
on DUC 2004 Task 1
Large-scale Simple Question Answering with Memory Networks
Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions.
Ranked #1 on
Question Answering
on WebQuestions
(F1 metric)
End-To-End Memory Networks
For the former our approach is competitive with Memory Networks, but with less supervision.
Ranked #6 on
Question Answering
on bAbi
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.
Question Answering with Subgraph Embeddings
Training our system using pairs of questions and structured representations of their answers, and pairs of question paraphrases, yields competitive results on a competitive benchmark of the literature.
Ranked #2 on
Question Answering
on WebQuestions
(F1 metric)
Open Question Answering with Weakly Supervised Embedding Models
Building computers able to answer questions on any subject is a long standing goal of artificial intelligence.
Ranked #1 on
Question Answering
on Reverb
Translating Embeddings for Modeling Multi-relational Data
We consider the problem of embedding entities and relationships of multi-relational data in low-dimensional vector spaces.
Ranked #5 on
Link Prediction
on FB122
Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction
This paper proposes a novel approach for relation extraction from free text which is trained to jointly use information from the text and from existing knowledge.
Irreflexive and Hierarchical Relations as Translations
We consider the problem of embedding entities and relations of knowledge bases in low-dimensional vector spaces.
A Semantic Matching Energy Function for Learning with Multi-relational Data
Large-scale relational learning becomes crucial for handling the huge amounts of structured data generated daily in many application domains ranging from computational biology or information retrieval, to natural language processing.
Large-Scale Music Annotation and Retrieval: Learning to Rank in Joint Semantic Spaces
Music prediction tasks range from predicting tags given a song or clip of audio, predicting the name of the artist, or predicting related songs given a song, clip, artist name or tag.
Natural Language Processing (almost) from Scratch
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling.
Label Embedding Trees for Large Multi-Class Tasks
Multi-class classification becomes challenging at test time when the number of classes is very large and testing against every possible class can become computationally infeasible.
Polynomial Semantic Indexing
We present a class of nonlinear (polynomial) models that are discriminatively trained to directly map from the word content in a query-document or document-document pair to a ranking score.
