Search Results for author:
Found 23 papers, 15 papers with code
Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
However, existing open math datasets either contain a small collection of high-quality, human-written problems or a large corpus of machine-generated problems of uncertain quality, forcing researchers to choose between quality and quantity.
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Though
We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT.
Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
Further, we emphasize the existence of Quality-Diversity trade-offs in training data and the downstream effects on model performance.
Generative Reward Models
We introduce GenRM, an iterative algorithm that trains an LLM on self-generated reasoning traces, leading to synthetic preference labels matching human preference judgments.
Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
Furthermore, while model performance improves across all tasks as LLM size increases, only factual question answering shows an increase in memorization, whereas machine translation and reasoning tasks exhibit greater generalization, producing more novel outputs.
A Mathematical Framework, a Taxonomy of Modeling Paradigms, and a Suite of Learning Techniques for Neural-Symbolic Systems
There is a pressing need for a unifying theory to illuminate the commonalities and differences in approaches and enable further progress.
The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources
Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications.
DataComp-LM: In search of the next generation of training sets for language models
We introduce DataComp for Language Models (DCLM), a testbed for controlled dataset experiments with the goal of improving language models.
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture.
A Survey on Data Selection for Language Models
A major factor in the recent success of large language models is the use of enormous and ever-growing text datasets for unsupervised pre-training.
Efficient Online Data Mixing For Language Model Pre-Training
The data used to pretrain large language models has a decisive impact on a model's downstream performance, which has led to a large body of work on data selection methods that aim to automatically determine the most suitable data to use for pretraining.
RWKV: Reinventing RNNs for the Transformer Era
This work presents a significant step towards reconciling trade-offs between computational efficiency and model performance in sequence processing tasks.
Ranked #22 on
Natural Language Inference
on WNLI
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations.
Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data
In this work, we focus on Few-shot Learning with Auxiliary Data (FLAD), a training paradigm that assumes access to auxiliary data during few-shot learning in hopes of improving generalization.
CausalDialogue: Modeling Utterance-level Causality in Conversations
Despite their widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans.
An Exploration of Data Efficiency in Intra-Dataset Task Transfer for Dialog Understanding
Transfer learning is an exciting area of Natural Language Processing that has the potential to both improve model performance and increase data efficiency.
Data-Efficiency with a Single GPU: An Exploration of Transfer Methods for Small Language Models
Multi-task learning (MTL), instruction tuning, and prompting have recently been shown to improve the generalizability of large language models to new tasks.
Emotion Recognition in Conversation using Probabilistic Soft Logic
Creating agents that can both appropriately respond to conversations and understand complex human linguistic tendencies and social cues has been a long standing challenge in the NLP community.
NeuPSL: Neural Probabilistic Soft Logic
In this paper, we introduce Neural Probabilistic Soft Logic (NeuPSL), a novel neuro-symbolic (NeSy) framework that unites state-of-the-art symbolic reasoning with the low-level perception of deep neural networks.
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue
Task transfer, transferring knowledge contained in related tasks, holds the promise of reducing the quantity of labeled data required to fine-tune language models.
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains.
D-REX: Dialogue Relation Extraction with Explanations
Existing research studies on cross-sentence relation extraction in long-form multi-party conversations aim to improve relation extraction without considering the explainability of such methods.
Ranked #7 on
Dialog Relation Extraction
on DialogRE
Modeling Disclosive Transparency in NLP Application Descriptions
Broader disclosive transparency$-$truth and clarity in communication regarding the function of AI systems$-$is widely considered desirable.
