Search Results for author:
Found 20 papers, 16 papers with code
Semiparametric Token-Sequence Co-Supervision
Especially, the robustness of parametric token space which is established during the pretraining step tends to effectively enhance the stability of nonparametric sequence embedding space, a new space established by another language model.
LangBridge: Multilingual Reasoning Without Multilingual Supervision
We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision.
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Since the release of T\"ULU [Wang et al., 2023b], open resources for instruction tuning have developed quickly, from better base models to new finetuning techniques.
How Well Do Large Language Models Truly Ground?
Reliance on the inherent knowledge of Large Language Models (LLMs) can cause issues such as hallucinations, lack of control, and difficulties in integrating variable knowledge.
Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging
In this work, we study Reinforcement Learning from Personalized Human Feedback (RLPHF) problem, wherein LLMs are aligned to multiple (sometimes conflicting) preferences by modeling alignment as a Multi-Objective Reinforcement Learning (MORL) problem.
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.
Gradient Ascent Post-training Enhances Language Model Generalization
In this work, we empirically show that updating pretrained LMs (350M, 1. 3B, 2. 7B) with just a few steps of Gradient Ascent Post-training (GAP) on random, unlabeled text corpora enhances its zero-shot generalization capabilities across diverse NLP tasks.
Exploring the Practicality of Generative Retrieval on Dynamic Corpora
Benchmarking the performance of information retrieval (IR) methods are mostly conducted with a fixed set of documents (static corpora); in realistic scenarios, this is rarely the case and the document to be retrieved are constantly updated and added.
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis
Previous works in prompt engineering for large language models have introduced different gradient-free probability-based prompt selection methods that aim to choose the optimal prompt among the candidates for a given task but have failed to provide a comprehensive and fair comparison between each other.
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
Furthermore, we show that instruction tuning with CoT Collection allows LMs to possess stronger few-shot learning capabilities on 4 domain-specific tasks, resulting in an improvement of +2. 24% (Flan-T5 3B) and +2. 37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations until the max length by a +13. 98% margin.
Ranked #1 on
on BIG-bench (SNARKS)
Common Sense Reasoning
Common Sense Reasoning (Zero-Shot)
+7
Exploring the Benefits of Training Expert Language Models over Instruction Tuning
Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known as multitask-prompted fine-tuning (MT), have shown the capability to generalize to unseen tasks.
Ranked #9 on
Question Answering
on StoryCloze
Efficiently Enhancing Zero-Shot Performance of Instruction Following Model via Retrieval of Soft Prompt
Enhancing the zero-shot performance of instruction-following models requires heavy computation, either by scaling the total number of training datasets or the model size.
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
Meta-training, which fine-tunes the language model (LM) on various downstream tasks by maximizing the likelihood of the target label given the task instruction and input instance, has improved the zero-shot task generalization performance.
Ranked #2 on
Question Answering
on StoryCloze
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities.
Ranked #3 on
Language Modelling
on The Pile
(Test perplexity metric)
Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts
Previous work has shown that there exists a scaling law between the size of Language Models (LMs) and their zero-shot performance on different downstream NLP tasks.
Prompt Injection: Parameterization of Fixed Inputs
Through these explorations, we show that PI can be a promising direction for conditioning language models, especially in scenarios with long and fixed prompts.
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models
Language Models (LMs) become outdated as the world changes; they often fail to perform tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment.
Towards Continual Knowledge Learning of Language Models
By highlighting the critical causes of knowledge forgetting, we show that CKL is a challenging and important problem that helps us better understand and train ever-changing LMs.
Learning to Balance with Incremental Learning
Classification tasks require balanced distribution of data in order to ensure the learner to be trained to generalize over all classes.
Sequential Targeting: an incremental learning approach for data imbalance in text classification
Classification tasks require a balanced distribution of data to ensure the learner to be trained to generalize over all classes.
