Search Results for author:
Found 35 papers, 28 papers with code
Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencoder
We prove that SAEs can serve as a good alternative to diversity measure and design our method to be scalable for potential industrial large-scale pruning, and we will also release our trained SAEs for use by the broader community.
Improving Model Factuality with Fine-grained Critique-based Evaluator
Factuality evaluation aims to detect factual errors produced by language models (LMs) and hence guide the development of more factual models.
M$^2$PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning
Instruction tuning has emerged as an effective strategy for achieving zero-shot generalization by finetuning pretrained models on diverse multimodal tasks.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Question Answering
on PeerQA
Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts
Rainbow Teaming casts adversarial prompt generation as a quality-diversity problem and uses open-ended search to generate prompts that are both effective and diverse.
RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training
Under a unified evaluation of fine-tuned LMs by incorporating four representative perspectives of model robustness, we demonstrate the effectiveness of RoAST compared to state-of-the-art fine-tuning methods on six different types of LMs, which indicates its usefulness in practice.
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
We introduce Llama Guard, an LLM-based input-output safeguard model geared towards Human-AI conversation use cases.
MART: Improving LLM Safety with Multi-round Automatic Red-Teaming
Specifically, an adversarial LLM and a target LLM interplay with each other in an iterative manner, where the adversarial LLM aims to generate challenging prompts that elicit unsafe responses from the target LLM, while the target LLM is fine-tuned with safety aligned data on these adversarial prompts.
Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
Ranked #2 on
Question Answering
on PubChemQA
LIMA: Less Is More for Alignment
Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and reinforcement learning, to better align to end tasks and user preferences.
Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization
In this work, we introduce Residual Prompt Tuning - a simple and efficient method that significantly improves the performance and stability of prompt tuning.
Representation Deficiency in Masked Language Modeling
In this work, we offer a new perspective on the consequence of such a discrepancy: We demonstrate empirically and theoretically that MLM pretraining allocates some model dimensions exclusively for representing $\texttt{[MASK]}$ tokens, resulting in a representation deficiency for real tokens and limiting the pretrained model's expressiveness when it is adapted to downstream data without $\texttt{[MASK]}$ tokens.
Progressive Prompts: Continual Learning for Language Models
We introduce Progressive Prompts - a simple and efficient approach for continual learning in language models.
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models
Large multilingual language models typically rely on a single vocabulary shared across 100+ languages.
Towards a Unified Multi-Dimensional Evaluator for Text Generation
We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions.
CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision
Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers.
Ranked #1 on
Extreme Summarization
on CiteSum
Unsupervised Multi-Granularity Summarization
In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum.
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction
Consequently, it is more challenging to encode the key information sources--relevant contexts and entity types.
Ranked #1 on
Relation Extraction
on CDR
Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion
Typical DocRE methods blindly take the full document as input, while a subset of the sentences in the document, noted as the evidence, are often sufficient for humans to predict the relation of an entity pair.
Ranked #5 on
Relation Extraction
on DocRED
Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation
In this paper, we present a systematic analysis that studies whether current seq2seq models, especially pre-trained language models, are good enough for preserving important input concepts and to what extent explicitly guiding generation with the concepts as lexical constraints is beneficial.
Taxonomy Completion via Triplet Matching Network
Previous approaches focus on the taxonomy expansion, i. e. finding an appropriate hypernym concept from the taxonomy for a new query concept.
Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering
Current open-domain question answering systems often follow a Retriever-Reader architecture, where the retriever first retrieves relevant passages and the reader then reads the retrieved passages to form an answer.
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation
Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document.
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning
Additionally, the explicit redundancy measure in MMR helps the neural representation of the summary to better capture redundancy.
Generation-Augmented Retrieval for Open-domain Question Answering
We demonstrate that the generated contexts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better performance than state-of-the-art dense retrieval methods such as DPR.
AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types
Can one build a knowledge graph (KG) for all products in the world?
Octet: Online Catalog Taxonomy Enrichment with Self-Supervision
We propose to distantly train a sequence labeling model for term extraction and employ graph neural networks (GNNs) to capture the taxonomy structure as well as the query-item-taxonomy interactions for term attachment.
Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning
Walk-based models have shown their advantages in knowledge graph (KG) reasoning by achieving decent performance while providing interpretable decisions.
Generating Representative Headlines for News Stories
In this work, we study the problem of generating representative headlines for news stories.
Hierarchical Text Classification with Reinforced Label Assignment
While existing hierarchical text classification (HTC) methods attempt to capture label hierarchies for model training, they either make local decisions regarding each label or completely ignore the hierarchy information during inference.
Ranked #1 on
Text Classification
on RCV1
(Macro F1 metric)
Facet-Aware Evaluation for Extractive Summarization
In this paper, we present a facet-aware evaluation setup for better assessment of the information coverage in extracted summaries.
End-to-End Hierarchical Text Classification with Label Assignment Policy
We present an end-to-end reinforcement learning approach to hierarchical text classification where documents are labeled by placing them at the right positions in a given hierarchy.
End-to-End Reinforcement Learning for Automatic Taxonomy Induction
We present a novel end-to-end reinforcement learning approach to automatic taxonomy induction from a set of terms.
