Search Results for author:
Found 31 papers, 21 papers with code
MMTEB: Massive Multilingual Text Embedding Benchmark
MMTEB includes a diverse set of challenging, novel tasks such as instruction following, long-document retrieval, and code retrieval, representing the largest multilingual collection of evaluation tasks for embedding models to date.
Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving
On the miniF2F benchmark, it achieves a 57. 6% success rate (Pass@32), exceeding the previous best open-source model by 7. 6%.
Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
We confirm that the MoA performance is rather sensitive to the quality, and mixing different LLMs often lowers the average quality of the models.
ICONS: Influence Consensus for Vision-Language Data Selection
In this work, we introduce ICONS, a gradient-driven Influence CONsensus approach for vision-language data Selection that selects a compact training dataset for efficient multi-task training.
BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
To better benchmark retrieval on such challenging queries, we introduce BRIGHT, the first text retrieval benchmark that requires intensive reasoning to retrieve relevant documents.
LitSearch: A Retrieval Benchmark for Scientific Literature Search
LitSearch is constructed using a combination of (1) questions generated by GPT-4 based on paragraphs containing inline citations from research papers and (2) questions manually written by authors about their recently published papers.
CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
All models lag far behind human performance of 80. 5%, underscoring weaknesses in the chart understanding capabilities of existing MLLMs.
SimPO: Simple Preference Optimization with a Reference-Free Reward
Direct Preference Optimization (DPO) is a widely used offline preference optimization algorithm that reparameterizes reward functions in reinforcement learning from human feedback (RLHF) to enhance simplicity and training stability.
Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
Mixture-of-experts (MoE) models facilitate efficient scaling; however, training the router network introduces the challenge of optimizing a non-differentiable, discrete objective.
What is in Your Safe Data? Identifying Benign Data that Breaks Safety
Training on just 100 of these seemingly benign datapoints surprisingly leads to the fine-tuned model affirmatively responding to >70% of tested harmful requests, compared to <20% after fine-tuning on randomly selected data.
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
We develop methods to identify critical regions that are vital for safety guardrails, and that are disentangled from utility-relevant regions at both the neuron and rank levels.
LESS: Selecting Influential Data for Targeted Instruction Tuning
Instruction tuning has unlocked powerful capabilities in large language models (LLMs), effectively using combined datasets to develop generalpurpose chatbots.
Detecting Pretraining Data from Large Language Models
Min-K% Prob can be applied without any knowledge about the pretraining corpus or any additional training, departing from previous detection methods that require training a reference model on data that is similar to the pretraining data.
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
In this work, we study structured pruning as an effective means to develop smaller LLMs from pre-trained, larger models.
Ranked #45 on
Question Answering
on PIQA
Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation
Finally, we propose an effective alignment method that explores diverse generation strategies, which can reasonably reduce the misalignment rate under our attack.
Trainable Transformer in Transformer
In this work, we propose an efficient construction, Transformer in Transformer (in short, TinT), that allows a transformer to simulate and fine-tune complex models internally during inference (e. g., pre-trained language models).
InstructEval: Systematic Evaluation of Instruction Selection Methods
In-context learning (ICL) performs tasks by prompting a large language model (LLM) using an instruction and a small set of annotated examples called demonstrations.
Training Trajectories of Language Models Across Scales
Why do larger language models demonstrate more desirable behaviors?
Don't Prompt, Search! Mining-based Zero-Shot Learning with Language Models
Our results suggest that the success of prompting can partly be explained by the model being exposed to similar examples during pretraining, which can be directly retrieved through regular expressions.
MABEL: Attenuating Gender Bias using Textual Entailment Data
To this end, we propose MABEL (a Method for Attenuating Gender Bias using Entailment Labels), an intermediate pre-training approach for mitigating gender bias in contextualized representations.
Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models
In this work, we adapt prompt-based few-shot learning to ELECTRA and show that it outperforms masked language models in a wide range of tasks.
Structured Pruning Learns Compact and Accurate Models
The growing size of neural language models has led to increased attention in model compression.
Non-Parametric Few-Shot Learning for Word Sense Disambiguation
One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-tail distribution.
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning
Extensive experiments on real-world low-resource languages - without access to large-scale monolingual corpora or large amounts of labeled data - for tasks like cross-lingual sentiment analysis and named entity recognition show the effectiveness of our approach.
Demoting Racial Bias in Hate Speech Detection
In current hate speech datasets, there exists a high correlation between annotators' perceptions of toxicity and signals of African American English (AAE).
Predicting Performance for Natural Language Processing Tasks
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting.
A Summary of the First Workshop on Language Technology for Language Documentation and Revitalization
Despite recent advances in natural language processing and other language technology, the application of such technology to language documentation and conservation has been limited.
Generalized Data Augmentation for Low-Resource Translation
Translation to or from low-resource languages LRLs poses challenges for machine translation in terms of both adequacy and fluency.
Domain Adaptation of Neural Machine Translation by Lexicon Induction
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift.
Choosing Transfer Languages for Cross-Lingual Learning
Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages.
