Search Results for author:
Found 18 papers, 12 papers with code
Detecting Pretraining Data from Large Language Models
Min-K% Prob can be applied without any knowledge about the pretraining corpus or any additional training, departing from previous detection methods that require training a reference model on data that is similar to the pretraining data.
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
In this work, we study structured pruning as an effective means to develop smaller LLMs from pre-trained, larger models.
Ranked #25 on
Question Answering
on PIQA
Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation
Finally, we propose an effective alignment method that explores diverse generation strategies, which can reasonably reduce the misalignment rate under our attack.
Trainable Transformer in Transformer
In this work, we propose an efficient construction, Transformer in Transformer (in short, TinT), that allows a transformer to simulate and fine-tune complex models internally during inference (e. g., pre-trained language models).
InstructEval: Systematic Evaluation of Instruction Selection Methods
In-context learning (ICL) performs tasks by prompting a large language model (LLM) using an instruction and a small set of annotated examples called demonstrations.
Training Trajectories of Language Models Across Scales
Why do larger language models demonstrate more desirable behaviors?
MABEL: Attenuating Gender Bias using Textual Entailment Data
To this end, we propose MABEL (a Method for Attenuating Gender Bias using Entailment Labels), an intermediate pre-training approach for mitigating gender bias in contextualized representations.
Don't Prompt, Search! Mining-based Zero-Shot Learning with Language Models
Our results suggest that the success of prompting can partly be explained by the model being exposed to similar examples during pretraining, which can be directly retrieved through regular expressions.
Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models
In this work, we adapt prompt-based few-shot learning to ELECTRA and show that it outperforms masked language models in a wide range of tasks.
Structured Pruning Learns Compact and Accurate Models
The growing size of neural language models has led to increased attention in model compression.
Non-Parametric Few-Shot Learning for Word Sense Disambiguation
One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-tail distribution.
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning
Extensive experiments on real-world low-resource languages - without access to large-scale monolingual corpora or large amounts of labeled data - for tasks like cross-lingual sentiment analysis and named entity recognition show the effectiveness of our approach.
Demoting Racial Bias in Hate Speech Detection
In current hate speech datasets, there exists a high correlation between annotators' perceptions of toxicity and signals of African American English (AAE).
Predicting Performance for Natural Language Processing Tasks
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting.
A Summary of the First Workshop on Language Technology for Language Documentation and Revitalization
Despite recent advances in natural language processing and other language technology, the application of such technology to language documentation and conservation has been limited.
Generalized Data Augmentation for Low-Resource Translation
Translation to or from low-resource languages LRLs poses challenges for machine translation in terms of both adequacy and fluency.
Domain Adaptation of Neural Machine Translation by Lexicon Induction
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift.
Choosing Transfer Languages for Cross-Lingual Learning
Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages.
