Search Results for author:
Found 22 papers, 13 papers with code
What’s in a Name? Answer Equivalence For Open-Domain Question Answering
We incorporate answers for two settings: evaluation with additional answers and model training with equivalent answers.
Best Practices and Lessons Learned on Synthetic Data for Language Models
The success of AI models relies on the availability of large, diverse, and high-quality datasets, which can be challenging to obtain due to data scarcity, privacy concerns, and high costs.
Design2Code: How Far Are We From Automating Front-End Engineering?
In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking.
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition
We also present a comprehensive taxonomical ontology of the types of adversarial prompts.
Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong
To reduce over-reliance on LLMs, we ask LLMs to provide contrastive information - explain both why the claim is true and false, and then we present both sides of the explanation to users.
Getting MoRE out of Mixture of Language Model Reasoning Experts
Beyond generalizability, the interpretable design of MoRE improves selective question answering results compared to baselines without incorporating inter-expert agreement.
Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations
We investigate the inductive biases of ICL from the perspective of feature bias: which feature ICL is more likely to use given a set of underspecified demonstrations in which two features are equally predictive of the labels.
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises
In order to fill this important gap, we construct READIN: a Chinese multi-task benchmark with REalistic And Diverse Input Noises.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Prompting GPT-3 To Be Reliable
While reliability is a broad and vaguely defined term, we decompose reliability into four main facets that correspond to the existing framework of ML safety and are well-recognized to be important: generalizability, social biases, calibration, and factuality.
Re-Examining Calibration: The Case of Question Answering
Building on those observations, we propose a new calibration metric, MacroCE, that better captures whether the model assigns low confidence to wrong predictions and high confidence to correct predictions.
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
What are the units of text that we want to model?
What's in a Name? Answer Equivalence For Open-Domain Question Answering
We incorporate answers for two settings: evaluation with additional answers and model training with equivalent answers.
Adversarial Training for Machine Reading Comprehension with Virtual Embeddings
Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks.
Sub-Character Tokenization for Chinese Pretrained Language Models
2) Pronunciation-based SubChar tokenizers can encode Chinese homophones into the same transliteration sequences and produce the same tokenization output, hence being robust to homophone typos.
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
In this work, we propose a simple and effective method to cover a much larger proportion of the attack search space, called Adversarial and Mixup Data Augmentation (AMDA).
CharBERT: Character-aware Pre-trained Language Model
Most pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations, by which OOV (out-of-vocab) words are almost avoidable.
Benchmarking Robustness of Machine Reading Comprehension Models
To fill this important gap, we construct AdvRACE (Adversarial RACE), a new model-agnostic benchmark for evaluating the robustness of MRC models under four different types of adversarial attacks, including our novel distractor extraction and generation attacks.
Sentiment Aware Neural Machine Translation
We conducted tests with both sentiment and non-sentiment bearing contexts to examine the effectiveness of our methods.
What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?
Based on our experiments on the 5 key MCRC datasets - RACE, MCTest, MCScript, MCScript2. 0, DREAM - we observe that 1) fine-tuned BERT mainly learns how keywords lead to correct prediction, instead of learning semantic understanding and reasoning; and 2) BERT does not need correct syntactic information to solve the task; 3) there exists artifacts in these datasets such that they can be solved even without the full context.
Dataset Mention Extraction and Classification
Datasets are integral artifacts of empirical scientific research.
