Search Results for author:
Found 19 papers, 12 papers with code
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Evaluating bias, fairness, and social impact in monolingual language models is a difficult task.
Extreme Miscalibration and the Illusion of Adversarial Robustness
Deep learning-based Natural Language Processing (NLP) models are vulnerable to adversarial attacks, where small perturbations can cause a model to misclassify.
Automatic Feature Fairness in Recommendation via Adversaries
We leverage the Adaptive Adversarial perturbation based on the widely-applied Factorization Machine (AAFM) as our backbone model.
Large Language Models of Code Fail at Completing Code with Potential Bugs
We find that the presence of potential bugs significantly degrades the generation performance of the high-performing Code-LLMs.
ReCode: Robustness Evaluation of Code Generation Models
Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.
BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems
We have open-sourced the toolkit at https://github. com/salesforce/botsim
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Whodunit? Learning to Contrast for Authorship Attribution
To the best of our knowledge, we are the first to integrate contrastive learning with pre-trained language model fine-tuning for authorship attribution.
Data Governance in the Age of Large-Scale Data-Driven Language Technology
The recent emergence and adoption of Machine Learning technology, and specifically of Large Language Models, has drawn attention to the need for systematic and transparent management of language data.
The Risks of Machine Learning Systems
We explore how different risks may manifest in various types of ML systems, the factors that affect each risk, and how first-order risks may lead to second-order effects when the system interacts with the real world.
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
What are the units of text that we want to model?
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.
TraVLR: Now You See It, Now You Don't! A Bimodal Dataset for Evaluating Visio-Linguistic Reasoning
Furthermore, existing V+L benchmarks often report global accuracy scores on the entire dataset, making it difficult to pinpoint the specific reasoning tasks that models fail and succeed at.
Interpreting the Robustness of Neural NLP Models to Textual Perturbations
In this work, we test the hypothesis that the extent to which a model is affected by an unseen textual perturbation (robustness) can be explained by the learnability of the perturbation (defined as how well the model learns to identify the perturbation with a small amount of evidence).
Reliability Testing for Natural Language Processing Systems
Questions of fairness, robustness, and transparency are paramount to address before deploying NLP systems.
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
Multilingual models have demonstrated impressive cross-lingual transfer performance.
Robustness Gym: Unifying the NLP Evaluation Landscape
Despite impressive performance on standard benchmarks, deep neural networks are often brittle when deployed in real-world systems.
It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
Training on only perfect Standard English corpora predisposes pre-trained neural networks to discriminate against minorities from non-standard linguistic backgrounds (e. g., African American Vernacular English, Colloquial Singapore English, etc.).
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding
Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English.
