Search Results for author:
Found 208 papers, 110 papers with code
On the Transferability of Adversarial Attacks against Neural Text Classifier
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
Robustness and Adversarial Examples in Natural Language Processing
In particular, we will review recent studies on analyzing the weakness of NLP systems when facing adversarial inputs and data with a distribution shift.
Using Item Response Theory to Measure Gender and Racial Bias of a BERT-based Automated English Speech Assessment System
This study addresses the need to examine potential biases of transformer-based models in the context of automated English speech assessment.
Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples
Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples.
A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints
Under such distributions, computing the likelihood of even simple constraints is #P-hard.
Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty
In this paper, we aim to tackle this problem with a unified framework consisting of an end-to-end trainable method and a planning algorithm.
A Reevaluation of Event Extraction: Past, Present, and Future Challenges
In this work, we first identify and discuss these evaluation challenges, including the unfair comparisons resulting from different assumptions about data or different data preprocessing steps, the incompleteness of the current evaluation framework leading to potential dataset bias or data split bias, and low reproducibility of prior studies.
JAB: Joint Adversarial Prompting and Belief Augmentation
With the recent surge of language models in different applications, attention to safety and robustness of these models has gained significant importance.
VideoCon: Robust Video-Language Alignment via Contrast Captions
Despite being (pre)trained on a massive amount of data, state-of-the-art video-language alignment models are not robust to semantically-plausible contrastive changes in the video captions.
Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs
Leveraging this unified data and modular design, Lumos not only achieves comparable or superior performance to current, state-of-the-art agents, but also exhibits several key advantages: (1) Lumos surpasses GPT-4/3. 5-based agents in complex question answering and web tasks, while equalling the performance of significantly larger LLM agents on math tasks; (2) Lumos outperforms open-source agents created through conventional training methods and those using chain-of-thoughts training; and (3) Lumos is capable of effectively generalizing to unseen interactive tasks, outperforming larger LLM-based agents and even exceeding performance of specialized agents.
On the steerability of large language models toward data-driven personas
Towards this goal, we first present a new conceptualization of a persona.
Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks
Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models (LLMs) on a massive amount of diverse tasks with instructions.
What's "up" with vision-language models? Investigating their struggle with spatial reasoning
Recent vision-language (VL) models are powerful, but can they reliably distinguish "right" from "left"?
Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond
Vision-language (VL) understanding tasks evaluate models' comprehension of complex visual scenes through multiple-choice questions.
An Exploration of In-Context Learning for Speech Language Model
Despite the success of ICL in NLP, little work is exploring the possibility of ICL in speech processing.
LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following
Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents.
Will the Prince Get True Love's Kiss? On the Model Sensitivity to Gender Perturbation over Fairytale Texts
Using counterfactual data augmentation to the FairytaleQA dataset, we evaluate model robustness against swapped gender character information, and then mitigate learned biases by introducing counterfactual gender stereotypes during training time.
Mitigating Bias for Question Answering Models by Tracking Bias Influence
Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance by observing its influence on another instance.
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters
Through benchmarking evaluation on 2 popular LLMs- ChatGPT and Alpaca, we reveal significant gender biases in LLM-generated recommendation letters.
Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models
DeSel improves greedy search by an average of 4. 7% semantic F1 across five datasets.
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems
Recent advancements in Large Language Models empower them to follow freeform instructions, including imitating generic or specific demographic personas in conversations.
Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Model
Notably, in the low-resource scenario, prompting consistently outperforms adapter tuning.
MathVista: Evaluating Math Reasoning in Visual Contexts with GPT-4V, Bard, and Other Large Multimodal Models
To bridge this gap, we present MathVista, a benchmark designed to combine challenges from diverse mathematical and visual tasks.
Towards General-Purpose Text-Instruction-Guided Voice Conversion
This paper introduces a novel voice conversion (VC) model, guided by text instructions such as "articulate slowly with a deep tone" or "speak in a cheerful boyish voice".
Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer
In this paper, we propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer of the multilingual pretrained language models without the help of such external data.
Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
To achieve comprehensive coverage of diverse speech tasks and harness instruction tuning, we invite the community to collaborate and contribute, facilitating the dynamic growth of the benchmark.
Contextual Label Projection for Cross-Lingual Structure Extraction
In this paper, we introduce CLAP, which first translates text to the target language and performs contextual translation on the labels using the translated text as the context, ensuring better accuracy for the translated labels.
FLIRT: Feedback Loop In-context Red Teaming
Here we propose an automatic red teaming framework that evaluates a given model and exposes its vulnerabilities against unsafe and inappropriate content generation.
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding
However, we find visual and textual fine-grained information, e. g., keywords in the sentence and objects in the image, can be fairly informative for semantics understanding.
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
We release our corpus of chain-of-thought samples and code.
SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts
In this paper, we present pioneering research that explores the application of prompt tuning to stimulate speech LMs for various generation tasks, within a unified framework called SpeechGen, with around 10M trainable parameters.
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models
In this paper, we study an interesting hypothesis: can we transfer the in-context learning ability from the language domain to VL domain?
Efficient Shapley Values Estimation by Amortization for Text Classification
In practice, Shapley Values are often estimated with a small number of stochastic model evaluations.
Red Teaming Language Model Detectors with Language Models
The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users.
MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Models
In this paper, we introduce MiniSUPERB, a lightweight benchmark that efficiently evaluates SSL speech models with comparable results to SUPERB but lower computational costs significantly.
ABC-KD: Attention-Based-Compression Knowledge Distillation for Deep Learning-Based Noise Suppression
To begin with, we are among the first to comprehensively investigate mainstream KD techniques on DNS models to resolve the two challenges.
ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation
Paraphrase generation is a long-standing task in natural language processing (NLP).
PIP: Parse-Instructed Prefix for Syntactically Controlled Paraphrase Generation
Existing fine-tuning methods for this task are costly as all the parameters of the model need to be updated during the training process.
IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models
Specifically, IdealGPT utilizes an LLM to generate sub-questions, a VLM to provide corresponding sub-answers, and another LLM to reason to achieve the final answer.
Text encoders bottleneck compositionality in contrastive vision-language models
We first curate CompPrompts, a set of increasingly compositional image captions that VL models should be able to capture (e. g., single object, to object+property, to multiple interacting objects).
Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis
Recent research has revealed that deep learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances.
Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation
Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses.
ChatGPT for Us: Preserving Data Privacy in ChatGPT via Dialogue Text Ambiguation to Expand Mental Health Care Delivery
Large language models have been useful in expanding mental health care delivery.
"I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation
Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life.
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response.
KPEval: Towards Fine-grained Semantic-based Evaluation of Keyphrase Extraction and Generation Systems
Despite the significant advancements in keyphrase extraction and keyphrase generation methods, the predominant approach for evaluation only relies on exact matching with human references and disregards reference-free attributes.
CoBIT: A Contrastive Bi-directional Image-Text Generation Model
The field of vision and language has witnessed a proliferation of pre-trained foundation models.
Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness
Through a critical review of how intersectionality is discussed in 30 papers from the AI fairness literature, we deductively and inductively: 1) map how intersectionality tenets operate within the AI fairness paradigm and 2) uncover gaps between the conceptualization and operationalization of intersectionality.
CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning
Multimodal contrastive pretraining has been used to train multimodal representation models, such as CLIP, on large amounts of paired image-text data.
SpeechPrompt v2: Prompt Tuning for Speech Classification Tasks
For speech processing, SpeechPrompt shows its high parameter efficiency and competitive performance on a few speech classification tasks.
Ranked #16 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
Semantic Strengthening of Neuro-Symbolic Learning
Numerous neuro-symbolic approaches have recently been proposed typically with the goal of adding symbolic knowledge to the output layer of a neural network.
Ensemble knowledge distillation of self-supervised speech models
We tried two different aggregation techniques, layerwise-average and layerwise-concatenation, to the representations of different teacher models and found that the former was more effective.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods
A key goal for the advancement of AI is to develop technologies that serve the needs not just of one group but of all communities regardless of their geographical region.
PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English
Privacy policies provide individuals with information about their rights and how their personal information is handled.
Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
However, there lacks a systematic study of how the two types of approaches compare and how different design choices can affect the performance of PLM-based models.
A Survey of Deep Learning for Mathematical Reasoning
Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life.
Find Someone Who: Visual Commonsense Understanding in Human-Centric Grounding
We present a new commonsense task, Human-centric Commonsense Grounding, that tests the models' ability to ground individuals given the context descriptions about what happened before, and their mental/physical states or intentions.
REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory
REVEAL consists of four key components: the memory, the encoder, the retriever and the generator.
Ranked #1 on
Visual Question Answering (VQA)
on A-OKVQA
(Accuracy metric)
Is the Elephant Flying? Resolving Ambiguities in Text-to-Image Generative Models
Natural language often contains ambiguities that can lead to misinterpretation and miscommunication.
Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN
Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm.
Empowering Language Models with Knowledge Graph Reasoning for Question Answering
Answering open-domain questions requires world knowledge about in-context entities.
Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense
Visual commonsense understanding requires Vision Language (VL) models to not only understand image and text but also cross-reference in-between to fully integrate and achieve comprehension of the visual scene described.
Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations
In this paper, we demonstrate that leveraging Abstract Meaning Representations (AMR) can greatly improve the performance of unsupervised syntactically controlled paraphrase generation.
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers
Large pre-trained language models have shown remarkable performance over the past few years.
How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?
To this end, we introduce an Ethical NaTural Language Interventions in Text-to-Image GENeration (ENTIGEN) benchmark dataset to evaluate the change in image generations conditional on ethical interventions across three social axes -- gender, skin color, and culture.
ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimation
Finally, our analysis shows that the two types of uncertainty provided by \textbf{ADDMU} can be leveraged to characterize adversarial examples and identify the ones that contribute most to model's robustness in adversarial training.
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks
How reliably can we trust the scores obtained from social bias benchmarks as faithful indicators of problematic social biases in a given language model?
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning
We present the SUPERB challenge at SLT 2022, which aims at learning self-supervised speech representation for better performance, generalization, and efficiency.
Watermarking Pre-trained Language Models with Backdooring
Large pre-trained language models (PLMs) have proven to be a crucial component of modern natural language processing systems.
An Analysis of the Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation
We present a systematic analysis of the impact of decoding algorithms on LM fairness, and analyze the trade-off between fairness, diversity and quality.
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data.
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
We further design language models to learn to generate lectures and explanations as the chain of thought (CoT) to mimic the multi-hop reasoning process when answering ScienceQA questions.
Ranked #4 on
Science Question Answering
on ScienceQA
Improving the Adversarial Robustness of NLP Models by Information Bottleneck
Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models.
Semantic Probabilistic Layers for Neuro-Symbolic Learning
We design a predictive layer for structured-output prediction (SOP) that can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints.
Controllable Text Generation with Neurally-Decomposed Oracle
We propose a general and efficient framework to control auto-regressive generation models with NeurAlly-Decomposed Oracle (NADO).
GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles
We utilize this ontology to further introduce GENEVA, a diverse generalizability benchmarking dataset comprising four test suites, aimed at evaluating models' ability to handle limited data and unseen event type generalization.
TAGPRIME: A Unified Framework for Relational Structure Extraction
In this work, we propose to take a unified view of all these tasks and introduce TAGPRIME to address relational structure extraction problems.
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation
Disinformation has become a serious problem on social media.
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models
In this paper, we introduce a benchmark dataset, Geo-Diverse Commonsense Multilingual Language Models Analysis (GeoMLAMA), for probing the diversity of the relational knowledge in multilingual PLMs.
Conditional Supervised Contrastive Learning for Fair Text Classification
Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data.
Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages
In code generation, the model learns to do the opposite.
On the Paradox of Learning to Reason from Data
Logical reasoning is needed in a wide range of NLP tasks.
Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks
Experiments demonstrate that MAD leads to consistent gains in the low-shot, domain-shifted, and fully-supervised conditions on VCR, SNLI-VE, and VQA, achieving SOTA performance on VCR compared to other single models pretrained with image-text data.
Ranked #4 on
Visual Question Answering (VQA)
on VCR (Q-A) test
Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies
Prior studies in privacy policies frame the question answering (QA) task as identifying the most relevant text segment or a list of sentences from a policy document given a user query.
SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Tasks
We report in this paper the first exploration of the prompt tuning paradigm for speech processing tasks based on Generative Spoken Language Model (GSLM).
On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations
Multiple metrics have been introduced to measure fairness in various natural language processing tasks.
Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal
Language models excel at generating coherent text, and model compression techniques such as knowledge distillation have enabled their use in resource-constrained settings.
Measuring Fairness of Text Classifiers via Prediction Sensitivity
With the rapid growth in language processing applications, fairness has emerged as an important consideration in data-driven solutions.
Representation Learning for Resource-Constrained Keyphrase Generation
State-of-the-art keyphrase generation methods generally depend on large annotated datasets, limiting their performance in domains with limited annotated data.
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction
We present a study on leveraging multilingual pre-trained generative language models for zero-shot cross-lingual event argument extraction (EAE).
A Survey of Knowledge-Intensive NLP with Pre-Trained Language Models
With the increasing of model capacity brought by pre-trained language models, there emerges boosting needs for more knowledgeable natural language processing (NLP) models with advanced functionalities including providing and making flexible use of encyclopedic and commonsense knowledge.
Neuro-Symbolic Entropy Regularization
We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
As for pre-training, a scene-graph-aware pre-training method is proposed to leverage structure knowledge extracted in the visual scene graph.
Grounded Language-Image Pre-training
The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.
Ranked #1 on
2D Object Detection
on RF100
Socially Aware Bias Measurements for Hindi Language Representations
Language representations are efficient tools used across NLP applications, but they are strife with encoded societal biases.
Toward Degradation-Robust Voice Conversion
However, in real-world scenarios, it is difficult to collect clean utterances of a speaker, and they are usually degraded by noises or reverberations.
Relation-Guided Pre-Training for Open-Domain Question Answering
Answering complex open-domain questions requires understanding the latent relations between involving entities.
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning
Commonsense is defined as the knowledge that is shared by everyone.
Ranked #1 on
Visual Commonsense Reasoning
on GD-VCR
Cultural Vocal Bursts Intensity Prediction
Visual Commonsense Reasoning
Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution
Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models.
DEGREE: A Data-Efficient Generation-Based Event Extraction Model
Given a passage and a manually designed prompt, DEGREE learns to summarize the events mentioned in the passage into a natural sentence that follows a predefined pattern.
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models.
Retrieval Augmented Code Generation and Summarization
To mimic developers' code or summary generation behavior, we propose a retrieval augmented framework, REDCODER, that retrieves relevant code or summaries from a retrieval database and provides them as a supplement to code generation or summarization models.
Ranked #1 on
Code Generation
on CodeXGLUE - CodeSearchNet
(using extra training data)
AVATAR: A Parallel Corpus for Java-Python Program Translation
Automating program translation is of paramount importance in software migration, and recently researchers explored unsupervised approaches due to the unavailability of parallel corpora.
BERTHop: An Effective Vision-and-Language Model for Chest X-ray Disease Diagnosis
Vision-and-language(V&L) models take image and text as input and learn to capture the associations between them.
On Measures of Biases and Harms in NLP
Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality.
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble
Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.
How Much Can CLIP Benefit Vision-and-Language Tasks?
Most existing Vision-and-Language (V&L) models rely on pre-trained visual encoders, using a relatively small set of manually-annotated data (as compared to web-crawled data), to perceive the visual world.
Ranked #4 on
Vision and Language Navigation
on RxR
(using extra training data)
Integrating topic modeling and word embedding to characterize violent deaths
Our method offers a flexible and broadly applicable approach to model topics in text data.
Clinical Named Entity Recognition using Contextualized Token Representations
To facilitate various downstream applications using clinical case reports (CCRs), we pre-train two deep contextualized language models, Clinical Embeddings from Language Model (C-ELMo) and Clinical Contextual String Embeddings (C-Flair) using the clinical-related corpus from the PubMed Central.
Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification
Existing bias mitigation methods to reduce disparities in model outcomes across cohorts have focused on data augmentation, debiasing model embeddings, or adding fairness-based optimization objectives during training.
Syntax-augmented Multilingual BERT for Cross-lingual Transfer
In recent years, we have seen a colossal effort in pre-training multilingual text encoders using large-scale corpora in many languages to facilitate cross-lingual transfer learning.
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?
We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social stereotypes.
``Nice Try, Kiddo'': Investigating Ad Hominems in Dialogue Responses
Ad hominem attacks are those that target some feature of a person{'}s character instead of the position the person is maintaining.
Societal Biases in Language Generation: Progress and Challenges
Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner.
SpeechNet: A Universal Modularized Model for Speech Processing Tasks
The multi-task learning of a wide variety of speech processing tasks with a universal model has not been studied.
Adapting Coreference Resolution for Processing Violent Death Narratives
Coreference resolution is an important component in analyzing narrative text from administrative data (e. g., clinical or police sources).
Evaluating the Values of Sources in Transfer Learning
Transfer learning that adapts a model trained on data-rich sources to low-resource targets has been widely applied in natural language processing (NLP).
Revealing Persona Biases in Dialogue Systems
Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives.
On the Sensitivity and Stability of Model Interpretations in NLP
We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existed removal-based criteria.
Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer.
Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
Our method is able to reveal the hidden model biases not directly shown in the test dataset.
Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models
In this work, we present ParaBART, a semantic sentence embedding model that learns to disentangle semantics and syntax in sentence embeddings obtained by pre-trained language models.
Leveraging Unlabeled Data for Entity-Relation Extraction through Probabilistic Constraint Satisfaction
We study the problem of entity-relation extraction in the presence of symbolic domain knowledge.
Unified Pre-training for Program Understanding and Generation
Experiments on code summarization in the English language, code generation, and code translation in seven programming languages show that PLBART outperforms or rivals state-of-the-art models.
CREATe: Clinical Report Extraction and Annotation Technology
Clinical case reports are written descriptions of the unique aspects of a particular clinical case, playing an essential role in sharing clinical experiences about atypical disease phenotypes and new therapies.
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
To systematically study and benchmark social biases in open-ended language generation, we introduce the Bias in Open-Ended Language Generation Dataset (BOLD), a large-scale dataset that consists of 23, 679 English text generation prompts for bias benchmarking across five domains: profession, gender, race, religion, and political ideology.
Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs
We also demonstrate that the performance of SynPG is competitive or even better than supervised models when the unannotated data is large.
Intent Classification and Slot Filling for Privacy Policies
We refer to predicting the privacy practice explained in a sentence as intent classification and identifying the text spans sharing specific information as slot filling.
Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference
There has been a steady need in the medical community to precisely extract the temporal relations between clinical events.
Generating Sports News from Live Commentary: A Chinese Dataset for Sports Game Summarization
To deeply study this task, we present SportsSum, a Chinese sports game summarization dataset which contains 5, 428 soccer games of live commentaries and the corresponding news articles.
On the Transferability of Adversarial Attacksagainst Neural Text Classifier
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
Cross-Lingual Dependency Parsing by POS-Guided Word Reordering
The words in each sentence of a source language corpus are rearranged to meet the word order in a target language under the guidance of a part-of-speech based language model (LM).
"Nice Try, Kiddo": Investigating Ad Hominems in Dialogue Responses
Ad hominem attacks are those that target some feature of a person's character instead of the position the person is maintaining.
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions
Pre-trained contextual vision-and-language (V&L) models have achieved impressive performance on various benchmarks.
LOGAN: Local Group Bias Detection by Clustering
Machine learning techniques have been widely used in natural language processing (NLP).
GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction
Recent progress in cross-lingual relation and event extraction use graph convolutional networks (GCNs) with universal dependency parses to learn language-agnostic sentence representations such that models trained on one language can be applied to other languages.
PolicyQA: A Reading Comprehension Dataset for Privacy Policies
Prior studies in this domain frame the QA task as retrieving the most relevant text segment or a list of sentences from the policy document given a question.
Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention
Natural language processing techniques have demonstrated promising results in keyphrase generation.
What Does BERT with Vision Look At?
Pre-trained visually grounded language models such as ViLBERT, LXMERT, and UNITER have achieved significant performance improvement on vision-and-language tasks but what they learn during pre-training remains unclear.
``The Boating Store Had Its Best Sail Ever'': Pronunciation-attentive Contextualized Pun Recognition
In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence.
GPT-GNN: Generative Pre-Training of Graph Neural Networks
Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data.
Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples.
An Integer Linear Programming Framework for Mining Constraints from Data
This raises a question -- \emph{can we mine constraints and rules from data based on a learning algorithm?}
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization
With little performance loss, our method can almost remove the bias amplification in the distribution.
On the Robustness of Language Encoders against Grammatical Errors
We conduct a thorough study to diagnose the behaviors of pre-trained language encoders (ELMo, BERT, and RoBERTa) when confronted with natural grammatical errors.
SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
We propose SentiBERT, a variant of BERT that effectively captures compositional sentiment semantics.
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
In this paper, we study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications.
A Transformer-based Approach for Source Code Summarization
Generating a readable summary that describes the functionality of a program is known as source code summarization.
Towards Controllable Biases in Language Generation
We present a general approach towards controllable societal biases in natural language generation (NLG).
"The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition
In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence.
Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
Linear relaxation based perturbation analysis (LiRPA) for neural networks, which computes provable linear bounds of output neurons given a certain amount of input perturbation, has become a core component in robustness verification and certified defense.
Robustness Verification for Transformers
Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees.
Towards Understanding Gender Bias in Relation Extraction
We use WikiGenderBias to evaluate systems for bias and find that NRE systems exhibit gender biased predictions and lay groundwork for future evaluation of bias in NRE.
Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages
We conduct experiments on cross-lingual dependency parsing where we train a dependency parser on a source language and transfer it to a wide range of target languages.
Visualizing Trends of Key Roles in News Articles
There are tons of news articles generated every day reflecting the activities of key roles such as people, organizations and political parties.
Retrofitting Contextualized Word Embeddings with Paraphrases
Contextualized word embedding models, such as ELMo, generate meaningful representations of words and their context.
BOSH: An Efficient Meta Algorithm for Decision-based Attacks
Adversarial example generation becomes a viable method for evaluating the robustness of a machine learning model.
Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification
To identify adversarial attacks, a perturbation discriminator validates how likely a token in the text is perturbed and provides a set of potential perturbations.
Examining Gender Bias in Languages with Grammatical Gender
Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora.