Search Results for author: Yangfeng Ji

Found 58 papers, 27 papers with code

“This is a Problem, Don’t You Agree?” Framing and Bias in Human Evaluation for Natural Language Generation

no code implementations ACL (EvalNLGEval, INLG) 2020 Stephanie Schoch, Diyi Yang, Yangfeng Ji

Despite recent efforts reviewing current human evaluation practices for natural language generation (NLG) research, the lack of reported question wording and potential for framing effects or cognitive biases influencing results has been widely overlooked.

Text Generation

Contrastive Data and Learning for Natural Language Processing

no code implementations NAACL (ACL) 2022 Rui Zhang, Yangfeng Ji, Yue Zhang, Rebecca J. Passonneau

We then survey the benefits and the best practices of contrastive learning for various downstream NLP applications including Text Classification, Question Answering, Summarization, Text Generation, Interpretability and Explainability, Commonsense Knowledge and Reasoning, Vision-and-Language. This tutorial intends to help researchers in the NLP and computational linguistics community to understand this emerging topic and promote future research directions of using contrastive learning for NLP applications.

Contrastive Learning Question Answering +5

Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generation

no code implementations2 Nov 2023 Wanyu Du, Yangfeng Ji

The development of trustworthy conversational information-seeking systems relies on dialogue models that can generate faithful and accurate responses based on relevant knowledge texts.

Dialogue Generation

Secure and Effective Data Appraisal for Machine Learning

no code implementations3 Oct 2023 Xu Ouyang, Changhong Yang, Felix Xiaozhu Lin, Yangfeng Ji

Essential for an unfettered data market is the ability to discreetly select and evaluate training data before finalizing a transaction between the data owner and model owner.

Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values

1 code implementation16 Jun 2023 Stephanie Schoch, Ritwick Mishra, Yangfeng Ji

Although Shapley values have been shown to be highly effective for identifying harmful training instances, dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models.

Data Valuation Language Modelling +1

Pre-training Transformers for Knowledge Graph Completion

no code implementations28 Mar 2023 Sanxing Chen, Hao Cheng, Xiaodong Liu, Jian Jiao, Yangfeng Ji, Jianfeng Gao

Learning transferable representation of knowledge graphs (KGs) is challenging due to the heterogeneous, multi-relational nature of graph structures.

Improving Interpretability via Explicit Word Interaction Graph Layer

1 code implementation3 Feb 2023 Arshdeep Sekhon, Hanjie Chen, Aman Shrivastava, Zhe Wang, Yangfeng Ji, Yanjun Qi

Recent NLP literature has seen growing interest in improving model interpretability.

CS-Shapley: Class-wise Shapley Values for Data Valuation in Classification

2 code implementations13 Nov 2022 Stephanie Schoch, Haifeng Xu, Yangfeng Ji

Our theoretical analysis shows the proposed value function is (essentially) the unique function that satisfies two desirable properties for evaluating data values in classification.

Data Valuation

Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models

1 code implementation20 Oct 2022 Hannah Chen, Yangfeng Ji, David Evans

Traditional (fickle) adversarial examples involve finding a small perturbation that does not change an input's true label but confuses the classifier into outputting a different prediction.

Contrastive Learning Natural Language Inference +1

REV: Information-Theoretic Evaluation of Free-Text Rationales

1 code implementation10 Oct 2022 Hanjie Chen, Faeze Brahman, Xiang Ren, Yangfeng Ji, Yejin Choi, Swabha Swayamdipta

More concretely, we propose a metric called REV (Rationale Evaluation with conditional V-information), to quantify the amount of new, label-relevant information in a rationale beyond the information already available in the input or the label.

Efficient NLP Model Finetuning via Multistage Data Filtering

1 code implementation28 Jul 2022 Xu Ouyang, Shahina Mohd Azam Ansari, Felix Xiaozhu Lin, Yangfeng Ji

To this end, we set to filter training examples in a streaming fashion, in tandem with training the target model.

text-classification Text Classification

PLAtE: A Large-scale Dataset for List Page Web Extraction

no code implementations24 May 2022 Aidan San, Yuan Zhuang, Jan Bakus, Colin Lockard, David Ciemiewicz, Sandeep Atluri, Yangfeng Ji, Kevin Small, Heba Elfardy

Recently, neural models have been leveraged to significantly improve the performance of information extraction from semi-structured websites.

Attribute Attribute Extraction

Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation

1 code implementation19 May 2022 Wanyu Du, Hanjie Chen, Yangfeng Ji

In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost of annotating MR-to-Text pairs.

Dialogue Generation Language Modelling +2

White-box Testing of NLP models with Mask Neuron Coverage

no code implementations Findings (NAACL) 2022 Arshdeep Sekhon, Yangfeng Ji, Matthew B. Dwyer, Yanjun Qi

Recent literature has seen growing interest in using black-box strategies like CheckList for testing the behavior of NLP models.

Data Augmentation Fault Detection

Pathologies of Pre-trained Language Models in Few-shot Fine-tuning

no code implementations insights (ACL) 2022 Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah, Yangfeng Ji

Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance gain comes from.

text-classification Text Classification

Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors

1 code implementation4 Apr 2022 Wanyu Du, Jianqiao Zhao, LiWei Wang, Yangfeng Ji

The proposed stochastic function is sampled from a Gaussian process prior to (1) provide infinite number of joint Gaussian distributions of random context variables (diversity-promoting) and (2) explicitly model dependency between context variables (accurate-encoding).

Gaussian Processes Paraphrase Generation +4

Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretation

1 code implementation23 Mar 2022 Hanjie Chen, Yangfeng Ji

Neural language models show vulnerability to adversarial examples which are semantically similar to their original counterparts with a few words replaced by their synonyms.

text-classification Text Classification

Synopses of Movie Narratives: a Video-Language Dataset for Story Understanding

no code implementations11 Mar 2022 Yidan Sun, Qin Chao, Yangfeng Ji, Boyang Li

Despite recent advances of AI, story understanding remains an open and under-investigated problem.

Retrieval Text Retrieval +1

FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows

no code implementations14 Feb 2022 Jianqiao Zhao, Yanyang Li, Wanyu Du, Yangfeng Ji, Dong Yu, Michael R. Lyu, LiWei Wang

Hence, we propose segment act, an extension of dialog act from utterance level to segment level, and crowdsource a large-scale dataset for it.

Dialogue Evaluation

Explaining Predictive Uncertainty by Looking Back at Model Explanations

no code implementations11 Jan 2022 Hanjie Chen, Wanyu Du, Yangfeng Ji

Explaining predictive uncertainty is an important complement to explaining prediction labels in helping users understand model decision making and gaining their trust on model predictions, while has been largely ignored in prior works.

Decision Making Natural Language Inference +2

Simple Text Detoxification by Identifying a Linear Toxic Subspace in Language Model Embeddings

no code implementations15 Dec 2021 Andrew Wang, Mohit Sudhakar, Yangfeng Ji

We hypothesize the existence of a low-dimensional toxic subspace in the latent space of pre-trained language models, the existence of which suggests that toxic features follow some underlying pattern and are thus removable.

Abusive Language Language Modelling +1

Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing

1 code implementation EMNLP (BlackboxNLP) 2021 Sanchit Sinha, Hanjie Chen, Arshdeep Sekhon, Yangfeng Ji, Yanjun Qi

Via a small portion of word-level swaps, these adversarial perturbations aim to make the resulting text semantically and spatially similar to its seed input (therefore sharing similar interpretations).

Language Modelling

Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

1 code implementation NAACL 2021 Hanjie Chen, Song Feng, Jatin Ganhotra, Hui Wan, Chulaka Gunasekara, Sachindra Joshi, Yangfeng Ji

Most existing methods generate post-hoc explanations for neural network models by identifying individual feature attributions or detecting interactions between adjacent features.

Natural Language Inference Paraphrase Identification +1

The Amazing World of Neural Language Generation

no code implementations EMNLP 2020 Yangfeng Ji, Antoine Bosselut, Thomas Wolf, Asli Celikyilmaz

Neural Language Generation (NLG) {--} using neural network models to generate coherent text {--} is among the most promising methods for automated text creation.

Language Modelling Text Generation +1

Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers

3 code implementations EMNLP 2020 Hanjie Chen, Yangfeng Ji

To build an interpretable neural text classifier, most of the prior work has focused on designing inherently interpretable models or finding faithful explanations.

General Classification text-classification +1

A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsing

1 code implementation COLING 2020 Sanxing Chen, Aidan San, Xiaodong Liu, Yangfeng Ji

In Text-to-SQL semantic parsing, selecting the correct entities (tables and columns) for the generated SQL query is both crucial and challenging; the parser is required to connect the natural language (NL) question and the SQL query to the structured knowledge in the database.

Semantic Parsing SQL Parsing +1

HittER: Hierarchical Transformers for Knowledge Graph Embeddings

2 code implementations EMNLP 2021 Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, Yangfeng Ji

Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block.

 Ranked #1 on Link Prediction on FB15k-237 (Hit@10 metric)

Knowledge Graph Embeddings Link Prediction +2

Pointwise Paraphrase Appraisal is Potentially Problematic

no code implementations ACL 2020 Hannah Chen, Yangfeng Ji, David Evans

The prevailing approach for training and evaluating paraphrase identification models is constructed as a binary classification problem: the model is given a pair of sentences, and is judged by how accurately it classifies pairs as either paraphrases or non-paraphrases.

Binary Classification Paraphrase Identification

Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection

2 code implementations ACL 2020 Hanjie Chen, Guangtao Zheng, Yangfeng Ji

Experiments show the effectiveness of the proposed method in providing explanations that are both faithful to models and interpretable to humans.

Decision Making General Classification +2

Improving the Explainability of Neural Sentiment Classifiers via Data Augmentation

no code implementations10 Sep 2019 Hanjie Chen, Yangfeng Ji

Experiments show the proposed data augmentation methods significantly improve the explainability of both neural classifiers.

Data Augmentation General Classification +3

Neural Discourse Structure for Text Categorization

1 code implementation ACL 2017 Yangfeng Ji, Noah Smith

We show that discourse structure, as defined by Rhetorical Structure Theory and provided by an existing discourse parser, benefits text categorization.

Text Categorization

DyNet: The Dynamic Neural Network Toolkit

4 code implementations15 Jan 2017 Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Kumar, Chaitanya Malaviya, Paul Michel, Yusuke Oda, Matthew Richardson, Naomi Saphra, Swabha Swayamdipta, Pengcheng Yin

In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives.

graph construction

LSTM based Conversation Models

1 code implementation31 Mar 2016 Yi Luan, Yangfeng Ji, Mari Ostendorf

In this paper, we present a conversational model that incorporates both context and participant role for two-party conversations.

Language Modelling Text Generation

A Latent Variable Recurrent Neural Network for Discourse Relation Language Models

1 code implementation7 Mar 2016 Yangfeng Ji, Gholamreza Haffari, Jacob Eisenstein

This paper presents a novel latent variable recurrent neural network architecture for jointly modeling sequences of words and (possibly latent) discourse relations between adjacent sentences.

Classification Dialog Act Classification +4

Document Context Language Models

1 code implementation12 Nov 2015 Yangfeng Ji, Trevor Cohn, Lingpeng Kong, Chris Dyer, Jacob Eisenstein

Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure.

Sentence

deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets

no code implementations IJCNLP 2015 Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, Bill Dolan

We introduce Discriminative BLEU (deltaBLEU), a novel metric for intrinsic evaluation of generated text in tasks that admit a diverse range of possible outputs.

Sentence

One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations

no code implementations TACL 2015 Yangfeng Ji, Jacob Eisenstein

A more subtle challenge is that it is not enough to represent the meaning of each argument of a discourse relation, because the relation may depend on links between lowerlevel components, such as entity mentions.

Question Answering Relation +1

Entity-Augmented Distributional Semantics for Discourse Relations

no code implementations17 Dec 2014 Yangfeng Ji, Jacob Eisenstein

A more subtle challenge is that it is not enough to represent the meaning of each sentence of a discourse relation, because the relation may depend on links between lower-level elements, such as entity mentions.

Relation Sentence

Extracting Lexically Divergent Paraphrases from Twitter

1 code implementation TACL 2014 Wei Xu, Alan Ritter, Chris Callison-Burch, William B. Dolan, Yangfeng Ji

We present MultiP (Multi-instance Learning Paraphrase Model), a new model suited to identify paraphrases within the short messages on Twitter.

Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.