Search Results for author: Yangyi Chen

Found 21 papers, 17 papers with code

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation

1 code implementation22 Nov 2023 Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji

We adopt a weakly-supervised approach to directly generate visual event structures from captions for ViStruct training, capitalizing on abundant image-caption pairs from the web.

DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback

no code implementations16 Nov 2023 Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, Ajay Divakaran

The critique NLF identifies the strengths and weaknesses of the responses and is used to align the LVLMs with human preferences.

Language Modelling

Prudent Silence or Foolish Babble? Examining Large Language Models' Responses to the Unknown

no code implementations16 Nov 2023 Genglin Liu, Xingyao Wang, Lifan Yuan, Yangyi Chen, Hao Peng

When presented with such unanswerable questions, an LLM should appropriately convey uncertainty, and be able to challenge the premise and refuse to generate a response.

Question Answering valid

R-Tuning: Teaching Large Language Models to Refuse Unknown Questions

no code implementations16 Nov 2023 Hanning Zhang, Shizhe Diao, Yong Lin, Yi R. Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, Tong Zhang

This approach is formalized by first identifying the knowledge gap between parametric knowledge and the instruction tuning data.

CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets

1 code implementation29 Sep 2023 Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R. Fung, Hao Peng, Heng Ji

It creates toolsets specifically curated for the tasks and equips LLMs with a component that retrieves tools from these sets to enhance their capability to solve complex tasks.

Language Modelling Mathematical Reasoning

MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback

no code implementations19 Sep 2023 Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, Heng Ji

However, current evaluation protocols often emphasize benchmark performance with single-turn exchanges, neglecting the nuanced interactions among the user, LLMs, and external tools, while also underestimating the importance of natural language feedback from users.

Decision Making

Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models

1 code implementation8 Sep 2023 Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, Ajay Divakaran

Based on this pipeline and the existing coarse-grained annotated dataset, we build the CURE benchmark to measure both the zero-shot reasoning performance and consistency of VLMs.

Visual Reasoning

Making Pre-trained Language Models both Task-solvers and Self-calibrators

1 code implementation21 Jul 2023 Yangyi Chen, Xingyao Wang, Heng Ji

In this work, we consider the practical scenario that we need to effectively utilize training samples to make PLMs both task-solvers and self-calibrators.

Adversarial Defense

Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations

1 code implementation7 Jun 2023 Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun

Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets.

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework

1 code implementation29 May 2023 Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji

In our experiments, we conduct a robustness evaluation of RoBERTa models to demonstrate the effectiveness of our evaluation framework, and further show the rationality of each component in the framework.

Adversarial Attack

A Close Look into the Calibration of Pre-trained Language Models

1 code implementation31 Oct 2022 Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, Heng Ji

We observe a consistent change in calibration performance across six factors.

Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP

1 code implementation19 Oct 2022 Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, Maosong Sun

We discuss the deficiencies in previous work and propose our suggestions that the research on the Security-oriented adversarial NLP (SoadNLP) should: (1) evaluate their methods on security tasks to demonstrate the real-world concerns; (2) consider real-world attackers' goals, instead of developing impractical methods.

Data Augmentation

A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks

1 code implementation17 Jun 2022 Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, Maosong Sun

However, we highlight two issues in previous backdoor learning evaluations: (1) The differences between real-world scenarios (e. g. releasing poisoned datasets or models) are neglected, and we argue that each scenario has its own constraints and concerns, thus requires specific evaluation protocols; (2) The evaluation metrics only consider whether the attacks could flip the models' predictions on poisoned samples and retain performances on benign samples, but ignore that poisoned samples should also be stealthy and semantic-preserving.

text similarity

Exploring the Universal Vulnerability of Prompt-based Learning Paradigm

1 code implementation Findings (NAACL) 2022 Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Zhiyuan Liu

Prompt-based learning paradigm bridges the gap between pre-training and fine-tuning, and works effectively under the few-shot setting.

Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework

1 code implementation28 Oct 2021 Lifan Yuan, Yichi Zhang, Yangyi Chen, Wei Wei

In this paper, we instantiate our framework with an attack algorithm named Textual Projected Gradient Descent (T-PGD).

Adversarial Attack Language Modelling

Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks

1 code implementation15 Oct 2021 Yangyi Chen, Fanchao Qi, Hongcheng Gao, Zhiyuan Liu, Maosong Sun

In this paper, we find two simple tricks that can make existing textual backdoor attacks much more harmful.

Vocal Bursts Valence Prediction

Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer

1 code implementation EMNLP 2021 Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, Maosong Sun

In this paper, we make the first attempt to conduct adversarial and backdoor attacks based on text style transfer, which is aimed at altering the style of a sentence while preserving its meaning.

Backdoor Attack Style Transfer +1

Multi-granularity Textual Adversarial Attack with Behavior Cloning

1 code implementation EMNLP 2021 Yangyi Chen, Jin Su, Wei Wei

Furthermore, we propose a reinforcement-learning based method to train a multi-granularity attack agent through behavior cloning with the expert knowledge from our MAYA algorithm to further reduce the query times.

Adversarial Attack

Automatic Construction of Sememe Knowledge Bases via Dictionaries

1 code implementation Findings (ACL) 2021 Fanchao Qi, Yangyi Chen, Fengyu Wang, Zhiyuan Liu, Xiao Chen, Maosong Sun

We use this method to build an English SKB and a French SKB, and conduct comprehensive evaluations from both intrinsic and extrinsic perspectives.

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

2 code implementations ACL 2021 Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun

As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort.

Backdoor Attack

Cannot find the paper you are looking for? You can Submit a new open access paper.