Search Results for author: Zhou Yu

Found 165 papers, 76 papers with code

CGIM: A Cycle Guided Interactive Learning Model for Consistency Identification in Task-oriented Dialogue

1 code implementation COLING 2022 Libo Qin, Qiguang Chen, Tianbao Xie, Qian Liu, Shijue Huang, Wanxiang Che, Zhou Yu

Consistency identification in task-oriented dialog (CI-ToD) usually consists of three subtasks, aiming to identify inconsistency between current system response and current user response, dialog history and the corresponding knowledge base.

DialogStitch: Synthetic Deeper and Multi-Context Task-Oriented Dialogs

1 code implementation SIGDIAL (ACL) 2021 Satwik Kottur, Chinnadhurai Sankar, Zhou Yu, Alborz Geramifard

Real-world conversational agents must effectively handle long conversations that span multiple contexts.

Towards Socially Intelligent Agents with Mental State Transition and Human Value

no code implementations SIGDIAL (ACL) 2022 Liang Qiu, Yizhou Zhao, Yuan Liang, Pan Lu, Weiyan Shi, Zhou Yu, Song-Chun Zhu

One of which is to track the agent’s mental state transition and teach the agent to make decisions guided by its value like a human.

ErAConD: Error Annotated Conversational Dialog Dataset for Grammatical Error Correction

1 code implementation NAACL 2022 Xun Yuan, Derek Pham, Sam Davidson, Zhou Yu

Currently available grammatical error correction (GEC) datasets are compiled using essays or other long-form text written by language learners, limiting the applicability of these datasets to other domains such as informal writing and conversational dialog.

Chatbot Grammatical Error Correction

Improving Conversational Recommendation Systems’ Quality with Context-Aware Item Meta-Information

no code implementations Findings (NAACL) 2022 Bowen Yang, Cong Han, Yu Li, Lei Zuo, Zhou Yu

In this paper, we propose a simple yet effective architecture comprising a pre-trained language model (PLM) and an item metadata encoder to integrate the recommendation and the dialog generation better.

Knowledge Graphs Language Modelling +2

KERS: A Knowledge-Enhanced Framework for Recommendation Dialog Systems with Multiple Subgoals

1 code implementation Findings (EMNLP) 2021 Jun Zhang, Yan Yang, Chencai Chen, Liang He, Zhou Yu

Recommendation dialogs require the system to build a social bond with users to gain trust and develop affinity in order to increase the chance of a successful recommendation.

Question Answering Recommendation Systems +1

Improving Named Entity Recognition in Spoken Dialog Systems by Context and Speech Pattern Modeling

no code implementations SIGDIAL (ACL) 2021 Minh Nguyen, Zhou Yu

Deployed spoken dialog systems receive user input in the form of automatic speech recognition (ASR) transcripts, and simply applying NER model trained on written text to ASR transcripts often leads to low accuracy because compared to written text, ASR transcripts lack important cues such as punctuation and capitalization.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Effective Unsupervised Constrained Text Generation based on Perturbed Masking

no code implementations Findings (ACL) 2022 Yingwen Fu, Wenjie Ou, Zhou Yu, Yue Lin

Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data.

Position Sentence +1

Parallel Structures in Pre-training Data Yield In-Context Learning

no code implementations19 Feb 2024 Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, He He

Pre-trained language models (LMs) are capable of in-context learning (ICL): they can adapt to a task with only a few examples given in the prompt without any parameter update.

In-Context Learning

The Mirrored Influence Hypothesis: Efficient Data Influence Estimation by Harnessing Forward Passes

no code implementations14 Feb 2024 Myeongseob Ko, Feiyang Kang, Weiyan Shi, Ming Jin, Zhou Yu, Ruoxi Jia

Inspired by this, we introduce a new method for estimating the influence of training data, which requires calculating gradients for specific test samples, paired with a forward pass for each training point.

Memorization Test

State Value Generation with Prompt Learning and Self-Training for Low-Resource Dialogue State Tracking

1 code implementation30 Jan 2024 Ming Gu, Yan Yang, Chengcai Chen, Zhou Yu

Experimental results on the MultiWOZ 2. 1 dataset show that our method which has only less than 1 billion parameters achieves state-of-the-art performance under the data ratio settings of 5%, 10%, and 25% when limited to models under 100 billion parameters.

Dialogue State Tracking

ConFit: Improving Resume-Job Matching using Data Augmentation and Contrastive Learning

no code implementations29 Jan 2024 Xiao Yu, Jinzhong Zhang, Zhou Yu

A reliable resume-job matching system helps a company find suitable candidates from a pool of resumes, and helps a job seeker find relevant jobs from a list of job posts.

Contrastive Learning Data Augmentation

GarchingSim: An Autonomous Driving Simulator with Photorealistic Scenes and Minimalist Workflow

1 code implementation28 Jan 2024 Liguo Zhou, Yinglei Song, Yichao Gao, Zhou Yu, Michael Sodamin, Hongshen Liu, Liang Ma, Lian Liu, Hao liu, Yang Liu, Haichuan Li, Guang Chen, Alois Knoll

However, the availability of free and open-source simulators is limited, and the installation and configuration process can be daunting for beginners and interdisciplinary researchers.

Autonomous Driving

ProLex: A Benchmark for Language Proficiency-oriented Lexical Substitution

no code implementations21 Jan 2024 Xuanming Zhang, Zixun Chen, Zhou Yu

To bridge this gap, we propose a new task, language proficiency-oriented lexical substitution.


SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space

1 code implementation13 Dec 2023 Yunchen Li, Zhou Yu, Gaoqi He, Yunhang Shen, Ke Li, Xing Sun, Shaohui Lin

On the other hand, the model unconditionally learns the probability distribution of the data $p(X)$ and generates samples that conform to this distribution.

Denoising Traffic Prediction

End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions

no code implementations15 Nov 2023 Libo Qin, Wenbo Pan, Qiguang Chen, Lizi Liao, Zhou Yu, Yue Zhang, Wanxiang Che, Min Li

End-to-end task-oriented dialogue (EToD) can directly generate responses in an end-to-end fashion without modular training, which attracts escalating popularity.

Teaching Language Models to Self-Improve through Interactive Demonstrations

1 code implementation20 Oct 2023 Xiao Yu, Baolin Peng, Michel Galley, Jianfeng Gao, Zhou Yu

The self-improving ability of large language models (LLMs), enabled by prompting them to analyze and revise their own outputs, has garnered significant interest in recent research.


Pragmatic Evaluation of Clarifying Questions with Fact-Level Masking

no code implementations17 Oct 2023 Matthew Toles, Yukun Huang, Zhou Yu, Luis Gravano

Here we present a definition and framework for natural language pragmatic asking of clarifying questions (PACQ), the problem of generating questions that result in answers useful for a reasoning task.

Chatbot Question Answering +2

Building Persona Consistent Dialogue Agents with Offline Reinforcement Learning

1 code implementation16 Oct 2023 Ryan Shea, Zhou Yu

Our automatic and human evaluations show that our framework improves both the persona consistency and dialogue quality of a state-of-the-art social chatbot.

Chatbot Offline RL +2

Curriculum-Driven Edubot: A Framework for Developing Language Learning Chatbots Through Synthesizing Conversational Data

no code implementations28 Sep 2023 Yu Li, Shang Qu, Jili Shen, Shangchao Min, Zhou Yu

Chatbots have become popular in educational settings, revolutionizing how students interact with material and how teachers teach.


ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer

1 code implementation29 Aug 2023 Zachary Horvitz, Ajay Patel, Chris Callison-Burch, Zhou Yu, Kathleen McKeown

Our parameter-efficient approach, ParaGuide, leverages paraphrase-conditioned diffusion models alongside gradient-based guidance from both off-the-shelf classifiers and strong existing style embedders to transform the style of text while preserving semantic information.

Style Transfer

MultiPA: a multi-task speech pronunciation assessment system for a closed and open response scenario

no code implementations24 Aug 2023 Yu-Wen Chen, Zhou Yu, Julia Hirschberg

The design of automatic speech pronunciation assessment can be categorized into closed and open response scenarios, each with strengths and limitations.


DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI

1 code implementation19 Jul 2023 JianGuo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong

Despite advancements in conversational AI, language models encounter challenges to handle diverse conversational tasks, and existing dialogue dataset collections often lack diversity and comprehensiveness.

Few-Shot Learning Language Modelling +1

Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations

no code implementations17 Jul 2023 Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, Kathleen McKeown

To answer these questions, we propose to evaluate $\textbf{counterfactual simulatability}$ of natural language explanations: whether an explanation can enable humans to precisely infer the model's outputs on diverse counterfactuals of the explained input.


Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler

1 code implementation1 Jul 2023 Shaohui Lin, Wenxuan Huang, Jiao Xie, Baochang Zhang, Yunhang Shen, Zhou Yu, Jungong Han, David Doermann

In this paper, we propose a novel Knowledge-driven Differential Filter Sampler~(KDFS) with Masked Filter Modeling~(MFM) framework for filter pruning, which globally prunes the redundant filters based on the prior knowledge of a pre-trained model in a differential and non-alternative optimization.

Image Classification Network Pruning

IdEALS: Idiomatic Expressions for Advancement of Language Skills

1 code implementation23 May 2023 Narutatsu Ri, Bill Sun, Sam Davidson, Zhou Yu

Although significant progress has been made in developing methods for Grammatical Error Correction (GEC), addressing word choice improvements has been notably lacking and enhancing sentence expressivity by replacing phrases with advanced expressions is an understudied aspect.

Grammatical Error Correction Sentence

Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning

1 code implementation23 May 2023 Xiao Yu, Maximillian Chen, Zhou Yu

Planning for goal-oriented dialogue often requires simulating future dialogue interactions and estimating task progress.

Language Modelling Large Language Model

Using Textual Interface to Align External Knowledge for End-to-End Task-Oriented Dialogue Systems

no code implementations23 May 2023 Qingyang Wu, Deema Alnuhait, Derek Chen, Zhou Yu

We demonstrate our paradigm in practice through MultiWOZ-Remake, including an interactive textual interface built for the MultiWOZ database and a correspondingly re-processed dataset.

Task-Oriented Dialogue Systems

Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment

1 code implementation23 May 2023 Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, Smaranda Muresan

Embedding Chain-of-Thought prompting in a human-AI collaborative framework, we build a high-quality dataset of 3, 069 social norms aligned with social situations across Chinese and American cultures alongside corresponding free-text explanations.

Descriptive In-Context Learning +1

Controllable Mixed-Initiative Dialogue Generation through Prompting

1 code implementation6 May 2023 Maximillian Chen, Xiao Yu, Weiyan Shi, Urvi Awasthi, Zhou Yu

The standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents.

Dialogue Generation

ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos

1 code implementation CVPR 2023 Zhou Yu, Lixiang Zheng, Zhou Zhao, Fei Wu, Jianping Fan, Kui Ren, Jun Yu

A recent benchmark AGQA poses a promising paradigm to generate QA pairs automatically from pre-annotated scene graphs, enabling it to measure diverse reasoning abilities with granular control.

Question Answering Spatio-temporal Scene Graphs +1

User Adaptive Language Learning Chatbots with a Curriculum

no code implementations11 Apr 2023 Kun Qian, Ryan Shea, Yu Li, Luke Kutszik Fryer, Zhou Yu

Along with the development of systems for natural language understanding and generation, dialog systems have been widely adopted for language learning and practicing.

Natural Language Understanding

PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs

1 code implementation15 Mar 2023 Rahul Goel, Waleed Ammar, Aditya Gupta, Siddharth Vashishtha, Motoki Sano, Faiz Surani, Max Chang, HyunJeong Choe, David Greene, Kyle He, Rattima Nitisaroj, Anna Trukhina, Shachi Paul, Pararth Shah, Rushin Shah, Zhou Yu

Research interest in task-oriented dialogs has increased as systems such as Google Assistant, Alexa and Siri have become ubiquitous in everyday life.

FaceChat: An Emotion-Aware Face-to-face Dialogue Framework

1 code implementation8 Mar 2023 Deema Alnuhait, Qingyang Wu, Zhou Yu

While current dialogue systems like ChatGPT have made significant advancements in text-based interactions, they often overlook the potential of other modalities in enhancing the overall user experience.

Mixture of Soft Prompts for Controllable Data Generation

1 code implementation2 Mar 2023 Derek Chen, Celine Lee, Yunan Lu, Domenic Rosati, Zhou Yu

Large language models (LLMs) effectively generate fluent text when the target output follows natural language patterns.

Data Augmentation Denoising +2

Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

no code implementations24 Feb 2023 Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, Jianfeng Gao

Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e. g., task-oriented dialog and question answering.

Informativeness Open-Domain Question Answering

Pre-Finetuning for Few-Shot Emotional Speech Recognition

no code implementations24 Feb 2023 Maximillian Chen, Zhou Yu

Speech models have long been known to overfit individual speakers for many classification tasks.

Few-Shot Learning speech-recognition +2

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking

no code implementations12 Feb 2023 Derek Chen, Kun Qian, Zhou Yu

Prompt-based methods with large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks.

Dialogue State Tracking In-Context Learning +2

PLACES: Prompting Language Models for Social Conversation Synthesis

1 code implementation7 Feb 2023 Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, Zhou Yu, Dilek Hakkani-Tur

Collecting high quality conversational data can be very expensive for most applications and infeasible for others due to privacy, ethical, or similar concerns.

Conversational Response Generation

In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models

no code implementations20 Dec 2022 Yukun Huang, Yanda Chen, Zhou Yu, Kathleen McKeown

We propose to combine in-context learning objectives with language modeling objectives to distill both the ability to read in-context examples and task knowledge to the smaller models.

Few-Shot Learning In-Context Learning +1

DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization

no code implementations20 Dec 2022 Yu Li, Baolin Peng, Pengcheng He, Michel Galley, Zhou Yu, Jianfeng Gao

In this work, we propose DIONYSUS (dynamic input optimization in pre-training for dialogue summarization), a pre-trained encoder-decoder model for summarizing dialogues in any new domain.

MIGA: A Unified Multi-task Generation Framework for Conversational Text-to-SQL

no code implementations19 Dec 2022 Yingwen Fu, Wenjie Ou, Zhou Yu, Yue Lin

Conversational text-to-SQL is designed to translate multi-turn natural language questions into their corresponding SQL queries.


Sources of Noise in Dialogue and How to Deal with Them

no code implementations6 Dec 2022 Derek Chen, Zhou Yu

Training dialogue systems often entails dealing with noisy training examples and unexpected user inputs.


Focus! Relevant and Sufficient Context Selection for News Image Captioning

no code implementations1 Dec 2022 Mingyang Zhou, Grace Luo, Anna Rohrbach, Zhou Yu

In our paper, we first demonstrate that by combining more fine-grained context that captures the key named entities (obtained via an oracle) and the global context that summarizes the news, we can dramatically improve the model's ability to generate accurate news captions.

Image Captioning Relation Extraction +1

KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning

1 code implementation30 Nov 2022 Xiao Yu, Qingyang Wu, Kun Qian, Zhou Yu

In task-oriented dialogs (TOD), reinforcement learning (RL) algorithms train a model to directly optimize response for task-related metrics.

Language Modelling reinforcement-learning +2

Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding

no code implementations25 Oct 2022 Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Andy Rosenbaum, Seokhwan Kim, Yang Liu, Zhou Yu, Dilek Hakkani-Tur

Dialogue understanding tasks often necessitate abundant annotated data to achieve good performance and that presents challenges in low-resource settings.

Data Augmentation Dialogue Understanding +2

Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems

1 code implementation22 Oct 2022 David Gros, Yu Li, Zhou Yu

Dialog systems are often designed or trained to output human-like responses.

Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks

no code implementations11 Oct 2022 Kushal Chawla, Weiyan Shi, Jingwen Zhang, Gale Lucas, Zhou Yu, Jonathan Gratch

Dialogue systems capable of social influence such as persuasion, negotiation, and therapy, are essential for extending the use of technology to numerous realistic scenarios.

On the Relation between Sensitivity and Accuracy in In-context Learning

no code implementations16 Sep 2022 Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, He He

In-context learning (ICL) suffers from oversensitivity to the prompt, making it unreliable in real-world scenarios.

In-Context Learning Relation

Stateful Memory-Augmented Transformers for Efficient Dialogue Modeling

1 code implementation15 Sep 2022 Qingyang Wu, Zhou Yu

Transformer encoder-decoder models have achieved great performance in dialogue generation tasks, however, their inability to process long dialogue history often leads to truncation of the context To address this problem, we propose a novel memory-augmented transformer that is compatible with existing pre-trained encoder-decoder models and enables efficient preservation of the dialogue history information.

Dialogue Generation Language Modelling

Using Chatbots to Teach Languages

no code implementations31 Jul 2022 Yu Li, Chun-Yen Chen, Dian Yu, Sam Davidson, Ryan Hou, Xun Yuan, Yinghua Tan, Derek Pham, Zhou Yu

This paper reports on progress towards building an online language learning tool to provide learners with conversational experience by using dialog systems as conversation practice partners.

reinforcement-learning Reinforcement Learning (RL)

Learning a Better Initialization for Soft Prompts via Meta-Learning

no code implementations25 May 2022 Yukun Huang, Kun Qian, Zhou Yu

So pre-trained prompt tuning (PPT) is proposed to initialize prompts by leveraging pre-training data.


Just Fine-tune Twice: Selective Differential Privacy for Large Language Models

1 code implementation15 Apr 2022 Weiyan Shi, Ryan Shea, Si Chen, Chiyuan Zhang, Ruoxi Jia, Zhou Yu

Utilizing the fact that sensitive information in language data tends to be sparse, Shi et al. (2021) formalized a DP notion extension called Selective Differential Privacy (SDP) to protect only the sensitive tokens defined by a policy function.

Bilaterally Slimmable Transformer for Elastic and Efficient Visual Question Answering

1 code implementation24 Mar 2022 Zhou Yu, Zitian Jin, Jun Yu, Mingliang Xu, Hongbo Wang, Jianping Fan

Recent advances in Transformer architectures [1] have brought remarkable improvements to visual question answering (VQA).

Question Answering Visual Question Answering

FastKASSIM: A Fast Tree Kernel-Based Syntactic Similarity Metric

1 code implementation15 Mar 2022 Maximillian Chen, Caitlyn Chen, Xiao Yu, Zhou Yu

Syntax is a fundamental component of language, yet few metrics have been employed to capture syntactic similarity or coherence at the utterance- and document-level.

Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue

no code implementations15 Mar 2022 Maximillian Chen, Weiyan Shi, Feifan Yan, Ryan Hou, Jingwen Zhang, Saurav Sahay, Zhou Yu

Complex conversation settings such as persuasion involve communicating changes in attitude or behavior, so users' perspectives need to be addressed, even when not directly related to the topic.


Random Forest Weighted Local Fréchet Regression with Random Objects

no code implementations10 Feb 2022 Rui Qiu, Zhou Yu, Ruoqing Zhu

Statistical analysis is increasingly confronted with complex data from metric spaces.


Optimal Model Averaging of Support Vector Machines in Diverging Model Spaces

no code implementations24 Dec 2021 Chaoxia Yuan, Chao Ying, Zhou Yu, Fang Fang

Support vector machine (SVM) is a powerful classification method that has achieved great success in many fields.

Model Selection

Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Information

1 code implementation15 Dec 2021 Bowen Yang, Cong Han, Yu Li, Lei Zuo, Zhou Yu

The encoder learns to map item metadata to embeddings that can reflect the semantic information in the dialog context.

Language Modelling Recommendation Systems +1

AllWOZ: Towards Multilingual Task-Oriented Dialog Systems for All

no code implementations15 Dec 2021 Lei Zuo, Kun Qian, Bowen Yang, Zhou Yu

A commonly observed problem of the state-of-the-art natural language technologies, such as Amazon Alexa and Apple Siri, is that their services do not extend to most developing countries' citizens due to language barriers.


Database Search Results Disambiguation for Task-Oriented Dialog Systems

no code implementations NAACL 2022 Kun Qian, Ahmad Beirami, Satwik Kottur, Shahin Shayandeh, Paul Crook, Alborz Geramifard, Zhou Yu, Chinnadhurai Sankar

We find that training on our augmented dialog data improves the model's ability to deal with ambiguous scenarios, without sacrificing performance on unmodified turns.

Multi-Task Learning

Insta-VAX: A Multimodal Benchmark for Anti-Vaccine and Misinformation Posts Detection on Social Media

no code implementations15 Dec 2021 Mingyang Zhou, Mahasweta Chakraborti, Sijia Qian, Zhou Yu, Jingwen Zhang

The dataset and classifiers contribute to monitoring and tracking of vaccine discussions for social scientific and public health efforts in combating the problem of vaccine misinformation.


DG2: Data Augmentation Through Document Grounded Dialogue Generation

no code implementations SIGDIAL (ACL) 2022 Qingyang Wu, Song Feng, Derek Chen, Sachindra Joshi, Luis A. Lastras, Zhou Yu

Collecting data for training dialog systems can be extremely expensive due to the involvement of human participants and need for extensive annotation.

Data Augmentation Dialogue Generation

ErAConD : Error Annotated Conversational Dialog Dataset for Grammatical Error Correction

1 code implementation15 Dec 2021 Xun Yuan, Derek Pham, Sam Davidson, Zhou Yu

Currently available grammatical error correction (GEC) datasets are compiled using well-formed written text, limiting the applicability of these datasets to other domains such as informal writing and dialog.

Chatbot Grammatical Error Correction

Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation

no code implementations NAACL 2022 Yu Li, Baolin Peng, Yelong Shen, Yi Mao, Lars Liden, Zhou Yu, Jianfeng Gao

To address these challenges, we present PLUG, a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks.

Dialogue Generation Language Modelling

IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning

1 code implementation25 Oct 2021 Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei zhang, Zhou Yu, Xiaodan Liang, Song-Chun Zhu

Also, we develop a strong IconQA baseline Patch-TRM that applies a pyramid cross-modal Transformer with input diagram embeddings pre-trained on the icon dataset.

Arithmetic Reasoning Math Word Problem Solving +2

Clean or Annotate: How to Spend a Limited Data Collection Budget

no code implementations DeepLo 2022 Derek Chen, Zhou Yu, Samuel R. Bowman

Crowdsourcing platforms are often used to collect datasets for training machine learning models, despite higher levels of inaccurate labeling compared to expert labeling.

Denoising Learning with noisy labels

Zero-Shot Dialogue State Tracking via Cross-Task Transfer

1 code implementation EMNLP 2021 Zhaojiang Lin, Bing Liu, Andrea Madotto, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Eunjoon Cho, Rajen Subba, Pascale Fung

Zero-shot transfer learning for dialogue state tracking (DST) enables us to handle a variety of task-oriented dialogue domains without the expense of collecting in-domain data.

Dialogue State Tracking Question Answering +1

GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation

2 code implementations EMNLP 2021 Derek Chen, Zhou Yu

Practical dialogue systems require robust methods of detecting out-of-scope (OOS) utterances to avoid conversational breakdowns and related failure modes.

Data Augmentation

Selective Differential Privacy for Language Modeling

1 code implementation NAACL 2022 Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, Zhou Yu

Given that the private information in natural language is sparse (for example, the bulk of an email might not carry personally identifiable information), we propose a new privacy notion, selective differential privacy, to provide rigorous privacy guarantees on the sensitive portion of the data to improve model utility.

Language Modelling Privacy Preserving

ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration

1 code implementation16 Aug 2021 Yuhao Cui, Zhou Yu, Chunqi Wang, Zhongzhou Zhao, Ji Zhang, Meng Wang, Jun Yu

Nevertheless, most existing VLP approaches have not fully utilized the intrinsic knowledge within the image-text pairs, which limits the effectiveness of the learned alignments and further restricts the performance of their models.

Visual Reasoning

Discovering Dialogue Slots with Weak Supervision

no code implementations ACL 2021 Vojt{\v{e}}ch Hude{\v{c}}ek, Ond{\v{r}}ej Du{\v{s}}ek, Zhou Yu

Our model demonstrates state-of-the-art performance in slot tagging without labeled training data on four different dialogue domains.

Response Generation Task-Oriented Dialogue Systems

The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity

no code implementations ACL 2021 David Gros, Yu Li, Zhou Yu

Humans are increasingly interacting with machines through language, sometimes in contexts where the user may not know they are talking to a machine (like over the phone or a text chatbot).


Towards Emotional Support Dialog Systems

1 code implementation ACL 2021 Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, Minlie Huang

Emotional support is a crucial ability for many conversation scenarios, including social interactions, mental health support, and customer service chats.

Annotation Inconsistency and Entity Bias in MultiWOZ

no code implementations SIGDIAL (ACL) 2021 Kun Qian, Ahmad Beirami, Zhouhan Lin, Ankita De, Alborz Geramifard, Zhou Yu, Chinnadhurai Sankar

In this work, we identify an overlooked issue with dialog state annotation inconsistencies in the dataset, where a slot type is tagged inconsistently across similar dialogs leading to confusion for DST modeling.

dialog state tracking Memorization +2

Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking

2 code implementations10 May 2021 Zhaojiang Lin, Bing Liu, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Andrea Madotto, Eunjoon Cho, Rajen Subba

Zero-shot cross-domain dialogue state tracking (DST) enables us to handle task-oriented dialogue in unseen domains without the expense of collecting in-domain data.

Dialogue State Tracking Transfer Learning

LEGOEval: An Open-Source Toolkit for Dialogue System Evaluation via Crowdsourcing

1 code implementation ACL 2021 Yu Li, Josh Arnold, Feifan Yan, Weiyan Shi, Zhou Yu

We present LEGOEval, an open-source toolkit that enables researchers to easily evaluate dialogue systems in a few lines of code using the online crowdsource platform, Amazon Mechanical Turk.

Revealing Persona Biases in Dialogue Systems

1 code implementation18 Apr 2021 Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, Nanyun Peng

Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives.

DEUX: An Attribute-Guided Framework for Sociable Recommendation Dialog Systems

no code implementations16 Apr 2021 Yu Li, Shirley Anugrah Hayati, Weiyan Shi, Zhou Yu

It is important for sociable recommendation dialog systems to perform as both on-task content and social content to engage users and gain their favor.

Attribute dialog state tracking +1

A Student-Teacher Architecture for Dialog Domain Adaptation under the Meta-Learning Setting

no code implementations6 Apr 2021 Kun Qian, Wei Wei, Zhou Yu

The most recent researches on domain adaption focus on giving the model a better initialization, rather than optimizing the adaptation process.

Domain Adaptation Meta-Learning

MIDAS: A Dialog Act Annotation Scheme for Open Domain HumanMachine Spoken Conversations

no code implementations EACL 2021 Dian Yu, Zhou Yu

To validate our scheme, we leveraged transfer learning methods to train a multi-label dialog act prediction model and reached an F1 score of 0. 79.

Transfer Learning

Attribute Alignment: Controlling Text Generation from Pre-trained Language Models

1 code implementation Findings (EMNLP) 2021 Dian Yu, Zhou Yu, Kenji Sagae

Large language models benefit from training with a large amount of unlabeled text, which gives them increasingly fluent and diverse generation capabilities.

Attribute Language Modelling +1

Towards Socially Intelligent Agents with Mental State Transition and Human Utility

no code implementations12 Mar 2021 Liang Qiu, Yizhou Zhao, Yuan Liang, Pan Lu, Weiyan Shi, Zhou Yu, Song-Chun Zhu

One of which is to track the agent's mental state transition and teach the agent to make decisions guided by its value like a human.

ChainCQG: Flow-Aware Conversational Question Generation

1 code implementation EACL 2021 Jing Gu, Mostafa Mirshekari, Zhou Yu, Aaron Sisto

Conversational systems enable numerous valuable applications, and question-answering is an important component underlying many of these.

Conversational Question Answering Question Generation +1

Refine and Imitate: Reducing Repetition and Inconsistency in Dialogue Generation via Reinforcement Learning and Human Demonstration

no code implementations1 Jan 2021 Weiyan Shi, Yu Li, Saurav Sahay, Zhou Yu

Despite the recent success of large-scale language models on various downstream NLP tasks, the repetition and inconsistency problems still persist in dialogue response generation.

Dialogue Generation Language Modelling +2

Sufficient and Disentangled Representation Learning

no code implementations1 Jan 2021 Jian Huang, Yuling Jiao, Xu Liao, Jin Liu, Zhou Yu

We provide strong statistical guarantees for the learned representation by establishing an upper bound on the excess error of the objective function and show that it reaches the nonparametric minimax rate under mild conditions.


Continual Learning in Task-Oriented Dialogue Systems

1 code implementation EMNLP 2021 Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Zhiguang Wang

Continual learning in task-oriented dialogue systems can allow us to add new domains and functionalities through time without incurring the high cost of a whole system retraining.

Continual Learning Intent Recognition +3

Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration

no code implementations Findings (EMNLP) 2021 Weiyan Shi, Yu Li, Saurav Sahay, Zhou Yu

Persuasion dialogue systems reflect the machine's ability to make strategic moves beyond verbal communication, and therefore differentiate themselves from task-oriented or open-domain dialogue systems and have their own unique values.

Language Modelling Reinforcement Learning (RL) +2

Code to Comment "Translation": Data, Metrics, Baselining & Evaluation

no code implementations3 Oct 2020 David Gros, Hariharan Sezhiyan, Prem Devanbu, Zhou Yu

We carefully examine the underlying assumption here: that the task of generating comments sufficiently resembles the task of translating between natural languages, and so similar models and evaluation metrics could be used.

Information Retrieval Retrieval +1

INSPIRED: Toward Sociable Recommendation Dialog Systems

1 code implementation EMNLP 2020 Shirley Anugrah Hayati, Dongyeop Kang, Qingxiaoyang Zhu, Weiyan Shi, Zhou Yu

To better understand how humans make recommendations in communication, we design an annotation scheme related to recommendation strategies based on social science theories and annotate these dialogs.

Movie Recommendation

ALICE: Active Learning with Contrastive Natural Language Explanations

no code implementations EMNLP 2020 Weixin Liang, James Zou, Zhou Yu

We propose Active Learning with Contrastive Explanations (ALICE), an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning.

Active Learning Classification +1

Structured Attention for Unsupervised Dialogue Structure Induction

1 code implementation EMNLP 2020 Liang Qiu, Yizhou Zhao, Weiyan Shi, Yuan Liang, Feng Shi, Tao Yuan, Zhou Yu, Song-Chun Zhu

Inducing a meaningful structural representation from one or a set of dialogues is a crucial but challenging task in computational linguistics.

Inductive Bias Sentence +1

Perception Score, A Learned Metric for Open-ended Text Generation Evaluation

no code implementations7 Aug 2020 Jing Gu, Qingyang Wu, Zhou Yu

Automatic evaluation for open-ended natural language generation tasks remains a challenge.

Text Generation

Deep Dimension Reduction for Supervised Representation Learning

1 code implementation10 Jun 2020 Jian Huang, Yuling Jiao, Xu Liao, Jin Liu, Zhou Yu

We propose a deep dimension reduction approach to learning representations with these characteristics.

Dimensionality Reduction Disentanglement

Deep Multimodal Neural Architecture Search

1 code implementation25 Apr 2020 Zhou Yu, Yuhao Cui, Jun Yu, Meng Wang, DaCheng Tao, Qi Tian

Most existing works focus on a single task and design neural architectures manually, which are highly task-specific and hard to generalize to different tasks.

Image-text matching Neural Architecture Search +4

A Tailored Pre-Training Model for Task-Oriented Dialog Generation

1 code implementation24 Apr 2020 Jing Gu, Qingyang Wu, Chongruo wu, Weiyan Shi, Zhou Yu

The recent success of large pre-trained language models such as BERT and GPT-2 has suggested the effectiveness of incorporating language priors in downstream dialog generation tasks.

Knowledge Distillation Language Modelling +1

Paraphrase Augmented Task-Oriented Dialog Generation

1 code implementation ACL 2020 Silin Gao, Yichi Zhang, Zhijian Ou, Zhou Yu

Neural generative models have achieved promising performance on dialog generation tasks if given a huge data set.

Data Augmentation Response Generation

TextGAIL: Generative Adversarial Imitation Learning for Text Generation

no code implementations7 Apr 2020 Qingyang Wu, Lei LI, Zhou Yu

Generative Adversarial Networks (GANs) for text generation have recently received many criticisms, as they perform worse than their MLE counterparts.

Conditional Text Generation Imitation Learning

Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos

no code implementations16 Mar 2020 Yijun Song, Jingwen Wang, Lin Ma, Zhou Yu, Jun Yu

The task of temporally grounding textual queries in videos is to localize one video segment that semantically corresponds to the given query.


Matching Text with Deep Mutual Information Estimation

no code implementations9 Mar 2020 Xixi Zhou, Chengxi Li, Jiajun Bu, Chengwei Yao, Keyue Shi, Zhi Yu, Zhou Yu

Our approach, Text matching with Deep Info Max (TIM), is integrated with a procedure of unsupervised learning of representations by maximizing the mutual information between text matching neural network's input and output.

Answer Selection Mutual Information Estimation +3

Distributed estimation of principal support vector machines for sufficient dimension reduction

no code implementations28 Nov 2019 Jun Jin, Chao Ying, Zhou Yu

The principal support vector machines method (Li et al., 2011) is a powerful tool for sufficient dimension reduction that replaces original predictors with their low-dimensional linear combinations without loss of information.

Binary Classification Dimensionality Reduction

Filling Conversation Ellipsis for Better Social Dialog Understanding

no code implementations25 Nov 2019 Xiyuan Zhang, Chengxi Li, Dian Yu, Samuel Davidson, Zhou Yu

We then train a prediction model using both utterances containing ellipsis and our automatically completed utterances.

Semantic Role Labeling Sentence +1

End-to-End Trainable Non-Collaborative Dialog System

1 code implementation25 Nov 2019 Yu Li, Kun Qian, Weiyan Shi, Zhou Yu

End-to-end task-oriented dialog models have achieved promising performance on collaborative tasks where users willingly coordinate with the system to complete a given task.

Sentence Test

Importance-Aware Learning for Neural Headline Editing

no code implementations25 Nov 2019 Qingyang Wu, Lei LI, Hao Zhou, Ying Zeng, Zhou Yu

We propose to automate this headline editing process through neural network models to provide more immediate writing support for these social media news writers.

Headline Generation

Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models

1 code implementation EACL 2021 Qingyang Wu, Yichi Zhang, Yu Li, Zhou Yu

Existing dialog system models require extensive human annotations and are difficult to generalize to different tasks.

Language Modelling Response Generation

MOSS: End-to-End Dialog System Framework with Modular Supervision

1 code implementation12 Sep 2019 Weixin Liang, Youzhi Tian, Chengcai Chen, Zhou Yu

To utilize limited training data more efficiently, we propose Modular Supervision Network (MOSS), an encoder-decoder training framework that could incorporate supervision from various intermediate dialog system modules including natural language understanding, dialog state tracking, dialog policy learning, and natural language generation.

dialog state tracking Natural Language Understanding +1

Dependency Parsing for Spoken Dialog Systems

no code implementations IJCNLP 2019 Sam Davidson, Dian Yu, Zhou Yu

Dependency parsing of conversational input can play an important role in language understanding for dialog systems by identifying the relationships between entities extracted from user utterances.

Dependency Parsing

How to Build User Simulators to Train RL-based Dialog Systems

1 code implementation IJCNLP 2019 Weiyan Shi, Kun Qian, Xuewei Wang, Zhou Yu

We propose a method of standardizing user simulator building that can be used by the community to compare dialog system quality using the same set of user simulators fairly.

Reinforcement Learning (RL) User Simulation

A Large-Scale User Study of an Alexa Prize Chatbot: Effect of TTS Dynamism on Perceived Quality of Social Dialog

no code implementations WS 2019 Michelle Cohn, Chun-Yen Chen, Zhou Yu

This study tests the effect of cognitive-emotional expression in an Alexa text-to-speech (TTS) voice on users{'} experience with a social dialog system.


MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations

1 code implementation27 Aug 2019 Dian Yu, Zhou Yu

Previous dialog act schemes, such as SWBD-DAMSL, are designed for human-human conversations, in which conversation partners have perfect language understanding ability.

Transfer Learning

Multimodal Unified Attention Networks for Vision-and-Language Interactions

no code implementations12 Aug 2019 Zhou Yu, Yuhao Cui, Jun Yu, DaCheng Tao, Qi Tian

Learning an effective attention mechanism for multimodal data is important in many vision-and-language tasks that require a synergic understanding of both the visual and textual contents.

Question Answering Visual Grounding +1

Deep Modular Co-Attention Networks for Visual Question Answering

7 code implementations CVPR 2019 Zhou Yu, Jun Yu, Yuhao Cui, DaCheng Tao, Qi Tian

In this paper, we propose a deep Modular Co-Attention Network (MCAN) that consists of Modular Co-Attention (MCA) layers cascaded in depth.

Question Answering Test +1

Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good

3 code implementations ACL 2019 Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, Zhou Yu

Developing intelligent persuasive conversational agents to change people's opinions and actions for social good is the frontier in advancing the ethical development of automated dialogue systems.

Persuasion Strategies Sentence

Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks

no code implementations11 Jun 2019 Qingyang Wu, He Li, Lexin Li, Zhou Yu

With the widespread success of deep neural networks in science and technology, it is becoming increasingly important to quantify the uncertainty of the predictions produced by deep learning.

Classification General Classification +1

Domain Adaptive Dialog Generation via Meta Learning

1 code implementation ACL 2019 Kun Qian, Zhou Yu

We train a dialog system model using multiple rich-resource single-domain dialog data by applying the model-agnostic meta-learning algorithm to dialog domain.

Domain Adaptation Meta-Learning

Multimodal Transformer with Multi-View Visual Representation for Image Captioning

no code implementations20 May 2019 Jun Yu, Jing Li, Zhou Yu, Qingming Huang

Despite the success of existing studies, current methods only model the co-attention that characterizes the inter-modal interactions while neglecting the self-attention that characterizes the intra-modal interactions.

Image Captioning Machine Translation +1

Single Pixel Reconstruction for One-stage Instance Segmentation

no code implementations16 Apr 2019 Jun Yu, Jinghan Yao, Jian Zhang, Zhou Yu, DaCheng Tao

In this paper, we propose a one-stage framework, SPRNet, which performs efficient instance segmentation by introducing a single pixel reconstruction (SPR) branch to off-the-shelf one-stage detectors.

Instance Segmentation Region Proposal +2

Unsupervised Dialog Structure Learning

1 code implementation NAACL 2019 Weiyan Shi, Tiancheng Zhao, Zhou Yu

The learned dialog structure can shed light on how to analyze human dialogs, and more importantly contribute to the design and evaluation of dialog systems.

Incorporating Structured Commonsense Knowledge in Story Completion

no code implementations1 Nov 2018 Jiaao Chen, Jianshu Chen, Zhou Yu

The ability to select an appropriate story ending is the first step towards perfect narrative comprehension.

Story Completion

Structured Content Preservation for Unsupervised Text Style Transfer

2 code implementations15 Oct 2018 Youzhi Tian, Zhiting Hu, Zhou Yu

Text style transfer aims to modify the style of a sentence while keeping its content unchanged.

Language Modelling Sentence +3

Cross-Lingual Cross-Platform Rumor Verification Pivoting on Multimedia Content

2 code implementations EMNLP 2018 Weiming Wen, Songwen Su, Zhou Yu

With the increasing popularity of smart devices, rumors with multimedia content become more and more common on social networks.

Semantic Similarity Semantic Textual Similarity

Overlapping Sliced Inverse Regression for Dimension Reduction

no code implementations23 Jun 2018 Ning Zhang, Zhou Yu, Qiang Wu

The new algorithm, called overlapping sliced inverse regression (OSIR), is able to estimate the effective dimension reduction space and determine the number of effective factors more accurately.

Dimensionality Reduction regression

Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog

no code implementations WS 2018 Jiaping Zhang, Tiancheng Zhao, Zhou Yu

We propose a multimodal hierarchical reinforcement learning framework that dynamically integrates vision and language for task-oriented visual dialog.

Hierarchical Reinforcement Learning reinforcement-learning +3

Sentiment Adaptive End-to-End Dialog Systems

no code implementations ACL 2018 Weiyan Shi, Zhou Yu

End-to-end learning framework is useful for building dialog systems for its simplicity in training and efficiency in model updating.

reinforcement-learning Reinforcement Learning (RL)

Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering

2 code implementations10 Aug 2017 Zhou Yu, Jun Yu, Chenchao Xiang, Jianping Fan, DaCheng Tao

For fine-grained image and question representations, a `co-attention' mechanism is developed by using a deep neural network architecture to jointly learn the attentions for both the image and the question, which can allow us to reduce the irrelevant features effectively and obtain more discriminative features for image and question representations.

Question Answering Visual Question Answering +1

Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering

6 code implementations ICCV 2017 Zhou Yu, Jun Yu, Jianping Fan, DaCheng Tao

For multi-modal feature fusion, here we develop a Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and effectively combine multi-modal features, which results in superior performance for VQA compared with other bilinear pooling approaches.

Question Answering Visual Question Answering