no code implementations • EMNLP 2021 • Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
no code implementations • Findings (ACL) 2022 • Jianhan Xu, Cenyuan Zhang, Xiaoqing Zheng, Linyang Li, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples.
no code implementations • EMNLP (ACL) 2021 • Kai-Wei Chang, He He, Robin Jia, Sameer Singh
In particular, we will review recent studies on analyzing the weakness of NLP systems when facing adversarial inputs and data with a distribution shift.
no code implementations • NAACL (BEA) 2022 • Alexander Kwako, Yixin Wan, Jieyu Zhao, Kai-Wei Chang, Li Cai, Mark Hansen
This study addresses the need to examine potential biases of transformer-based models in the context of automated English speech assessment.
no code implementations • 30 May 2024 • Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai-Wei Chang, Wei Wang
To further self-improve reasoning on the extracted visual information, we let the model reuse a small portion of existing instruction-tuning data and append its self-generated image descriptions to the prompts.
1 code implementation • 29 May 2024 • WenBo Hu, Zi-Yi Dou, Liunian Harold Li, Amita Kamath, Nanyun Peng, Kai-Wei Chang
This raises the question: can we achieve flexibility in the number of visual tokens to suit different tasks and computational resources?
no code implementations • 18 May 2024 • Junzhang Liu, Zhecan Wang, Hammad Ayyubi, Haoxuan You, Chris Thomas, Rui Sun, Shih-Fu Chang, Kai-Wei Chang
Despite the widespread adoption of Vision-Language Understanding (VLU) benchmarks such as VQA v2, OKVQA, A-OKVQA, GQA, VCR, SWAG, and VisualCOMET, our analysis reveals a pervasive issue affecting their integrity: these benchmarks contain samples where answers rely on assumptions unsupported by the provided context.
no code implementations • 12 May 2024 • Kareem Ahmed, Stefano Teso, Paolo Morettin, Luca Di Liello, Pierfrancesco Ardino, Jacopo Gobbi, Yitao Liang, Eric Wang, Kai-Wei Chang, Andrea Passerini, Guy Van Den Broeck
We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training by minimizing the network's violation of such dependencies, steering the network towards predicting distributions satisfying the underlying structure.
no code implementations • 7 May 2024 • Hritik Bansal, Yonatan Bitton, Michal Yarom, Idan Szpektor, Aditya Grover, Kai-Wei Chang
As a result, we show that the pretrained T2V model can generate multi-scene videos that adhere to the multi-scene text descriptions and be visually consistent (e. g., w. r. t entity and background).
no code implementations • 27 Apr 2024 • Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang
Vision-language models have become increasingly powerful for tasks that require an understanding of both visual and linguistic elements, bridging the gap between these modalities.
no code implementations • 16 Apr 2024 • Yixin Wan, Kai-Wei Chang
Social biases can manifest in language agency.
no code implementations • 10 Apr 2024 • Masoud Monajatipoor, Jiaxin Yang, Joel Stremmel, Melika Emami, Fazlolah Mohaghegh, Mozhdeh Rouhsedaghat, Kai-Wei Chang
Large Language Models (LLMs) demonstrate remarkable versatility in various NLP tasks but encounter distinct challenges in biomedicine due to medical language complexities and data scarcity.
no code implementations • 7 Apr 2024 • Hritik Bansal, Po-Nien Kung, P. Jeffrey Brantingham, Kai-Wei Chang, Nanyun Peng
In this paper, we propose GenEARL, a training-free generative framework that harness the power of the modern generative models to understand event task descriptions given image contexts to perform the EARL task.
no code implementations • 4 Apr 2024 • Jooyoung Lee, Fan Yang, Thanh Tran, Qian Hu, Emre Barut, Kai-Wei Chang, Chengwei Su
The Frozen large LM is then prompted to predict a task output based on the rationale generated by the lightweight LM.
1 code implementation • 2 Apr 2024 • Tanmay Parekh, Anh Mac, Jiarui Yu, Yuxuan Dong, Syed Shahriar, Bonnie Liu, Eric Yang, Kuan-Hao Huang, Wei Wang, Nanyun Peng, Kai-Wei Chang
In our work, we pioneer exploiting Event Detection (ED) for better preparedness and early warnings of any upcoming epidemic by developing a framework to extract and analyze epidemic-related events from social media posts.
no code implementations • 1 Apr 2024 • Yixin Wan, Arjun Subramonian, Anaelia Ovalle, Zongyu Lin, Ashima Suvarna, Christina Chance, Hritik Bansal, Rebecca Pattichis, Kai-Wei Chang
In this survey, we review prior studies on dimensions of bias: Gender, Skintone, and Geo-Culture.
1 code implementation • 31 Mar 2024 • Hritik Bansal, Ashima Suvarna, Gantavya Bhatt, Nanyun Peng, Kai-Wei Chang, Aditya Grover
A common technique for aligning large language models (LLMs) relies on acquiring human preferences by comparing multiple generations conditioned on a fixed context.
no code implementations • 21 Mar 2024 • Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, Hongsheng Li
To this end, we introduce MathVerse, an all-around visual math benchmark designed for an equitable and in-depth evaluation of MLLMs.
1 code implementation • 4 Mar 2024 • Xueqing Wu, Rui Zheng, Jingzhen Sha, Te-Lin Wu, Hanyu Zhou, Mohan Tang, Kai-Wei Chang, Nanyun Peng, Haoran Huang
We construct the DACO dataset, containing (1) 440 databases (of tabular data) collected from real-world scenarios, (2) ~2k query-answer pairs that can serve as weak supervision for model training, and (3) a concentrated but high-quality test set with human refined annotations that serves as our main evaluation benchmark.
no code implementations • 28 Feb 2024 • Fan Yin, Jayanth Srinivasa, Kai-Wei Chang
We study how to characterize and predict the truthfulness of texts generated from large language models (LLMs), which serves as a crucial step in building trust between humans and LLMs.
1 code implementation • 27 Feb 2024 • Xiao Liu, Zirui Wu, Xueqing Wu, Pan Lu, Kai-Wei Chang, Yansong Feng
To address this gap, we introduce the Quantitative Reasoning with Data (QRData) benchmark, aiming to evaluate Large Language Models' capability in statistical and causal reasoning with real-world data.
1 code implementation • 21 Feb 2024 • Di wu, Wasi Uddin Ahmad, Kai-Wei Chang
This study addresses the application of encoder-only Pre-trained Language Models (PLMs) in keyphrase generation (KPG) amidst the broader availability of domain-tailored encoder-only models compared to encoder-decoder models.
no code implementations • 20 Feb 2024 • Haibin Wu, Xuanjun Chen, Yi-Cheng Lin, Kai-Wei Chang, Ho-Lam Chung, Alexander H. Liu, Hung-Yi Lee
Neural audio codecs are initially introduced to compress audio data into compact codes to reduce transmission latency.
1 code implementation • 20 Feb 2024 • Haibin Wu, Huang-Cheng Chou, Kai-Wei Chang, Lucas Goncalves, Jiawei Du, Jyh-Shing Roger Jang, Chi-Chun Lee, Hung-Yi Lee
Speech emotion recognition (SER) is a pivotal technology for human-computer interaction systems.
1 code implementation • 20 Feb 2024 • Haibin Wu, Ho-Lam Chung, Yi-Cheng Lin, Yuan-Kuei Wu, Xuanjun Chen, Yu-Chi Pai, Hsiu-Hsuan Wang, Kai-Wei Chang, Alexander H. Liu, Hung-Yi Lee
The sound codec's dual roles in minimizing data transmission latency and serving as tokenizers underscore its critical importance.
no code implementations • 16 Feb 2024 • Yixin Wan, Kai-Wei Chang
Recent large-scale Text-To-Image (T2I) models such as DALLE-3 demonstrate great potential in new applications, but also face unprecedented fairness challenges.
1 code implementation • 31 Jan 2024 • Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie zhou, Kai-Wei Chang, Minlie Huang, Nanyun Peng
In this work, we investigate how LLMs' behavior (i. e., complying with or refusing user queries) is affected by safety prompts from the perspective of model representation.
no code implementations • 24 Jan 2024 • Rohan Wadhawan, Hritik Bansal, Kai-Wei Chang, Nanyun Peng
Our findings reveal a significant performance gap of 30. 8% between the best-performing LMM, GPT-4V(ision), and human capabilities using human evaluation indicating substantial room for improvement in context-sensitive text-rich visual reasoning.
1 code implementation • 19 Jan 2024 • Yiwei Wang, Muhao Chen, Nanyun Peng, Kai-Wei Chang
To enforce these constraints, we utilize a depth-first search to adaptively substitute new knowledge for the LLMs' original reasoning steps, greedily seeking the optimal path of multi-hop reasoning with new knowledge.
1 code implementation • 10 Jan 2024 • Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao liu, Heng Ji, Hongyi Wang, huan zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
1 code implementation • 10 Jan 2024 • Xiao Liu, Yansong Feng, Kai-Wei Chang
Motivated by the definition of probability of sufficiency (PS) in the causal literature, we proposeCASA, a zero-shot causality-driven argument sufficiency assessment framework.
1 code implementation • 9 Jan 2024 • Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, Nanyun Peng
One critical challenge that has emerged is the presence of hallucinations in the output of large language models (LLMs) due to false or outdated knowledge.
no code implementations • 19 Dec 2023 • Anaelia Ovalle, Ninareh Mehrabi, Palash Goyal, Jwala Dhamala, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Yuval Pinter, Rahul Gupta
Our paper is the first to link LLM misgendering to tokenization and deficient neopronoun grammar, indicating that LLMs unable to correctly treat neopronouns as pronouns are more prone to misgender.
no code implementations • NeurIPS 2023 • Kareem Ahmed, Kai-Wei Chang, Guy Van Den Broeck
Under such distributions, computing the likelihood of even simple constraints is #P-hard.
1 code implementation • 2 Dec 2023 • Cheng-Fu Yang, Haoyang Xu, Te-Lin Wu, Xiaofeng Gao, Kai-Wei Chang, Feng Gao
In this paper, we aim to tackle this problem with a unified framework consisting of an end-to-end trainable method and a planning algorithm.
1 code implementation • 16 Nov 2023 • Kuan-Hao Huang, I-Hung Hsu, Tanmay Parekh, Zhiyu Xie, Zixuan Zhang, Premkumar Natarajan, Kai-Wei Chang, Nanyun Peng, Heng Ji
In this work, we identify and address evaluation challenges, including inconsistency due to varying data assumptions or preprocessing steps, the insufficiency of current evaluation frameworks that may introduce dataset or data split bias, and the low reproducibility of some previous approaches.
no code implementations • 16 Nov 2023 • Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Jwala Dhamala, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta
With the recent surge of language models in different applications, attention to safety and robustness of these models has gained significant importance.
1 code implementation • 15 Nov 2023 • Hritik Bansal, Yonatan Bitton, Idan Szpektor, Kai-Wei Chang, Aditya Grover
Despite being (pre)trained on a massive amount of data, state-of-the-art video-language alignment models are not robust to semantically-plausible contrastive changes in the video captions.
1 code implementation • 9 Nov 2023 • Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
To foster generalizable agent learning, we collect large-scale, unified, and high-quality training annotations derived from diverse ground-truth reasoning rationales across various complex interactive tasks.
no code implementations • 8 Nov 2023 • Junyi Li, Ninareh Mehrabi, Charith Peris, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, Rahul Gupta
Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented.
1 code implementation • 1 Nov 2023 • Po-Nien Kung, Fan Yin, Di wu, Kai-Wei Chang, Nanyun Peng
Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models (LLMs) on a massive amount of diverse tasks with instructions.
1 code implementation • 30 Oct 2023 • Amita Kamath, Jack Hessel, Kai-Wei Chang
Recent vision-language (VL) models are powerful, but can they reliably distinguish "right" from "left"?
no code implementations • 23 Oct 2023 • Zhecan Wang, Long Chen, Haoxuan You, Keyang Xu, Yicheng He, Wenhao Li, Noel Codella, Kai-Wei Chang, Shih-Fu Chang
Vision-language (VL) understanding tasks evaluate models' comprehension of complex visual scenes through multiple-choice questions.
no code implementations • 19 Oct 2023 • Ming-Hao Hsu, Kai-Wei Chang, Shang-Wen Li, Hung-Yi Lee
Despite the success of ICL in NLP, little work is exploring the possibility of ICL in speech processing.
1 code implementation • 18 Oct 2023 • Cheng-Fu Yang, Yen-Chun Chen, Jianwei Yang, Xiyang Dai, Lu Yuan, Yu-Chiang Frank Wang, Kai-Wei Chang
Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents.
no code implementations • 16 Oct 2023 • Christina Chance, Da Yin, Dakuo Wang, Kai-Wei Chang
Using counterfactual data augmentation to the FairytaleQA dataset, we evaluate model robustness against swapped gender character information, and then mitigate learned biases by introducing counterfactual gender stereotypes during training time.
1 code implementation • 13 Oct 2023 • Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, Nanyun Peng
Through benchmarking evaluation on 2 popular LLMs- ChatGPT and Alpaca, we reveal significant gender biases in LLM-generated recommendation letters.
no code implementations • 13 Oct 2023 • Mingyu Derek Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng
Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance by observing its influence on another instance.
1 code implementation • 10 Oct 2023 • Di wu, Wasi Uddin Ahmad, Kai-Wei Chang
DeSel improves greedy search by an average of 4. 7% semantic F1 across five datasets.
1 code implementation • 8 Oct 2023 • Yixin Wan, Jieyu Zhao, Aman Chadha, Nanyun Peng, Kai-Wei Chang
Recent advancements in Large Language Models empower them to follow freeform instructions, including imitating generic or specific demographic personas in conversations.
no code implementations • 4 Oct 2023 • Kai-Wei Chang, Ming-Hsin Chen, Yun-Ping Lin, Jing Neng Hsu, Paul Kuo-Ming Huang, Chien-yu Huang, Shang-Wen Li, Hung-Yi Lee
Notably, in the low-resource scenario, prompting consistently outperforms adapter tuning.
1 code implementation • 3 Oct 2023 • Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, Jianfeng Gao
To bridge this gap, we present MathVista, a benchmark designed to combine challenges from diverse mathematical and visual tasks.
no code implementations • 25 Sep 2023 • Chun-Yi Kuan, Chen An Li, Tsu-Yuan Hsu, Tse-Yang Lin, Ho-Lam Chung, Kai-Wei Chang, Shuo-Yiin Chang, Hung-Yi Lee
This paper introduces a novel voice conversion (VC) model, guided by text instructions such as "articulate slowly with a deep tone" or "speak in a cheerful boyish voice".
no code implementations • 19 Sep 2023 • Fei Wang, Kuan-Hao Huang, Kai-Wei Chang, Muhao Chen
In this paper, we propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer of the multilingual pretrained language models without the help of such external data.
1 code implementation • 18 Sep 2023 • Chien-yu Huang, Ke-Han Lu, Shih-Heng Wang, Chi-Yuan Hsiao, Chun-Yi Kuan, Haibin Wu, Siddhant Arora, Kai-Wei Chang, Jiatong Shi, Yifan Peng, Roshan Sharma, Shinji Watanabe, Bhiksha Ramakrishnan, Shady Shehata, Hung-Yi Lee
To achieve comprehensive coverage of diverse speech tasks and harness instruction tuning, we invite the community to collaborate and contribute, facilitating the dynamic growth of the benchmark.
1 code implementation • 16 Sep 2023 • Tanmay Parekh, I-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, Nanyun Peng
Label projection, which involves obtaining translated labels and texts jointly, is essential for leveraging machine translation to facilitate cross-lingual transfer in structured prediction tasks.
no code implementations • 8 Aug 2023 • Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta
Here we propose an automatic red teaming framework that evaluates a given model and exposes its vulnerabilities against unsafe and inappropriate content generation.
1 code implementation • 3 Jul 2023 • Rui Sun, Zhecan Wang, Haoxuan You, Noel Codella, Kai-Wei Chang, Shih-Fu Chang
However, we find visual and textual fine-grained information, e. g., keywords in the sentence and objects in the image, can be fairly informative for semantics understanding.
1 code implementation • 24 Jun 2023 • Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang, Yejin Choi
We release our corpus of chain-of-thought samples and code.
no code implementations • 3 Jun 2023 • Haibin Wu, Kai-Wei Chang, Yuan-Kuei Wu, Hung-Yi Lee
In this paper, we present pioneering research that explores the application of prompt tuning to stimulate speech LMs for various generation tasks, within a unified framework called SpeechGen, with around 10M trainable parameters.
no code implementations • 2 Jun 2023 • Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin F. Yang, Kai-Wei Chang
In this paper, we study an interesting hypothesis: can we transfer the in-context learning ability from the language domain to VL domain?
2 code implementations • 31 May 2023 • Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, Cho-Jui Hsieh
The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users.
1 code implementation • 31 May 2023 • Chenghao Yang, Fan Yin, He He, Kai-Wei Chang, Xiaofei Ma, Bing Xiang
In practice, Shapley Values are often estimated with a small number of stochastic model evaluations.
1 code implementation • 30 May 2023 • Yu-Hsiang Wang, Huang-Yu Chen, Kai-Wei Chang, Winston Hsu, Hung-Yi Lee
In this paper, we introduce MiniSUPERB, a lightweight benchmark that efficiently evaluates SSL speech models with comparable results to SUPERB but lower computational costs significantly.
1 code implementation • 26 May 2023 • Kuan-Hao Huang, Varun Iyer, I-Hung Hsu, Anoop Kumar, Kai-Wei Chang, Aram Galstyan
Paraphrase generation is a long-standing task in natural language processing (NLP).
no code implementations • 26 May 2023 • Yixin Wan, Yuan Zhou, Xiulian Peng, Kai-Wei Chang, Yan Lu
To begin with, we are among the first to comprehensively investigate mainstream KD techniques on DNS models to resolve the two challenges.
1 code implementation • 26 May 2023 • Yixin Wan, Kuan-Hao Huang, Kai-Wei Chang
Existing fine-tuning methods for this task are costly as all the parameters of the model need to be updated during the training process.
1 code implementation • 24 May 2023 • Amita Kamath, Jack Hessel, Kai-Wei Chang
We first curate CompPrompts, a set of increasingly compositional image captions that VL models should be able to capture (e. g., single object, to object+property, to multiple interacting objects).
2 code implementations • 24 May 2023 • Haoxuan You, Rui Sun, Zhecan Wang, Long Chen, Gengyu Wang, Hammad A. Ayyubi, Kai-Wei Chang, Shih-Fu Chang
Specifically, IdealGPT utilizes an LLM to generate sub-questions, a VLM to provide corresponding sub-answers, and another LLM to reason to achieve the final answer.
1 code implementation • 23 May 2023 • Oscar Chew, Hsuan-Tien Lin, Kai-Wei Chang, Kuan-Hao Huang
Recent research has revealed that machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances.
1 code implementation • 23 May 2023 • Da Yin, Xiao Liu, Fan Yin, Ming Zhong, Hritik Bansal, Jiawei Han, Kai-Wei Chang
Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses.
no code implementations • 19 May 2023 • Anaelia Ovalle, Mehrab Beikzadeh, Parshan Teimouri, Kai-Wei Chang, Majid Sarrafzadeh
Large language models have been useful in expanding mental health care delivery.
no code implementations • 17 May 2023 • Anaelia Ovalle, Palash Goyal, Jwala Dhamala, Zachary Jaggers, Kai-Wei Chang, Aram Galstyan, Richard Zemel, Rahul Gupta
Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life.
1 code implementation • NeurIPS 2023 • Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao
At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response.
1 code implementation • 27 Mar 2023 • Di wu, Da Yin, Kai-Wei Chang
Despite the significant advancements in keyphrase extraction and keyphrase generation methods, the predominant approach for evaluation mainly relies on exact matching with human references.
no code implementations • 23 Mar 2023 • Haoxuan You, Mandy Guo, Zhecan Wang, Kai-Wei Chang, Jason Baldridge, Jiahui Yu
The field of vision and language has witnessed a proliferation of pre-trained foundation models.
no code implementations • 16 Mar 2023 • Anaelia Ovalle, Arjun Subramonian, Vagrant Gautam, Gilbert Gee, Kai-Wei Chang
Through a critical review of how intersectionality is discussed in 30 papers from the AI fairness literature, we deductively and inductively: 1) map how intersectionality tenets operate within the AI fairness paradigm and 2) uncover gaps between the conceptualization and operationalization of intersectionality.
1 code implementation • ICCV 2023 • Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin, Aditya Grover, Kai-Wei Chang
Multimodal contrastive pretraining has been used to train multimodal representation models, such as CLIP, on large amounts of paired image-text data.
no code implementations • 1 Mar 2023 • Kai-Wei Chang, Yu-Kai Wang, Hua Shen, Iu-thing Kang, Wei-Cheng Tseng, Shang-Wen Li, Hung-Yi Lee
For speech processing, SpeechPrompt shows its high parameter efficiency and competitive performance on a few speech classification tasks.
Ranked #17 on Spoken Language Understanding on Fluent Speech Commands (using extra training data)
no code implementations • 28 Feb 2023 • Kareem Ahmed, Kai-Wei Chang, Guy Van Den Broeck
Numerous neuro-symbolic approaches have recently been proposed typically with the goal of adding symbolic knowledge to the output layer of a neural network.
no code implementations • 24 Feb 2023 • Kuan-Po Huang, Tzu-hsun Feng, Yu-Kuan Fu, Tsu-Yuan Hsu, Po-Chieh Yen, Wei-Cheng Tseng, Kai-Wei Chang, Hung-Yi Lee
We tried two different aggregation techniques, layerwise-average and layerwise-concatenation, to the representations of different teacher models and found that the former was more effective.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +4
no code implementations • CVPR 2023 • Da Yin, Feng Gao, Govind Thattai, Michael Johnston, Kai-Wei Chang
A key goal for the advancement of AI is to develop technologies that serve the needs not just of one group but of all communities regardless of their geographical region.
1 code implementation • 20 Dec 2022 • Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, Kai-Wei Chang
Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life.
1 code implementation • 20 Dec 2022 • Jianfeng Chi, Wasi Uddin Ahmad, Yuan Tian, Kai-Wei Chang
Privacy policies provide individuals with information about their rights and how their personal information is handled.
1 code implementation • 20 Dec 2022 • Di wu, Wasi Uddin Ahmad, Kai-Wei Chang
However, there lacks a systematic study of how the two types of approaches compare and how different design choices can affect the performance of PLM-based models.
no code implementations • 14 Dec 2022 • Haoxuan You, Rui Sun, Zhecan Wang, Kai-Wei Chang, Shih-Fu Chang
We present a new commonsense task, Human-centric Commonsense Grounding, that tests the models' ability to ground individuals given the context descriptions about what happened before, and their mental/physical states or intentions.
1 code implementation • CVPR 2023 • Ziniu Hu, Ahmet Iscen, Chen Sun, ZiRui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, Alireza Fathi
REVEAL consists of four key components: the memory, the encoder, the retriever and the generator.
Ranked #9 on Visual Question Answering (VQA) on OK-VQA
no code implementations • 17 Nov 2022 • Ninareh Mehrabi, Palash Goyal, Apurv Verma, Jwala Dhamala, Varun Kumar, Qian Hu, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Rahul Gupta
Natural language often contains ambiguities that can lead to misinterpretation and miscommunication.
no code implementations • 16 Nov 2022 • Anaelia Ovalle, Sunipa Dev, Jieyu Zhao, Majid Sarrafzadeh, Kai-Wei Chang
Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm.
no code implementations • 15 Nov 2022 • Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, ZiYi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
Answering open-domain questions requires world knowledge about in-context entities.
no code implementations • 10 Nov 2022 • Zhecan Wang, Haoxuan You, Yicheng He, Wenhao Li, Kai-Wei Chang, Shih-Fu Chang
Visual commonsense understanding requires Vision Language (VL) models to not only understand image and text but also cross-reference in-between to fully integrate and achieve comprehension of the visual scene described.
no code implementations • 2 Nov 2022 • Kuan-Hao Huang, Varun Iyer, Anoop Kumar, Sriram Venkatapathy, Kai-Wei Chang, Aram Galstyan
In this paper, we demonstrate that leveraging Abstract Meaning Representations (AMR) can greatly improve the performance of unsupervised syntactically controlled paraphrase generation.
no code implementations • 28 Oct 2022 • Jieyu Zhao, Xuezhi Wang, Yao Qin, Jilin Chen, Kai-Wei Chang
Large pre-trained language models have shown remarkable performance over the past few years.
1 code implementation • 27 Oct 2022 • Hritik Bansal, Da Yin, Masoud Monajatipoor, Kai-Wei Chang
To this end, we introduce an Ethical NaTural Language Interventions in Text-to-Image GENeration (ENTIGEN) benchmark dataset to evaluate the change in image generations conditional on ethical interventions across three social axes -- gender, skin color, and culture.
Cultural Vocal Bursts Intensity Prediction Text-to-Image Generation
1 code implementation • 22 Oct 2022 • Fan Yin, Yao Li, Cho-Jui Hsieh, Kai-Wei Chang
Finally, our analysis shows that the two types of uncertainty provided by \textbf{ADDMU} can be leveraged to characterize adversarial examples and identify the ones that contribute most to model's robustness in adversarial training.
1 code implementation • 18 Oct 2022 • Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi, Tushar Khot, Kai-Wei Chang
How reliably can we trust the scores obtained from social bias benchmarks as faithful indicators of problematic social biases in a given language model?
no code implementations • 16 Oct 2022 • Tzu-hsun Feng, Annie Dong, Ching-Feng Yeh, Shu-wen Yang, Tzu-Quan Lin, Jiatong Shi, Kai-Wei Chang, Zili Huang, Haibin Wu, Xuankai Chang, Shinji Watanabe, Abdelrahman Mohamed, Shang-Wen Li, Hung-Yi Lee
We present the SUPERB challenge at SLT 2022, which aims at learning self-supervised speech representation for better performance, generalization, and efficiency.
no code implementations • 14 Oct 2022 • Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, Cho-Jui Hsieh
Large pre-trained language models (PLMs) have proven to be a crucial component of modern natural language processing systems.
no code implementations • 7 Oct 2022 • Jwala Dhamala, Varun Kumar, Rahul Gupta, Kai-Wei Chang, Aram Galstyan
We present a systematic analysis of the impact of decoding algorithms on LM fairness, and analyze the trade-off between fairness, diversity and quality.
2 code implementations • 29 Sep 2022 • Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, Ashwin Kalyan
However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data.
1 code implementation • 20 Sep 2022 • Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, Ashwin Kalyan
We further design language models to learn to generate lectures and explanations as the chain of thought (CoT) to mimic the multi-hop reasoning process when answering ScienceQA questions.
Ranked #6 on Science Question Answering on ScienceQA
1 code implementation • Findings (ACL) 2022 • Cenyuan Zhang, Xiang Zhou, Yixin Wan, Xiaoqing Zheng, Kai-Wei Chang, Cho-Jui Hsieh
Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models.
1 code implementation • 1 Jun 2022 • Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van Den Broeck, Antonio Vergari
We design a predictive layer for structured-output prediction (SOP) that can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints.
1 code implementation • 27 May 2022 • Tao Meng, Sidi Lu, Nanyun Peng, Kai-Wei Chang
We propose a general and efficient framework to control auto-regressive generation models with NeurAlly-Decomposed Oracle (NADO).
1 code implementation • 25 May 2022 • Tanmay Parekh, I-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, Nanyun Peng
We utilize this ontology to further introduce GENEVA, a diverse generalizability benchmarking dataset comprising four test suites, aimed at evaluating models' ability to handle limited data and unseen event type generalization.
1 code implementation • 25 May 2022 • I-Hung Hsu, Kuan-Hao Huang, Shuning Zhang, Wenxin Cheng, Premkumar Natarajan, Kai-Wei Chang, Nanyun Peng
In this work, we propose to take a unified view of all these tasks and introduce TAGPRIME to address relational structure extraction problems.
no code implementations • 25 May 2022 • Jingnong Qu, Liunian Harold Li, Jieyu Zhao, Sunipa Dev, Kai-Wei Chang
Disinformation has become a serious problem on social media.
1 code implementation • 24 May 2022 • Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, Kai-Wei Chang
In this paper, we introduce a benchmark dataset, Geo-Diverse Commonsense Multilingual Language Models Analysis (GeoMLAMA), for probing the diversity of the relational knowledge in multilingual PLMs.
1 code implementation • 23 May 2022 • Jianfeng Chi, William Shand, Yaodong Yu, Kai-Wei Chang, Han Zhao, Yuan Tian
Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data.
1 code implementation • 23 May 2022 • Honghua Zhang, Liunian Harold Li, Tao Meng, Kai-Wei Chang, Guy Van Den Broeck
Logical reasoning is needed in a wide range of NLP tasks.
1 code implementation • 23 May 2022 • Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang
In code generation, the model learns to do the opposite.
no code implementations • 22 Apr 2022 • Zhecan Wang, Noel Codella, Yen-Chun Chen, Luowei Zhou, Xiyang Dai, Bin Xiao, Jianwei Yang, Haoxuan You, Kai-Wei Chang, Shih-Fu Chang, Lu Yuan
Experiments demonstrate that MAD leads to consistent gains in the low-shot, domain-shifted, and fully-supervised conditions on VCR, SNLI-VE, and VQA, achieving SOTA performance on VCR compared to other single models pretrained with image-text data.
Ranked #4 on Visual Question Answering (VQA) on VCR (Q-A) test
no code implementations • 19 Apr 2022 • Md Rizwan Parvez, Jianfeng Chi, Wasi Uddin Ahmad, Yuan Tian, Kai-Wei Chang
Prior studies in privacy policies frame the question answering (QA) task as identifying the most relevant text segment or a list of sentences from a policy document given a user query.
1 code implementation • 31 Mar 2022 • Kai-Wei Chang, Wei-Cheng Tseng, Shang-Wen Li, Hung-Yi Lee
We report in this paper the first exploration of the prompt tuning paradigm for speech processing tasks based on Generative Spoken Language Model (GSLM).
no code implementations • ACL 2022 • Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang, Rahul Gupta, Varun Kumar, Jwala Dhamala, Aram Galstyan
Multiple metrics have been introduced to measure fairness in various natural language processing tasks.
no code implementations • Findings (ACL) 2022 • Umang Gupta, Jwala Dhamala, Varun Kumar, Apurv Verma, Yada Pruksachatkun, Satyapriya Krishna, Rahul Gupta, Kai-Wei Chang, Greg Ver Steeg, Aram Galstyan
Language models excel at generating coherent text, and model compression techniques such as knowledge distillation have enabled their use in resource-constrained settings.
no code implementations • ACL 2022 • Satyapriya Krishna, Rahul Gupta, Apurv Verma, Jwala Dhamala, Yada Pruksachatkun, Kai-Wei Chang
With the rapid growth in language processing applications, fairness has emerged as an important consideration in data-driven solutions.
1 code implementation • ACL 2022 • Kuan-Hao Huang, I-Hung Hsu, Premkumar Natarajan, Kai-Wei Chang, Nanyun Peng
We present a study on leveraging multilingual pre-trained generative language models for zero-shot cross-lingual event argument extraction (EAE).
1 code implementation • 15 Mar 2022 • Di wu, Wasi Uddin Ahmad, Sunipa Dev, Kai-Wei Chang
State-of-the-art keyphrase generation methods generally depend on large annotated datasets, limiting their performance in domains with limited annotated data.
no code implementations • 17 Feb 2022 • Da Yin, Li Dong, Hao Cheng, Xiaodong Liu, Kai-Wei Chang, Furu Wei, Jianfeng Gao
With the increasing of model capacity brought by pre-trained language models, there emerges boosting needs for more knowledgeable natural language processing (NLP) models with advanced functionalities including providing and making flexible use of encyclopedic and commonsense knowledge.
no code implementations • 25 Jan 2022 • Kareem Ahmed, Eric Wang, Kai-Wei Chang, Guy Van Den Broeck
We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
no code implementations • 16 Dec 2021 • Zhecan Wang, Haoxuan You, Liunian Harold Li, Alireza Zareian, Suji Park, Yiqing Liang, Kai-Wei Chang, Shih-Fu Chang
As for pre-training, a scene-graph-aware pre-training method is proposed to leverage structure knowledge extracted in the visual scene graph.
2 code implementations • CVPR 2022 • Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, Jianfeng Gao
The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.
Ranked #1 on 2D Object Detection on RF100
2 code implementations • NAACL 2022 • Vijit Malik, Sunipa Dev, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang
Language representations are efficient tools used across NLP applications, but they are strife with encoded societal biases.
no code implementations • 14 Oct 2021 • Chien-yu Huang, Kai-Wei Chang, Hung-Yi Lee
However, in real-world scenarios, it is difficult to collect clean utterances of a speaker, and they are usually degraded by noises or reverberations.
no code implementations • Findings (EMNLP) 2021 • Ziniu Hu, Yizhou Sun, Kai-Wei Chang
Answering complex open-domain questions requires understanding the latent relations between involving entities.
1 code implementation • EMNLP 2021 • Da Yin, Liunian Harold Li, Ziniu Hu, Nanyun Peng, Kai-Wei Chang
Commonsense is defined as the knowledge that is shared by everyone.
Ranked #1 on Visual Commonsense Reasoning on GD-VCR
Cultural Vocal Bursts Intensity Prediction Visual Commonsense Reasoning
2 code implementations • NAACL 2022 • I-Hung Hsu, Kuan-Hao Huang, Elizabeth Boschee, Scott Miller, Prem Natarajan, Kai-Wei Chang, Nanyun Peng
Given a passage and a manually designed prompt, DEGREE learns to summarize the events mentioned in the passage into a natural sentence that follows a predefined pattern.
1 code implementation • EMNLP 2021 • Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, Cho-Jui Hsieh
Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models.
1 code implementation • EMNLP 2021 • Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff M Phillips, Kai-Wei Chang
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models.
1 code implementation • 26 Aug 2021 • Wasi Uddin Ahmad, Md Golam Rahman Tushar, Saikat Chakraborty, Kai-Wei Chang
Automating program translation is of paramount importance in software migration, and recently researchers explored unsupervised approaches due to the unavailability of parallel corpora.
1 code implementation • Findings (EMNLP) 2021 • Md Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang
To mimic developers' code or summary generation behavior, we propose a retrieval augmented framework, REDCODER, that retrieves relevant code or summaries from a retrieval database and provides them as a supplement to code generation or summarization models.
Ranked #1 on Code Generation on CodeXGLUE - CodeSearchNet (using extra training data)
1 code implementation • 10 Aug 2021 • Masoud Monajatipoor, Mozhdeh Rouhsedaghat, Liunian Harold Li, Aichi Chien, C. -C. Jay Kuo, Fabien Scalzo, Kai-Wei Chang
Vision-and-language(V&L) models take image and text as input and learn to capture the associations between them.
no code implementations • 7 Aug 2021 • Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang
Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality.
1 code implementation • ACL 2021 • Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.
4 code implementations • 13 Jul 2021 • Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, Kurt Keutzer
Most existing Vision-and-Language (V&L) models rely on pre-trained visual encoders, using a relatively small set of manually-annotated data (as compared to web-crawled data), to perceive the visual world.
Ranked #4 on Vision and Language Navigation on RxR (using extra training data)
1 code implementation • 28 Jun 2021 • Alina Arseniev-Koehler, Susan D. Cochran, Vickie M. Mays, Kai-Wei Chang, Jacob Gates Foster
Our method offers a flexible and broadly applicable approach to model topics in text data.
no code implementations • 23 Jun 2021 • Yichao Zhou, Chelsea Ju, J. Harry Caufield, Kevin Shih, Calvin Chen, Yizhou Sun, Kai-Wei Chang, Peipei Ping, Wei Wang
To facilitate various downstream applications using clinical case reports (CCRs), we pre-train two deep contextualized language models, Clinical Embeddings from Language Model (C-ELMo) and Clinical Contextual String Embeddings (C-Flair) using the clinical-related corpus from the PubMed Central.
no code implementations • Findings (ACL) 2021 • Yada Pruksachatkun, Satyapriya Krishna, Jwala Dhamala, Rahul Gupta, Kai-Wei Chang
Existing bias mitigation methods to reduce disparities in model outcomes across cohorts have focused on data augmentation, debiasing model embeddings, or adding fairness-based optimization objectives during training.
1 code implementation • ACL 2021 • Wasi Uddin Ahmad, Haoran Li, Kai-Wei Chang, Yashar Mehdad
In recent years, we have seen a colossal effort in pre-training multilingual text encoders using large-scale corpora in many languages to facilitate cross-lingual transfer learning.
1 code implementation • Findings (ACL) 2021 • Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Kai-Wei Chang
We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social stereotypes.
no code implementations • NAACL 2021 • Emily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun Peng
Ad hominem attacks are those that target some feature of a person{'}s character instead of the position the person is maintaining.
1 code implementation • ACL 2021 • Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng
Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner.
1 code implementation • 7 May 2021 • Yi-Chen Chen, Po-Han Chi, Shu-wen Yang, Kai-Wei Chang, Jheng-Hao Lin, Sung-Feng Huang, Da-Rong Liu, Chi-Liang Liu, Cheng-Kuang Lee, Hung-Yi Lee
The multi-task learning of a wide variety of speech processing tasks with a universal model has not been studied.
no code implementations • NAACL 2021 • Ankith Uppunda, Susan D. Cochran, Jacob G. Foster, Alina Arseniev-Koehler, Vickie M. Mays, Kai-Wei Chang
Coreference resolution is an important component in analyzing narrative text from administrative data (e. g., clinical or police sources).
1 code implementation • NAACL 2021 • Md Rizwan Parvez, Kai-Wei Chang
Transfer learning that adapts a model trained on data-rich sources to low-resource targets has been widely applied in natural language processing (NLP).
1 code implementation • 18 Apr 2021 • Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, Nanyun Peng
Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives.
1 code implementation • ACL 2022 • Fan Yin, Zhouxing Shi, Cho-Jui Hsieh, Kai-Wei Chang
We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existed removal-based criteria.
1 code implementation • EMNLP 2021 • Kuan-Hao Huang, Wasi Uddin Ahmad, Nanyun Peng, Kai-Wei Chang
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer.
1 code implementation • NAACL 2021 • Chong Zhang, Jieyu Zhao, huan zhang, Kai-Wei Chang, Cho-Jui Hsieh
Our method is able to reveal the hidden model biases not directly shown in the test dataset.
1 code implementation • NAACL 2021 • James Y. Huang, Kuan-Hao Huang, Kai-Wei Chang
In this work, we present ParaBART, a semantic sentence embedding model that learns to disentangle semantics and syntax in sentence embeddings obtained by pre-trained language models.
no code implementations • 20 Mar 2021 • Kareem Ahmed, Eric Wang, Guy Van Den Broeck, Kai-Wei Chang
We study the problem of entity-relation extraction in the presence of symbolic domain knowledge.
1 code implementation • NAACL 2021 • Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang
Experiments on code summarization in the English language, code generation, and code translation in seven programming languages show that PLBART outperforms or rivals state-of-the-art models.
no code implementations • 28 Feb 2021 • Yichao Zhou, Wei-Ting Chen, BoWen Zhang, David Lee, J. Harry Caufield, Kai-Wei Chang, Yizhou Sun, Peipei Ping, Wei Wang
Clinical case reports are written descriptions of the unique aspects of a particular clinical case, playing an essential role in sharing clinical experiences about atypical disease phenotypes and new therapies.
1 code implementation • 27 Jan 2021 • Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, Rahul Gupta
To systematically study and benchmark social biases in open-ended language generation, we introduce the Bias in Open-Ended Language Generation Dataset (BOLD), a large-scale dataset that consists of 23, 679 English text generation prompts for bias benchmarking across five domains: profession, gender, race, religion, and political ideology.
1 code implementation • EACL 2021 • Kuan-Hao Huang, Kai-Wei Chang
We also demonstrate that the performance of SynPG is competitive or even better than supervised models when the unannotated data is large.
1 code implementation • ACL 2021 • Wasi Uddin Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, Kai-Wei Chang
We refer to predicting the privacy practice explained in a sentence as intent classification and identifying the text spans sharing specific information as slot filling.
2 code implementations • 16 Dec 2020 • Yichao Zhou, Yu Yan, Rujun Han, J. Harry Caufield, Kai-Wei Chang, Yizhou Sun, Peipei Ping, Wei Wang
There has been a steady need in the medical community to precisely extract the temporal relations between clinical events.
1 code implementation • Asian Chapter of the Association for Computational Linguistics 2020 • Kuan-Hao Huang, Chen Li, Kai-Wei Chang
To deeply study this task, we present SportsSum, a Chinese sports game summarization dataset which contains 5, 428 soccer games of live commentaries and the corresponding news articles.
no code implementations • 17 Nov 2020 • Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Lu Liu, Yi Zhou, Jianhan Xu, Xiaoqing Zheng, Kai-Wei Chang, Xuanjing Huang
The words in each sentence of a source language corpus are rearranged to meet the word order in a target language under the guidance of a part-of-speech based language model (LM).
1 code implementation • 24 Oct 2020 • Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng
Ad hominem attacks are those that target some feature of a person's character instead of the position the person is maintaining.
1 code implementation • NAACL 2021 • Liunian Harold Li, Haoxuan You, Zhecan Wang, Alireza Zareian, Shih-Fu Chang, Kai-Wei Chang
Pre-trained contextual vision-and-language (V&L) models have achieved impressive performance on various benchmarks.
1 code implementation • EMNLP 2020 • Jieyu Zhao, Kai-Wei Chang
Machine learning techniques have been widely used in natural language processing (NLP).
1 code implementation • 6 Oct 2020 • Wasi Uddin Ahmad, Nanyun Peng, Kai-Wei Chang
Recent progress in cross-lingual relation and event extraction use graph convolutional networks (GCNs) with universal dependency parses to learn language-agnostic sentence representations such that models trained on one language can be applied to other languages.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Wasi Uddin Ahmad, Jianfeng Chi, Yuan Tian, Kai-Wei Chang
Prior studies in this domain frame the QA task as retrieving the most relevant text segment or a list of sentences from the policy document given a question.
no code implementations • ACL 2021 • Wasi Uddin Ahmad, Xiao Bai, Soomin Lee, Kai-Wei Chang
Natural language processing techniques have demonstrated promising results in keyphrase generation.
no code implementations • ACL 2020 • Yichao Zhou, Jyun-Yu Jiang, Jieyu Zhao, Kai-Wei Chang, Wei Wang
In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence.
no code implementations • ACL 2020 • Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang
Pre-trained visually grounded language models such as ViLBERT, LXMERT, and UNITER have achieved significant performance improvement on vision-and-language tasks but what they learn during pre-training remains unclear.
2 code implementations • 27 Jun 2020 • Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, Yizhou Sun
Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data.
1 code implementation • 20 Jun 2020 • Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples.
1 code implementation • 18 Jun 2020 • Tao Meng, Kai-Wei Chang
This raises a question -- \emph{can we mine constraints and rules from data based on a learning algorithm?}
1 code implementation • ACL 2020 • Shengyu Jia, Tao Meng, Jieyu Zhao, Kai-Wei Chang
With little performance loss, our method can almost remove the bias amplification in the distribution.
1 code implementation • ACL 2020 • Fan Yin, Quanyu Long, Tao Meng, Kai-Wei Chang
We conduct a thorough study to diagnose the behaviors of pre-trained language encoders (ELMo, BERT, and RoBERTa) when confronted with natural grammatical errors.
2 code implementations • ACL 2020 • Da Yin, Tao Meng, Kai-Wei Chang
We propose SentiBERT, a variant of BERT that effectively captures compositional sentiment semantics.
1 code implementation • ACL 2020 • Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, Ahmed Hassan Awadallah
In this paper, we study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications.
2 code implementations • Findings of the Association for Computational Linguistics 2020 • Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng
We present a general approach towards controllable societal biases in natural language generation (NLG).
9 code implementations • ACL 2020 • Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang
Generating a readable summary that describes the functionality of a program is known as source code summarization.
1 code implementation • 29 Apr 2020 • Yichao Zhou, Jyun-Yu Jiang, Jieyu Zhao, Kai-Wei Chang, Wei Wang
In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence.
5 code implementations • NeurIPS 2020 • Kaidi Xu, Zhouxing Shi, huan zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, Cho-Jui Hsieh
Linear relaxation based perturbation analysis (LiRPA) for neural networks, which computes provable linear bounds of output neurons given a certain amount of input perturbation, has become a core component in robustness verification and certified defense.
1 code implementation • ICLR 2020 • Zhouxing Shi, huan zhang, Kai-Wei Chang, Minlie Huang, Cho-Jui Hsieh
Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees.
1 code implementation • ACL 2020 • Andrew Gaut, Tony Sun, Shirlyn Tang, Yuxin Huang, Jing Qian, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang Wang
We use WikiGenderBias to evaluate systems for bias and find that NRE systems exhibit gender biased predictions and lay groundwork for future evaluation of bias in NRE.
1 code implementation • CONLL 2019 • Wasi Uddin Ahmad, Zhisong Zhang, Xuezhe Ma, Kai-Wei Chang, Nanyun Peng
We conduct experiments on cross-lingual dependency parsing where we train a dependency parser on a source language and transfer it to a wide range of target languages.
1 code implementation • IJCNLP 2019 • Chen Xia, Haoxiang Zhang, Jacob Moghtader, Allen Wu, Kai-Wei Chang
There are tons of news articles generated every day reflecting the activities of key roles such as people, organizations and political parties.
no code implementations • IJCNLP 2019 • Weijia Shi, Muhao Chen, Pei Zhou, Kai-Wei Chang
Contextualized word embedding models, such as ELMo, generate meaningful representations of words and their context.
no code implementations • 10 Sep 2019 • Zhenxin Xiao, Puyudi Yang, Yuchen Jiang, Kai-Wei Chang, Cho-Jui Hsieh
Adversarial example generation becomes a viable method for evaluating the robustness of a machine learning model.
1 code implementation • IJCNLP 2019 • Yichao Zhou, Jyun-Yu Jiang, Kai-Wei Chang, Wei Wang
To identify adversarial attacks, a perturbation discriminator validates how likely a token in the text is perturbed and provides a set of potential perturbations.
1 code implementation • IJCNLP 2019 • Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, Kai-Wei Chang
Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora.
1 code implementation • IJCNLP 2019 • Tao Meng, Nanyun Peng, Kai-Wei Chang
Experiments show that the Lagrangian relaxation and posterior regularization inference improve the performances on 15 and 17 out of 19 target languages, respectively.
1 code implementation • IJCNLP 2019 • Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng
We present a systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups.
7 code implementations • 9 Aug 2019 • Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang
We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.
Ranked #1 on Visual Reasoning on NLVR
1 code implementation • ACL 2019 • Ziniu Hu, Ting Chen, Kai-Wei Chang, Yizhou Sun
Existing approaches for learning word embeddings often assume there are sufficient occurrences for each word in the corpus, such that the representation of words can be accurately estimated from their contexts.
no code implementations • WS 2019 • Weijia Shi, Muhao Chen, Yingtao Tian, Kai-Wei Chang
Bilingual word embeddings, which representlexicons of different languages in a shared em-bedding space, are essential for supporting se-mantic and knowledge transfers in a variety ofcross-lingual NLP tasks.
1 code implementation • ACL 2019 • Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang Wang
In this paper, we review contemporary studies on recognizing and mitigating gender bias in NLP.
5 code implementations • 5 Jun 2019 • Wasi Uddin Ahmad, Kai-Wei Chang, Hongning Wang
We present a context-aware neural ranking model to exploit users' on-task search activities and enhance retrieval performance.
no code implementations • 31 May 2019 • Ziniu Hu, Changjun Fan, Ting Chen, Kai-Wei Chang, Yizhou Sun
With the proposed pre-training procedure, the generic structural information is learned and preserved, thus the pre-trained GNN requires less amount of labeled data and fewer domain-specific features to achieve high performance on different downstream tasks.
no code implementations • 10 May 2019 • Everett Fall, Kai-Wei Chang, Liang-Gee Chen
Marked progress has been made in video quality, compression, and computational efficiency.
2 code implementations • NAACL 2019 • Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, Kai-Wei Chang
In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo's contextualized word vectors.
no code implementations • TACL 2019 • Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang
Contextual representation models have achieved great success in improving various downstream natural language processing tasks.
no code implementations • 28 Feb 2019 • Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.