no code implementations • CCL 2021 • Libo Geng, Zixuan Xue, Yiming Yang
“文章使用ERP技术, 对比分析了安静、白噪声、汉语噪声、英语噪声四种听觉条件下, 听力正常的汉语母语者加工汉语句子的情况, 以探究信息掩蔽条件下语义加工的神经机制。研究发现不同噪声条件下诱发的N100、N400、LPC等ERPs成分具有不同的波形表现, 据此本文得出以下结论:首先, 在语音掩蔽条件下, 对于难度较大的语义加工, 目标语音与掩蔽噪声在知觉层面的相似性并非主要影响因素, 而掩蔽噪声语义内容上的可懂度发挥着更关键的作用。其次, 当言语噪声为听者极其熟悉或完全陌生的语言, 其对语义加工的掩蔽干扰较小, 而当掩蔽噪声为听者接触过的语言但不是母语或主要语言, 其掩蔽效应可能更强。最后, 不熟悉的言语噪声中所包含的出现频率较小但能够被听者理解的语义内容, 与听者的预期相冲突, 引发听者的注意转移, 这些语义信息被传输至听觉中枢神经, 占用了原本用于目标刺激的认知资源, 从而增强了信息掩蔽的效果。”
1 code implementation • EMNLP 2021 • Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Peter Clark, Yiming Yang, Eduard Hovy
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
1 code implementation • 25 Jan 2025 • Yiqun Chen, Lingyong Yan, Weiwei Sun, Xinyu Ma, Yi Zhang, Shuaiqiang Wang, Dawei Yin, Yiming Yang, Jiaxin Mao
Specifically, we present MMOA-RAG, a Multi-Module joint Optimization Algorithm for RAG, which employs multi-agent reinforcement learning to harmonize all agents' goals towards a unified reward, such as the F1 score of the final answer.
no code implementations • 24 Jan 2025 • Haowei Lin, Shanda Li, Haotian Ye, Yiming Yang, Stefano Ermon, Yitao Liang, Jianzhu Ma
Given an unconditional generative model and a predictor for a target property (e. g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training.
no code implementations • 23 Jan 2025 • Yiming Yang, Xiaoyuan Cheng, Daniel Giles, Sibo Cheng, Yi He, Xiao Xue, Boli Chen, Yukun Hu
In this paper, we propose \textit{Tensor-Var} to address these challenges using kernel Conditional Mean Embedding (CME).
no code implementations • 26 Dec 2024 • Ruixing Wu, Yiming Yang, Jiakai He, Haifeng Hu
Unsupervised learning visible-infrared person re-identification (USL-VI-ReID) aims to learn modality-invariant features from unlabeled cross-modality datasets and reduce the inter-modality gap.
no code implementations • 20 Dec 2024 • Shengyu Feng, Yiming Yang
It utilizes suboptimal demonstrations through both offline reinforcement learning on the demonstrations generated by suboptimal heuristics and self-imitation learning on past good experiences sampled by itself.
no code implementations • 11 Dec 2024 • Wenxuan Sun, Zixuan Yang, Yunli Wang, Zhen Zhang, Zhiqiang Wang, Yu Li, Jian Yang, Yiming Yang, Shiyang Wen, Peng Jiang, Kun Gai
To the best of our knowledge, Adaptive$^2$ is the first approach to automatically learn both domain identification and adaptation in online advertising, opening new research directions for this area.
no code implementations • 11 Dec 2024 • Yiming Yang, Weipeng Hu, Haifeng Hu
Specifically, our DMIC naturally integrates Modality-Camera Invariant Expansion (MIE), Dynamic Neighborhood Clustering (DNC) and Hybrid Modality Contrastive Learning (HMCL) into a unified framework, which eliminates both the cross-modality and cross-camera discrepancies in clustering.
no code implementations • 25 Nov 2024 • Peng Cui, Yiming Yang, Fusheng Jin, Siyuan Tang, Yunli Wang, Fukang Yang, Yalong Jia, Qingpeng Cai, Fei Pan, Changcheng Li, Peng Jiang
To alleviate the issue of discontinuity in one-hot hard labels, the Bucket Classification Module with label Smoothing method (BCMS) converts one-hot hard labels into non-normalized soft labels, then fits these soft labels by minimizing classification loss and regression loss.
2 code implementations • 21 Oct 2024 • Ruohong Zhang, BoWen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, Yiming Yang
This work emphasizes the importance of incorporating detailed rationales in training and leveraging reinforcement learning to strengthen the reasoning capabilities of VLMs.
no code implementations • 2 Oct 2024 • Shengyu Feng, Xiang Kong, Shuang Ma, Aonan Zhang, Dong Yin, Chong Wang, Ruoming Pang, Yiming Yang
Augmenting the multi-step reasoning abilities of Large Language Models (LLMs) has been a persistent challenge.
no code implementations • 1 Aug 2024 • Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, Yiming Yang
As a first step towards understanding and designing compute-optimal inference methods, we studied cost-performance trade-offs for inference strategies such as greedy search, majority voting, best-of-$n$, weighted voting, and two different tree search algorithms, using different model sizes and compute budgets.
no code implementations • 14 Jul 2024 • Haohan Lin, Zhiqing Sun, Yiming Yang, Sean Welleck
We present Lean-STaR, a framework for training language models to produce informal thoughts prior to each step of a proof, thereby boosting the model's theorem-proving capabilities.
1 code implementation • 26 Jun 2024 • Jaehyung Kim, Yiming Yang
As the diversity of users increases, the capability of providing personalized responses by large language models (LLMs) has become increasingly important.
1 code implementation • 26 Jun 2024 • Jaehyung Kim, Dongyoung Kim, Yiming Yang
It uses a trained adaptation model to perform a seq2seq mapping from the often-imperfect reasonings of the original black-box LLM to the correct or improved reasonings.
1 code implementation • 1 May 2024 • Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, Quanquan Gu
In this paper, we propose a self-play-based method for language model alignment, which treats the problem as a constant-sum two-player game aimed at identifying the Nash equilibrium policy.
no code implementations • 30 Apr 2024 • Yiming Yang, Mohammad Vaseem, Ruiqi Wang, Behrooz Makki, Atif Shamim
The RIS array comprises 20x20 unit cells (4. 54x4. 54{\lambda}^2 at 29. 5 GHz).
1 code implementation • 1 Apr 2024 • Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, Yiming Yang
Preference modeling techniques, such as direct preference optimization (DPO), has shown effective in enhancing the generalization abilities of large language model (LLM).
1 code implementation • 14 Mar 2024 • Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, Chuang Gan
This paper answers this question in the context of tackling hard reasoning tasks (e. g., level 4-5 MATH problems) via learning from human annotations on easier tasks (e. g., level 1-3 MATH problems), which we term as easy-to-hard generalization.
1 code implementation • 7 Mar 2024 • Zhiying Zhu, Yiming Yang, Zhiqing Sun
Hallucinations pose a significant challenge to the reliability of large language models (LLMs) in critical domains.
1 code implementation • 8 Feb 2024 • Tianjun Zhang, Aman Madaan, Luyu Gao, Steven Zheng, Swaroop Mishra, Yiming Yang, Niket Tandon, Uri Alon
We evaluate LEAP on a wide range of benchmarks, including multi-hop question answering (Hotpot QA), textual QA (DROP), Big-Bench Hard reasoning, and math problems (GSM8K and MATH); in all these benchmarks, LEAP improves the strongest available LLMs such as GPT-3. 5-turbo, GPT-4, GPT-4 turbo and Claude-2. 1.
no code implementations • 16 Jan 2024 • Syeda Nahida Akter, Aman Madaan, Sangwu Lee, Yiming Yang, Eric Nyberg
The potential of Vision-Language Models (VLMs) often remains underutilized in handling complex text-based problems, particularly when these problems could benefit from visual representation.
no code implementations • 4 Dec 2023 • Xiaoyuan Cheng, Yiming Yang, Wei Jiang, Yukun Hu
The core idea is to establish an equivariant geometry that is diffeomorphic to the manifold defined by a dynamical system and to perform optimal control within this corresponding geometry, which is a non-trivial task.
no code implementations • 17 Nov 2023 • Ruohong Zhang, Luyu Gao, Chen Zheng, Zhen Fan, Guokun Lai, Zheng Zhang, Fangzhou Ai, Yiming Yang, Hongxia Yang
This paper introduces a novel approach to enhance LLMs by effectively extracting the relevant knowledge from domain-specific textual sources, and the adaptive training of a chatbot with domain-specific inquiries.
1 code implementation • 19 Oct 2023 • Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, Mausam
Large language models (LLMs) are now available from cloud API providers in various sizes and configurations.
1 code implementation • 9 Oct 2023 • Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Cox, Yiming Yang, Chuang Gan
Supervised Fine-Tuning (SFT) on response demonstrations combined with Reinforcement Learning from Human Feedback (RLHF) constitutes a powerful paradigm for aligning LLM-based AI agents.
no code implementations • 6 Oct 2023 • Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontanon, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Preventing the performance decay of Transformers on inputs longer than those used for training has been an important challenge in extending the context length of these models.
no code implementations • 25 Sep 2023 • Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, Trevor Darrell
Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounded by the multimodal information in context.
no code implementations • 12 Aug 2023 • Junwei Huang, Zhiqing Sun, Yiming Yang
Graph-based diffusion models have shown promising results in terms of generating high-quality solutions to NP-complete (NPC) combinatorial optimization (CO) problems.
1 code implementation • 7 Aug 2023 • Renjie Liang, Yiming Yang, Hui Lu, Li Li
To tackle this problem, we propose a novel efficient multi-teacher model (EMTM) based on knowledge distillation to transfer diverse knowledge from both heterogeneous and isomorphic networks.
1 code implementation • 22 Jul 2023 • Qingyang Zhang, Yiming Yang, Jingqing Ruan, Xuantang Xiong, Dengpeng Xing, Bo Xu
However, existing works often overlook the temporal coherence in GCHRL when learning latent subgoal representations and lack an efficient subgoal selection strategy that balances exploration and exploitation.
no code implementations • 24 May 2023 • Yau-Shian Wang, Ta-Chung Chi, Ruohong Zhang, Yiming Yang
We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification.
1 code implementation • 22 May 2023 • Long Yang, Zhixiong Huang, Fenghao Lei, Yucun Zhong, Yiming Yang, Cong Fang, Shiting Wen, Binbin Zhou, Zhouchen Lin
Popular reinforcement learning (RL) algorithms tend to produce a unimodal policy distribution, which weakens the expressiveness of complicated policy and decays the ability of exploration.
1 code implementation • 19 May 2023 • Pranjal Aggarwal, Aman Madaan, Yiming Yang, Mausam
A popular approach for improving the correctness of output from large language models (LLMs) is Self-Consistency - poll the LLM multiple times and output the most frequent solution.
2 code implementations • 11 May 2023 • Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig
In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation.
1 code implementation • NeurIPS 2023 • Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, Chuang Gan
Recent AI-assistant agents, such as ChatGPT, predominantly rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback (RLHF) to align the output of large language models (LLMs) with human intentions, ensuring they are helpful, ethical, and reliable.
1 code implementation • 24 Apr 2023 • Ruohong Zhang, Yau-Shian Wang, Yiming Yang
To overcome these limitations, we introduce a novel method, namely GenCo, which leverages the strong generative power of LLMs to assist in training a smaller and more adaptable language model.
no code implementations • 23 Apr 2023 • Ruiqi Wang, Yiming Yang, Behrooz Makki, Atif Shamim
The measurement results demonstrate that the proposed RIS could maintain a 3 dB peak gain variation bandwidth among various array configurations within 22. 5 to 29. 5 GHz (26. 9%) and with a beam scanning capability of 50{\deg}, making this design a good candidate for 5G mm-wave applications.
3 code implementations • NeurIPS 2023 • Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, Peter Clark
Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement.
1 code implementation • 16 Feb 2023 • Zhiqing Sun, Yiming Yang, Shinjae Yoo
Numerical simulation of non-linear partial differential equations plays a crucial role in modeling physical science and engineering phenomena, such as weather, climate, and aerodynamics.
1 code implementation • NeurIPS 2023 • Zhiqing Sun, Yiming Yang
We evaluate our methods on two well-studied NPC combinatorial optimization problems: Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS).
no code implementations • 7 Feb 2023 • Yiming Yang, Ruiqi Wang, Mohammad Vaseem, Behrooz Makki, Atif Shamim
In this paper, we propose a via-less fully screen-printed reconfigurable intelligent surface which can establish a second line-of-sight communication from 23. 5GHz to 29. 5GHz.
no code implementations • 3 Feb 2023 • Krzysztof Marcin Choromanski, Shanda Li, Valerii Likhosherstov, Kumar Avinava Dubey, Shengjie Luo, Di He, Yiming Yang, Tamas Sarlos, Thomas Weingarten, Adrian Weller
We propose a new class of linear Transformers called FourierLearner-Transformers (FLTs), which incorporate a wide range of relative positional encoding mechanisms (RPEs).
3 code implementations • 18 Nov 2022 • Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, PengFei Liu, Yiming Yang, Jamie Callan, Graham Neubig
Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem.
Ranked #43 on
Math Word Problem Solving
on MATH
2 code implementations • 13 Oct 2022 • Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, Graham Neubig
In all these natural language tasks, we show that using our approach, a code generation LM (CODEX) outperforms natural-LMs that are fine-tuned on the target task (e. g., T5) and other strong LMs such as GPT-3 in the few-shot setting.
1 code implementation • 8 Oct 2022 • Ruizhong Qiu, Zhiqing Sun, Yiming Yang
Recently, deep reinforcement learning (DRL) models have shown promising results in solving NP-hard Combinatorial Optimization (CO) problems.
1 code implementation • 4 Oct 2022 • Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, Denny Zhou
We propose a new paradigm to help Large Language Models (LLMs) generate more accurate factual knowledge without retrieving from an external corpus, called RECITation-augmented gEneration (RECITE).
no code implementations • 15 Jul 2022 • Aman Madaan, Yiming Yang
Machine learning systems typically apply the same model to both easy and tough cases.
no code implementations • 25 May 2022 • Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, Antoine Bosselut
Conditional set generation learns a mapping from an input sequence of tokens to a set.
no code implementations • 2 Apr 2022 • Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Donghan Yu, Tom Vu, Likun Lei
Extreme Multi-label Text Classification (XMTC) has been a tough challenge in machine learning research and applications due to the sheer sizes of the label spaces and the severe data scarce problem associated with the long tail of rare labels in highly skewed distributions.
Multi Label Text Classification
Multi-Label Text Classification
+3
no code implementations • 2 Apr 2022 • Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Tom Vu, Likun Lei
Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels from a very large space of predefined categories.
Multi Label Text Classification
Multi-Label Text Classification
+1
1 code implementation • 31 Mar 2022 • Christian Eichenberger, Moritz Neun, Henry Martin, Pedro Herruzo, Markus Spanring, Yichao Lu, Sungbin Choi, Vsevolod Konyakhin, Nina Lukashina, Aleksei Shpilman, Nina Wiedemann, Martin Raubal, Bo wang, Hai L. Vu, Reza Mohajerpoor, Chen Cai, Inhi Kim, Luca Hermes, Andrew Melnik, Riza Velioglu, Markus Vieth, Malte Schilling, Alabi Bojesomo, Hasan Al Marzouqi, Panos Liatsis, Jay Santokhi, Dylan Hillier, Yiming Yang, Joned Sarwar, Anna Jordan, Emil Hewage, David Jonietz, Fei Tang, Aleksandra Gruca, Michael Kopp, David Kreil, Sepp Hochreiter
The IARAI Traffic4cast competitions at NeurIPS 2019 and 2020 showed that neural networks can successfully predict future traffic conditions 1 hour into the future on simply aggregated GPS probe data in time and space bins.
1 code implementation • 16 Jan 2022 • Aman Madaan, Niket Tandon, Peter Clark, Yiming Yang
Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to humans.
1 code implementation • Findings (NAACL) 2022 • Niket Tandon, Aman Madaan, Peter Clark, Yiming Yang
Our goal is for an LM to continue to improve after deployment, without retraining, using feedback from the user.
1 code implementation • 15 Dec 2021 • Niket Tandon, Aman Madaan, Peter Clark, Keisuke Sakaguchi, Yiming Yang
We present a new dataset, Interscript, containing user feedback on a deployed model that generates complex everyday tasks.
1 code implementation • 24 Oct 2021 • Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Peter Clark, Yiming Yang, Eduard Hovy
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
1 code implementation • 21 Oct 2021 • Jay Santokhi, Dylan Hillier, Yiming Yang, Joned Sarwar, Anna Jordan, Emil Hewage
The landscape of city-wide mobility behaviour has altered significantly over the past 18 months.
no code implementations • ACL 2022 • Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
The recent proposed Fusion-in-Decoder (FiD), which is built on top of the pretrained generative model T5, achieves the state-of-the-art performance in the reading module.
no code implementations • ICLR 2022 • Zhiqing Sun, Yiming Yang, Shinjae Yoo
To overcome these issues, this paper proposes a new strategy for sparse attention, namely LHA (Learning-to-Hash Attention), which directly learns separate parameterized hash functions for queries and keys, respectively.
1 code implementation • AKBC Workshop CSKB 2021 • Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, Eduard Hovy
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
no code implementations • 18 Apr 2021 • Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Yiming Yang, Peter Clark, Keisuke Sakaguchi, Ed Hovy
A class of explainable NLP models for reasoning tasks support their decisions by generating free-form or structured explanations, but what happens when these supporting structures contain errors?
1 code implementation • 16 Apr 2021 • Donghan Yu, Yiming Yang
Different from traditional knowledge graphs (KGs) where facts are represented as entity-relation-entity triplets, hyper-relational KGs (HKGs) allow triplets to be associated with additional relation-entity pairs (a. k. a qualifiers) to convey more complex information.
1 code implementation • CSRR (ACL) 2022 • Dheeraj Rajagopal, Aman Madaan, Niket Tandon, Yiming Yang, Shrimai Prabhumoye, Abhilasha Ravichander, Peter Clark, Eduard Hovy
Recently, models have been shown to predict the effects of unexpected situations, e. g., would cloudy skies help or hinder plant growth?
1 code implementation • ICLR 2021 • Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig
Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data.
no code implementations • 25 Nov 2020 • Yongquan Yang, Yiming Yang, Jie Chen, Jiayi Zheng, Zhongxi Zheng
Learning from noisy labels is an important concern in plenty of real-world scenarios.
1 code implementation • ICCV 2021 • Zhiqing Sun, Shengcao Cao, Yiming Yang, Kris Kitani
DETR is a recently proposed Transformer-based method which views object detection as a set prediction problem and achieves state-of-the-art performance but demands extra-long training time to converge.
3 code implementations • EMNLP 2020 • Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, Lei LI
Pre-trained contextual representations like BERT have achieved great success in natural language processing.
Ranked #16 on
Semantic Textual Similarity
on STS16
no code implementations • 22 Oct 2020 • Aman Madaan, Dheeraj Rajagopal, Yiming Yang, Abhilasha Ravichander, Eduard Hovy, Shrimai Prabhumoye
Reasoning about events and tracking their influences is fundamental to understanding processes.
1 code implementation • NAACL 2021 • Aman Madaan, Yiming Yang
We address this challenge by using existing IE/NLP tools to automatically generate a large quantity (89, 000) of system-produced document-graph pairs, and propose a novel formulation of the contextualized graph generation problem as a sequence-to-sequence mapping task.
no code implementations • 2 Oct 2020 • Donghan Yu, Chenguang Zhu, Yiming Yang, Michael Zeng
Knowledge graphs (KGs) contain rich information about world knowledge, entities and relations.
no code implementations • 18 Sep 2020 • Guokun Lai, Zihang Dai, Yiming Yang
In contrast, there is a large-scale of parallel corpus created by humans on the Internet.
no code implementations • 6 Jul 2020 • Wei-Cheng Chang, Chun-Liang Li, Youssef Mroueh, Yiming Yang
NCK is crucial for successful inference with SVGD in high dimension, as it adapts the kernel to the noise level of the score estimate.
1 code implementation • ICML 2020 • Zhiqing Sun, Yiming Yang
Autoregressive (AR) models have been the dominating approach to conditional sequence generation, but are suffering from the issue of high inference latency.
1 code implementation • 12 Jun 2020 • Donghan Yu, Yiming Yang, Ruohong Zhang, Yuexin Wu
Recently, a considerable literature has grown up around the theme of Graph Convolutional Network (GCN).
3 code implementations • NeurIPS 2020 • Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le
With the success of language pretraining, it is highly desirable to develop more efficient architectures of good scalability that can exploit the abundant unlabeled data at a lower cost.
Ranked #6 on
Reading Comprehension
on RACE
1 code implementation • ACL 2020 • Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu, Yiming Yang, Graham Neubig
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting.
1 code implementation • ACL 2020 • Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W. black, Shrimai Prabhumoye
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.
1 code implementation • 24 Apr 2020 • Ruohong Zhang, Yu Hao, Donghan Yu, Wei-Cheng Chang, Guokun Lai, Yiming Yang
Keywords: Multivariate Time Series, Change-point Detection, Graph Neural Networks
1 code implementation • 24 Apr 2020 • Aman Madaan, Shruti Rijhwani, Antonios Anastasopoulos, Yiming Yang, Graham Neubig
We propose a method of curating high-quality comparable training data for low-resource languages with monolingual annotators.
7 code implementations • ACL 2020 • Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, Denny Zhou
Then, we conduct knowledge transfer from this teacher to MobileBERT.
Ranked #20 on
Semantic Textual Similarity
on MRPC
1 code implementation • CVPR 2020 • Jingzhou Liu, Wenhu Chen, Yu Cheng, Zhe Gan, Licheng Yu, Yiming Yang, Jingjing Liu
We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text.
1 code implementation • AKBC 2020 • Zhengbao Jiang, Jun Araki, Donghan Yu, Ruohong Zhang, Wei Xu, Yiming Yang, Graham Neubig
We propose several methods that incorporate both structured and textual information to represent relations for this task.
no code implementations • ICLR 2020 • Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, Sanjiv Kumar
We consider the large-scale query-document retrieval problem: given a query (e. g., a question), return the set of relevant documents (e. g., paragraphs containing the answer) from a large document corpus.
4 code implementations • 17 Nov 2019 • Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, Yiming Yang
Graph Convolutional Networks (GCNs) have received increasing attention in the machine learning community for effectively leveraging both the content features of nodes and the linkage patterns across graphs in various applications.
2 code implementations • ACL 2020 • Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, Yiming Yang
Knowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs.
Ranked #25 on
Link Prediction
on FB15k-237
(MR metric)
no code implementations • 19 Oct 2019 • Yong-Siang Shih, Wei-Cheng Chang, Yiming Yang
While neural sequence generation models achieve initial success for many NLP applications, the canonical decoding procedure with left-to-right generation order (i. e., autoregressive) in one-pass can not reflect the true nature of human revising a sentence to obtain a refined result.
no code implementations • 16 Oct 2019 • Yuexin Wu, Yichong Xu, Aarti Singh, Yiming Yang, Artur Dubrawski
Graph Neural Networks (GNNs) for prediction tasks like node classification or edge prediction have received increasing attention in recent machine learning from graphically structured data.
2 code implementations • ICLR 2020 • Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang, Graham Neubig, Jaime Carbonell
Learning multilingual representations of text has proven a successful method for many cross-lingual transfer learning tasks.
no code implementations • 25 Sep 2019 • Yuexin Wu, Yichong Xu, Aarti Singh, Artur Dubrawski, Yiming Yang
Graph Neural Networks (GNNs) for prediction tasks like node classification or edge prediction have received increasing attention in recent machine learning from graphically structured data.
1 code implementation • 16 Sep 2019 • Guokun Lai, Barlas Oguz, Yiming Yang, Veselin Stoyanov
We consider the setting of semi-supervised cross-lingual understanding, where labeled data is available in a source language (English), but only unlabeled data is available in the target language.
1 code implementation • IJCNLP 2019 • Bohan Li, Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick, Yiming Yang
In this paper, we investigate a simple fix for posterior collapse which yields surprisingly effective results.
no code implementations • WS 2019 • Sasha Spala, Nicholas A. Miller, Yiming Yang, Franck Dernoncourt, Carl Dockhorn
Definition extraction has been a popular topic in NLP research for well more than a decade, but has been historically limited to well-defined, structured, and narrow conditions.
27 code implementations • NeurIPS 2019 • Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
Ranked #1 on
Question Answering
on SQuAD2.0 dev
2 code implementations • 7 May 2019 • Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, Inderjit Dhillon
However, naively applying deep transformer models to the XMC problem leads to sub-optimal performance due to the large output space and the label sparsity issue.
Extreme Multi-Label Classification
General Classification
+4
no code implementations • ICLR 2019 • Zihang Dai*, Zhilin Yang*, Yiming Yang, William W. Cohen, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov
Moreover, Transformer-XL is up to 1, 800+ times faster than vanilla Transformer during evaluation.
no code implementations • 26 Feb 2019 • Chun-Liang Li, Wei-Cheng Chang, Youssef Mroueh, Yiming Yang, Barnabás Póczos
While learning the kernel in a data driven way has been investigated, in this paper we explore learning the spectral distribution of kernel via implicit generative models parametrized by deep neural networks.
no code implementations • 24 Feb 2019 • Aditi Chaudhary, Siddharth Dalmia, Junjie Hu, Xinjian Li, Austin Matthews, Aldrian Obaja Muis, Naoki Otani, Shruti Rijhwani, Zaid Sheikh, Nidhi Vyas, Xinyi Wang, Jiateng Xie, Ruochen Xu, Chunting Zhou, Peter J. Jansen, Yiming Yang, Lori Levin, Florian Metze, Teruko Mitamura, David R. Mortensen, Graham Neubig, Eduard Hovy, Alan W. black, Jaime Carbonell, Graham V. Horwood, Shabnam Tafreshi, Mona Diab, Efsun S. Kayi, Noura Farra, Kathleen McKeown
This paper describes the ARIEL-CMU submissions to the Low Resource Human Language Technologies (LoReHLT) 2018 evaluations for the tasks Machine Translation (MT), Entity Discovery and Linking (EDL), and detection of Situation Frames in Text and Speech (SF Text and Speech).
1 code implementation • NeurIPS 2019 • Zihang Dai, Guokun Lai, Yiming Yang, Shinjae Yoo
With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence.
no code implementations • 22 Jan 2019 • Xiang Kong, Bohan Li, Graham Neubig, Eduard Hovy, Yiming Yang
In this work, we propose a method for neural dialogue response generation that allows not only generating semantically reasonable responses according to the dialogue history, but also explicitly controlling the sentiment of the response via sentiment labels.
2 code implementations • ICLR 2019 • Wei-Cheng Chang, Chun-Liang Li, Yiming Yang, Barnabás Póczos
Detecting the emergence of abrupt property changes in time series is a challenging problem.
37 code implementations • ACL 2019 • Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov
Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.
Ranked #4 on
Language Modelling
on One Billion Word
1 code implementation • 19 Nov 2018 • Yuexin Wu, Xiujun Li, Jingjing Liu, Jianfeng Gao, Yiming Yang
Training task-completion dialogue agents with reinforcement learning usually requires a large number of real user experiences.
no code implementations • NIPS Workshop CDNNRIA 2018 • Wei-Cheng Chang, Hsiang-Fu Yu, Inderjit S. Dhillon, Yiming Yang
To circumvent the softmax bottleneck, SeCSeq compresses labels into sequences of semantic-aware compact codes, on which Seq2Seq models are trained.
Extreme Multi-Label Classification
MUlTI-LABEL-ClASSIFICATION
1 code implementation • EMNLP 2018 • Ruochen Xu, Yiming Yang, Naoki Otani, Yuexin Wu
Supervised methods for this problem rely on the availability of cross-lingual supervision, either using parallel corpora or bilingual lexicons as the labeled data for training, which may not be available for many low resource languages.
58 code implementations • ICLR 2019 • Hanxiao Liu, Karen Simonyan, Yiming Yang
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner.
1 code implementation • 15 Jun 2018 • Guokun Lai, Bohan Li, Guoqing Zheng, Yiming Yang
In this paper, we combine the ideas from both stochastic latent variables and dilated convolutions, and propose a new architecture to model sequential data, termed as Stochastic WaveNet, where stochastic latent variables are injected into the WaveNet structure.
no code implementations • ICLR 2018 • Guokun Lai, Hanxiao Liu, Yiming Yang
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features.
1 code implementation • 20 Nov 2017 • Guoqing Zheng, Yiming Yang, Jaime Carbonell
However, freely enriching the family of variational distribution is challenging since the ELBO requires variational likelihood evaluations of the latent variables.
1 code implementation • ICLR 2018 • Guoqing Zheng, Yiming Yang, Jaime Carbonell
Variational inference provides one way to approximate the posterior distribution, however its expressive power is limited and so is the accuracy of resulting approximation.
no code implementations • 31 Oct 2017 • Guokun Lai, Hanxiao Liu, Yiming Yang
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features.
3 code implementations • NeurIPS 2017 • Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, Barnabás Póczos
In this paper, we propose to improve both the model expressiveness of GMMN and its computational efficiency by introducing adversarial kernel learning techniques, as the replacement of a fixed Gaussian kernel in the original GMMN.
no code implementations • 23 May 2017 • Wei-Cheng Chang, Chun-Liang Li, Yiming Yang, Barnabas Poczos
Large-scale kernel approximation is an important problem in machine learning research.
1 code implementation • ICML 2017 • Hanxiao Liu, Yuexin Wu, Yiming Yang
Large-scale multi-relational embedding refers to the task of learning the latent representations for entities and relations in large knowledge graphs.
Ranked #24 on
Link Prediction
on WN18
1 code implementation • ACL 2017 • Ruochen Xu, Yiming Yang
Using soft probabilistic predictions for the documents in a label-rich language as the (induced) supervisory labels in a parallel corpus of documents, we train classifiers successfully for new languages in which labeled training data are not available.
2 code implementations • EMNLP 2017 • Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, Eduard Hovy
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task.
21 code implementations • 21 Mar 2017 • Guokun Lai, Wei-Cheng Chang, Yiming Yang, Hanxiao Liu
Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation.
Ranked #1 on
Univariate Time Series Forecasting
on Solar-Power
no code implementations • 3 Mar 2017 • Keerthiram Murugesan, Jaime Carbonell, Yiming Yang
This paper presents a new multitask learning framework that learns a shared representation among the tasks, incorporating both task and feature clusters.
no code implementations • COLING 2016 • Andrew Hsi, Yiming Yang, Jaime Carbonell, Ruochen Xu
Event extraction has become one of the most important topics in information extraction, but to date, there is very limited work on leveraging cross-lingual training to boost performance.
no code implementations • NeurIPS 2016 • Keerthiram Murugesan, Hanxiao Liu, Jaime Carbonell, Yiming Yang
This paper addresses the challenge of jointly learning both the per-task model parameters and the inter-task relationships in a multi-task online learning setting.
no code implementations • 6 May 2016 • Hanxiao Liu, Yiming Yang
Cross-graph Relational Learning (CGRL) refers to the problem of predicting the strengths or labels of multi-relational tuples of heterogeneous object types, through the joint inference over multiple graphs which specify the internal connections among each type of objects.
no code implementations • NeurIPS 2012 • Siddharth Gopal, Yiming Yang, Bing Bai, Alexandru Niculescu-Mizil
A challenging problem in hierarchical classification is to leverage the hierarchical relations among classes for improving classification performance.