Search Results for author: Xiaodong He

Found 130 papers, 45 papers with code

Learn to Copy from the Copying History: Correlational Copy Network for Abstractive Summarization

no code implementations EMNLP 2021 Haoran Li, Song Xu, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, BoWen Zhou

It thereby takes advantage of prior copying distributions and, at each time step, explicitly encourages the model to copy the input word that is relevant to the previously copied one.

Abstractive Text Summarization News Summarization

Flow Completion Network: Inferring the Fluid Dynamics from Incomplete Flow Information using Graph Neural Networks

no code implementations10 May 2022 Xiaodong He, Yinan Wang, Juan Li

This paper introduces a novel neural network -- the flow completion network (FCN) -- to infer the fluid dynamics, including the flow field and the force acting on the body, from the incomplete data based on Graph Convolution Attention Network.

BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog

1 code implementation5 May 2022 Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He

To enhance the denoising capability of the model to reduce the impact of error propagation, denoising reconstruction is used to reconstruct the corrupted dialog state and response.

Denoising

LUNA: Learning Slot-Turn Alignment for Dialogue State Tracking

1 code implementation5 May 2022 Yifan Wang, Jing Zhao, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He

Dialogue state tracking (DST) aims to predict the current dialogue state given the dialogue history.

Dialogue State Tracking

OPERA:Operation-Pivoted Discrete Reasoning over Text

no code implementations29 Apr 2022 Yongwei Zhou, Junwei Bao, Chaoqun Duan, Haipeng Sun, Jiahui Liang, Yifan Wang, Jing Zhao, Youzheng Wu, Xiaodong He, Tiejun Zhao

To inherit the advantages of these two types of methods, we propose OPERA, an operation-pivoted discrete reasoning framework, where lightweight symbolic operations (compared with logical forms) as neural modules are utilized to facilitate the reasoning ability and interpretability.

Machine Reading Comprehension Semantic Parsing

The low-entropy hydration shell at the binding site of spike RBD determines the contagiousness of SARS-CoV-2 variants

no code implementations27 Apr 2022 Lin Yang, Shuai Guo, Chengyu Houc, Jiacheng Lia, Liping Shi, Chenchen Liao, Rongchun Shi, Xiaoliang Ma, Bing Zheng, Yi Fang, Lin Ye, Xiaodong He

The low-entropy level of hydration shells at the binding site of a spike protein is found to be an important indicator of the contagiousness of the coronavirus.

Label Anchored Contrastive Learning for Language Understanding

no code implementations26 Apr 2022 Zhenyu Zhang, Yuming Zhao, Meng Chen, Xiaodong He

Motivated by this, we propose a novel label anchored contrastive learning approach (denoted as LaCon) for language understanding.

Contrastive Learning Data Augmentation +2

SE-GAN: Skeleton Enhanced GAN-based Model for Brush Handwriting Font Generation

no code implementations22 Apr 2022 Shaozu Yuan, Ruixue Liu, Meng Chen, Baoyang Chen, Zhijie Qiu, Xiaodong He

There is rare research on brush handwriting font generation, which involves holistic structure changes and complex strokes transfer.

Font Generation

Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue

no code implementations18 Apr 2022 Jiudong Yang, Peiying Wang, Yi Zhu, Mingchao Feng, Meng Chen, Xiaodong He

Turn-taking, aiming to decide when the next speaker can start talking, is an essential component in building human-robot spoken dialogue systems.

Contrastive Learning Data Augmentation +1

Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis

no code implementations22 Mar 2022 Zexun Wang, Yuquan Le, Yi Zhu, Yuming Zhao, Mingchao Feng, Meng Chen, Xiaodong He

Building Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors is an essential issue for various voice-enabled virtual assistants.

Automatic Speech Recognition Natural Language Understanding +2

Fine- and Coarse-Granularity Hybrid Self-Attention for Efficient BERT

1 code implementation ACL 2022 Jing Zhao, Yifan Wang, Junwei Bao, Youzheng Wu, Xiaodong He

To confront this, we propose FCA, a fine- and coarse-granularity hybrid self-attention that reduces the computation cost through progressively shortening the computational sequence length in self-attention.

Informativeness

Space Layout of Low-entropy Hydration Shells Guides Protein Binding

no code implementations22 Feb 2022 Lin Yang, Shuai Guo, Chengyu Hou, Chencheng Liao, Jiacheng Li, Liping Shi, Xiaoliang Ma, Shenda Jiang, Bing Zheng, Yi Fang, Lin Ye, Xiaodong He

According to an analysis of determined protein complex structures, shape matching between the largest low-entropy hydration shell region of a protein and that of its partner at the binding sites is revealed as a regular pattern.

Cross-modal Contrastive Distillation for Instructional Activity Anticipation

no code implementations18 Jan 2022 Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo

In this study, we aim to predict the plausible future action steps given an observation of the past and study the task of instructional activity anticipation.

Knowledge Distillation

ViDA-MAN: Visual Dialog with Digital Humans

no code implementations26 Oct 2021 Tong Shen, Jiawei Zuo, Fan Shi, Jin Zhang, Liqin Jiang, Meng Chen, Zhengchen Zhang, Wei zhang, Xiaodong He, Tao Mei

We demonstrate ViDA-MAN, a digital-human agent for multi-modal interaction, which offers realtime audio-visual responses to instant speech inquiries.

Speech Recognition Video Generation +1

SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition

no code implementations8 Oct 2021 Li Fu, Xiaoxiao Li, Runyu Wang, Lu Fan, Zhengchen Zhang, Meng Chen, Youzheng Wu, Xiaodong He

End-to-end Automatic Speech Recognition (ASR) models are usually trained to optimize the loss of the whole token sequence, while neglecting explicit phonemic-granularity supervision.

Automatic Speech Recognition Contrastive Learning +1

The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service

no code implementations27 Sep 2021 Nan Zhao, Haoran Li, Youzheng Wu, Xiaodong He, BoWen Zhou

We present the solutions of top-5 teams participating in the JDDC multimodal dialogue challenge based on this dataset, which provides valuable insights for further researches on the multimodal dialogue task.

CUSTOM: Aspect-Oriented Product Summarization for E-Commerce

no code implementations18 Aug 2021 Jiahui Liang, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, BoWen Zhou

To address the problem, we propose CUSTOM, aspect-oriented product summarization for e-commerce, which generates diverse and controllable summaries towards different product aspects.

Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation

1 code implementation29 Jun 2021 Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu

Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.

Machine Translation Style Transfer +3

Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

no code implementations9 Jun 2021 Zichuan Lin, Jing Huang, BoWen Zhou, Xiaodong He, Tengyu Ma

Recent work (Takanobu et al., 2020) proposed the system-wise evaluation on dialog systems and found that improvement on individual components (e. g., NLU, policy) in prior work may not necessarily bring benefit to pipeline systems in system-wise evaluation.

Data Augmentation Goal-Oriented Dialog

RevCore: Review-augmented Conversational Recommendation

no code implementations Findings (ACL) 2021 Yu Lu, Junwei Bao, Yan Song, Zichen Ma, Shuguang Cui, Youzheng Wu, Xiaodong He

Existing conversational recommendation (CR) systems usually suffer from insufficient item information when conducted on short dialogue history and unfamiliar items.

Response Generation

Conversational AI Systems for Social Good: Opportunities and Challenges

no code implementations13 May 2021 Peng Qi, Jing Huang, Youzheng Wu, Xiaodong He, BoWen Zhou

Conversational artificial intelligence (ConvAI) systems have attracted much academic and commercial attention recently, making significant progress on both fronts.

SGG: Learning to Select, Guide, and Generate for Keyphrase Generation

1 code implementation NAACL 2021 Jing Zhao, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, BoWen Zhou

Keyphrases, that concisely summarize the high-level topics discussed in a document, can be categorized into present keyphrase which explicitly appears in the source text, and absent keyphrase which does not match any contiguous subsequence but is highly semantically related to the source.

Keyphrase Generation Text Generation

K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce

1 code implementation Findings (EMNLP) 2021 Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, BoWen Zhou

K-PLUG achieves new state-of-the-art results on a suite of domain-specific NLP tasks, including product knowledge base completion, abstractive product summarization, and multi-turn dialogue, significantly outperforms baselines across the board, which demonstrates that the proposed method effectively learns a diverse set of domain-specific knowledge for both language understanding and generation tasks.

Knowledge Base Completion Language Modelling +2

Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification

no code implementations NAACL 2021 Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, BoWen Zhou

Recent work on aspect-level sentiment classification has demonstrated the efficacy of incorporating syntactic structures such as dependency trees with graph neural networks(GNN), but these approaches are usually vulnerable to parsing errors.

Ensemble Learning General Classification +1

Hydrophobic interaction determines docking affinity of SARS CoV 2 variants with antibodies

no code implementations28 Feb 2021 Jiacheng Li, Chengyu Hou, Menghao Wang, Chencheng Liao, Shuai Guo, Liping Shi, Xiaoliang Ma, Hongchi Zhang, Shenda Jiang, Bing Zheng, Lin Ye, Lin Yang, Xiaodong He

Preliminary epidemiologic, phylogenetic and clinical findings suggest that several novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants have increased transmissibility and decreased efficacy of several existing vaccines.

Conversational Query Rewriting with Self-supervised Learning

no code implementations9 Feb 2021 Hang Liu, Meng Chen, Youzheng Wu, Xiaodong He, BoWen Zhou

Conversational Query Rewriting (CQR) aims to simplify the multi-turn dialogue modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance.

Self-Supervised Learning

K-PLUG: KNOWLEDGE-INJECTED PRE-TRAINED LANGUAGE MODEL FOR NATURAL LANGUAGE UNDERSTANDING AND GENERATION

1 code implementation1 Jan 2021 Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, BoWen Zhou

K-PLUG achieves new state-of-the-art results on a suite of domain-specific NLP tasks, including product knowledge base completion, abstractive product summarization, and multi-turn dialogue, significantly outperforms baselines across the board, which demonstrates that the proposed method effectively learns a diverse set of domain-specific knowledge for both language understanding and generation tasks.

Chatbot Knowledge Base Completion +4

Multimodal Sentence Summarization via Multimodal Selective Encoding

no code implementations COLING 2020 Haoran Li, Junnan Zhu, Jiajun Zhang, Xiaodong He, Chengqing Zong

Thus, we propose a multimodal selective gate network that considers reciprocal relationships between textual and multi-level visual features, including global image descriptor, activation grids, and object proposals, to select highlights of the event when encoding the source sentence.

Sentence Summarization

Group Contextual Encoding for 3D Point Clouds

1 code implementation NeurIPS 2020 Xu Liu, Chengtao Li, Jian Wang, Jingbo Wang, Boxin Shi, Xiaodong He

In this work, we extended the contextual encoding layer that was originally designed for 2D tasks to 3D Point Cloud scenarios.

Scene Understanding

Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesis

no code implementations6 Nov 2020 Guanghui Xu, Wei Song, Zhengchen Zhang, Chao Zhang, Xiaodong He, BoWen Zhou

Despite prosody is related to the linguistic information up to the discourse structure, most text-to-speech (TTS) systems only take into account that within each sentence, which makes it challenging when converting a paragraph of texts into natural and expressive speech.

Sentence Embeddings Speech Synthesis

Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product

1 code implementation EMNLP 2020 Tiangang Zhu, Yue Wang, Haoran Li, Youzheng Wu, Xiaodong He, Bo-Wen Zhou

We annotate a multimodal product attribute value dataset that contains 87, 194 instances, and the experimental results on this dataset demonstrate that explicitly modeling the relationship between attributes and values facilitates our method to establish the correspondence between them, and selectively utilizing visual product information is necessary for the task.

Attribute Value Extraction

A hydrophobic-interaction-based mechanism trigger docking between the SARS CoV 2 spike and angiotensin-converting enzyme 2

no code implementations27 Aug 2020 Jiacheng Li, Xiaoliang Ma, Shuai Guo, Chengyu Hou, Liping Shi, Hongchi Zhang, Bing Zheng, Chencheng Liao, Lin Yang, Lin Ye, Xiaodong He

The hydrophobic interaction between the SARS-CoV-2 S and ACE2 protein is found to be significantly greater than that between SARS-CoV S and ACE2.

Neural Kalman Filtering for Speech Enhancement

no code implementations28 Jul 2020 Wei Xue, Gang Quan, Chao Zhang, Guohong Ding, Xiaodong He, BoWen Zhou

Statistical signal processing based speech enhancement methods adopt expert knowledge to design the statistical models and linear filters, which is complementary to the deep neural network (DNN) based methods which are data-driven.

Automatic Speech Recognition Speech Enhancement

Self-Attention Guided Copy Mechanism for Abstractive Summarization

no code implementations ACL 2020 Song Xu, Haoran Li, Peng Yuan, Youzheng Wu, Xiaodong He, Bo-Wen Zhou

Copy module has been widely equipped in the recent abstractive summarization models, which facilitates the decoder to extract words from the source into the summary.

Abstractive Text Summarization

Incremental Learning for End-to-End Automatic Speech Recognition

no code implementations11 May 2020 Li Fu, Xiaoxiao Li, Libo Zi, Zhengchen Zhang, Youzheng Wu, Xiaodong He, BoWen Zhou

In this paper, we propose an incremental learning method for end-to-end Automatic Speech Recognition (ASR) which enables an ASR system to perform well on new tasks while maintaining the performance on its originally learned ones.

Automatic Speech Recognition Incremental Learning +1

Speaker Diarization with Lexical Information

no code implementations13 Apr 2020 Tae Jin Park, Kyu J. Han, Jing Huang, Xiaodong He, Bo-Wen Zhou, Panayiotis Georgiou, Shrikanth Narayanan

This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition.

Automatic Speech Recognition Speaker Diarization

Graph Sequential Network for Reasoning over Sequences

no code implementations4 Apr 2020 Ming Tu, Jing Huang, Xiaodong He, Bo-Wen Zhou

We validate the proposed GSN on two NLP tasks: interpretable multi-hop reading comprehension on HotpotQA and graph based fact verification on FEVER.

Fact Verification Machine Reading Comprehension +1

DR-GAN: Conditional Generative Adversarial Network for Fine-Grained Lesion Synthesis on Diabetic Retinopathy Images

no code implementations10 Dec 2019 Yi Zhou, Boyang Wang, Xiaodong He, Shanshan Cui, Ling Shao

In this paper, we propose a diabetic retinopathy generative adversarial network (DR-GAN) to synthesize high-resolution fundus images which can be manipulated with arbitrary grading and lesion information.

Data Augmentation Lesion Segmentation

Multimodal Intelligence: Representation Learning, Information Fusion, and Applications

no code implementations10 Nov 2019 Chao Zhang, Zichao Yang, Xiaodong He, Li Deng

This review provides a comprehensive analysis of recent works on multimodal deep learning from three perspectives: learning multimodal representations, fusing multimodal signals at various levels, and multimodal applications.

Multimodal Deep Learning Question Answering +5

Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding

no code implementations ACL 2020 Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, Bo-Wen Zhou

Translational distance-based knowledge graph embedding has shown progressive improvements on the link prediction task, from TransE to the latest state-of-the-art RotatE.

Knowledge Graph Embedding Link Prediction

Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents

1 code implementation1 Nov 2019 Ming Tu, Kevin Huang, Guangtao Wang, Jing Huang, Xiaodong He, Bo-Wen Zhou

Interpretable multi-hop reading comprehension (RC) over multiple documents is a challenging problem because it demands reasoning over multiple information sources and explaining the answer prediction by providing supporting evidences.

Learning-To-Rank Multi-Hop Reading Comprehension +1

Relation Module for Non-answerable Prediction on Question Answering

no code implementations23 Oct 2019 Kevin Huang, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou

In this paper, we aim to improve a MRC model's ability to determine whether a question has an answer in a given context (e. g. the recently proposed SQuAD 2. 0 task).

Machine Reading Comprehension Question Answering

Zero-shot Text-to-SQL Learning with Auxiliary Task

1 code implementation29 Aug 2019 Shuaichen Chang, PengFei Liu, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou

Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task.

Text-To-Sql

Multiple instance learning with graph neural networks

no code implementations12 Jun 2019 Ming Tu, Jing Huang, Xiaodong He, Bo-Wen Zhou

In this paper, we propose a new end-to-end graph neural network (GNN) based algorithm for MIL: we treat each bag as a graph and use GNN to learn the bag embedding, in order to explore the useful structural information among instances in bags.

Multiple Instance Learning

Mappa Mundi: An Interactive Artistic Mind Map Generator with Artificial Imagination

no code implementations9 May 2019 Ruixue Liu, Baoyang Chen, Meng Chen, Youzheng Wu, Zhijie Qiu, Xiaodong He

We present a novel real-time, collaborative, and interactive AI painting system, Mappa Mundi, for artistic Mind Map creation.

From Knowledge Map to Mind Map: Artificial Imagination

no code implementations4 Mar 2019 Ruixue Liu, Baoyang Chen, XIAOYU GUO, Yan Dai, Meng Chen, Zhijie Qiu, Xiaodong He

Imagination is one of the most important factors which makes an artistic painting unique and impressive.

Object-driven Text-to-Image Synthesis via Adversarial Training

1 code implementation CVPR 2019 Wenbo Li, Pengchuan Zhang, Lei Zhang, Qiuyuan Huang, Xiaodong He, Siwei Lyu, Jianfeng Gao

In this paper, we propose Object-driven Attentive Generative Adversarial Newtorks (Obj-GANs) that allow object-centered text-to-image synthesis for complex scenes.

Image Generation

End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion

1 code implementation11 Nov 2018 Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, Bo-Wen Zhou

The recent graph convolutional network (GCN) provides another way of learning graph node embedding by successfully utilizing graph connectivity structure.

Knowledge Base Completion Knowledge Graph Embedding +2

Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations

no code implementations EMNLP 2018 Dipendra Misra, Ming-Wei Chang, Xiaodong He, Wen-tau Yih

Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e. g., answers), search for good candidate semantic parses, and (2) choose the best model update algorithm.

Question Answering Semantic Parsing

Deep Reinforcement Learning for NLP

no code implementations ACL 2018 William Yang Wang, Jiwei Li, Xiaodong He

Many Natural Language Processing (NLP) tasks (including generation, language grounding, reasoning, information extraction, coreference resolution, and dialog) can be formulated as deep reinforcement learning (DRL) problems.

Atari Games Coreference Resolution +6

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

no code implementations21 May 2018 Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Dapeng Wu, Jian-Feng Wang, Xiaodong He

We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytelling task.

reinforcement-learning Story Generation +1

Discourse-Aware Neural Rewards for Coherent Text Generation

no code implementations NAACL 2018 Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, Yejin Choi

In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text.

reinforcement-learning Sentence Ordering +1

Generating Diverse and Accurate Visual Captions by Comparative Adversarial Learning

1 code implementation3 Apr 2018 Dianqi Li, Qiuyuan Huang, Xiaodong He, Lei Zhang, Ming-Ting Sun

By contrasting with human-written captions and image-mismatched captions, the caption generator effectively exploits the inherent characteristics of human languages, and generates more discriminative captions.

Deep Communicating Agents for Abstractive Summarization

no code implementations NAACL 2018 Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, Yejin Choi

We present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization.

Ranked #21 on Abstractive Text Summarization on CNN / Daily Mail (using extra training data)

Abstractive Text Summarization reinforcement-learning

Stacked Cross Attention for Image-Text Matching

3 code implementations ECCV 2018 Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, Xiaodong He

Prior work either simply aggregates the similarity of all possible pairs of regions and words without attending differentially to more and less important words or regions, or uses a multi-step attentional process to capture limited number of semantic alignments which is less interpretable.

Cross-Modal Retrieval Image Retrieval +2

Attentive Tensor Product Learning

no code implementations20 Feb 2018 Qiuyuan Huang, Li Deng, Dapeng Wu, Chang Liu, Xiaodong He

This paper proposes a new architecture - Attentive Tensor Product Learning (ATPL) - to represent grammatical structures in deep learning models.

Constituency Parsing Image Captioning +2

From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots

no code implementations6 Jan 2018 Heung-Yeung Shum, Xiaodong He, Di Li

Conversational systems have come a long way since their inception in the 1960s.

Chatbot

Reinforcement Learning To Adapt Speech Enhancement to Instantaneous Input Signal Quality

no code implementations29 Nov 2017 Rasool Fakoor, Xiaodong He, Ivan Tashev, Shuayb Zarar

Today, the optimal performance of existing noise-suppression algorithms, both data-driven and those based on classic statistical methods, is range bound to specific levels of instantaneous input signal-to-noise ratios.

Frame reinforcement-learning +1

AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

17 code implementations CVPR 2018 Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He

In this paper, we propose an Attentional Generative Adversarial Network (AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation.

Text Matching Text to image generation +1

CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise

1 code implementation CVPR 2018 Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang

We demonstrate the effectiveness of the proposed algorithm on both of the label noise detection task and the image classification on noisy data task on several large-scale datasets.

 Ranked #1 on Image Classification on Food-101N (using extra training data)

Classification General Classification +2

On the Discrimination-Generalization Tradeoff in GANs

no code implementations ICLR 2018 Pengchuan Zhang, Qiang Liu, Dengyong Zhou, Tao Xu, Xiaodong He

When evaluated with neural distance, our bounds show that generalization is guaranteed as long as the discriminator set is small enough, regardless of the size of the generator or hypothesis set.

Generalization Bounds

A Neural-Symbolic Approach to Design of CAPTCHA

no code implementations29 Oct 2017 Qiuyuan Huang, Paul Smolensky, Xiaodong He, Li Deng, Dapeng Wu

To address this, this paper promotes image/visual captioning based CAPTCHAs, which is robust against machine-learning-based attacks.

Image Captioning

Tensor Product Generation Networks for Deep NLP Modeling

2 code implementations NAACL 2018 Qiuyuan Huang, Paul Smolensky, Xiaodong He, Li Deng, Dapeng Wu

We present a new approach to the design of deep networks for natural language processing (NLP), based on the general technique of Tensor Product Representations (TPRs) for encoding and processing symbol structures in distributed neural networks.

Multiple-Kernel Based Vehicle Tracking Using 3D Deformable Model and Camera Self-Calibration

no code implementations22 Aug 2017 Zheng Tang, Gaoang Wang, Tao Liu, Young-Gun Lee, Adwin Jahn, Xu Liu, Xiaodong He, Jenq-Neng Hwang

In this challenge, we propose a model-based vehicle localization method, which builds a kernel at each patch of the 3D deformable vehicle model and associates them with constraints in 3D space.

Ensemble Learning Object Detection

Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering

62 code implementations CVPR 2018 Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang

Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through fine-grained analysis and even multiple steps of reasoning.

Image Captioning Visual Question Answering +1

StyleNet: Generating Attractive Visual Captions With Styles

no code implementations CVPR 2017 Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, Li Deng

We propose a novel framework named StyleNet to address the task of generating attractive captions for images and videos with different styles.

Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension

2 code implementations EMNLP 2017 David Golub, Po-Sen Huang, Xiaodong He, Li Deng

We develop a technique for transfer learning in machine comprehension (MC) using a novel two-stage synthesis network (SynNet).

Reading Comprehension Transfer Learning

Adversarial Ranking for Language Generation

1 code implementation NeurIPS 2017 Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, Ming-Ting Sun

Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group.

Text Generation

Question-Answering with Grammatically-Interpretable Representations

no code implementations23 May 2017 Hamid Palangi, Paul Smolensky, Xiaodong He, Li Deng

In our application of TPRN, internal representations learned by end-to-end optimization in a deep neural network performing a textual question-answering (QA) task can be interpreted using basic concepts from linguistic theory.

Question Answering

Reinforcement Learning with External Knowledge and Two-Stage Q-functions for Predicting Popular Reddit Threads

no code implementations20 Apr 2017 Ji He, Mari Ostendorf, Xiaodong He

This paper addresses the problem of predicting popularity of comments in an online discussion forum using reinforcement learning, particularly addressing two challenges that arise from having natural language state and action spaces.

Q-Learning reinforcement-learning

Character-level Deep Conflation for Business Data Analytics

2 code implementations8 Feb 2017 Zhe Gan, P. D. Singh, Ameet Joshi, Xiaodong He, Jianshu Chen, Jianfeng Gao, Li Deng

Connecting different text attributes associated with the same entity (conflation) is important in business data analytics since it could help merge two different tables in a database to provide a more comprehensive profile of an entity.

Learning Generic Sentence Representations Using Convolutional Neural Networks

no code implementations EMNLP 2017 Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin

We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes.

Semantic Compositional Networks for Visual Captioning

1 code implementation CVPR 2017 Zhe Gan, Chuang Gan, Xiaodong He, Yunchen Pu, Kenneth Tran, Jianfeng Gao, Lawrence Carin, Li Deng

The degree to which each member of the ensemble is used to generate an image caption is tied to the image-dependent probability of the corresponding tag.

Image Captioning Semantic Composition +1

Bi-directional Attention with Agreement for Dependency Parsing

1 code implementation EMNLP 2016 Hao Cheng, Hao Fang, Xiaodong He, Jianfeng Gao, Li Deng

We develop a novel bi-directional attention model for dependency parsing, which learns to agree on headword predictions from the forward and backward parsing directions.

Dependency Parsing

MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition

10 code implementations27 Jul 2016 Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, Jianfeng Gao

In this paper, we design a benchmark task and provide the associated datasets for recognizing face images and link them to corresponding entity keys in a knowledge base.

Face Recognition Image Captioning

Unsupervised Learning of Predictors from Unpaired Input-Output Samples

no code implementations15 Jun 2016 Jianshu Chen, Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng

In particular, we show that with regularization via a generative model, learning with the proposed unsupervised objective function converges to an optimal solution.

Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads

1 code implementation EMNLP 2016 Ji He, Mari Ostendorf, Xiaodong He, Jianshu Chen, Jianfeng Gao, Lihong Li, Li Deng

We introduce an online popularity prediction and tracking task as a benchmark task for reinforcement learning with a combinatorial, natural language action space.

reinforcement-learning

Character-Level Question Answering with Attention

1 code implementation EMNLP 2016 David Golub, Xiaodong He

We show that a character-level encoder-decoder framework can be successfully applied to question answering with a structured knowledge base.

Data Augmentation Question Answering

Rich Image Captioning in the Wild

no code implementations30 Mar 2016 Kenneth Tran, Xiaodong He, Lei Zhang, Jian Sun, Cornelia Carapcea, Chris Thrasher, Chris Buehler, Chris Sienkiewicz

We present an image caption system that addresses new challenges of automatically describing images in the wild.

Image Captioning

Generating Natural Questions About an Image

2 code implementations ACL 2016 Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Margaret Mitchell, Xiaodong He, Lucy Vanderwende

There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering questions about images.

Image Captioning Question Generation

Basic Reasoning with Tensor Product Representations

no code implementations12 Jan 2016 Paul Smolensky, Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng

In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)).

Question Answering

Deep Reinforcement Learning with a Natural Language Action Space

3 code implementations ACL 2016 Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng, Mari Ostendorf

This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games.

Q-Learning reinforcement-learning +1

Recurrent Reinforcement Learning: A Hybrid Approach

no code implementations10 Sep 2015 Xiujun Li, Lihong Li, Jianfeng Gao, Xiaodong He, Jianshu Chen, Li Deng, Ji He

Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states.

reinforcement-learning

End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture

1 code implementation NeurIPS 2015 Jianshu Chen, Ji He, Yelong Shen, Lin Xiao, Xiaodong He, Jianfeng Gao, Xinying Song, Li Deng

We develop a fully discriminative learning approach for supervised Latent Dirichlet Allocation (LDA) model using Back Propagation (i. e., BP-sLDA), which maximizes the posterior probability of the prediction variable given the input document.

General Classification Topic Models

A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems

1 code implementation WWW 2015 Ali Elkahky, Yang song, Xiaodong He

We extend the model to jointly learn from features of items from different domains and user features by introducing a multi-view Deep Learning model.

News Recommendation Recommendation Systems

A Deep Embedding Model for Co-occurrence Learning

no code implementations11 Apr 2015 Yelong Shen, Ruoming Jin, Jianshu Chen, Xiaodong He, Jianfeng Gao, Li Deng

Co-occurrence Data is a common and important information source in many areas, such as the word co-occurrence in the sentences, friends co-occurrence in social networks and products co-occurrence in commercial transaction data, etc, which contains rich correlation and clustering information about the items.

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

8 code implementations20 Dec 2014 Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng

We consider learning representations of entities and relations in KBs using the neural-embedding approach.

Link Prediction

Learning Multi-Relational Semantics Using Neural-Embedding Models

no code implementations14 Nov 2014 Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng

In this paper we present a unified framework for modeling multi-relational representations, scoring, and learning, and conduct an empirical study of several recent multi-relational embedding models under the framework.

Knowledge Base Completion

Learning Semantic Representations for the Phrase Translation Model

no code implementations28 Nov 2013 Jianfeng Gao, Xiaodong He, Wen-tau Yih, Li Deng

The results show that the new semantic-based phrase translation model significantly improves the performance of a state-of-the-art phrase-based statistical machine translation sys-tem, leading to a gain of 0. 7-1. 0 BLEU points.

Learning Semantic Representations Machine Translation +1

Learning deep structured semantic models for web search using clickthrough data

3 code implementations CIKM 2013 Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck

The proposed deep structured semantic models are discriminatively trained by maximizing the conditional likelihood of the clicked documents given a query using the clickthrough data.

Document Ranking

Cannot find the paper you are looking for? You can Submit a new open access paper.