Search Results for author: ChengXiang Zhai

Found 47 papers, 18 papers with code

Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries

1 code implementation EMNLP 2021 Carl Edwards, ChengXiang Zhai, Heng Ji

Moreover, this can be viewed as an especially challenging cross-lingual retrieval problem by considering the molecules as a language with a very unique grammar.

Cross-Modal Retrieval Natural Language Queries +1

C-PMI: Conditional Pointwise Mutual Information for Turn-level Dialogue Evaluation

1 code implementation27 Jun 2023 Liliang Ren, Mankeerat Sidhu, Qi Zeng, Revanth Gangi Reddy, Heng Ji, ChengXiang Zhai

Existing reference-free turn-level evaluation metrics for chatbots inadequately capture the interaction between the user and the system.

Dialogue Evaluation

Sparse Modular Activation for Efficient Sequence Modeling

1 code implementation19 Jun 2023 Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai

Linear State Space Models (SSMs) have demonstrated strong performance in a variety of sequence modeling tasks due to their efficient encoding of the recurrent structure.

Chunking Long-range modeling +1

User Simulation for Evaluating Information Access Systems

no code implementations14 Jun 2023 Krisztian Balog, ChengXiang Zhai

Information access systems, such as search engines, recommender systems, and conversational assistants, have become integral to our daily lives as they help us satisfy our information needs.

Recommendation Systems User Simulation

Measuring the Effect of Influential Messages on Varying Personas

1 code implementation25 May 2023 Chenkai Sun, Jinning Li, Hou Pong Chan, ChengXiang Zhai, Heng Ji

Our analysis shows that the best-performing models are capable of predicting responses that are consistent with the personas, and as a byproduct, the task formulation also enables many interesting applications in the analysis of social network groups and their opinions, such as the discovery of extreme opinion groups.

Noise-Robust Dense Retrieval via Contrastive Alignment Post Training

no code implementations6 Apr 2023 Daniel Campos, ChengXiang Zhai, Alessandro Magnani

The success of contextual word representations and advances in neural information retrieval have made dense vector-based retrieval a standard approach for passage and document ranking.

Data Augmentation Document Ranking +3

To Asymmetry and Beyond: Structured Pruning of Sequence to Sequence Models for Improved Inference Efficiency

no code implementations5 Apr 2023 Daniel Campos, ChengXiang Zhai

Sequence-to-sequence language models can be used to produce abstractive summaries which are coherent, relevant, and concise.

Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders

no code implementations31 Mar 2023 Daniel Campos, Alessandro Magnani, ChengXiang Zhai

In this paper, we consider the problem of improving the inference latency of language model-based dense retrieval systems by introducing structural compression and model size asymmetry between the context and query encoders.

Knowledge Distillation Language Modelling +3

Dense Sparse Retrieval: Using Sparse Language Models for Inference Efficient Dense Retrieval

no code implementations31 Mar 2023 Daniel Campos, ChengXiang Zhai

Vector-based retrieval systems have become a common staple for academic and industrial search applications because they provide a simple and scalable way of extending the search to leverage contextual representations for documents and queries.

Retrieval TriviaQA

oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes

no code implementations30 Mar 2023 Daniel Campos, Alexandre Marques, Mark Kurtz, ChengXiang Zhai

In this paper, we introduce the range of oBERTa language models, an easy-to-use set of language models which allows Natural Language Processing (NLP) practitioners to obtain between 3. 8 and 24. 3 times faster models without expertise in model compression.

Knowledge Distillation Model Compression +3

Competence-Based Analysis of Language Models

no code implementations1 Mar 2023 Adam Davies, Jize Jiang, ChengXiang Zhai

Despite the recent success of large pretrained language models (LMs) on a variety of prompting tasks, these models can be alarmingly brittle to small changes in inputs or application contexts.

Learning by Applying: A General Framework for Mathematical Reasoning via Enhancing Explicit Knowledge Learning

no code implementations11 Feb 2023 Jiayu Liu, Zhenya Huang, ChengXiang Zhai, Qi Liu

In LeAp, we perform knowledge learning in a novel problem-knowledge-expression paradigm, with a Knowledge Encoder to acquire knowledge from problem data and a Knowledge Decoder to apply knowledge for expression reasoning.

Mathematical Reasoning

Entity Set Co-Expansion in StackOverflow

no code implementations5 Dec 2022 Yu Zhang, Yunyi Zhang, Yucheng Jiang, Martin Michalski, Yu Deng, Lucian Popa, ChengXiang Zhai, Jiawei Han

Given a few seed entities of a certain type (e. g., Software or Programming Language), entity set expansion aims to discover an extensive set of entities that share the same type as the seeds.

graph construction Management

Language Model Pre-Training with Sparse Latent Typing

1 code implementation23 Oct 2022 Liliang Ren, Zixuan Zhang, Han Wang, Clare R. Voss, ChengXiang Zhai, Heng Ji

Modern large-scale Pre-trained Language Models (PLMs) have achieved tremendous success on a wide range of downstream tasks.

Ranked #3 on Few-shot NER on Few-NERD (INTRA) (using extra training data)

Few-shot NER Language Modelling

Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT

1 code implementation9 Oct 2022 Bhavya Bhavya, JinJun Xiong, ChengXiang Zhai

We propose a novel application of prompting Pre-trained Language Models (PLMs) to generate analogies and study how to design effective prompts for two task settings: generating a source concept analogous to a given target concept (aka Analogous Concept Generation or ACG), and generating an explanation of the similarity between a given pair of target concept and source concept (aka Analogous Explanation Generation or AEG).

Explanation Generation

CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval

1 code implementation COLING 2022 Kung-Hsiang Huang, ChengXiang Zhai, Heng Ji

Given the absence of cross-lingual information retrieval datasets with claim-like queries, we train the retriever with our proposed Cross-lingual Inverse Cloze Task (X-ICT), a self-supervised algorithm that creates training instances by translating the title of a passage.

Cross-lingual Fact-checking Cross-Lingual Information Retrieval +4

Incorporating Task-specific Concept Knowledge into Script Learning

1 code implementation31 Aug 2022 Chenkai Sun, Tie XU, ChengXiang Zhai, Heng Ji

In this paper, we present Tetris, a new task of Goal-Oriented Script Completion.

Contrastive Learning

Sparse*BERT: Sparse Models Generalize To New tasks and Domains

no code implementations25 May 2022 Daniel Campos, Alexandre Marques, Tuan Nguyen, Mark Kurtz, ChengXiang Zhai

Our experimentation shows that models that are pruned during pretraining using general domain masked language models can transfer to novel domains and tasks without extensive hyperparameter exploration or specialized approaches.


Domain Representative Keywords Selection: A Probabilistic Approach

1 code implementation Findings (ACL) 2022 Pritom Saha Akash, Jie Huang, Kevin Chen-Chuan Chang, Yunyao Li, Lucian Popa, ChengXiang Zhai

We propose a probabilistic approach to select a subset of a \textit{target domain representative keywords} from a candidate set, contrasting with a context domain.

DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization

no code implementations13 May 2021 Safa Messaoud, Ismini Lourentzou, Assma Boughoula, Mona Zehni, Zhizhen Zhao, ChengXiang Zhai, Alexander G. Schwing

The recent growth of web video sharing platforms has increased the demand for systems that can efficiently browse, retrieve and summarize video content.

Video Summarization

No Feature Is An Island: Adaptive Collaborations Between Features Improve Adversarial Robustness

no code implementations1 Jan 2021 Yufeng Zhang, Yunan Zhang, ChengXiang Zhai

To classify images, neural networks extract features from raw inputs and then sum them up with fixed weights via the fully connected layer.

Adversarial Robustness

Towards Dark Jargon Interpretation in Underground Forums

no code implementations5 Nov 2020 Dominic Seyler, Wei Liu, XiaoFeng Wang, ChengXiang Zhai

Dark jargons are benign-looking words that have hidden, sinister meanings and are used by participants of underground forums for illicit behavior.

AutoML to Date and Beyond: Challenges and Opportunities

no code implementations21 Oct 2020 Shubhra Kanti Karmaker Santu, Md. Mahadi Hassan, Micah J. Smith, Lei Xu, ChengXiang Zhai, Kalyan Veeramachaneni

AutoML tools aim to make machine learning accessible for non-machine learning experts (domain experts), to improve the efficiency of machine learning, and to accelerate machine learning research.

AutoML BIG-bench Machine Learning

Multi-task Learning for Multilingual Neural Machine Translation

no code implementations EMNLP 2020 Yiren Wang, ChengXiang Zhai, Hany Hassan Awadalla

In this work, we propose a multi-task learning (MTL) framework that jointly trains the model with the translation task on bitext data and two denoising tasks on the monolingual data.

Cross-Lingual Transfer Denoising +4

Towards a Soft Faceted Browsing Scheme for Information Access

no code implementations20 Feb 2020 Yinan Zhang, Parikshit Sondhi, Anjan Goswami, ChengXiang Zhai

Faceted browsing is a commonly supported feature of user interfaces for access to information.


Cooperative Reasoning on Knowledge Graph and Corpus: A Multi-agentReinforcement Learning Approach

no code implementations4 Dec 2019 Yunan Zhang, Xiang Cheng, Heting Gao, ChengXiang Zhai

We model the question answering on KG as a cooperative task between two agents, a knowledge graph reasoning agent and an information extraction agent.

Question Answering

Improving N-gram Language Models with Pre-trained Deep Transformer

no code implementations22 Nov 2019 Yiren Wang, Hongzhao Huang, Zhe Liu, Yutong Pang, Yongqiang Wang, ChengXiang Zhai, Fuchun Peng

Although n-gram language models (LMs) have been outperformed by the state-of-the-art neural LMs, they are still widely used in speech recognition due to its high efficiency in inference.

Data Augmentation speech-recognition +2

TILM: Neural Language Models with Evolving Topical Influence

no code implementations CONLL 2019 Shubhra Kanti Karmaker Santu, Kalyan Veeramachaneni, ChengXiang Zhai

Specifically, we propose a novel language model called Topical Influence Language Model (TILM), which is a novel extension of a neural language model to capture the influences on the contents in one text stream by the evolving topics in another related (or possibly same) text stream.

Language Modelling

Multi-Agent Dual Learning

no code implementations ICLR 2019 Yiren Wang, Yingce Xia, Tianyu He, Fei Tian, Tao Qin, ChengXiang Zhai, Tie-Yan Liu

Dual learning has attracted much attention in machine learning, computer vision and natural language processing communities.

Machine Translation Translation

Adapting Sequence to Sequence models for Text Normalization in Social Media

1 code implementation12 Apr 2019 Ismini Lourentzou, Kabir Manghnani, ChengXiang Zhai

Social media offer an abundant source of valuable raw data, however informal writing can quickly become a bottleneck for many natural language processing (NLP) tasks.

Lexical Normalization

On Application of Learning to Rank for E-Commerce Search

no code implementations1 Mar 2019 Shubhra Kanti Karmaker Santu, Parikshit Sondhi, ChengXiang Zhai

In this paper, we discuss the practical challenges in applying learning to rank methods to E-Com search, including the challenges in feature representation, obtaining reliable relevance judgments, and optimally exploiting multiple user feedback signals such as click rates, add-to-cart ratios, order rates, and revenue.

Information Retrieval Learning-To-Rank +1

JIM: Joint Influence Modeling for Collective Search Behavior

no code implementations1 Mar 2019 Shubhra Kanti Karmaker Santu, Liangda Li, Yi Chang, ChengXiang Zhai

This assumption is unrealistic as there are many correlated events in the real world which influence each other and thus, would pose a joint influence on the user search behavior rather than posing influence independently.

Non-Autoregressive Machine Translation with Auxiliary Regularization

no code implementations22 Feb 2019 Yiren Wang, Fei Tian, Di He, Tao Qin, ChengXiang Zhai, Tie-Yan Liu

However, the high efficiency has come at the cost of not capturing the sequential dependency on the target side of translation, which causes NAT to suffer from two kinds of translation errors: 1) repeated translations (due to indistinguishable adjacent decoder hidden states), and 2) incomplete translations (due to incomplete transfer of source side information via the decoder hidden states).

Machine Translation Translation

Modeling Diverse Relevance Patterns in Ad-hoc Retrieval

2 code implementations SIGIR '18 2018 Yixing Fan, Jiafeng Guo, Yanyan Lan, Jun Xu, ChengXiang Zhai, Xue-Qi Cheng

The local matching layer focuses on producing a set of local relevance signals by modeling the semantic matching between a query and each passage of a document.


Semantic Text Analysis for Detection of Compromised Accounts on Social Networks

1 code implementation19 Apr 2018 Dominic Seyler, Lunan Li, ChengXiang Zhai

We propose to use the difference of language models of users and adversaries to define novel interpretable semantic features for measuring semantic incoherence in a message stream.

Language Modelling

Identifying Humor in Reviews using Background Text Sources

no code implementations EMNLP 2017 Alex Morales, ChengXiang Zhai

We study the problem of automatically identifying humorous text from a new kind of text data, i. e., online reviews.

Language Modelling

High-Dimensional Variance-Reduced Stochastic Gradient Expectation-Maximization Algorithm

no code implementations ICML 2017 Rongda Zhu, Lingxiao Wang, ChengXiang Zhai, Quanquan Gu

We apply our generic algorithm to two illustrative latent variable models: Gaussian mixture model and mixture of linear regression, and demonstrate the advantages of our algorithm by both theoretical analysis and numerical experiments.

Vocal Bursts Intensity Prediction

Cannot find the paper you are looking for? You can Submit a new open access paper.