Search Results for author: Shafiq Joty

Found 176 papers, 71 papers with code

Response Selection for Multi-Party Conversations with Dynamic Topic Tracking

no code implementations EMNLP 2020 Weishi Wang, Steven C.H. Hoi, Shafiq Joty

While participants in a multi-party multi-turn conversation simultaneously engage in multiple conversation topics, existing response selection methods are developed mainly focusing on a two-party single-conversation scenario.

Disentanglement Multi-Task Learning +1

Effective Fine-Tuning Methods for Cross-lingual Adaptation

no code implementations EMNLP 2021 Tao Yu, Shafiq Joty

In this work, we propose a novel fine-tuning method based on co-training that aims to learn more generalized semantic equivalences as a complementary to multilingual language modeling using the unlabeled data in the target language.

Contrastive Learning Language Modelling +1

ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning

no code implementations14 Mar 2024 Ahmed Masry, Mehrad Shahmohammadi, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty

Further evaluation shows that our instruction-tuning approach supports a wide array of real-world chart comprehension and reasoning scenarios, thereby expanding the scope and applicability of our models to new kinds of tasks.

Instruction Following Question Answering

Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and Challenges

no code implementations5 Mar 2024 Bosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Anh Tuan Luu, Shafiq Joty

In the rapidly evolving field of machine learning (ML), data augmentation (DA) has emerged as a pivotal technique for enhancing model performance by diversifying training examples without the need for additional data collection.

Data Augmentation

Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing

1 code implementation1 Feb 2024 Fangkai Jiao, Chengwei Qin, Zhengyuan Liu, Nancy F. Chen, Shafiq Joty

Large Language Models (LLMs) have demonstrated significant potential in handling complex reasoning tasks through step-by-step rationale generation.

Hallucination Logical Reasoning

BootPIG: Bootstrapping Zero-shot Personalized Image Generation Capabilities in Pretrained Diffusion Models

no code implementations25 Jan 2024 Senthil Purushwalkam, Akash Gokul, Shafiq Joty, Nikhil Naik

We propose a novel architecture (BootPIG) that allows a user to provide reference images of an object in order to guide the appearance of a concept in the generated images.

Image Segmentation Semantic Segmentation +1

Improving In-context Learning via Bidirectional Alignment

no code implementations28 Dec 2023 Chengwei Qin, Wenhan Xia, Fangkai Jiao, Shafiq Joty

Large language models (LLMs) have shown impressive few-shot generalization on many tasks via in-context learning (ICL).

In-Context Learning

Do LLMs Work on Charts? Designing Few-Shot Prompts for Chart Question Answering and Summarization

no code implementations17 Dec 2023 Xuan Long Do, Mohammad Hassanpour, Ahmed Masry, Parsa Kavehzadeh, Enamul Hoque, Shafiq Joty

However, their application to chart-related tasks is not trivial as these tasks typically involve considering not only the underlying data but also the visual features in the chart image.

Chart Question Answering Question Answering

X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning

1 code implementation30 Nov 2023 Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, ran Xu, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles

Vision-language pre-training and instruction tuning have demonstrated general-purpose capabilities in 2D visual reasoning tasks by aligning visual encoders with state-of-the-art large language models (LLMs).

Visual Reasoning

Diffusion Model Alignment Using Direct Preference Optimization

no code implementations21 Nov 2023 Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, Nikhil Naik

Large language models (LLMs) are fine-tuned using human comparison data with Reinforcement Learning from Human Feedback (RLHF) methods to make them better aligned with users' preferences.

Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization

1 code implementation15 Nov 2023 Yixin Liu, Alexander R. Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq Joty, PengFei Liu, Dragomir Radev, Chien-Sheng Wu, Arman Cohan

Our study reveals that instruction controllable text summarization remains a challenging task for LLMs, since (1) all LLMs evaluated still make factual and other types of errors in their summaries; (2) all LLM-based evaluation methods cannot achieve a strong alignment with human annotators when judging the quality of candidate summaries; (3) different LLMs show large performance gaps in summary generation and evaluation.

Benchmarking Text Summarization

DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text

no code implementations31 Oct 2023 Wenting Zhao, Ye Liu, Tong Niu, Yao Wan, Philip S. Yu, Shafiq Joty, Yingbo Zhou, Semih Yavuz

Moreover, a significant gap in the current landscape is the absence of a realistic benchmark for evaluating the effectiveness of grounding LLMs on heterogeneous knowledge sources (e. g., knowledge base and text).

Knowledge Graphs Open-Domain Question Answering +2

Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation

1 code implementation28 Oct 2023 Hailin Chen, Amrita Saha, Steven Hoi, Shafiq Joty

With the rise of powerful closed-sourced LLMs (ChatGPT, GPT-4), there are increasing interests in distilling the capabilies of close-sourced LLMs to smaller open-sourced LLMs.

Code Generation

Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation

no code implementations15 Oct 2023 Chengwei Qin, Chen Chen, Shafiq Joty

Inspired by the learning paradigm of humans, we propose Dynamic Module Expansion and Adaptation (DMEA), which enables the model to dynamically determine the architecture for acquiring new knowledge based on task correlation and select the most similar previous tasks to facilitate adaptation to new tasks.

Continual Learning Transfer Learning

CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules

1 code implementation13 Oct 2023 Hung Le, Hailin Chen, Amrita Saha, Akash Gokul, Doyen Sahoo, Shafiq Joty

We find that by naturally encouraging the LLM to reuse the previously developed and verified sub-modules, CodeChain can significantly boost both modularity as well as correctness of the generated solutions, achieving relative pass@1 improvements of 35% on APPS and 76% on CodeContests.

Ranked #2 on Code Generation on CodeContests (Test Set pass@1 metric)

Code Generation

Hierarchical Evaluation Framework: Best Practices for Human Evaluation

no code implementations3 Oct 2023 Iva Bojic, Jessica Chen, Si Yuan Chang, Qi Chwen Ong, Shafiq Joty, Josip Car

Human evaluation plays a crucial role in Natural Language Processing (NLP) as it assesses the quality and relevance of developed systems, thereby facilitating their enhancement.

Machine Reading Comprehension

L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models

no code implementations29 Sep 2023 Ansong Ni, Pengcheng Yin, Yilun Zhao, Martin Riddell, Troy Feng, Rui Shen, Stephen Yin, Ye Liu, Semih Yavuz, Caiming Xiong, Shafiq Joty, Yingbo Zhou, Dragomir Radev, Arman Cohan

Recently, large language models (LLMs), especially those that are pretrained on code, have demonstrated strong capabilities in generating programs from natural language inputs in a few-shot or even zero-shot manner.

Code Generation Math +1

RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair

no code implementations12 Sep 2023 Weishi Wang, Yue Wang, Shafiq Joty, Steven C. H. Hoi

Automatic program repair (APR) is crucial to reduce manual debugging efforts for developers and improve software reliability.

Language Modelling Program Repair +1

XGen-7B Technical Report

1 code implementation7 Sep 2023 Erik Nijkamp, Tian Xie, Hiroaki Hayashi, Bo Pang, Congying Xia, Chen Xing, Jesse Vig, Semih Yavuz, Philippe Laban, Ben Krause, Senthil Purushwalkam, Tong Niu, Wojciech Kryściński, Lidiya Murakhovs'ka, Prafulla Kumar Choubey, Alex Fabbri, Ye Liu, Rui Meng, Lifu Tu, Meghana Bhat, Chien-Sheng Wu, Silvio Savarese, Yingbo Zhou, Shafiq Joty, Caiming Xiong

Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.

Exploring the Integration Strategies of Retriever and Large Language Models

no code implementations24 Aug 2023 Ye Liu, Semih Yavuz, Rui Meng, Meghana Moorthy, Shafiq Joty, Caiming Xiong, Yingbo Zhou

This paper aims to fill this gap by investigating different methods of combining retrieved passages with LLMs to enhance answer generation.

Answer Generation Open-Domain Question Answering

PromptSum: Parameter-Efficient Controllable Abstractive Summarization

no code implementations6 Aug 2023 Mathieu Ravaut, Hailin Chen, Ruochen Zhao, Chengwei Qin, Shafiq Joty, Nancy Chen

Prompt tuning (PT), a parameter-efficient technique that only tunes the additional prompt embeddings while keeping the backbone pre-trained language model (PLM) frozen, has shown promising results in language understanding tasks, especially in low-resource scenarios.

Abstractive Text Summarization Language Modelling

Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning

1 code implementation1 Jun 2023 Fan Yin, Jesse Vig, Philippe Laban, Shafiq Joty, Caiming Xiong, Chien-Sheng Jason Wu

Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks.

Building Extractive Question Answering System to Support Human-AI Health Coaching Model for Sleep Domain

no code implementations31 May 2023 Iva Bojic, Qi Chwen Ong, Shafiq Joty, Josip Car

Non-communicable diseases (NCDs) are a leading cause of global deaths, necessitating a focus on primary prevention and lifestyle behavior change.

Extractive Question-Answering Passage Retrieval +2

SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages

1 code implementation30 May 2023 Philippe Laban, Jesse Vig, Wojciech Kryscinski, Shafiq Joty, Caiming Xiong, Chien-Sheng Wu

Text simplification research has mostly focused on sentence-level simplification, even though many desirable edits - such as adding relevant background information or reordering content - may require document-level context.

Sentence Text Simplification

UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning

1 code implementation24 May 2023 Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, Shafiq Joty

Charts are very popular for analyzing data, visualizing key insights and answering complex reasoning questions about data.

Ranked #15 on Chart Question Answering on ChartQA (using extra training data)

Chart Question Answering Question Answering

Unlocking Temporal Question Answering for Large Language Models Using Code Execution

1 code implementation24 May 2023 Xingxuan Li, Liying Cheng, Qingyu Tan, Hwee Tou Ng, Shafiq Joty, Lidong Bing

Our preliminary experiments show that generating intermediate reasoning steps does not always boost the performance of complex temporal question-answering tasks.

Logical Reasoning Math +1

Exploring Self-supervised Logic-enhanced Training for Large Language Models

2 code implementations23 May 2023 Fangkai Jiao, Zhiyang Teng, Bosheng Ding, Zhengyuan Liu, Nancy F. Chen, Shafiq Joty

Existing efforts to improve logical reasoning ability of language models have predominantly relied on supervised fine-tuning, hindering generalization to new domains and/or tasks.

In-Context Learning Logical Reasoning

LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond

1 code implementation23 May 2023 Philippe Laban, Wojciech Kryściński, Divyansh Agarwal, Alexander R. Fabbri, Caiming Xiong, Shafiq Joty, Chien-Sheng Wu

To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits.

Misinformation

Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework

1 code implementation5 May 2023 Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, Lidong Bing

As large language models (LLMs) have become the norm in NLP, demonstrating good performance in generation and reasoning tasks, one of its most fatal disadvantages is the lack of factual correctness.

Open-Domain Question Answering

Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation

1 code implementation4 May 2023 Xuan Long Do, Bowei Zou, Shafiq Joty, Anh Tai Tran, Liangming Pan, Nancy F. Chen, Ai Ti Aw

In addition, we propose Conv-Distinct, a novel evaluation metric for CQG, to evaluate the diversity of the generated conversation from a context.

Question Generation Question-Generation +1

Explaining Language Models' Predictions with High-Impact Concepts

no code implementations3 May 2023 Ruochen Zhao, Shafiq Joty, Yongjie Wang, Tan Wang

The emergence of large-scale pretrained language models has posed unprecedented challenges in deriving explanations of why the model has made some predictions.

Fairness Vocal Bursts Intensity Prediction

Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning

1 code implementation3 Apr 2023 Lifu Tu, Jin Qu, Semih Yavuz, Shafiq Joty, Wenhao Liu, Caiming Xiong, Yingbo Zhou

Our results demonstrate the strong and efficient modeling ability of NLI-based classifiers and the large cross-lingual transfer improvements achieved by our aligned prompts, particularly in few-shot settings.

Cross-Lingual Transfer intent-classification +4

A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets

1 code implementation2 Apr 2023 Iva Bojic, Josef Halim, Verena Suharman, Sreeja Tar, Qi Chwen Ong, Duy Phung, Mathieu Ravaut, Shafiq Joty, Josip Car

We applied the proposed framework to four biomedical datasets and showed relative improvement of up to 33%/40% for fine-tuning of retrieval/reader models on the BioASQ dataset when using back translation to enhance the original dataset quality.

Machine Reading Comprehension Retrieval

Retrieving Multimodal Information for Augmented Generation: A Survey

no code implementations20 Mar 2023 Ruochen Zhao, Hailin Chen, Weishi Wang, Fangkai Jiao, Xuan Long Do, Chengwei Qin, Bosheng Ding, Xiaobao Guo, Minzhi Li, Xingxuan Li, Shafiq Joty

As Large Language Models (LLMs) become popular, there emerged an important trend of using multimodality to augment the LLMs' generation ability, which enables LLMs to better interact with the world.

Retrieval

Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation

1 code implementation7 Mar 2023 Yixin Liu, Alexander R. Fabbri, Yilun Zhao, PengFei Liu, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, Dragomir Radev

Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics.

xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval

2 code implementations6 Mar 2023 Mohammad Abdullah Matin Khan, M Saiful Bari, Xuan Long Do, Weishi Wang, Md Rizwan Parvez, Shafiq Joty

Recently, pre-trained large language models (LLMs) have shown impressive abilities in generating codes from natural language descriptions, repairing buggy codes, translating codes between languages, and retrieving relevant code segments.

Program Repair Retrieval

Dynamic Scheduled Sampling with Imitation Loss for Neural Text Generation

no code implementations31 Jan 2023 Xiang Lin, Prathyusha Jwalapuram, Shafiq Joty

Scheduled sampling is a curriculum learning strategy that gradually exposes the model to its own predictions during training to mitigate this bias.

Machine Translation Text Generation

Is GPT-3 a Good Data Annotator?

no code implementations20 Dec 2022 Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia, Shafiq Joty, Boyang Li, Lidong Bing

In this paper, we evaluate the performance of GPT-3 as a data annotator by comparing it with traditional data annotation methods and analyzing its output on a range of tasks.

Language Modelling

Evaluating Psychological Safety of Large Language Models

no code implementations20 Dec 2022 Xingxuan Li, Yutong Li, Lin Qiu, Shafiq Joty, Lidong Bing

Despite being instruction fine-tuned with safety metrics to reduce toxicity, InstructGPT, GPT-3. 5, and GPT-4 still showed dark personality patterns; these models scored higher than self-supervised GPT-3 on the Machiavellianism and narcissism traits on SD-3.

Unsupervised Summarization Re-ranking

2 code implementations19 Dec 2022 Mathieu Ravaut, Shafiq Joty, Nancy Chen

With the rise of task-specific pre-training objectives, abstractive summarization models like PEGASUS offer appealing zero-shot performance on downstream summarization tasks.

Abstractive Text Summarization Re-Ranking

Learning Label Modular Prompts for Text Classification in the Wild

1 code implementation30 Nov 2022 Hailin Chen, Amrita Saha, Shafiq Joty, Steven C. H. Hoi

Machine learning models usually assume i. i. d data during training and testing, but data and tasks in real world often change over time.

text-classification Text Classification

BotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systems

1 code implementation29 Nov 2022 Guangsen Wang, Shafiq Joty, Junnan Li, Steven Hoi

BotSIM adopts a layered design comprising the infrastructure layer, the adaptor layer and the application layer.

User Simulation

Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations

1 code implementation16 Nov 2022 Linlin Liu, Xingxuan Li, Megh Thakkar, Xin Li, Shafiq Joty, Luo Si, Lidong Bing

Due to the huge amount of parameters, fine-tuning of pretrained language models (PLMs) is prone to overfitting in the low resource scenarios.

Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning

1 code implementation8 Nov 2022 Qian Li, Shafiq Joty, Daling Wang, Shi Feng, Yifei Zhang

Sparsity of formal knowledge and roughness of non-ontological construction make sparsity problem particularly prominent in Open Knowledge Graphs (OpenKGs).

Contrastive Learning Knowledge Graphs +1

Towards Summary Candidates Fusion

1 code implementation17 Oct 2022 Mathieu Ravaut, Shafiq Joty, Nancy F. Chen

To bypass this limitation, we propose a new paradigm in second-stage abstractive summarization called SummaFusion that fuses several summary candidates to produce a novel abstractive second-stage summary.

Abstractive Text Summarization Re-Ranking

Improving Conversational Recommender System via Contextual and Time-Aware Modeling with Less Domain-Specific Knowledge

no code implementations23 Sep 2022 Lingzhi Wang, Shafiq Joty, Wei Gao, Xingshan Zeng, Kam-Fai Wong

In addition to conducting experiments on a popular dataset (ReDial), we also include a multi-domain dataset (OpenDialKG) to show the effectiveness of our model.

Recommendation Systems

CoHS-CQG: Context and History Selection for Conversational Question Generation

1 code implementation COLING 2022 Xuan Long Do, Bowei Zou, Liangming Pan, Nancy F. Chen, Shafiq Joty, Ai Ti Aw

While previous studies mainly focus on how to model the flow and alignment of the conversation, there has been no thorough study to date on which parts of the context and history are necessary for the model.

Question Generation Question-Generation +1

Refining Low-Resource Unsupervised Translation by Language Disentanglement of Multilingual Model

1 code implementation31 May 2022 Xuan-Phi Nguyen, Shafiq Joty, Wu Kui, Ai Ti Aw

Numerous recent work on unsupervised machine translation (UMT) implies that competent unsupervised translations of low-resource and unrelated languages, such as Nepali or Sinhala, are only possible if the model is trained in a massive multilingual environment, where these low-resource languages are mixed with high-resource counterparts.

Disentanglement Translation +1

Data Selection Curriculum for Neural Machine Translation

no code implementations25 Mar 2022 Tasnim Mohiuddin, Philipp Koehn, Vishrav Chaudhary, James Cross, Shruti Bhosale, Shafiq Joty

In this work, we introduce a two-stage curriculum training framework for NMT where we fine-tune a base NMT model on subsets of data, selected by both deterministic scoring using pre-trained methods and online scoring that considers prediction scores of the emerging NMT model.

Machine Translation NMT +1

ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

1 code implementation Findings (ACL) 2022 Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, Enamul Hoque

To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions.

Chart Question Answering Logical Reasoning +1

SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization

1 code implementation ACL 2022 Mathieu Ravaut, Shafiq Joty, Nancy F. Chen

Sequence-to-sequence neural networks have recently achieved great success in abstractive summarization, especially through fine-tuning large pre-trained language models on the downstream dataset.

Abstractive Text Summarization Document Summarization +1

Chart-to-Text: A Large-Scale Benchmark for Chart Summarization

2 code implementations ACL 2022 Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, Shafiq Joty

We also introduce a number of state-of-the-art neural models as baselines that utilize image captioning and data-to-text generation techniques to tackle two problem variations: one assumes the underlying data table of the chart is available while the other needs to extract data from chart images.

Data-to-Text Generation Image Captioning

Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation

1 code implementation ACL 2022 Chengwei Qin, Shafiq Joty

Existing continual relation learning (CRL) methods rely on plenty of labeled training data for learning a new task, which can be hard to acquire in real scenario as getting large and representative labeled data is often expensive and time-consuming.

Data Augmentation Relation

Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples

1 code implementation22 Nov 2021 Linlin Liu, Xin Li, Ruidan He, Lidong Bing, Shafiq Joty, Luo Si

In this work, we explore methods to make better use of the multilingual annotation and language agnostic property of KG triples, and present novel knowledge based multilingual language models (KMLMs) trained directly on the knowledge triples.

Knowledge Graphs Language Modelling +9

A Unified Speaker Adaptation Approach for ASR

1 code implementation EMNLP 2021 Yingzhu Zhao, Chongjia Ni, Cheung-Chi Leung, Shafiq Joty, Eng Siong Chng, Bin Ma

For model adaptation, we use a novel gradual pruning method to adapt to target speakers without changing the model architecture, which to the best of our knowledge, has never been explored in ASR.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Cascaded Fast and Slow Models for Efficient Semantic Code Search

no code implementations15 Oct 2021 Akhilesh Deepak Gotmare, Junnan Li, Shafiq Joty, Steven C. H. Hoi

The goal of natural language semantic code search is to retrieve a semantically relevant code snippet from a fixed set of candidates using a natural language query.

Code Search Re-Ranking +1

Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling

no code implementations ACL 2022 Prathyusha Jwalapuram, Shafiq Joty, Xiang Lin

Given the claims of improved text generation quality across various pre-trained neural models, we consider the coherence evaluation of machine generated text to be one of the principal applications of coherence models that needs to be investigated.

Coherence Evaluation Contrastive Learning +1

LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5

1 code implementation ICLR 2022 Chengwei Qin, Shafiq Joty

Existing approaches to lifelong language learning rely on plenty of labeled data for learning a new task, which is hard to obtain in most real scenarios.

Few-Shot Learning

Contrastive Clustering to Mine Pseudo Parallel Data for Unsupervised Translation

no code implementations ICLR 2022 Xuan-Phi Nguyen, Hongyu Gong, Yun Tang, Changhan Wang, Philipp Koehn, Shafiq Joty

Modern unsupervised machine translation systems mostly train their models by generating synthetic parallel training data from large unlabeled monolingual corpora of different languages through various means, such as iterative back-translation.

Clustering Translation +1

CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation

5 code implementations EMNLP 2021 Yue Wang, Weishi Wang, Shafiq Joty, Steven C. H. Hoi

We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers.

Clone Detection Code Summarization +4

MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER

no code implementations ACL 2021 Linlin Liu, Bosheng Ding, Lidong Bing, Shafiq Joty, Luo Si, Chunyan Miao

With the source-language data as well as the translated data, a generation-based multilingual data augmentation method is introduced to further increase diversity by generating synthetic labeled data in multiple languages.

Cross-Lingual NER Data Augmentation +5

Remember What You have drawn: Semantic Image Manipulation with Memory

no code implementations27 Jul 2021 Xiangxi Shi, Zhonghua Wu, Guosheng Lin, Jianfei Cai, Shafiq Joty

Therefore, in this paper, we propose a memory-based Image Manipulation Network (MIM-Net), where a set of memories learned from images is introduced to synthesize the texture information with the guidance of the textual description.

Image Manipulation

A Conditional Splitting Framework for Efficient Constituency Parsing

no code implementations ACL 2021 Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, XiaoLi Li

We introduce a generic seq2seq parsing framework that casts constituency parsing problems (syntactic and discourse parsing) into a series of conditional splitting decisions.

Constituency Parsing Discourse Segmentation +1

Straight to the Gradient: Learning to Use Novel Tokens for Neural Text Generation

1 code implementation14 Jun 2021 Xiang Lin, Simeng Han, Shafiq Joty

Advanced large-scale neural language models have led to significant success in many language generation tasks.

Text Generation

AUGVIC: Exploiting BiText Vicinity for Low-Resource NMT

no code implementations Findings (ACL) 2021 Tasnim Mohiuddin, M Saiful Bari, Shafiq Joty

We show that AUGVIC helps to attenuate the discrepancies between relevant and distant-domain monolingual data in traditional back-translation.

Data Augmentation Machine Translation +2

RST Parsing from Scratch

1 code implementation NAACL 2021 Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, XiaoLi Li

We introduce a novel top-down end-to-end formulation of document-level discourse parsing in the Rhetorical Structure Theory (RST) framework.

Discourse Segmentation Segmentation

Addressing the Vulnerability of NMT in Input Perturbations

1 code implementation NAACL 2021 Weiwen Xu, Ai Ti Aw, Yang Ding, Kui Wu, Shafiq Joty

Neural Machine Translation (NMT) has achieved significant breakthrough in performance but is known to suffer vulnerability to input perturbations.

Machine Translation NMT +1

Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning

no code implementations28 Jan 2021 Amrita Saha, Shafiq Joty, Steven C. H. Hoi

Neural Module Networks (NMNs) have been quite successful in incorporating explicit reasoning as learnable modules in various question answering tasks, including the most generic form of numerical reasoning over text in Machine Reading Comprehension (MRC).

Dependency Parsing Language Modelling +2

DiP Benchmark Tests: Evaluation Benchmarks for Discourse Phenomena in MT

no code implementations1 Jan 2021 Prathyusha Jwalapuram, Barbara Rychalska, Shafiq Joty, Dominika Basaj

Despite increasing instances of machine translation (MT) systems including extrasentential context information, the evidence for translation quality improvement is sparse, especially for discourse phenomena.

Machine Translation Translation

Self-Supervised Relationship Probing

no code implementations NeurIPS 2020 Jiuxiang Gu, Jason Kuen, Shafiq Joty, Jianfei Cai, Vlad Morariu, Handong Zhao, Tong Sun

Structured representations of images that model visual relationships are beneficial for many vision and vision-language applications.

Contrastive Learning Language Modelling +1

Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses

1 code implementation EMNLP 2020 Prathyusha Jwalapuram, Shafiq Joty, Youlin Shen

Our sentence-level model shows a 0. 5 BLEU improvement on both the WMT14 and the IWSLT13 De-En testsets, while our contextual model achieves the best results, improving from 31. 81 to 32 BLEU on WMT14 De-En testset, and from 32. 10 to 33. 13 on the IWSLT13 De-En testset, with corresponding improvements in pronoun translation.

Machine Translation NMT +2

Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading

1 code implementation EMNLP 2020 Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven C. H. Hoi, Caiming Xiong, Irwin King, Michael R. Lyu

Based on the learned EDU and entailment representations, we either reply to the user our final decision "yes/no/irrelevant" of the initial question, or generate a follow-up question to inquiry more information.

Decision Making Discourse Segmentation +3

UNISON: Unpaired Cross-lingual Image Captioning

no code implementations3 Oct 2020 Jiahui Gao, Yi Zhou, Philip L. H. Yu, Shafiq Joty, Jiuxiang Gu

In this work, we present a novel unpaired cross-lingual method to generate image captions without relying on any caption corpus in the source or the target language.

Image Captioning Machine Translation +2

Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning

no code implementations ECCV 2020 Xiangxi Shi, Xu Yang, Jiuxiang Gu, Shafiq Joty, Jianfei Cai

In this paper, we propose a novel visual encoder to explicitly distinguish viewpoint changes from semantic changes in the change captioning task.

Reinforcement Learning (RL)

GeDi: Generative Discriminator Guided Sequence Generation

3 code implementations Findings (EMNLP) 2021 Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, Nazneen Fatema Rajani

While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate.

Attribute Linguistic Acceptability +1

Efficient Constituency Parsing by Pointing

no code implementations ACL 2020 Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, Xiao-Li Li

We propose a novel constituency parsing model that casts the parsing problem into a series of pointing tasks.

Constituency Parsing

Cross-model Back-translated Distillation for Unsupervised Machine Translation

1 code implementation3 Jun 2020 Xuan-Phi Nguyen, Shafiq Joty, Thanh-Tung Nguyen, Wu Kui, Ai Ti Aw

Recent unsupervised machine translation (UMT) systems usually employ three main principles: initialization, language modeling and iterative back-translation, though they may apply them differently.

Denoising Language Modelling +2

Unsupervised Word Translation with Adversarial Autoencoder

no code implementations CL 2020 Tasnim Mohiuddin, Shafiq Joty

Crosslingual word embeddings learned from monolingual embeddings have a crucial role in many downstream tasks, ranging from machine translation to transfer learning.

Machine Translation Transfer Learning +3

EMT: Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading

1 code implementation26 May 2020 Yifan Gao, Chien-Sheng Wu, Shafiq Joty, Caiming Xiong, Richard Socher, Irwin King, Michael R. Lyu, Steven C. H. Hoi

The goal of conversational machine reading is to answer user questions given a knowledge base text which may require asking clarification questions.

Decision Making Reading Comprehension +1

It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations

1 code implementation ACL 2020 Samson Tan, Shafiq Joty, Min-Yen Kan, Richard Socher

Training on only perfect Standard English corpora predisposes pre-trained neural networks to discriminate against minorities from non-standard linguistic backgrounds (e. g., African American Vernacular English, Colloquial Singapore English, etc.).

Rethinking Coherence Modeling: Synthetic vs. Downstream Tasks

no code implementations EACL 2021 Tasnim Mohiuddin, Prathyusha Jwalapuram, Xiang Lin, Shafiq Joty

Although coherence modeling has come a long way in developing novel models, their evaluation on downstream applications for which they are purportedly developed has largely been neglected.

Benchmarking Coherence Evaluation +7

Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding

1 code implementation EMNLP 2020 Samson Tan, Shafiq Joty, Lav R. Varshney, Min-Yen Kan

Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English.

Morphological Inflection Translation

Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation

no code implementations30 Apr 2020 Prathyusha Jwalapuram, Barbara Rychalska, Shafiq Joty, Dominika Basaj

Despite increasing instances of machine translation (MT) systems including contextual information, the evidence for translation quality improvement is sparse, especially for discourse phenomena.

Machine Translation Translation

LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space

no code implementations EMNLP 2020 Tasnim Mohiuddin, M Saiful Bari, Shafiq Joty

Most of the successful and predominant methods for bilingual lexicon induction (BLI) are mapping-based, where a linear mapping function is learned with the assumption that the word embedding spaces of different languages exhibit similar geometric structures (i. e., approximately isomorphic).

Bilingual Lexicon Induction Cross-Lingual Word Embeddings +1

VD-BERT: A Unified Vision and Dialog Transformer with BERT

1 code implementation EMNLP 2020 Yue Wang, Shafiq Joty, Michael R. Lyu, Irwin King, Caiming Xiong, Steven C. H. Hoi

By contrast, in this work, we propose VD-BERT, a simple yet effective framework of unified vision-dialog Transformer that leverages the pretrained BERT language models for Visual Dialog tasks.

Answer Generation Visual Dialog

Tree-structured Attention with Hierarchical Accumulation

no code implementations ICLR 2020 Xuan-Phi Nguyen, Shafiq Joty, Steven C. H. Hoi, Richard Socher

Incorporating hierarchical structures like constituency trees has been shown to be effective for various natural language processing (NLP) tasks.

text-classification Text Classification +1

Pairwise Neural Machine Translation Evaluation

no code implementations IJCNLP 2015 Francisco Guzman, Shafiq Joty, Lluis Marquez, Preslav Nakov

We present a novel framework for machine translation evaluation using neural networks in a pairwise setting, where the goal is to select the better translation from a pair of hypotheses, given the reference translation.

Machine Translation Sentence +2

DiscoTK: Using Discourse Structure for Machine Translation Evaluation

no code implementations WS 2014 Shafiq Joty, Francisco Guzman, Lluis Marquez, Preslav Nakov

We present novel automatic metrics for machine translation evaluation that use discourse structure and convolution kernels to compare the discourse tree of an automatic translation with that of the human reference.

Machine Translation Translation

Zero-Resource Cross-Lingual Named Entity Recognition

1 code implementation22 Nov 2019 M Saiful Bari, Shafiq Joty, Prathyusha Jwalapuram

Recently, neural methods have achieved state-of-the-art (SOTA) results in Named Entity Recognition (NER) tasks for many languages without the need for manually crafted features.

Cross-Lingual NER Low Resource Named Entity Recognition +3

Global Thread-Level Inference for Comment Classification in Community Question Answering

no code implementations EMNLP 2015 Shafiq Joty, Alberto Barrón-Cedeño, Giovanni Da San Martino, Simone Filice, Lluís Màrquez, Alessandro Moschitti, Preslav Nakov

Community question answering, a recent evolution of question answering in the Web context, allows a user to quickly consult the opinion of a number of people on a particular topic, thus taking advantage of the wisdom of the crowd.

Community Question Answering General Classification

Resurrecting Submodularity for Neural Text Generation

no code implementations8 Nov 2019 Simeng Han, Xiang Lin, Shafiq Joty

The resulting attention module offers an architecturally simple and empirically effective method to improve the coverage of neural text generation.

Abstractive Text Summarization Text Generation

Data Diversification: A Simple Strategy For Neural Machine Translation

2 code implementations NeurIPS 2020 Xuan-Phi Nguyen, Shafiq Joty, Wu Kui, Ai Ti Aw

Our method achieves state-of-the-art BLEU scores of 30. 7 and 43. 7 in the WMT'14 English-German and English-French translation tasks, respectively.

Knowledge Distillation Machine Translation +2

Enhancing Attention with Explicit Phrasal Alignments

no code implementations25 Sep 2019 Xuan-Phi Nguyen, Shafiq Joty, Thanh-Tung Nguyen

The attention mechanism is an indispensable component of any state-of-the-art neural machine translation system.

Language Modelling Machine Translation +1

A Unified Neural Coherence Model

no code implementations IJCNLP 2019 Han Cheol Moon, Tasnim Mohiuddin, Shafiq Joty, Xu Chi

In this paper, we propose a unified coherence model that incorporates sentence grammar, inter-sentence coherence relations, and global coherence patterns into a common neural framework.

Machine Translation Sentence +1

Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite

2 code implementations IJCNLP 2019 Prathyusha Jwalapuram, Shafiq Joty, Irina Temnikova, Preslav Nakov

The ongoing neural revolution in machine translation has made it easier to model larger contexts beyond the sentence-level, which can potentially help resolve some discourse-level ambiguities such as pronominal anaphora, thus enabling better translations.

Machine Translation Sentence +1

Hierarchical Pointer Net Parsing

1 code implementation IJCNLP 2019 Linlin Liu, Xiang Lin, Shafiq Joty, Simeng Han, Lidong Bing

Transition-based top-down parsing with pointer networks has achieved state-of-the-art results in multiple parsing tasks, while having a linear time complexity.

Discourse Parsing Inductive Bias +1

Watch It Twice: Video Captioning with a Refocused Video Encoder

no code implementations21 Jul 2019 Xiangxi Shi, Jianfei Cai, Shafiq Joty, Jiuxiang Gu

With the rapid growth of video data and the increasing demands of various applications such as intelligent video search and assistance toward visually-impaired people, video captioning task has received a lot of attention recently in computer vision and natural language processing fields.

Video Captioning

Sentence-Level Evidence Embedding for Claim Verification with Hierarchical Attention Networks

no code implementations ACL 2019 Jing Ma, Wei Gao, Shafiq Joty, Kam-Fai Wong

Claim verification is generally a task of verifying the veracity of a given claim, which is critical to many downstream applications.

Claim Verification Sentence

Discourse Analysis and Its Applications

no code implementations ACL 2019 Shafiq Joty, Giuseppe Carenini, Raymond Ng, Gabriel Murray

Discourse processing is a suite of Natural Language Processing (NLP) tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications.

Machine Translation Question Answering +3

A Unified Linear-Time Framework for Sentence-Level Discourse Parsing

2 code implementations ACL 2019 Xiang Lin, Shafiq Joty, Prathyusha Jwalapuram, M Saiful Bari

We propose an efficient neural framework for sentence-level discourse analysis in accordance with Rhetorical Structure Theory (RST).

Discourse Parsing Sentence

Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training

1 code implementation NAACL 2019 Tasnim Mohiuddin, Shafiq Joty

Adversarial training has shown impressive success in learning bilingual dictionary without any parallel data by mapping monolingual embeddings to a shared space.

Translation Word Translation

Modeling Speech Acts in Asynchronous Conversations: A Neural-CRF Approach

no code implementations CL 2018 Shafiq Joty, Tasnim Mohiuddin

Participants in an asynchronous conversation (e. g., forum, e-mail) interact with each other at different times, performing certain communicative acts, called speech acts (e. g., question, request).

Sentence Word Embeddings

Adversarial Unsupervised Representation Learning for Activity Time-Series

no code implementations14 Nov 2018 Karan Aggarwal, Shafiq Joty, Luis Fernandez-Luque, Jaideep Srivastava

Sufficient physical activity and restful sleep play a major role in the prevention and cure of many chronic conditions.

Representation Learning Time Series +1

Reuse and Adaptation for Entity Resolution through Transfer Learning

no code implementations28 Sep 2018 Saravanan Thirumuruganathan, Shameem A Puthiya Parambath, Mourad Ouzzani, Nan Tang, Shafiq Joty

Entity resolution (ER) is one of the fundamental problems in data integration, where machine learning (ML) based classifiers often provide the state-of-the-art results.

Entity Resolution Feature Engineering +1

Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings

no code implementations EMNLP 2018 Shafiq Joty, Lluis Marquez, Preslav Nakov

We address jointly two important tasks for Question Answering in community forums: given a new question, (i) find related existing questions, and (ii) find relevant answers to this new question.

Community Question Answering

Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction

no code implementations8 Jul 2018 Xiangxi Shi, Jianfei Cai, Jiuxiang Gu, Shafiq Joty

In this paper, we propose a boundary-aware hierarchical language decoder for video captioning, which consists of a high-level GRU based language decoder, working as a global (caption-level) language model, and a low-level GRU based language decoder, working as a local (phrase-level) language model.

Language Modelling Sentence +3

Domain Adaptation with Adversarial Training and Graph Embeddings

1 code implementation ACL 2018 Firoj Alam, Shafiq Joty, Muhammad Imran

In such scenarios, a DNN model can leverage labeled and unlabeled data from a related domain, but it has to deal with the shift in data distributions between the source and the target domains.

Domain Adaptation

Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach

1 code implementation ACL 2018 Tasnim Mohiuddin, Shafiq Joty, Dat Tien Nguyen

We propose a novel coherence model for written asynchronous conversations (e. g., forums, emails), and show its applications in coherence assessment and thread reconstruction tasks.

Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets

no code implementations2 May 2018 Firoj Alam, Shafiq Joty, Muhammad Imran

During time-critical situations such as natural disasters, rapid classification of data posted on social networks by affected people is useful for humanitarian organizations to gain situational awareness and to plan response efforts.

General Classification Humanitarian

Models for Capturing Temporal Smoothness in Evolving Networks for Learning Latent Representation of Nodes

1 code implementation16 Apr 2018 Tanay Kumar Saha, Thomas Williams, Mohammad Al Hasan, Shafiq Joty, Nicholas K. Varberg

However, existing models for learning latent representation are inadequate for obtaining the representation vectors of the vertices for different time-stamps of a dynamic network in a meaningful way.

Link Prediction Representation Learning

Unpaired Image Captioning by Language Pivoting

no code implementations ECCV 2018 Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Gang Wang

Image captioning is a multimodal task involving computer vision and natural language processing, where the goal is to learn a mapping from the image to its natural language description.

Image Captioning Sentence

Hyperedge2vec: Distributed Representations for Hyperedges

no code implementations ICLR 2018 Ankit Sharma, Shafiq Joty, Himanshu Kharkwal, Jaideep Srivastava

We present a number of interesting baselines, some of which adapt existing node-level embedding models to the hyperedge-level, as well as sequence based language techniques which are adapted for set structured hypergraph topology.

Probabilistic Deep Learning Sentence

Cross-Language Question Re-Ranking

no code implementations4 Oct 2017 Giovanni Da San Martino, Salvatore Romeo, Alberto Barron-Cedeno, Shafiq Joty, Lluis Marquez, Alessandro Moschitti, Preslav Nakov

We compare a kernel-based system with a feed-forward neural network in a scenario where a large parallel corpus is available for training a machine translation system, bilingual dictionaries, and cross-language word embeddings.

Machine Translation Re-Ranking +1

Discourse Structure in Machine Translation Evaluation

no code implementations CL 2017 Shafiq Joty, Francisco Guzmán, Lluís Màrquez, Preslav Nakov

In this article, we explore the potential of using sentence-level discourse structure for machine translation evaluation.

Machine Translation Sentence +1

DeepER -- Deep Entity Resolution

3 code implementations2 Oct 2017 Muhammad Ebraheem, Saravanan Thirumuruganathan, Shafiq Joty, Mourad Ouzzani, Nan Tang

word embeddings), we present a novel ER system, called DeepER, that achieves good accuracy, high efficiency, as well as ease-of-use (i. e., much less human efforts).

Databases

Cross-language Learning with Adversarial Neural Networks

no code implementations CONLL 2017 Shafiq Joty, Preslav Nakov, Llu{\'\i}s M{\`a}rquez, Israa Jaradat

We address the problem of cross-language adaptation for question-question similarity reranking in community question answering, with the objective to port a system trained on one input language to another input language given labeled training data for the first language and only unlabeled data for the second language.

Community Question Answering Domain Adaptation +3

A Neural Local Coherence Model

1 code implementation ACL 2017 Dat Tien Nguyen, Shafiq Joty

We propose a local coherence model based on a convolutional neural network that operates over the entity grid representation of a text.

Text Generation

Cross-language Learning with Adversarial Neural Networks: Application to Community Question Answering

no code implementations21 Jun 2017 Shafiq Joty, Preslav Nakov, Lluís Màrquez, Israa Jaradat

We address the problem of cross-language adaptation for question-question similarity reranking in community question answering, with the objective to port a system trained on one input language to another input language given labeled training data for the first language and only unlabeled data for the second language.

Community Question Answering Question Similarity

Dis-S2V: Discourse Informed Sen2Vec

1 code implementation25 Oct 2016 Tanay Kumar Saha, Shafiq Joty, Naeemul Hassan, Mohammad Al Hasan

Our first approach retrofits (already trained) Sen2Vec vectors with respect to the network in two different ways: (1) using the adjacency relations of a node, and (2) using a stochastic sampling method which is more flexible in sampling neighbors of a node.

Clustering Computational Efficiency +1

Applications of Online Deep Learning for Crisis Response Using Social Media Information

no code implementations4 Oct 2016 Dat Tien Nguyen, Shafiq Joty, Muhammad Imran, Hassan Sajjad, Prasenjit Mitra

During natural or man-made disasters, humanitarian response organizations look for useful information to support their decision-making processes.

Decision Making Disaster Response +3

Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks

no code implementations12 Aug 2016 Dat Tien Nguyen, Kamela Ali Al Mannai, Shafiq Joty, Hassan Sajjad, Muhammad Imran, Prasenjit Mitra

The current state-of-the-art classification methods require a significant amount of labeled data specific to a particular event for training plus a lot of feature engineering to achieve best results.

BIG-bench Machine Learning Classification +2

Cannot find the paper you are looking for? You can Submit a new open access paper.