Search Results for author:
Found 62 papers, 28 papers with code
Evaluating LLMs at Detecting Errors in LLM Responses
This work introduces ReaLMistake, the first error detection benchmark consisting of objective, realistic, and diverse errors made by LLMs.
X-Shot: A Unified System to Handle Frequent, Few-shot and Zero-shot Learning Simultaneously in Classification
In recent years, few-shot and zero-shot learning, which learn to predict labels with limited annotated instances, have garnered significant attention.
FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability
This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats, a crucial yet underexamined capability for their application as AI agents.
Multimodal Instruction Tuning with Conditional Mixture of LoRA
Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in diverse tasks across different domains, with an increasing focus on improving their zero-shot generalization capabilities for unseen multimodal tasks.
Contrastive Instruction Tuning
Instruction tuning has been used as a promising approach to improve the performance of large language models (LLMs) on unseen tasks.
Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models
Our in-depth study advocates more careful use of ME in real-world scenarios.
Large Language Models for Mathematical Reasoning: Progresses and Challenges
Mathematical reasoning serves as a cornerstone for assessing the fundamental cognitive capabilities of human intelligence.
MT-Ranker: Reference-free machine translation evaluation by inter-system ranking
Traditionally, Machine Translation (MT) Evaluation has been treated as a regression problem -- producing an absolute translation-quality score.
GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models
In this work, we propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs (e. g., GPT-4 \cite{openai2023gpt4}) to assess bias in models.
MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Following
In the realm of large language models (LLMs), enhancing instruction-following capability often involves curating expansive training data.
Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning
Unfortunately, this requires formatting them into specialized augmented format unknown to the base pretrained language model (PLMs) necessitating finetuning to the target format.
All Labels Together: Low-shot Intent Detection with an Efficient Label Semantic Encoding Paradigm
In intent detection tasks, leveraging meaningful semantic information from intent labels can be particularly beneficial for few-shot scenarios.
Toward Zero-Shot Instruction Following
This work proposes a challenging yet more realistic setting for zero-shot cross-task generalization: zero-shot instruction following, presuming the existence of a paragraph-style task definition while no demonstrations exist.
A Comprehensive Survey on Instruction Following
This survey paper tries to summarize and provide insights to the current research on instruction following, particularly, by answering the following questions: (i) What is task instruction, and what instruction types exist?
Robustness of Learning from Task Instructions
To our knowledge, this is the first work that systematically studies how robust a PLM is when it is supervised by instructions with different factors of variability.
Learning to Select from Multiple Options
To deal with the two issues, this work first proposes a contextualized TE model (Context-TE) by appending other k options as the context of the current (P, H) modeling.
OpenStance: Real-world Zero-shot Stance Detection
To our knowledge, this is the first work that studies stance detection under the open-domain zero-shot setting.
Anti-Overestimation Dialogue Policy Learning for Task-Completion Dialogue System
This problem is detrimental to RL-based dialogue policy learning.
Converse: A Tree-Based Modular Task-Oriented Dialogue System
At the core of the struggle is the need to script every single turn of interactions between the bot and the human user.
ConTinTin: Continual Learning from Task Instructions
This work defines a new learning paradigm ConTinTin (Continual Learning from Task Instructions), in which a system should learn a sequence of new tasks one by one, each task is explained by a piece of textual instruction.
Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference
The task of ultra-fine entity typing (UFET) seeks to predict diverse and free-form words or phrases that describe the appropriate types of entities mentioned in sentences.
Ranked #1 on
Entity Typing
on FIGER
Event Linking: Grounding Event Mentions to Wikipedia
Third, we retrain and evaluate two state-of-the-art (SOTA) entity linking models, showing the challenges of event linking, and we propose an event-specific linking system EVELINK to set a competitive result for the new task.
Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution
The distribution gap between training datasets and data encountered in production is well acknowledged.
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference
It has been studied intensively in the past few years thanks to the availability of large-scale labeled datasets.
Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System
Two major challenges exist in this new task: (i) For the learning process, the system should incrementally learn new classes round by round without re-training on the examples of preceding classes; (ii) For the performance, the system should perform well on new classes without much loss on preceding classes.
Learning to Synthesize Data for Semantic Parsing
Moreover, explicitly modeling compositions using PCFG leads to a better exploration of unseen programs, thus generate more diverse data.
Pairwise Representation Learning for Event Coreference
Natural Language Processing tasks such as resolving the coreference of events require understanding the relations between two text snippets.
Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start
We demonstrate that this framework enables a pretrained entailment model to work well on new entailment domains in a few-shot setting, and show its effectiveness as a unified solver for several downstream NLP tasks such as question answering and coreference resolution when the end-task annotations are limited.
Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks
Our studies show that mixup is a domain-independent data augmentation technique to pre-trained language models, resulting in significant performance improvement for transformer-based models.
Meta-learning for Few-shot Natural Language Processing: A Survey
If the target task itself cannot provide more information, how about collecting more tasks equipped with rich annotations to help the model learning?
CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization
The COVID-19 global pandemic has resulted in international efforts to understand, track, and mitigate the disease, yielding a significant corpus of COVID-19 and SARS-CoV-2-related publications across scientific disciplines.
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
There is an increasing amount of literature that claims the brittleness of deep neural networks in dealing with adversarial examples that are created maliciously.
Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
0Shot-TC aims to associate an appropriate label with a piece of text, irrespective of the text domain and the aspect (e. g., topic, emotion, event, etc.)
Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing
Multi-Task Learning (MTL) aims at boosting the overall performance of each individual task by leveraging useful information contained in multiple related tasks.
Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims
Inherently, this is a natural language understanding task, and we propose to address it as such.
Seeing Things from a Different Angle:Discovering Diverse Perspectives about Claims
Inherently, this is a natural language understanding task, and we propose to address it as such.
TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification
We develop TwoWingOS (two-wing optimization strategy), a system that, while identifying appropriate evidence for a claim, also determines whether or not the claim is supported by the evidence.
Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs
Large scale knowledge graphs (KGs) such as Freebase are generally incomplete.
Term Definitions Help Hypernymy Detection
Existing methods of hypernymy detection mainly rely on statistics over a big corpus, either mining some co-occurring patterns like "animals such as cats" or embedding words of interest into context-aware vectors.
End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions
This work deals with SciTail, a natural entailment challenge derived from a multi-choice question answering problem.
Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
We hypothesize that this is because the attention in CNNs has been mainly implemented as attentive pooling (i. e., it is applied to pooling) rather than as attentive convolution (i. e., it is integrated into convolution).
Improved Neural Relation Detection for Knowledge Base Question Answering
Relation detection is a core component for many NLP applications including Knowledge Base Question Answering (KBQA).
Comparative Study of CNN and RNN for Natural Language Processing
Deep neural networks (DNN) have revolutionized the field of natural language processing (NLP).
Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching
This work studies comparatively two typical sentence matching tasks: textual entailment (TE) and answer selection (AS), observing that weaker phrase alignments are more critical in TE, while stronger phrase alignments deserve more attention in AS.
Simple Question Answering by Attentive Convolutional Neural Network
In fact selection, we match the subject entity in a fact candidate with the entity mention in the question by a character-level convolutional neural network (char-CNN), and match the predicate in that fact with the question by a word-level CNN (word-CNN).
Why and How to Pay Different Attention to Phrase Alignments of Different Intensities
We address the problems of identifying phrase alignments of flexible granularity and pooling alignments of different intensities for these tasks.
Discriminative Phrase Embedding for Paraphrase Identification
This work, concerning paraphrase identification task, on one hand contributes to expanding deep learning embeddings to include continuous and discontinuous linguistic phrases.
Online Updating of Word Representations for Part-of-Speech Tagging
We propose online unsupervised domain adaptation (DA), which is performed incrementally as data comes in and is applicable when batch DA is not possible.
Online unsupervised domain adaptation
Part-Of-Speech Tagging
+2
Multichannel Variable-Size Convolution for Sentence Classification
We propose MVCNN, a convolution neural network (CNN) architecture for sentence classification.
Attention-Based Convolutional Neural Network for Machine Comprehension
Understanding open-domain text is one of the primary challenges in natural language processing (NLP).
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
(ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart.
Learning Meta-Embeddings by Using Ensembles of Embedding Sets
Word embeddings -- distributed representations of words -- in deep learning are beneficial for many tasks in natural language processing (NLP).
Deep Learning Embeddings for Discontinuous Linguistic Units
Deep learning embeddings have been successfully used for many natural language processing problems.
