Search Results for author:
Found 60 papers, 27 papers with code
RNR: Teaching Large Language Models to Follow Roles and Rules
However, existing models trained on open-source IFT datasets only have the ability to follow instructions from users, and often fail to follow complex role and rules specified by developers, a. k. a.
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
For a LLM to be trustworthy, its confidence level should be well-calibrated with its actual performance.
Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
We consider the problem of multi-objective alignment of foundation models with human preferences, which is a critical step towards helpful and harmless AI systems.
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
Recognizing the need for a comprehensive evaluation of LMM chart understanding, we also propose a MultiModal Chart Benchmark (\textbf{MMC-Benchmark}), a comprehensive human-annotated benchmark with nine distinct tasks evaluating reasoning capabilities over charts.
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
In this work, we investigate how the instruction tuning adjusts pre-trained models with a focus on intrinsic changes.
Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
We investigate how to elicit compositional generalization capabilities in large language models (LLMs).
Ranked #34 on
Math Word Problem Solving
on MATH
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
Specifically, the detection technique achieves a recall of ~88% and the mitigation technique successfully mitigates 57. 6% of the correctly detected hallucinations.
PIVOINE: Instruction Tuning for Open-world Information Extraction
In particular, we construct INSTRUCTOPENWIKI, a substantial instruction tuning dataset for Open-world IE enriched with a comprehensive corpus, extensive annotations, and diverse instructions.
CEO: Corpus-based Open-Domain Event Ontology Induction
Existing event-centric NLP models often only apply to the pre-defined ontology, which significantly restricts their generalization capabilities.
Learning Language Representations with Logical Inductive Bias
We construct a set of neural logic operators as learnable Horn clauses, which are further forward-chained into a fully differentiable neural architecture (FOLNet).
ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base.
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
In this paper, we develop a novel semi-parametric language model architecture, Knowledge-in-Context (KiC), which empowers a parametric text-to-text language model with a knowledge-rich external memory.
Ranked #5 on
Question Answering
on StoryCloze
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks.
Ranked #2 on
Visual Commonsense Tests
on ViComTe-color
Explanations from Large Language Models Make Small Reasoners Better
Integrating free-text explanations to in-context learning of large language models (LLM) is shown to elicit strong reasoning capabilities along with reasonable explanations.
Join-Chain Network: A Logical Reasoning View of the Multi-head Attention in Transformer
Developing neural architectures that are capable of logical reasoning has become increasingly important for a wide range of applications (e. g., natural language processing).
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Notably, our proposed $\text{Zemi}_\text{LARGE}$ outperforms T0-3B by 16% on all seven evaluation tasks while being 3. 9x smaller in model size.
Blessing of Class Diversity in Pre-training
Here, $n$ is the number of pre-training data and $m$ is the number of data in the downstream task, and typically $n \gg m$.
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations.
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
And with our pretrained reader, the entire system improves by up to 4% in exact match.
PRIMA: Planner-Reasoner Inside a Multi-task Reasoning Agent
We consider the problem of multi-task reasoning (MTR), where an agent can solve multiple tasks via (first-order) logic reasoning.
Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories
We then train a model to identify semantic equivalence between a target word in context and one of its glosses using these aligned inventories, which exhibits strong transfer capability to many WSD tasks.
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge
In this paper, we aim to extract commonsense knowledge to improve machine reading comprehension.
Comprehensive Image Captioning via Scene Graph Decomposition
We address the challenging problem of image captioning by revisiting the representation of image scene graph.
ZPR2: Joint Zero Pronoun Recovery and Resolution using Multi-Task Learning and BERT
Zero pronoun recovery and resolution aim at recovering the dropped pronoun and pointing out its anaphoric mentions, respectively.
On Effective Parallelization of Monte Carlo Tree Search
Second, and more importantly, we demonstrate how the proposed necessary conditions can be adopted to design more effective parallel MCTS algorithms.
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
In this paper, we study machine reading comprehension (MRC) on long texts, where a model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer.
Logical Natural Language Generation from Open-Domain Tables
To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset \cite{chen2019tabfact} featured with a wide range of logical/symbolic inferences as our testbed, and propose new automatic metrics to evaluate the fidelity of generation models w. r. t.\ logical inference.
Improving Pre-Trained Multilingual Model with Vocabulary Expansion
However, in multilingual setting, it is extremely resource-consuming to pre-train a deep language model over large-scale corpora for each language.
Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach
Recently, pretrained language models (e. g., BERT) have achieved great success on many downstream natural language understanding tasks and exhibit a certain level of commonsense reasoning ability.
Learning to Recover Sparse Signals
In compressed sensing, a primary problem to solve is to reconstruct a high dimensional sparse signal from a small number of observations.
TabFact: A Large-scale Dataset for Table-based Fact Verification
To this end, we construct a large-scale dataset called TabFact with 16k Wikipedia tables as the evidence for 118k human-annotated natural language statements, which are labeled as either ENTAILED or REFUTED.
Ranked #12 on
Table-based Fact Verification
on TabFact
Stochastic Variance Reduced Primal Dual Algorithms for Empirical Composition Optimization
We consider a generic empirical composition optimization problem, where there are empirical averages present both outside and inside nonlinear loss functions.
From Caesar Cipher to Unsupervised Learning: A New Method for Classifier Parameter Estimation
To remove the need for the parallel training corpora has practical significance for real-world applications, and it is one of the main goals of unsupervised learning.
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Semantically controlled neural response generation on limited-domain has achieved great performance.
Ranked #5 on
Data-to-Text Generation
on MULTIWOZ 2.1
DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension
We present DREAM, the first dialogue-based multiple-choice reading comprehension data set.
Evidence Sentence Extraction for Machine Reading Comprehension
Remarkable success has been achieved in the last few years on some limited machine reading comprehension (MRC) tasks.
Improving Question Answering with External Knowledge
We focus on multiple-choice question answering (QA) tasks in subject areas such as science, where we require both broad background knowledge and the facts from the given subject-area reference corpus.
DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension
DREAM is likely to present significant challenges for existing reading comprehension systems: 84% of answers are non-extractive, 85% of questions require reasoning beyond a single sentence, and 34% of questions also involve commonsense knowledge.
Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching
We consider the problem of training speech recognition systems without using any labeled data, under the assumption that the learner can only access to the input utterances and a phoneme language model estimated from a non-overlapping corpus.
Coupled Variational Bayes via Optimization Embedding
This flexible function class couples the variational distribution with the original parameters in the graphical models, allowing end-to-end learning of the graphical models by back-propagation through the variational distribution.
Incorporating Structured Commonsense Knowledge in Story Completion
The ability to select an appropriate story ending is the first step towards perfect narrative comprehension.
Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search
Monte Carlo Tree Search (MCTS) algorithms have achieved great success on many challenging benchmarks (e. g., Computer Go).
XL-NBT: A Cross-lingual Neural Belief Tracking Framework
Then, we pre-train a state tracker for the source language as a teacher, which is able to exploit easy-to-access parallel data.
M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search
In order to effectively train the agent from sparse rewards, we combine MCTS with the neural policy to generate trajectories yielding more positive rewards.
Ranked #45 on
Link Prediction
on WN18RR
(Hits@3 metric)
SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
When function approximation is used, solving the Bellman optimality equation with stability guarantees has remained a major open problem in reinforcement learning for decades.
Q-LDA: Uncovering Latent Patterns in Text-based Sequential Decision Processes
In sequential decision making, it is often important and useful for end users to understand the underlying patterns or causes that lead to the corresponding decisions.
A Learning-to-Infer Method for Real-Time Power Grid Multi-Line Outage Identification
Identifying a potentially large number of simultaneous line outages in power transmission networks in real time is a computationally hard problem.
Stochastic Variance Reduction Methods for Policy Evaluation
Policy evaluation is a crucial step in many reinforcement-learning procedures, which estimates a value function that predicts states' long-term value under a given policy.
Unsupervised Sequence Classification using Sequential Output Statistics
Although it is harder to optimize in its functional form, a stochastic primal-dual gradient method is developed to effectively solve the problem.
Character-level Deep Conflation for Business Data Analytics
Connecting different text attributes associated with the same entity (conflation) is important in business data analytics since it could help merge two different tables in a database to provide a more comprehensive profile of an entity.
Unsupervised Learning of Predictors from Unpaired Input-Output Samples
In particular, we show that with regularization via a generative model, learning with the proposed unsupervised objective function converges to an optimal solution.
Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads
We introduce an online popularity prediction and tracking task as a benchmark task for reinforcement learning with a combinatorial, natural language action space.
Deep Reinforcement Learning with a Natural Language Action Space
This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games.
Recurrent Reinforcement Learning: A Hybrid Approach
Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states.
End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture
We develop a fully discriminative learning approach for supervised Latent Dirichlet Allocation (LDA) model using Back Propagation (i. e., BP-sLDA), which maximizes the posterior probability of the prediction variable given the input document.
A Deep Embedding Model for Co-occurrence Learning
Co-occurrence Data is a common and important information source in many areas, such as the word co-occurrence in the sentences, friends co-occurrence in social networks and products co-occurrence in commercial transaction data, etc, which contains rich correlation and clustering information about the items.
Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval
The results show that the proposed method in this paper significantly outperforms it for web document retrieval task.
Dictionary Learning over Distributed Models
In this paper, we consider learning dictionary models over a network of agents, where each agent is only in charge of a portion of the dictionary elements.
Distributed Policy Evaluation Under Multiple Behavior Strategies
We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment.
A Primal-Dual Method for Training Recurrent Neural Networks Constrained by the Echo-State Property
We present an architecture of a recurrent neural network (RNN) with a fully-connected deep neural network (DNN) as its feature extractor.
