Search Results for author:
Found 59 papers, 26 papers with code
A Primal-Dual Method for Training Recurrent Neural Networks Constrained by the Echo-State Property
We present an architecture of a recurrent neural network (RNN) with a fully-connected deep neural network (DNN) as its feature extractor.
Distributed Policy Evaluation Under Multiple Behavior Strategies
We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment.
Dictionary Learning over Distributed Models
In this paper, we consider learning dictionary models over a network of agents, where each agent is only in charge of a portion of the dictionary elements.
Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval
The results show that the proposed method in this paper significantly outperforms it for web document retrieval task.
A Deep Embedding Model for Co-occurrence Learning
Co-occurrence Data is a common and important information source in many areas, such as the word co-occurrence in the sentences, friends co-occurrence in social networks and products co-occurrence in commercial transaction data, etc, which contains rich correlation and clustering information about the items.
End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture
We develop a fully discriminative learning approach for supervised Latent Dirichlet Allocation (LDA) model using Back Propagation (i. e., BP-sLDA), which maximizes the posterior probability of the prediction variable given the input document.
Recurrent Reinforcement Learning: A Hybrid Approach
Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states.
Deep Reinforcement Learning with a Natural Language Action Space
This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games.
Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads
We introduce an online popularity prediction and tracking task as a benchmark task for reinforcement learning with a combinatorial, natural language action space.
Unsupervised Learning of Predictors from Unpaired Input-Output Samples
In particular, we show that with regularization via a generative model, learning with the proposed unsupervised objective function converges to an optimal solution.
Character-level Deep Conflation for Business Data Analytics
Connecting different text attributes associated with the same entity (conflation) is important in business data analytics since it could help merge two different tables in a database to provide a more comprehensive profile of an entity.
Unsupervised Sequence Classification using Sequential Output Statistics
Although it is harder to optimize in its functional form, a stochastic primal-dual gradient method is developed to effectively solve the problem.
Stochastic Variance Reduction Methods for Policy Evaluation
Policy evaluation is a crucial step in many reinforcement-learning procedures, which estimates a value function that predicts states' long-term value under a given policy.
A Learning-to-Infer Method for Real-Time Power Grid Multi-Line Outage Identification
Identifying a potentially large number of simultaneous line outages in power transmission networks in real time is a computationally hard problem.
Q-LDA: Uncovering Latent Patterns in Text-based Sequential Decision Processes
In sequential decision making, it is often important and useful for end users to understand the underlying patterns or causes that lead to the corresponding decisions.
SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
When function approximation is used, solving the Bellman optimality equation with stability guarantees has remained a major open problem in reinforcement learning for decades.
M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search
In order to effectively train the agent from sparse rewards, we combine MCTS with the neural policy to generate trajectories yielding more positive rewards.
Ranked #44 on
Link Prediction
on WN18RR
(Hits@3 metric)
XL-NBT: A Cross-lingual Neural Belief Tracking Framework
Then, we pre-train a state tracker for the source language as a teacher, which is able to exploit easy-to-access parallel data.
Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search
Monte Carlo Tree Search (MCTS) algorithms have achieved great success on many challenging benchmarks (e. g., Computer Go).
Incorporating Structured Commonsense Knowledge in Story Completion
The ability to select an appropriate story ending is the first step towards perfect narrative comprehension.
Coupled Variational Bayes via Optimization Embedding
This flexible function class couples the variational distribution with the original parameters in the graphical models, allowing end-to-end learning of the graphical models by back-propagation through the variational distribution.
Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching
We consider the problem of training speech recognition systems without using any labeled data, under the assumption that the learner can only access to the input utterances and a phoneme language model estimated from a non-overlapping corpus.
DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension
DREAM is likely to present significant challenges for existing reading comprehension systems: 84% of answers are non-extractive, 85% of questions require reasoning beyond a single sentence, and 34% of questions also involve commonsense knowledge.
Improving Question Answering with External Knowledge
We focus on multiple-choice question answering (QA) tasks in subject areas such as science, where we require both broad background knowledge and the facts from the given subject-area reference corpus.
Evidence Sentence Extraction for Machine Reading Comprehension
Remarkable success has been achieved in the last few years on some limited machine reading comprehension (MRC) tasks.
DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension
We present DREAM, the first dialogue-based multiple-choice reading comprehension data set.
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Semantically controlled neural response generation on limited-domain has achieved great performance.
Ranked #5 on
Data-to-Text Generation
on MULTIWOZ 2.1
From Caesar Cipher to Unsupervised Learning: A New Method for Classifier Parameter Estimation
To remove the need for the parallel training corpora has practical significance for real-world applications, and it is one of the main goals of unsupervised learning.
Stochastic Variance Reduced Primal Dual Algorithms for Empirical Composition Optimization
We consider a generic empirical composition optimization problem, where there are empirical averages present both outside and inside nonlinear loss functions.
TabFact: A Large-scale Dataset for Table-based Fact Verification
To this end, we construct a large-scale dataset called TabFact with 16k Wikipedia tables as the evidence for 118k human-annotated natural language statements, which are labeled as either ENTAILED or REFUTED.
Ranked #10 on
Table-based Fact Verification
on TabFact
Learning to Recover Sparse Signals
In compressed sensing, a primary problem to solve is to reconstruct a high dimensional sparse signal from a small number of observations.
Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach
Recently, pretrained language models (e. g., BERT) have achieved great success on many downstream natural language understanding tasks and exhibit a certain level of commonsense reasoning ability.
Improving Pre-Trained Multilingual Model with Vocabulary Expansion
However, in multilingual setting, it is extremely resource-consuming to pre-train a deep language model over large-scale corpora for each language.
Logical Natural Language Generation from Open-Domain Tables
To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset \cite{chen2019tabfact} featured with a wide range of logical/symbolic inferences as our testbed, and propose new automatic metrics to evaluate the fidelity of generation models w. r. t.\ logical inference.
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
In this paper, we study machine reading comprehension (MRC) on long texts, where a model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer.
On Effective Parallelization of Monte Carlo Tree Search
Second, and more importantly, we demonstrate how the proposed necessary conditions can be adopted to design more effective parallel MCTS algorithms.
ZPR2: Joint Zero Pronoun Recovery and Resolution using Multi-Task Learning and BERT
Zero pronoun recovery and resolution aim at recovering the dropped pronoun and pointing out its anaphoric mentions, respectively.
Comprehensive Image Captioning via Scene Graph Decomposition
We address the challenging problem of image captioning by revisiting the representation of image scene graph.
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge
In this paper, we aim to extract commonsense knowledge to improve machine reading comprehension.
Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories
We then train a model to identify semantic equivalence between a target word in context and one of its glosses using these aligned inventories, which exhibits strong transfer capability to many WSD tasks.
PRIMA: Planner-Reasoner Inside a Multi-task Reasoning Agent
We consider the problem of multi-task reasoning (MTR), where an agent can solve multiple tasks via (first-order) logic reasoning.
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
And with our pretrained reader, the entire system improves by up to 4% in exact match.
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations.
Blessing of Class Diversity in Pre-training
Here, $n$ is the number of pre-training data and $m$ is the number of data in the downstream task, and typically $n \gg m$.
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Notably, our proposed $\text{Zemi}_\text{LARGE}$ outperforms T0-3B by 16% on all seven evaluation tasks while being 3. 9x smaller in model size.
Join-Chain Network: A Logical Reasoning View of the Multi-head Attention in Transformer
Developing neural architectures that are capable of logical reasoning has become increasingly important for a wide range of applications (e. g., natural language processing).
Explanations from Large Language Models Make Small Reasoners Better
Integrating free-text explanations to in-context learning of large language models (LLM) is shown to elicit strong reasoning capabilities along with reasonable explanations.
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks.
Ranked #2 on
Visual Commonsense Tests
on ViComTe-color
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
In this paper, we develop a novel semi-parametric language model architecture, Knowledge-in-Context (KiC), which empowers a parametric text-to-text language model with a knowledge-rich external memory.
Ranked #5 on
Question Answering
on StoryCloze
ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base.
Learning Language Representations with Logical Inductive Bias
We construct a set of neural logic operators as learnable Horn clauses, which are further forward-chained into a fully differentiable neural architecture (FOLNet).
CEO: Corpus-based Open-Domain Event Ontology Induction
Existing event-centric NLP models often only apply to the pre-defined ontology, which significantly restricts their generalization capabilities.
PIVOINE: Instruction Tuning for Open-world Information Extraction
In particular, we construct INSTRUCTOPENWIKI, a substantial instruction tuning dataset for Open-world IE enriched with a comprehensive corpus, extensive annotations, and diverse instructions.
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
Specifically, the detection technique achieves a recall of ~88% and the mitigation technique successfully mitigates 57. 6% of the correctly detected hallucinations.
Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
Compositional generalization empowers the LLMs to solve problems that are harder than the ones they have seen (i. e., easy-to-hard generalization), which is a critical reasoning capability of human-like intelligence.
Ranked #17 on
Math Word Problem Solving
on MATH
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
In this work, we investigate how the instruction tuning adjusts pre-trained models with a focus on intrinsic changes.
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
Recognizing the need for a comprehensive evaluation of LMM chart understanding, we also propose a MultiModal Chart Benchmark (\textbf{MMC-Benchmark}), a comprehensive human-annotated benchmark with nine distinct tasks evaluating reasoning capabilities over charts.
Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
We consider the problem of multi-objective alignment of foundation models with human preferences, which is a critical step towards helpful and harmless AI systems.
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
For a LLM to be trustworthy, its confidence level should be well-calibrated with its actual performance.
