Search Results for author:
Found 277 papers, 145 papers with code
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
Ranked #3 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
To democratize this, we train and release a family of large language models up to 16. 1B parameters, called CODEGEN, on natural language and programming language data, and open source the training library JAXFORMER.
Ranked #81 on
Code Generation
on HumanEval
CTRL: A Conditional Transformer Language Model for Controllable Generation
Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text.
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
Ranked #5 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions
This work presents Moonshot, a new video generation model that conditions simultaneously on multimodal inputs of image and text.
CodeGen2: Lessons for Training LLMs on Programming and Natural Languages
In this study, we attempt to render the training of LLMs for program synthesis more efficient by unifying four key components: (1) model architectures, (2) learning methods, (3) infill sampling, and, (4) data distributions.
Robustness Evaluation of Transformer-based Form Field Extractors via Form Attacks
We propose a novel framework to evaluate the robustness of transformer-based form field extraction methods via form attacks.
OpenAgents: An Open Platform for Language Agents in the Wild
Language agents show potential in being capable of utilizing natural language for varied and intricate tasks in diverse environments, particularly when built upon large language models (LLMs).
Merlion: A Machine Learning Library for Time Series
We introduce Merlion, an open-source machine learning library for time series.
The Natural Language Decathlon: Multitask Learning as Question Answering
Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting.
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
A significant amount of the world's knowledge is stored in relational databases.
Ranked #9 on
Code Generation
on WikiSQL
Quasi-Recurrent Neural Networks
Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism and makes RNNs unwieldy for very long sequences.
Ranked #15 on
Machine Translation
on IWSLT2015 German-English
Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems
Prior works mainly focus on adopting advanced RL techniques to train the ToD agents, while the design of the reward function is not well studied.
MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation
Counterfactual explanation is an important Explainable AI technique to explain machine learning predictions.
XGen-7B Technical Report
Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.
Robustness Gym: Unifying the NLP Evaluation Landscape
Despite impressive performance on standard benchmarks, deep neural networks are often brittle when deployed in real-world systems.
UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild
Visual generative foundation models such as Stable Diffusion show promise in navigating these goals, especially when prompted with arbitrary languages.
Prototypical Contrastive Learning of Unsupervised Representations
This paper presents Prototypical Contrastive Learning (PCL), an unsupervised representation learning method that addresses the fundamental limitations of instance-wise contrastive learning.
Ranked #5 on
Contrastive Learning
on imagenet-1k
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.
Ranked #1 on
Task-Oriented Dialogue Systems
on KVRET
A Deep Reinforced Model for Abstractive Summarization
We introduce a neural network model with a novel intra-attention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL).
Ranked #6 on
Text Summarization
on CNN / Daily Mail (Anonymized)
Lemur: Harmonizing Natural Language and Code for Language Agents
We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents.
Learned in Translation: Contextualized Word Vectors
For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.
Ranked #9 on
Text Classification
on TREC-6
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI
Despite advancements in conversational AI, language models encounter challenges to handle diverse conversational tasks, and existing dialogue dataset collections often lack diversity and comprehensiveness.
Unified Training of Universal Time Series Forecasting Transformers
Deep learning for time series forecasting has traditionally operated within a one-model-per-dataset framework, limiting its potential to leverage the game-changing impact of large pre-trained models.
ESPRIT: Explaining Solutions to Physical Reasoning Tasks
Our framework learns to generate explanations of how the physical simulation will causally evolve so that an agent or a human can easily reason about a solution using those interpretable descriptions.
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question.
Ranked #26 on
Question Answering
on HotpotQA
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Over-dependence on domain ontology and lack of knowledge sharing across domains are two practical and yet less studied problems of dialogue state tracking.
Ranked #15 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.0
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+2
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding
Then, ULIP learns a 3D representation space aligned with the common image-text space, using a small number of automatically synthesized triplets.
Ranked #3 on
Training-free 3D Point Cloud Classification
on ModelNet40
(using extra training data)
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding
It achieves a new SOTA of 50. 6% (top-1) on Objaverse-LVIS and 84. 7% (top-1) on ModelNet40 in zero-shot classification.
Ranked #6 on
3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
SummEval: Re-evaluating Summarization Evaluation
The scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continue to inhibit progress.
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
The model decides whether to attend to the image and where, in order to extract meaningful information for sequential word generation.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
BERTology Meets Biology: Interpreting Attention in Protein Language Models
Transformer architectures have proven to learn useful representations for protein classification and generation tasks.
Multi-Hop Knowledge Graph Reasoning with Reward Shaping
Multi-hop reasoning is an effective approach for query answering (QA) over incomplete knowledge graphs (KGs).
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
The massive successes of large language models (LLMs) encourage the emerging exploration of LLM-augmented Autonomous Agents (LAAs).
AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System
Thus, we open-source a new AI agent library, AgentLite, which simplifies this process by offering a lightweight, user-friendly platform for innovating LLM agent reasoning, architectures, and applications with ease.
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue
The underlying difference of linguistic patterns between general text and task-oriented dialogue makes existing pre-trained language models less useful in practice.
Binding Language Models in Symbolic Languages
We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e. g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations.
Ranked #4 on
Table-based Fact Verification
on TabFact
Non-Autoregressive Neural Machine Translation
Existing approaches to neural machine translation condition each output word on previously generated outputs.
Ranked #3 on
Machine Translation
on IWSLT2015 English-German
Dynamic Memory Networks for Visual and Textual Question Answering
Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering.
Ranked #4 on
Visual Question Answering (VQA)
on VQA v1 test-std
Salesforce CausalAI Library: A Fast and Scalable Framework for Causal Analysis of Time Series and Tabular Data
Finally, we provide a user interface (UI) that allows users to perform causal analysis on data without coding.
Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing
We present BRIDGE, a powerful sequential architecture for modeling dependencies between natural language questions and relational databases in cross-DB semantic parsing.
Pointer Sentinel Mixture Models
Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies.
SParC: Cross-Domain Semantic Parsing in Context
The best model obtains an exact match accuracy of 20. 2% over all questions and less than10% over all interaction sequences, indicating that the cross-domain setting and the con-textual phenomena of the dataset present significant challenges for future research.
Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions
We focus on the cross-domain context-dependent text-to-SQL generation task.
Ranked #5 on
Text-To-SQL
on SParC
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases
We present CoSQL, a corpus for building cross-domain, general-purpose database (DB) querying dialogue systems.
Ranked #8 on
Dialogue State Tracking
on CoSQL
Global-Locally Self-Attentive Dialogue State Tracker
Dialogue state tracking, which estimates user goals and requests given the dialogue context, is an essential part of task-oriented dialogue systems.
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+1
End-to-End Dense Video Captioning with Masked Transformer
To address this problem, we propose an end-to-end transformer model for dense video captioning.
Ranked #11 on
Video Captioning
on YouCook2
BookSum: A Collection of Datasets for Long-form Narrative Summarization
The majority of available text summarization datasets include short-form source documents that lack long-range causal and temporal dependencies, and often contain strong layout and stylistic biases.
Global-to-local Memory Pointer Networks for Task-Oriented Dialogue
In our model, a global memory encoder and a local memory decoder are proposed to share external knowledge.
Ranked #4 on
Task-Oriented Dialogue Systems
on KVRET
CTRLsum: Towards Generic Controllable Text Summarization
Our approach enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts.
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of world-knowledge or reasoning over information not immediately present in the input.
Ranked #22 on
Common Sense Reasoning
on CommonsenseQA
DART: Open-Domain Structured Data Record to Text Generation
Data-to-Text annotations can be a costly process, especially when dealing with tables which are the major source of structured data and contain nontrivial structures.
Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework
Current contrastive learning frameworks focus on leveraging a single supervisory signal to learn representations, which limits the efficacy on unseen data and downstream tasks.
Converse: A Tree-Based Modular Task-Oriented Dialogue System
At the core of the struggle is the need to script every single turn of interactions between the bot and the human user.
Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
The Regretful Navigation Agent for Vision-and-Language Navigation
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
CoMatch: Semi-supervised Learning with Contrastive Graph Regularization
CoMatch jointly learns two representations of the training data, their class probabilities and low-dimensional embeddings.
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
Natural Language Visual Grounding
Vision and Language Navigation
+2
RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering
We present RnG-KBQA, a Rank-and-Generate approach for KBQA, which remedies the coverage issue with a generation model while preserving a strong generalization capability.
FOLIO: Natural Language Reasoning with First-Order Logic
We present FOLIO, a human-annotated, open-domain, and logically complex and diverse dataset for reasoning in natural language (NL), equipped with first order logic (FOL) annotations.
ERASER: A Benchmark to Evaluate Rationalized NLP Models
We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i. e., the degree to which provided rationales influenced the corresponding predictions).
Improved Online Conformal Prediction via Strongly Adaptive Online Learning
We prove that our methods achieve near-optimal strongly adaptive regret for all interval lengths simultaneously, and approximately valid coverage.
Learning to Synthesize Data for Semantic Parsing
Moreover, explicitly modeling compositions using PCFG leads to a better exploration of unseen programs, thus generate more diverse data.
MoPro: Webly Supervised Learning with Momentum Prototypes
We propose momentum prototypes (MoPro), a simple contrastive learning method that achieves online label noise correction, out-of-distribution sample removal, and representation learning.
Ranked #12 on
Image Classification
on OmniBenchmark
(using extra training data)
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
With the availability of the fast influence functions, we demonstrate their usefulness in four applications.
QAConv: Question Answering on Informative Conversations
This paper introduces QAConv, a new question answering (QA) dataset that uses conversations as a knowledge source.
Intent Contrastive Learning for Sequential Recommendation
Specifically, we introduce a latent variable to represent users' intents and learn the distribution function of the latent variable via clustering.
HIVE: Harnessing Human Feedback for Instructional Visual Editing
Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences.
Contrastive Self-supervised Sequential Recommendation with Robust Augmentation
In this paper, we investigate the application of contrastive Self-Supervised Learning (SSL) to the sequential recommendation, as a way to alleviate some of these issues.
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference
It has been studied intensively in the past few years thanks to the availability of large-scale labeled datasets.
LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer
Graphic layout designs play an essential role in visual communication.
FeTaQA: Free-form Table Question Answering
Existing table question answering datasets contain abundant factual questions that primarily evaluate the query and schema comprehension capability of a system, but they fail to include questions that require complex reasoning and integration of information due to the constraint of the associated short-form answers.
ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?
Upon its release in late 2022, ChatGPT has brought a seismic shift in the entire landscape of AI, both in research and commerce.
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits.
Predicting with High Correlation Features
In this paper, we consider distribution shift as a shift in the distribution of input features during test time that exhibit low correlation with targets in the training set.
GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation
Empirical results show that GlueNet can be trained efficiently and enables various capabilities beyond previous state-of-the-art models: 1) multilingual language models such as XLM-Roberta can be aligned with existing T2I models, allowing for the generation of high-quality images from captions beyond English; 2) GlueNet can align multi-modal encoders such as AudioCLIP with the Stable Diffusion model, enabling sound-to-image generation; 3) it can also upgrade the current text encoder of the latent diffusion model for challenging case generation.
CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers
Dialogue state trackers have made significant progress on benchmark datasets, but their generalization capability to novel and realistic scenarios beyond the held-out conversations is less understood.
Ranked #2 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.1
(using extra training data)
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels
To enlarge the set of base classes, we propose a method to automatically generate pseudo bounding-box annotations of diverse objects from large-scale image-caption pairs.
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
Transfer and multi-task learning have traditionally focused on either a single source-target pair or very few, similar tasks.
Ranked #3 on
Chunking
on Penn Treebank
VD-BERT: A Unified Vision and Dialog Transformer with BERT
By contrast, in this work, we propose VD-BERT, a simple yet effective framework of unified vision-dialog Transformer that leverages the pretrained BERT language models for Visual Dialog tasks.
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference
Intent detection is one of the core components of goal-oriented dialog systems, and detecting out-of-scope (OOS) intents is also a practically important skill.
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Human evaluation is the foundation upon which the evaluation of both summarization systems and automatic metrics rests.
Efficient and Robust Question Answering from Minimal Context over Documents
Neural models for question answering (QA) over documents have achieved significant performance improvements.
Ranked #3 on
Question Answering
on NewsQA
DialFact: A Benchmark for Fact-Checking in Dialogue
Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation.
X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning
Vision-language pre-training and instruction tuning have demonstrated general-purpose capabilities in 2D visual reasoning tasks by aligning visual encoders with state-of-the-art large language models (LLMs).
EMT: Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
The goal of conversational machine reading is to answer user questions given a knowledge base text which may require asking clarification questions.
Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
The goal of conversational machine reading is to answer user questions given a knowledge base text which may require asking clarification questions.
Controllable Abstractive Dialogue Summarization with Sketch Supervision
In this paper, we aim to improve abstractive dialogue summarization quality and, at the same time, enable granularity control.
AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning
It meticulously standardizes and unifies these trajectories into a consistent format, streamlining the creation of a generic data loader optimized for agent training.
Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading
Based on the learned EDU and entailment representations, we either reply to the user our final decision "yes/no/irrelevant" of the initial question, or generate a follow-up question to inquiry more information.
Structured Scene Memory for Vision-Language Navigation
Recently, numerous algorithms have been developed to tackle the problem of vision-language navigation (VLN), i. e., entailing an agent to navigate 3D environments through following linguistic instructions.
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization
Factual consistency is an essential quality of text summarization models in practical settings.
A High-Quality Multilingual Dataset for Structured Documentation Translation
We build and evaluate translation models for seven target languages from English, with several different copy mechanisms and an XML-constrained beam search.
Structure Extraction in Task-Oriented Dialogues with Slot Clustering
Extracting structure information from dialogue data can help us better understand user and system behaviors.
MixQG: Neural Question Generation with Mixed Answer Types
Asking good questions is an essential ability for both human and machine intelligence.
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing
We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data.
Ranked #8 on
Semantic Parsing
on spider
Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking
Dialog state tracking (DST) is a core component in task-oriented dialog systems.
Ranked #4 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.0
dialog state tracking
Multi-domain Dialogue State Tracking
+1
Unsupervised Out-of-Domain Detection via Pre-trained Transformers
Deployed real-world machine learning applications are often subject to uncontrolled and even potentially malicious inputs.
How Much are LLMs Contaminated? A Comprehensive Survey and the LLMSanitize Library
With the rise of Large Language Models (LLMs) in recent years, new opportunities are emerging, but also new challenges, and contamination is quickly becoming critical.
Neural Abstract Style Transfer for Chinese Traditional Painting
In experiments, the proposed method shows more appealing stylized results in transferring the style of Chinese traditional painting than state-of-the-art neural style transfer methods.
Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation
Word embeddings derived from human-generated corpora inherit strong gender bias which can be further amplified by downstream models.
Learning from Noisy Anchors for One-stage Object Detection
State-of-the-art object detectors rely on regressing and classifying an extensive list of possible anchors, which are divided into positive and negative samples based on their intersection-over-union (IoU) with corresponding groundtruth objects.
A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis
Our evaluation results on the single-task polarity prediction show that our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+4
Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning
Disjoint Manifold Labeling: Neural Bayes allows us to formulate an objective which can optimally label samples from disjoint manifolds present in the support of a continuous distribution.
Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
We first show that this chaotic behavior exists even along the training optimization trajectory of a single model, and propose a simple model averaging protocol that both significantly boosts domain generalization and diminishes the impact of stochasticity by improving the rank correlation between the in-domain validation accuracy and out-domain test accuracy, which is crucial for reliable early stopping.
Ranked #4 on
Domain Generalization
on TerraIncognita
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks.
Evaluating the Factual Consistency of Abstractive Text Summarization
Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents.
RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems
Robust machine learning is an increasingly important topic that focuses on developing models resilient to various forms of imperfect data.
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Text simplification research has mostly focused on sentence-level simplification, even though many desirable edits - such as adding relevant background information or reordering content - may require document-level context.
Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewards
For instance, in tasks where the agent must achieve some goal state, simple distance-to-goal reward shaping often fails, as it renders learning vulnerable to local optima.
Learning From Noisy Data With Robust Representation Learning
In contrast to most existing methods, we combat noise by learning robust representation.
Improving Contrastive Learning with Model Augmentation
However, existing methods all construct views by adopting augmentation from data perspectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals.
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models
Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks.
Causal-aware Safe Policy Improvement for Task-oriented dialogue
This method gives guarantees on dialogue policy's performance and also learns to shape rewards according to intentions behind human responses, rather than just mimicking demonstration data; this couple with batch-RL helps overall with sample efficiency of the framework.
Field Extraction from Forms with Unlabeled Data
We propose a novel framework to conduct field extraction from forms with unlabeled data.
Value Retrieval with Arbitrary Queries for Form-like Documents
Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form.
DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents
Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form.
Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start
We demonstrate that this framework enables a pretrained entailment model to work well on new entailment domains in a few-shot setting, and show its effectiveness as a unified solver for several downstream NLP tasks such as question answering and coreference resolution when the end-task annotations are limited.
ELECRec: Training Sequential Recommenders as Discriminators
A generator, as an auxiliary model, is trained jointly with the discriminator to sample plausible alternative next items and will be thrown out after training.
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics.
Representation Learning for Sequence Data with Deep Autoencoding Predictive Components
We propose Deep Autoencoding Predictive Components (DAPC) -- a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Dense Hierarchical Retrieval for Open-Domain Question Answering
In this work, we propose Dense Hierarchical Retrieval (DHR), a hierarchical framework that can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic semantics specific to each passage.
Theory-Inspired Path-Regularized Differential Network Architecture Search
Then we propose a theory-inspired path-regularized DARTS that consists of two key modules: (i) a differential group-structured sparse binary gate introduced for each operation to avoid unfair competition among operations, and (ii) a path-depth-wise regularization used to incite search exploration for deep architectures that often converge slower than shallow ones as shown in our theory and are not well explored during the search.
GAEA: Graph Augmentation for Equitable Access via Reinforcement Learning
Similarly, social networks within universities and organizations may enable certain groups to more easily access people with valuable information or influence.
BERT is Not an Interlingua and the Bias of Tokenization
Multilingual transfer learning can benefit both high- and low-resource languages, but the source of these improvements is not well understood.
Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models
In this work, we explore joint energy-based model (EBM) training during the finetuning of pretrained text encoders (e. g., Roberta) for natural language understanding (NLU) tasks.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We propose to use sentence-compression data to train the post-editing model to take a summary with extrinsic entity errors marked with special tokens and output a compressed, well-formed summary with those errors removed.
Fair Abstractive Summarization of Diverse Perspectives
However, current work in summarization metrics and Large Language Models (LLMs) evaluation has not explored fair abstractive summarization.
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets
Precisely assessing the progress in natural language generation (NLG) tasks is challenging, and human evaluation to establish a preference in a model's output over another is often necessary.
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage
There are many potential benefits to news readers accessing diverse sources.
FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability
This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats, a crucial yet underexamined capability for their application as AI agents.
What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases
Vision-language (VL) models, pretrained on colossal image-text datasets, have attained broad VL competence that is difficult to evaluate.
Efficient and Differentiable Conformal Prediction with General Function Classes
Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly over existing approaches in several applications such as prediction intervals with improved length, minimum-volume prediction sets for multi-output regression, and label prediction sets for image classification.
Evaluating State-of-the-Art Classification Models Against Bayes Optimality
Moreover, we show that by varying the temperature of the learned flow models, we can generate synthetic datasets that closely resemble standard benchmark datasets, but with almost any desired Bayes error.
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles
In this paper, we propose a new task of summarizing diverse information encountered in multiple news articles encompassing the same event.
Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills
We perform an extensive evaluation of skill discovery methods on controlled environments and show that EDL offers significant advantages, such as overcoming the coverage problem, reducing the dependence of learned skills on the initial state, and allowing the user to define a prior over which behaviors should be learned.
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
This demonstrates that using policy gradient optimization to improve language agents, for which we believe our work is one of the first, seems promising and can be applied to optimize other models in the agent architecture to enhance agent performances over time.
Dynamic Coattention Networks For Question Answering
Several deep learning models have been proposed for question answering.
Ranked #2 on
Open-Domain Question Answering
on SQuAD1.1
DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
Traditional models for question answering optimize using cross entropy loss, which encourages exact answers at the cost of penalizing nearby or overlapping answers that are sometimes equally accurate.
Ranked #28 on
Question Answering
on SQuAD1.1 dev
Learning to Play General-Sum Games Against Multiple Boundedly Rational Agents
We study the problem of training a principal in a multi-agent general-sum game using reinforcement learning (RL).
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
Making big purchases requires consumers to research or consult a salesperson to gain domain expertise.
Augmented Cyclic Adversarial Learning for Low Resource Domain Adaptation
In low-resource supervised setting, the results show that our approach improves absolute performance by 14% and 4% when adapting SVHN to MNIST and vice versa, respectively, which outperforms unsupervised domain adaptation methods that require high-resource unlabeled target domain.
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning
Our results demonstrate the strong and efficient modeling ability of NLI-based classifiers and the large cross-lingual transfer improvements achieved by our aligned prompts, particularly in few-shot settings.
A Multi-Discriminator CycleGAN for Unsupervised Non-Parallel Speech Domain Adaptation
Domain adaptation plays an important role for speech recognition models, in particular, for domains that have low resources.
Interpretable Counting for Visual Question Answering
Questions that require counting a variety of objects in images remain a major challenge in visual question answering (VQA).
Block-diagonal Hessian-free Optimization for Training Neural Networks
Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence.
Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning
In order to help the agent learn the complex temporal dependencies necessary for the hierarchical policy, we provide it with a stochastic temporal grammar that modulates when to rely on previously learned skills and when to execute new skills.
Improved Regularization Techniques for End-to-End Speech Recognition
We augment audio data through random perturbations of tempo, pitch, volume, temporal alignment, and adding random noise. We further investigate the effect of dropout when applied to the inputs of all layers of the network.
Improving End-to-End Speech Recognition with Policy Learning
However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e. g., word error rate (WER).
Ranked #49 on
Speech Recognition
on LibriSpeech test-clean
Action Understanding with Multiple Classes of Actors
Despite the rapid progress, existing works on action understanding focus strictly on one type of action agent, which we call actor---a human adult, ignoring the diversity of actions performed by other actors.
A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs
LSTMs have become a basic building block for many deep NLP models.
Recognizing Car Fluents from Video
Since there are no publicly related dataset, we collect and annotate a car fluent dataset consisting of car videos with diverse fluents.
Compositional Structure Learning for Action Understanding
The focus of the action understanding literature has predominately been classification, how- ever, there are many applications demanding richer action understanding such as mobile robotics and video search, with solutions to classification, localization and detection.
Semi-Supervised Nonlinear Distance Metric Learning via Forests of Max-Margin Cluster Hierarchies
By introducing randomness during hierarchy training and combining the output of many of the resulting semi-random weak hierarchy metrics, we can obtain a powerful and robust nonlinear metric model.
Active Clustering with Model-Based Uncertainty Reduction
Semi-supervised clustering seeks to augment traditional clustering methods by incorporating side information provided via human expertise in order to increase the semantic meaningfulness of the resulting clusters.
Using Mode Connectivity for Loss Landscape Analysis
Mode connectivity is a recently introduced frame- work that empirically establishes the connected- ness of minima by finding a high accuracy curve between two independently trained models.
Improving Abstraction in Text Summarization
Abstractive text summarization aims to shorten long text documents into a human readable form that contains the most important facts from the original document.
Ranked #4 on
Text Summarization
on CNN / Daily Mail (Anonymized)
Identifying Generalization Properties in Neural Networks
In particular, we prove that model generalization ability is related to the Hessian, the higher-order "smoothness" terms characterized by the Lipschitz constant of the Hessian, and the scales of the parameters.
Interactive Agent Modeling by Learning to Probe
In particular, the probing agent (i. e. a learner) learns to interact with the environment and with a target agent (i. e., a demonstrator) to maximize the change in the observed behaviors of that agent.
AdaFrame: Adaptive Frame Selection for Fast Video Recognition
We present AdaFrame, a framework that adaptively selects relevant frames on a per-input basis for fast video recognition.
Global-Locally Self-Attentive Encoder for Dialogue State Tracking
Dialogue state tracking, which estimates user goals and requests given the dialogue context, is an essential part of task-oriented dialogue systems.
Automatic Speech Recognition (ASR)
Dialogue State Tracking
+3
Continual Learning via Explicit Structure Learning
During structure learning, the model optimizes for the best structure for the current task.
Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering
End-to-end neural models have made significant progress in question answering, however recent studies show that these models implicitly assume that the answer and evidence appear close together in a single document.
Ranked #5 on
Question Answering
on WikiHop
Actionness Ranking with Lattice Conditional Ordinal Random Fields
Action analysis in image and video has been attracting more and more attention in computer vision.
Joint Action Recognition and Pose Estimation From Video
Action recognition and pose estimation from video are closely related tasks for understanding human motion, most methods, however, learn separate models and combine them sequentially.
Can Humans Fly? Action Understanding With Multiple Classes of Actors
There is no work we know of on simultaneously inferring actors and actions in the video, not to mention a dataset to experiment with.
Competitive Experience Replay
We propose a novel method called competitive experience replay, which efficiently supplements a sparse reward by placing learning in the context of an exploration competition between a pair of agents.
StartNet: Online Detection of Action Start in Untrimmed Videos
We propose StartNet to address Online Detection of Action Start (ODAS) where action starts and their associated categories are detected in untrimmed, streaming videos.
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks.
Unifying Question Answering, Text Classification, and Regression via Span Extraction
Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different.
XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
XLDA is in contrast to, and performs markedly better than, a more naive approach that aggregates examples in various languages in a way that each example is solely in one language.
Cross-Lingual Natural Language Inference
Data Augmentation
+3
On the Generalization Gap in Reparameterizable Reinforcement Learning
For this problem class, estimating the expected return is efficient and the trajectory can be computed deterministically given peripheral random variables, which enables us to study reparametrizable RL using supervised learning and transfer learning theory.
Private Deep Learning with Teacher Ensembles
Privacy-preserving deep learning is crucial for deploying deep neural network based solutions, especially when the model works on data that contains sensitive information.
Learning World Graphs to Accelerate Hierarchical Reinforcement Learning
We perform a thorough ablation study to evaluate our approach on a suite of challenging maze tasks, demonstrating significant advantages from the proposed framework over baselines that lack world graph knowledge in terms of performance and efficiency.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
In particular, we explore knowledge distillation and learning rate heuristics of (cosine) restarts and warmup using mode connectivity and CCA.
Neural Text Summarization: A Critical Evaluation
Text summarization aims at compressing long documents into a shorter form that conveys the most important parts of the original document.
WSLLN: Weakly Supervised Natural Language Localization Networks
We propose weakly supervised language localization networks (WSLLN) to detect events in long, untrimmed videos given language queries.
Deleter: Leveraging BERT to Perform Unsupervised Successive Text Compression
In this work, we propose a fully unsupervised model, Deleter, that is able to discover an "optimal deletion path" for an arbitrary sentence, where each intermediate sequence along the path is a coherent subsequence of the previous one.
Global Capacity Measures for Deep ReLU Networks via Path Sampling
Classical results on the statistical complexity of linear models have commonly identified the norm of the weights $\|w\|$ as a fundamental capacity measure.
Sketch-Fill-A-R: A Persona-Grounded Chit-Chat Generation Framework
Human-like chit-chat conversation requires agents to generate responses that are fluent, engaging and consistent.
WSLLN:Weakly Supervised Natural Language Localization Networks
We propose weakly supervised language localization networks (WSLLN) to detect events in long, untrimmed videos given language queries.
MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
Our approach is model agnostic and can be easily applied on different future teacher model architectures.
Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
We test the verifier network on out-of-distribution detection and adversarial example detection problems, as well as anomaly detection problems in structured prediction tasks such as image caption generation.
LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition
This paper presents LiteEval, a simple yet effective coarse-to-fine framework for resource efficient video recognition, suitable for both online and offline scenarios.
Proposal Learning for Semi-Supervised Object Detection
two-stage object detectors) by training on both labeled and unlabeled data.
Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width
We propose \emph{Taylorized training} as an initiative towards better understanding neural network training at finite width.
Differentially Private Deep Learning with Smooth Sensitivity
Ensuring the privacy of sensitive data used to train modern machine learning models is of paramount importance in many areas of practice.
Towards Noise-resistant Object Detection with Noisy Annotations
We address the challenging problem of training object detectors with noisy annotations, where the noise contains a mixture of label noise and bounding box noise.
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
There is an increasing amount of literature that claims the brittleness of deep neural networks in dealing with adversarial examples that are created maliciously.
WOAD: Weakly Supervised Online Action Detection in Untrimmed Videos
Online action detection in untrimmed videos aims to identify an action as it happens, which makes it very important for real-time applications.
Ranked #5 on
Online Action Detection
on THUMOS'14
Towards Understanding Hierarchical Learning: Benefits of Neural Representations
When the trainable network is the quadratic Taylor model of a wide two-layer network, we show that neural representation can achieve improved sample complexities compared with the raw input: For learning a low-rank degree-$p$ polynomial ($p \geq 4$) in $d$ dimension, neural representation requires only $\tilde{O}(d^{\lceil p/2 \rceil})$ samples, while the best-known sample complexity upper bound for the raw input is $\tilde{O}(d^{p-1})$.
An investigation of phone-based subword units for end-to-end speech recognition
For Switchboard, our phone-based BPE system achieves 6. 8\%/14. 4\% word error rate (WER) on the Switchboard/CallHome portion of the test set while joint decoding achieves 6. 3\%/13. 3\% WER.
Photon: A Robust Cross-Domain Text-to-SQL System
Natural language interfaces to databases (NLIDB) democratize end user access to relational data.
Composed Variational Natural Language Generation for Few-shot Intents
In this paper, we focus on generating training examples for few-shot intents in the realistic imbalanced scenario.
CorDial: Coarse-to-fine Abstractive Dialogue Summarization with Controllable Granularity
2) A simple strategy to control the granularity of the final summary.
Neural Bayes: A Generic Parameterization Method for Unsupervised Learning
Disjoint Manifold Separation: Neural Bayes allows us to formulate an objective which can optimally label samples from disjoint manifolds present in the support of a continuous distribution.
Momentum Contrastive Autoencoder
Quantitatively, we show that our algorithm achieves a new state-of-the-art FID of 54. 36 on CIFAR-10, and performs competitively with existing models on CelebA in terms of FID score.
