Search Results for author:
Found 231 papers, 117 papers with code
Guiding Instruction-based Image Editing via Multimodal Large Language Models
Extensive experimental results demonstrate that expressive instructions are crucial to instruction-based image editing, and our MGIE can lead to a notable improvement in automatic metrics and human evaluation while maintaining competitive inference efficiency.
Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text
We release Multimodal C4, an augmentation of the popular text-only C4 corpus with images interleaved.
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Semantically controlled neural response generation on limited-domain has achieved great performance.
Ranked #5 on
Data-to-Text Generation
on MULTIWOZ 2.1
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning
We study the problem of learning to reason in large scale knowledge graphs (KGs).
Ranked #1 on
Link Prediction
on NELL-995
(Mean AP metric)
Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
In this paper, we take a radical approach to bridge the gap between synthetic studies and real-world practices---We propose a novel, planned-ahead hybrid reinforcement learning model that combines model-free and model-based reinforcement learning to solve a real-world vision-language navigation task.
Model-based Reinforcement Learning
reinforcement-learning
+4
TabFact: A Large-scale Dataset for Table-based Fact Verification
To this end, we construct a large-scale dataset called TabFact with 16k Wikipedia tables as the evidence for 118k human-annotated natural language statements, which are labeled as either ENTAILED or REFUTED.
Ranked #10 on
Table-based Fact Verification
on TabFact
WikiHow: A Large Scale Text Summarization Dataset
Sequence-to-sequence models have recently gained the state of the art performance in summarization.
Ranked #3 on
Text Summarization
on WikiHow
Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
Large language models (LLMs) have demonstrated remarkable performance across a wide array of NLP tasks.
Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis
In this work, we improve the compositional skills of T2I models, specifically more accurate attribute binding and better image compositions.
One-Shot Relational Learning for Knowledge Graphs
Knowledge graphs (KGs) are the key components of various natural language processing applications.
LayoutGPT: Compositional Visual Planning and Generation with Large Language Models
When combined with a downstream image generation model, LayoutGPT outperforms text-to-image models/systems by 20-40% and achieves comparable performance as human users in designing visual layouts for numerical and spatial correctness.
KBGAN: Adversarial Learning for Knowledge Graph Embeddings
This framework is independent of the concrete form of generator and discriminator, and therefore can utilize a wide variety of knowledge graph embedding models as its building blocks.
Ranked #24 on
Link Prediction
on WN18
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.
Ranked #14 on
Question Answering
on HotpotQA
FinQA: A Dataset of Numerical Reasoning over Financial Data
In contrast to existing tasks on general domain, the finance domain includes complex numerical reasoning and understanding of heterogeneous representations.
Ranked #4 on
Question Answering
on FinQA
Few-Shot NLG with Pre-Trained Language Model
Neural-based end-to-end approaches to natural language generation (NLG) from structured data or knowledge are data-hungry, making their adoption for real-world applications difficult with limited data.
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations.
Logical Natural Language Generation from Open-Domain Tables
To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset \cite{chen2019tabfact} featured with a wide range of logical/symbolic inferences as our testbed, and propose new automatic metrics to evaluate the fidelity of generation models w. r. t.\ logical inference.
KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation
We propose a knowledge-grounded pre-training (KGPT), which consists of two parts, 1) a general knowledge-grounded generation model to generate knowledge-enriched text.
Ranked #9 on
KG-to-Text Generation
on WebNLG 2.0 (Unconstrained)
r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection
We construct hybrid text+image models and perform extensive experiments for multiple variations of classification, demonstrating the importance of the novel aspect of multimodality and fine-grained classification unique to Fakeddit.
Classification
Cultural Vocal Bursts Intensity Prediction
+2
Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader
We propose a new end-to-end question answering model, which learns to aggregate answer evidence from an incomplete knowledge base (KB) and a set of retrieved text snippets.
No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling
Though impressive results have been achieved in visual captioning, the task of generating abstract stories from photo streams is still a little-tapped problem.
Ranked #13 on
Visual Storytelling
on VIST
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
Further, unlike previous studies that found pre-training tasks on video inputs (e. g., masked frame modeling) not very effective, we design a new pre-training task, Masked Visual-token Modeling (MVM), for better video modeling.
Ranked #20 on
Zero-Shot Video Retrieval
on DiDeMo
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
In this paper, we present liar: a new, publicly available dataset for fake news detection.
Ranked #1 on
Fake News Detection
on LIAR
``Liar, Liar Pants on Fire'': A New Benchmark Dataset for Fake News Detection
In this paper, we present LIAR: a new, publicly available dataset for fake news detection.
MojiTalk: Generating Emotional Responses at Scale
In this paper, we take a more radical approach: we exploit the idea of leveraging Twitter data that are naturally labeled with emojis.
A Survey on Detection of LLMs-Generated Content
The burgeoning capabilities of advanced large language models (LLMs) such as ChatGPT have led to an increase in synthetic content generation with implications across a variety of sectors, including media, cybersecurity, public discourse, and education.
LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation
Existing automatic evaluation on text-to-image synthesis can only provide an image-text matching score, without considering the object-level compositionality, which results in poor correlation with human judgments.
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
One of the long-term challenges of robotics is to enable robots to interact with humans in the visual world via natural language, as humans are visual animals that communicate through language.
Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing
Existing entity typing systems usually exploit the type hierarchy provided by knowledge base (KB) schema to model label correlations and thus improve the overall performance.
Ranked #3 on
Entity Typing
on Ontonotes v5 (English)
Multimodal Procedural Planning via Dual Text-Image Prompting
The key challenges of MPP are to ensure the informativeness, temporal coherence, and accuracy of plans across modalities.
Unsupervised Multi-hop Question Answering by Question Generation
Obtaining training data for multi-hop question answering (QA) is time-consuming and resource-intensive.
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
Most existing video-and-language (VidL) research focuses on a single dataset, or multiple datasets of a single task.
Environment-agnostic Multitask Learning for Natural Language Grounded Navigation
Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e. g., following natural language instructions or dialog.
ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering
With the recent advance in large pre-trained language models, researchers have achieved record performances in NLP tasks that mostly focus on language pattern matching.
Ranked #2 on
Question Answering
on ConvFinQA
Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning
The experimental results show that the proposed strategy significantly improves the performance of distant supervision comparing to state-of-the-art systems.
A Benchmark Dataset for Learning to Intervene in Online Hate Speech
In this paper, we also analyze the datasets to understand the common intervention strategies and explore the performance of common automatic response generation methods on these new datasets to provide a benchmark for future research.
Logic2Text: High-Fidelity Natural Language Generation from Logical Forms
If only provided with the table, it is hard for existing models to produce controllable and high-fidelity logical generations.
Zero-shot Fact Verification by Claim Generation
However, for each new domain that requires fact verification, creating a dataset by manually writing claims and linking them to their supporting evidence is expensive.
A Survey on Data Selection for Language Models
A major factor in the recent success of large language models is the use of enormous and ever-growing text datasets for unsupervised pre-training.
Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
The RNF transforms a latent variable into a space that respects the geometric characteristics of input space, which makes posterior impossible to collapse to the non-informative prior.
Sentence Embedding Alignment for Lifelong Relation Extraction
We formulate such a challenging problem as lifelong relation extraction and investigate memory-efficient incremental learning methods without catastrophically forgetting knowledge learned from previous tasks.
Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models.
INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback
By harnessing both explicit human instruction and the implicit knowledge of GPT-4, we fine-tune a text evaluation metric based on LLaMA, producing both a score for generated text and a human readable diagnostic report.
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
We also introduce two tasks for video-and-language research based on VATEX: (1) Multilingual Video Captioning, aimed at describing a video in various languages with a compact unified captioning model, and (2) Video-guided Machine Translation, to translate a source language description into the target language using the video information as additional spatiotemporal context.
Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs
Large-scale knowledge graphs (KGs) are shown to become more important in current information systems.
Weak-to-Strong Jailbreaking on Large Language Models
In this paper, we propose the weak-to-strong jailbreaking attack, an efficient method to attack aligned LLMs to produce harmful text.
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering
In this work, we propose a simple and resource-efficient method to pretrain the paragraph encoder.
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text
However, this progress also presents a significant challenge in detecting the origin of a given text, and current research on detection methods lags behind the rapid evolution of LLMs.
Fact-Checking Complex Claims with Program-Guided Reasoning
Fact-checking real-world claims often requires collecting multiple pieces of evidence and applying complex multi-step reasoning.
Self-Supervised Learning for Contextualized Extractive Summarization
Existing models for extractive summarization are usually trained from scratch with a cross-entropy loss, which does not explicitly capture the global context at the document level.
XL-NBT: A Cross-lingual Neural Belief Tracking Framework
Then, we pre-train a state tracker for the source language as a teacher, which is able to exploit easy-to-access parallel data.
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling
Masked visual modeling (MVM) has been recently proven effective for visual pre-training.
Ranked #1 on
Video Question Answering
on LSMDC-MC
Scheduled Policy Optimization for Natural Language Communication with Intelligent Agents
We investigate the task of learning to follow natural language instructions by jointly reasoning with visual observations and language inputs.
Deep Reinforcement Learning for Chinese Zero pronoun Resolution
In this study, we show how to integrate local and global decision-making by exploiting deep reinforcement learning models.
EDIS: Entity-Driven Image Search over Multimodal Web Content
Making image retrieval methods practical for real-world search applications requires significant progress in dataset scales, entity comprehension, and multimodal information fusion.
ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers
To address these challenges, we propose ALGO, a framework that synthesizes Algorithmic programs with LLM-Generated Oracles to guide the generation and verify their correctness.
Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning
Furthermore, for the first time, we validate the superior performance of the deep audio features on the video captioning task.
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Neural text-to-SQL models have achieved remarkable performance in translating natural language questions into SQL queries.
Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data
In this work, we focus on Few-shot Learning with Auxiliary Data (FLAD), a training paradigm that assumes access to auxiliary data during few-shot learning in hopes of improving generalization.
Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation
Outdoor vision-and-language navigation (VLN) is such a task where an agent follows natural language instructions and navigates a real-life urban environment.
Ranked #4 on
Vision and Language Navigation
on Touchdown Dataset
(using extra training data)
SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoning
In this paper, we introduce a Self-Supervised Counterfactual Reasoning (SSCR) framework that incorporates counterfactual thinking to overcome data scarcity.
Language-Driven Image Style Transfer
We propose contrastive language visual artist (CLVA) that learns to extract visual semantics from style instructions and accomplish LDAST by the patch-wise style discriminator.
Neural Correction Model for Open-Domain Named Entity Recognition
In this work, to address the low precision and recall problems, we first utilize DBpedia as the source of distant supervision to annotate abstracts from Wikipedia and design a neural correction model trained with a human-annotated NER dataset, DocRED, to correct the false entity labels.
Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation
Inspired by this, we introduce a novel task, text-guided video completion (TVC), which requests the model to generate a video from partial frames guided by an instruction.
Ranked #3 on
Video Prediction
on BAIR Robot Pushing
Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media
While social media empowers freedom of expression and individual voices, it also enables anti-social behavior, online harassment, cyberbullying, and hate speech.
Zero Pronoun Resolution with Attention-based Neural Network
Recent neural network methods for zero pronoun resolution explore multiple models for generating representation vectors for zero pronouns and their candidate antecedents.
Local Explanation of Dialogue Response Generation
To gain insights into the reasoning process of a generation model, we propose a new method, local explanation of response generation (LERG) that regards the explanations as the mutual interaction of segments in input and output sentences.
Imagination-Augmented Natural Language Understanding
Human brains integrate linguistic and perceptual information simultaneously to understand natural language, and hold the critical ability to render imaginations.
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation
Recent advances in text-to-image synthesis make it possible to visualize machine imaginations for a given context.
Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems
Our proposed method allows a single transformer model to directly walk on a large-scale knowledge graph to generate responses.
Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesis
Is it possible to build a general and automatic natural language generation (NLG) evaluation metric?
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue
Task transfer, transferring knowledge contained in related tasks, holds the promise of reducing the quantity of labeled data required to fine-tune language models.
SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes
In this paper, we propose SESCORE2, a self-supervised approach for training a model-based metric for text generation evaluation.
ReDi: Efficient Learning-Free Diffusion Inference via Trajectory Retrieval
Diffusion models show promising generation capability for a variety of data.
Cross-Lingual Vision-Language Navigation
Commanding a robot to navigate with natural language instructions is a long-term goal for grounded language understanding and robotics.
Multilingual Conceptual Coverage in Text-to-Image Models
We propose "Conceptual Coverage Across Languages" (CoCo-CroLa), a technique for benchmarking the degree to which any generative text-to-image system provides multilingual parity to its training language in terms of tangible nouns.
Counterfactual Maximum Likelihood Estimation for Training Deep Networks
Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive clues.
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization
Missing information is a common issue of dialogue summarization where some information in the reference summaries is not covered in the generated summaries.
Discriminative Diffusion Models as Few-shot Vision and Language Learners
Diffusion models, such as Stable Diffusion, have shown incredible performance on text-to-image generation.
Simple Models for Word Formation in English Slang
We propose generative models for three types of extra-grammatical word formation phenomena abounding in English slang: Blends, Clippings, and Reduplicatives.
Simple Models for Word Formation in Slang
We propose the first generative models for three types of extra-grammatical word formation phenomena abounding in slang: Blends, Clippings, and Reduplicatives.
VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View
In this work, we propose VELMA, an embodied LLM agent that uses a verbalization of the trajectory and of visual environment observations as contextual prompt for the next action.
Causal Balancing for Domain Generalization
While machine learning models rapidly advance the state-of-the-art on various real-world tasks, out-of-domain (OOD) generalization remains a challenging problem given the vulnerability of these models to spurious correlations.
On the Risk of Misinformation Pollution with Large Language Models
In this paper, we comprehensively investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation and its subsequent impact on information-intensive applications, particularly Open-Domain Question Answering (ODQA) systems.
MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models
Language Models (LMs) have shown impressive performance in various natural language tasks.
Diagnosing Vision-and-Language Navigation: What Really Matters
Results show that indoor navigation agents refer to both object and direction tokens when making decisions.
D-REX: Dialogue Relation Extraction with Explanations
Existing research studies on cross-sentence relation extraction in long-form multi-party conversations aim to improve relation extraction without considering the explainability of such methods.
Ranked #7 on
Dialog Relation Extraction
on DialogRE
Open-Domain Question-Answering for COVID-19 and Other Emergent Domains
In this work, we present such a system for the emergent domain of COVID-19.
Investigating African-American Vernacular English in Transformer-Based Text Generation
The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts.
ULN: Towards Underspecified Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a task to guide an embodied agent moving to a target position using language instructions.
Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learning
In this work, we present Knowledge-Grounded RL (KGRL), an RL paradigm fusing multiple knowledge policies and aiming for human-like efficiency and flexibility.
On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond
To this end, we propose Coupled-VAE, which couples a VAE model with a deterministic autoencoder with the same structure and improves the encoder and decoder parameterizations via encoder weight sharing and decoder signal matching.
Semi-Supervised Policy Initialization for Playing Games with Language Hints
Using natural language as a hint can supply an additional reward for playing sparse-reward games.
Language Agnostic Multilingual Information Retrieval with Contrastive Learning
Multilingual information retrieval (IR) is challenging since annotated training data is costly to obtain in many languages.
Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation
To understand how pre-training with a next-token prediction objective contributes to the emergence of such reasoning capability, we propose that we can view an LM as deriving new conclusions by aggregating indirect reasoning paths seen at pre-training time.
Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation
Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains.
CausalDialogue: Modeling Utterance-level Causality in Conversations
Despite their widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans.
Zero-Shot Detection of Machine-Generated Codes
We then modify the previous zero-shot text detection method, DetectGPT (Mitchell et al., 2023) by utilizing a surrogate white-box model to estimate the probability of the rightmost tokens, allowing us to identify code snippets generated by language models.
Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)
With advances in the quality of text-to-image (T2I) models has come interest in benchmarking their prompt faithfulness-the semantic coherence of generated images to the prompts they were conditioned on.
A Survey on Natural Language Processing for Fake News Detection
We also highlight the difference between fake news detection and other related tasks, and the importance of NLP solutions for fake news detection.
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space
We verify some of our linguistic findings by looking at the effect of morphological segmentation on English-Inuktitut alignment, in addition to examining the effect of word order agreement on isomorphism for 66 zero-shot language pairs from a different corpus.
Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI
Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury.
Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought
Despite exciting recent results showing vision-language systems' capacity to reason about images using natural language, their capacity for video reasoning remains under-explored.
Mitigating Gender Bias in Natural Language Processing: Literature Review
In this paper, we review contemporary studies on recognizing and mitigating gender bias in NLP.
Towards Understanding Gender Bias in Relation Extraction
We use WikiGenderBias to evaluate systems for bias and find that NRE systems exhibit gender biased predictions and lay groundwork for future evaluation of bias in NRE.
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer
This is a particularly notable issue in the medical domain, where layman are often confused by medical text online.
Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting
Mental illness remains one of the most critical public health issues of our time, due to the severe scarcity and accessibility limit of professionals.
ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models
As large language models are integrated into society, robustness toward a suite of prompts is increasingly important to maintain reliability in a high-variance environment. Robustness evaluations must comprehensively encapsulate the various settings in which a user may invoke an intelligent system.
SafeRoute: Learning to Navigate Streets Safely in an Urban Environment
We propose SafeRoute, a novel solution to the problem of navigating cities and avoiding street harassment and crime.
Investigating Memorization of Conspiracy Theories in Text Generation
In this work, we investigate the capability of language models to generate conspiracy theory text.
Attacking Open-domain Question Answering by Injecting Misinformation
We curate both human-written and model-generated false documents that we inject into the evidence corpus of QA models and assess the impact on the performance of these systems.
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains.
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks
We show that, with consistent tokenization, the model performs better in both in-domain and out-of-domain datasets, with a notable average of +1. 7 F2 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets.
DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction
Distant supervision can effectively label data for relation extraction, but suffers from the noise labeling problem.
Reinforced Co-Training
However, the selection of samples in existing co-training methods is based on a predetermined policy, which ignores the sampling bias between the unlabeled and the labeled subsets, and fails to explore the data space.
Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection
Hate speech detection is a critical, yet challenging problem in Natural Language Processing (NLP).
Video Captioning via Hierarchical Reinforcement Learning
Video captioning is the task of automatically generating a textual description of the actions in a video.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
IcoRating: A Deep-Learning System for Scam ICO Identification
For the best setting, the proposed system is able to identify scam ICO projects with 0. 83 precision.
TFW, DamnGina, Juvie, and Hotsie-Totsie: On the Linguistic and Social Aspects of Internet Slang
In this work, we use UrbanDictionary to conduct the first large-scale linguistic analysis of slang and its social aspects on the Internet to yield insights into this variety of language that is increasingly used all over the world online.
Learning to Explain Non-Standard English Words and Phrases
We describe a data-driven approach for automatically explaining new, non-standard English expressions in a given sentence, building on a large dataset that includes 15 years of crowdsourced examples from UrbanDictionary. com.
Deep Residual Learning for Weakly-Supervised Relation Extraction
Deep residual learning (ResNet) is a new method for training very deep neural networks using identity map-ping for shortcut connections.
Efficient Inference and Learning in a Large Knowledge Base: Reasoning with Extracted Information using a Locally Groundable First-Order Probabilistic Logic
We show that the problem of constructing proofs for this logic is related to computation of personalized PageRank (PPR) on a linearized version of the proof space, and using on this connection, we develop a proveably-correct approximate grounding scheme, based on the PageRank-Nibble algorithm.
Programming with Personalized PageRank: A Locally Groundable First-Order Probabilistic Logic
In many probabilistic first-order representation systems, inference is performed by "grounding"---i. e., mapping it to a propositional representation, and then performing propositional inference.
Hierarchical CVAE for Fine-Grained Hate Speech Classification
Existing work on automated hate speech detection typically focuses on binary classification or on differentiating among a small set of categories.
Multi-view Models for Political Ideology Detection of News Articles
A news article's title, content and link structure often reveal its political ideology.
Analyzing and Interpreting Convolutional Neural Networks in NLP
Convolutional neural networks have been successfully applied to various NLP tasks.
Towards Explainable NLP: A Generative Explanation Framework for Text Classification
Building explainable systems is a critical problem in the field of Natural Language Processing (NLP), since most machine learning models provide no explanations for the predictions.
DOLORES: Deep Contextualized Knowledge Graph Embeddings
We introduce a new method DOLORES for learning knowledge graph embeddings that effectively captures contextual cues and dependencies among entities and relations.
Dirichlet Variational Autoencoder for Text Modeling
We introduce an improved variational autoencoder (VAE) for text modeling with topic information explicitly modeled as a Dirichlet latent variable.
Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications
To bridge this gap, we study the role of SGD implicit regularization in deep learning systems.
Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning
Although promising results have been achieved in video captioning, existing models are limited to the fixed inventory of activities in the training corpus, and do not generalize to open vocabulary scenarios.
Quantifying Uncertainties in Natural Language Processing Tasks
Reliable uncertainty quantification is a first step towards building explainable, transparent, and accountable artificial intelligent systems.
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
Vision-language navigation (VLN) is the task of navigating an embodied agent to carry out natural language instructions inside real 3D environments.
Ranked #2 on
Vision-Language Navigation
on Room2Room
Deep Reinforcement Learning for NLP
Many Natural Language Processing (NLP) tasks (including generation, language grounding, reasoning, information extraction, coreference resolution, and dialog) can be formulated as deep reinforcement learning (DRL) problems.
Scalable Construction and Reasoning of Massive Knowledge Bases
In today{'}s information-based society, there is abundant knowledge out there carried in the form of natural language texts (e. g., news articles, social media posts, scientific publications), which spans across various domains (e. g., corporate documents, advertisements, legal acts, medical reports), which grows at an astonishing rate.
Learning to Decipher Hate Symbols
Furthermore, we propose a novel Variational Decipher and show how it can generalize better to unseen hate symbols in a more challenging testing setting.
Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation
The overreliance on large parallel corpora significantly limits the applicability of machine translation systems to the majority of language pairs.
Deep Adversarial Learning for NLP
Adversarial learning is a game-theoretic learning paradigm, which has achieved huge successes in the field of Computer Vision recently.
Self-Supervised Dialogue Learning
The sequential order of utterances is often meaningful in coherent dialogues, and the order changes of utterances could lead to low-quality and incoherent conversations.
TWEETQA: A Social Media Focused Question Answering Dataset
With social media becoming increasingly pop-ular on which lots of news and real-time eventsare reported, developing automated questionanswering systems is critical to the effective-ness of many applications that rely on real-time knowledge.
What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog.
In this work, we focus on the task of goal-oriented visual dialogue, aiming to automatically generate a series of questions about an image with a single objective.
What Should I Ask? Using Conversationally Informative Rewards for Goal-Oriented Visual Dialog
In this work, we focus on the task of goal-oriented visual dialogue, aiming to automatically generate a series of questions about an image with a single objective.
Meta Reasoning over Knowledge Graphs
The ability to reason over learned knowledge is an innate ability for humans and humans can easily master new reasoning rules with only a few demonstrations.
Multi-Task Self-Supervised Learning for Disfluency Detection
The pre-trained network is then fine-tuned using human-annotated disfluency detection training data.
Text Modeling with Syntax-Aware Variational Autoencoders
We propose syntax-aware variational autoencoders (SAVAEs) that dedicate a subspace in the latent dimensions dubbed syntactic latent to represent syntactic structures of sentences.
Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarization
Deep reinforcement learning (RL) has been a commonly-used strategy for the abstractive summarization task to address both the exposure bias and non-differentiable task issues.
Neural Gaussian Copula for Variational Autoencoder
We argue that this would cause a typical training problem called posterior collapse observed in all other variational language models.
Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification
Many tasks in natural language processing can be viewed as multi-label classification problems.
Simple yet Effective Bridge Reasoning for Open-Domain Multi-Hop Question Answering
To resolve this issue, we introduce a new sub-problem of open-domain multi-hop QA, which aims to recognize the bridge (\emph{i. e.}, the anchor that links to the answer passage) from the context of a set of start passages with a reading comprehension model.
Table-to-Text Natural Language Generation with Unseen Schemas
In this paper, we propose the new task of table-to-text NLG with unseen schemas, which specifically aims to test the generalization of NLG for input tables with attribute types that never appear during training.
Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
Visual navigation is a task of training an embodied agent by intelligently navigating to a target object (e. g., television) using only visual observations.
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampling
In particular, we present a model-agnostic adversarial path sampler (APS) that learns to sample challenging paths that force the navigator to improve based on the navigation performance.
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Models trained with our new objective yield significant improvements on the fact completion task.
Disentangled Representation Learning with Wasserstein Total Correlation
However, Kullback-Leibler (KL) divergence-based total correlation is metric-agnostic and sensitive to data samples.
HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing
We introduce HULK, a multi-task energy efficiency benchmarking platform for responsible natural language processing.
Evaluating Transformer-Based Multilingual Text Classification
As NLP tools become ubiquitous in today's technological landscape, they are increasingly applied to languages with a variety of typological structures.
Knowledge Injection into Dialogue Generation via Language Models
First, we train a large-scale language model and query it as textual knowledge.
Counterfactual Off-Policy Training for Neural Response Generation
Open-domain dialogue generation suffers from the data insufficiency problem due to the vast size of potential responses.
Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection
We construct hybrid text+image models and perform extensive experiments for multiple variations of classification, demonstrating the importance of the novel aspect of multimodality and fine-grained classification unique to Fakeddit.
Classification
Cultural Vocal Bursts Intensity Prediction
+2
Cross-lingual Transfer Learning for COVID-19 Outbreak Alignment
The spread of COVID-19 has become a significant and troubling aspect of society in 2020.
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampler
In particular, we present a model-agnostic adversarial path sampler (APS) that learns to sample challenging paths that force the navigator to improve based on the navigation performance.
Learning to Stop: A Simple yet Effective Approach to Urban Vision-Language Navigation
Vision-and-Language Navigation (VLN) is a natural language grounding task where an agent learns to follow language instructions and navigate to specified destinations in real-world environments.
Ranked #3 on
Vision and Language Navigation
on Touchdown Dataset
Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations
A major challenge in visually grounded language generation is to build robust benchmark datasets and models that can generalize well in real-world settings.
Counterfactual Off-Policy Training for Neural Dialogue Generation
Open-domain dialogue generation suffers from the data insufficiency problem due to the vast size of potential responses.
Modeling Disclosive Transparency in NLP Application Descriptions
Broader disclosive transparency$-$truth and clarity in communication regarding the function of AI systems$-$is widely considered desirable.
DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents
Creating presentation materials requires complex multimodal reasoning skills to summarize key concepts and arrange them in a logical and visually pleasing manner.
L2C: Describing Visual Differences Needs Semantic Understanding of Individuals
Recent advances in language and vision push forward the research of captioning a single image to describing visual differences between image pairs.
They, Them, Theirs: Rewriting with Gender-Neutral English
Responsible development of technology involves applications being inclusive of the diverse set of users they hope to support.
On Hallucination and Predictive Uncertainty in Conditional Language Generation
Despite improvements in performances on different natural language generation tasks, deep neural models are prone to hallucinating facts that are incorrect or nonexistent.
M3L: Language-based Video Editing via Multi-Modal Multi-Level Transformers
LBVE contains two features: 1) the scenario of the source video is preserved instead of generating a completely different video; 2) the semantic is presented differently in the target video, and all changes are controlled by the given instruction.
Gaze Perception in Humans and CNN-Based Model
Making accurate inferences about other individuals' locus of attention is essential for human social interactions and will be important for AI to effectively interact with humans.
Comparing Visual Reasoning in Humans and AI
Results show that the machine/human agreement scene descriptions are much lower than human/human agreement for our complex scenes.
FoveaTer: Foveated Transformer for Image Classification
We propose Foveated Transformer (FoveaTer) model, which uses pooling regions and eye movements to perform object classification tasks using a Vision Transformer architecture.
ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation
Automatic evaluations for natural language generation (NLG) conventionally rely on token-level or embedding-level comparisons with text references.
Neural Stylistic Response Generation with Disentangled Latent Variables
Generating open-domain conversational responses in the desired style usually suffers from the lack of parallel data in the style.
MIC: Model-agnostic Integrated Cross-channel Recommenders
However, existing work are primarily built upon pre-defined retrieval channels, including User-CF (U2U), Item-CF (I2I), and Embedding-based Retrieval (U2I), thus access to the limited correlation between users and items which solely entail from partial information of latent interactions.
Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction
Improving the sample efficiency of reinforcement learning algorithms requires effective exploration.
Relational Graph Learning for Grounded Video Description Generation
Such a setting can help explain the decisions of captioning models and prevents the model from hallucinating object words in its description.
PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outliers
Building natural language inference (NLI) benchmarks that are both challenging for modern techniques, and free from shortcut biases is difficult.
End-to-end Dense Video Captioning as Sequence Generation
Dense video captioning aims to identify the events of interest in an input video, and generate descriptive captions for each event.
Ranked #3 on
Dense Video Captioning
on ViTT
(CIDEr metric, using extra
training data)
HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data
A pressing challenge in current dialogue systems is to successfully converse with users on topics with information distributed across different modalities.
KETOD: Knowledge-Enriched Task-Oriented Dialogue
We also propose two new models, SimpleToDPlus and Combiner, for the proposed task.
Neuro-Symbolic Procedural Planning with Commonsense Prompting
Procedural planning aims to implement complex high-level goals by decomposition into sequential simpler low-level steps.
Anticipating the Unseen Discrepancy for Vision and Language Navigation
In this paper, we propose an Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
Learning to Prioritize: Precision-Driven Sentence Filtering for Long Text Summarization
Neural text summarization has shown great potential in recent years.
Dynamic Latent Separation for Deep Learning
A core problem in machine learning is to learn expressive latent variables for model prediction on complex data that involves multiple sub-components in a flexible and interpretable fashion.
