Search Results for author:
Found 49 papers, 23 papers with code
Multimodal Speech Emotion Recognition Using Audio and Text
Speech emotion recognition is a challenging task, and extensive reliance has been placed on models that use audio features in building well-performing classifiers.
Fine-grained Image Captioning with CLIP Reward
Toward more descriptive and distinctive caption generation, we propose using CLIP, a multimodal encoder trained on huge image-text pairs from web, to calculate multimodal similarity and use it as a reward function.
Ranked #26 on
Image Captioning
on COCO Captions
Comparative Studies of Detecting Abusive Language on Twitter
However, this dataset has not been comprehensively studied to its potential.
Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning
In this work, we focus on a more challenging few-shot intent detection scenario where many intents are fine-grained and semantically similar.
Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning
Even though BERT achieves successful performance improvements in various supervised learning tasks, applying BERT for unsupervised tasks still holds a limitation that it requires repetitive inference for computing contextual language representations.
Simple Questions Generate Named Entity Recognition Datasets
Recent named entity recognition (NER) models often rely on human-annotated datasets, requiring the significant engagement of professional knowledge on the target domain and entities.
Speech Emotion Recognition Using Multi-hop Attention Mechanism
As opposed to using knowledge from both the modalities separately, we propose a framework to exploit acoustic information in tandem with lexical data.
Detecting Incongruity Between News Headline and Body Text via a Deep Hierarchical Encoder
Some news headlines mislead readers with overrated or false information, and identifying them in advance will better assist readers in choosing proper news stories to consume.
Learning to Rank Question-Answer Pairs using Hierarchical Recurrent Encoder with Latent Topic Clustering
In this paper, we propose a novel end-to-end neural architecture for ranking candidate answers, that adapts a hierarchical recurrent neural network and a latent topic clustering module.
Ranked #1 on
Answer Selection
on Ubuntu Dialogue (v1, Ranking)
Attentive Modality Hopping Mechanism for Speech Emotion Recognition
In this work, we explore the impact of visual modality in addition to speech and text for improving the accuracy of the emotion detection system.
KPQA: A Metric for Generative Question Answering Using Keyphrase Weights
To evaluate our metric, we create high-quality human judgments of correctness on two GenQA datasets.
UMIC: An Unreferenced Metric for Image Captioning via Contrastive Learning
Also, we observe critical problems of the previous benchmark dataset (i. e., human annotations) on image captioning metric, and introduce a new collection of human annotations on the generated captions.
ViLBERTScore: Evaluating Image Caption Using Vision-and-Language BERT
In this paper, we propose an evaluation metric for image captioning systems using both image and text information.
Moment Detection in Long Tutorial Videos
To study this problem, we propose the first dataset of untrimmed, long-form tutorial videos for the task of Moment Detection called the Behance Moment Detection (BMD) dataset.
A Gradually Soft Multi-Task and Data-Augmented Approach to Medical Question Understanding
Users of medical question answering systems often submit long and detailed questions, making it hard to achieve high recall in answer retrieval.
How does fake news use a thumbnail? CLIP-based Multimodal Detection on the Unrepresentative News Image
This study investigates how fake news uses a thumbnail for a news article with a focus on whether a news article's thumbnail represents the news content correctly.
CAISE: Conversational Agent for Image Search and Editing
To our knowledge, this is the first dataset that provides conversational image search and editing annotations, where the agent holds a grounded conversation with users and helps them to search and edit images according to their requests.
PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search
While contextualized word embeddings have been a de-facto standard, learning contextualized phrase embeddings is less explored and being hindered by the lack of a human-annotated benchmark that tests machine understanding of phrase semantics given a context sentence or paragraph (instead of phrases alone).
PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck
CLIP-based classifiers rely on the prompt containing a {class name} that is known to the text encoder.
Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks
In this study, we propose a novel graph neural network called propagate-selector (PS), which propagates information over sentences to understand information that cannot be inferred when considering sentences in isolation.
Virtual Knowledge Graph Construction for Zero-Shot Domain-Specific Document Retrieval
We conduct three experiments 1) domain-specific document retrieval, 2) comparison of our virtual knowledge graph construction method with previous approaches, and 3) ablation study on each component of our virtual knowledge graph.
Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision
The system first matches the summarized user question with an FAQ from a trusted medical knowledge base, and then retrieves a fixed number of relevant sentences from the corresponding answer document.
Synonym Discovery with Etymology-based Word Embeddings
We test our model in the Chinese and Sino-Korean vocabularies.
Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network
In this paper, we propose an efficient transfer leaning methods for training a personalized language model using a recurrent neural network with long short-term memory architecture.
A Compare-Aggregate Model with Latent Clustering for Answer Selection
In this paper, we propose a novel method for a sentence-level answer-selection task that is a fundamental problem in natural language processing.
Ranked #8 on
Question Answering
on TrecQA
Surf at MEDIQA 2019: Improving Performance of Natural Language Inference in the Clinical Domain by Adopting Pre-trained Language Model
While deep learning techniques have shown promising results in many natural language processing (NLP) tasks, it has not been widely applied to the clinical domain.
BaitWatcher: A lightweight web interface for the detection of incongruent news headlines
In digital environments where substantial amounts of information are shared online, news headlines play essential roles in the selection and diffusion of news articles.
Drug-disease Graph: Predicting Adverse Drug Reaction Signals via Graph Neural Network with Clinical Data
In this study, we develop a novel graph-based framework for ADR signal detection using healthcare claims data.
DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator
Audio Visual Scene-aware Dialog (AVSD) is the task of generating a response for a question with a given scene, video, audio, and the history of previous turns in the dialog.
Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation
Applying generative adversarial networks (GANs) to text-related tasks is challenging due to the discrete nature of language.
UCSD-Adobe at MEDIQA 2021: Transfer Learning and Answer Sentence Selection for Medical Summarization
We show that both transfer learning methods combined achieve the highest ROUGE scores.
MACRONYM: A Large-Scale Dataset for Multilingual and Multi-Domain Acronym Extraction
Acronym extraction is the task of identifying acronyms and their expanded forms in texts that is necessary for various NLP applications.
Factual Error Correction for Abstractive Summaries Using Entity Retrieval
In this paper, we propose an efficient factual error correction system RFEC based on entities retrieval post-editing process.
Multimodal Intent Discovery from Livestream Videos
We construct and present a new multimodal dataset consisting of software instructional livestreams and containing manual annotations for both detailed and abstract procedural intent that enable training and evaluation of joint video and text understanding models.
Multimodal Speech Emotion Recognition using Cross Attention with Aligned Audio and Text
Then, the attention weights of each modality are applied directly to the other modality in a crossed way, so that the CAN gathers the audio and text information from the same time steps based on each modality.
Offensive Content Detection via Synthetic Code-Switched Text
The prevalent use of offensive content in social media has become an important reason for concern for online platforms (customer service chat-boxes, social media platforms, etc).
Keyphrase Prediction from Video Transcripts: New Dataset and Directions
To this end, this work studies new challenges of KP in transcripts of videos, an understudied domain for KP that involves informal texts and non-cohesive presentation styles.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations
Most weakly supervised named entity recognition (NER) models rely on domain-specific dictionaries provided by experts.
PR-MCS: Perturbation Robust Metric for MultiLingual Image Captioning
Vulnerability to lexical perturbation is a critical weakness of automatic evaluation metrics for image captioning.
Boosting Punctuation Restoration with Data Generation and Reinforcement Learning
Punctuation restoration is an important task in automatic speech recognition (ASR) which aim to restore the syntactic structure of generated ASR texts to improve readability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multilingual Sentence-Level Semantic Search using Meta-Distillation Learning
Our approach leverages meta-distillation learning based on MAML, an optimization-based Model-Agnostic Meta-Learner.
Is it Really Negative? Evaluating Natural Language Video Localization Performance on Multiple Reliable Videos Pool
With the explosion of multimedia content in recent years, Video Corpus Moment Retrieval (VCMR), which aims to detect a video moment that matches a given natural language query from multiple videos, has become a critical problem.
Aspect-based Meeting Transcript Summarization: A Two-Stage Approach with Weak Supervision on Sentence Classification
Then we merge the sentences selected for a specific aspect as the input for the summarizer to produce the aspect-based summary.
Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation
Video topic segmentation unveils the coarse-grained semantic structure underlying videos and is essential for other video understanding tasks.
Understanding News Thumbnail Representativeness by Counterfactual Text-Guided Contrastive Language-Image Pretraining
To serve the challenge, we introduce NewsTT, a manually annotated dataset of news thumbnail image and text pairs.
Fine-tuning CLIP Text Encoders with Two-step Paraphrasing
Contrastive language-image pre-training (CLIP) models have demonstrated considerable success across various vision-language tasks, such as text-to-image retrieval, where the model is required to effectively process natural language input to produce an accurate visual output.
Scaling Up Video Summarization Pretraining with Large Language Models
Long-form video content constitutes a significant portion of internet traffic, making automated video summarization an essential research problem.
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document
These atomic facts represent a more fine-grained unit of information, facilitating detailed understanding and interpretability of the summary's factual inconsistency.
