no code implementations • COLING 2022 • Cesa Salaam, Franck Dernoncourt, Trung Bui, Danda Rawat, Seunghyun Yoon
The prevalent use of offensive content in social media has become an important reason for concern for online platforms (customer service chat-boxes, social media platforms, etc).
1 code implementation • EMNLP (Eval4NLP) 2020 • Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Kyomin Jung
In this paper, we propose an evaluation metric for image captioning systems using both image and text information.
no code implementations • Findings (NAACL) 2022 • Adyasha Maharana, Quan Tran, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Mohit Bansal
We construct and present a new multimodal dataset consisting of software instructional livestreams and containing manual annotations for both detailed and abstract procedural intent that enable training and evaluation of joint video and text understanding models.
1 code implementation • COLING 2022 • Yeon Seonwoo, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, Alice Oh
We conduct three experiments 1) domain-specific document retrieval, 2) comparison of our virtual knowledge graph construction method with previous approaches, and 3) ablation study on each component of our virtual knowledge graph.
no code implementations • NAACL (BioNLP) 2021 • Khalil Mrini, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole
We show that both transfer learning methods combined achieve the highest ROUGE scores.
no code implementations • COLING 2022 • Amir Pouran Ben Veyseh, Quan Hung Tran, Seunghyun Yoon, Varun Manjunatha, Hanieh Deilamsalehy, Rajiv Jain, Trung Bui, Walter W. Chang, Franck Dernoncourt, Thien Huu Nguyen
To this end, this work studies new challenges of KP in transcripts of videos, an understudied domain for KP that involves informal texts and non-cohesive presentation styles.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +1
no code implementations • 4 Nov 2024 • Dang Nguyen, Viet Dac Lai, Seunghyun Yoon, Ryan A. Rossi, Handong Zhao, Ruiyi Zhang, Puneet Mathur, Nedim Lipka, Yu Wang, Trung Bui, Franck Dernoncourt, Tianyi Zhou
Existing LLM agent systems typically select actions from a fixed and predefined set at every step.
no code implementations • 16 Oct 2024 • Yejun Yoon, JaeYoon Jung, Seunghyun Yoon, Kunwoo Park
To tackle the AVeriTeC shared task hosted by the FEVER-24, we introduce a system that only employs publicly available large language models (LLMs) for each step of automated fact-checking, dubbed the Herd of Open LLMs for verifying real-world claims (HerO).
no code implementations • 22 Jul 2024 • Swetha Eppalapally, Daksh Dangi, Chaithra Bhat, Ankita Gupta, Ruiyi Zhang, Shubham Agarwal, Karishma Bagga, Seunghyun Yoon, Nedim Lipka, Ryan A. Rossi, Franck Dernoncourt
Question-answering for domain-specific applications has recently attracted much interest due to the latest advancements in large language models (LLMs).
1 code implementation • 16 Jul 2024 • Minh Nguyen, Franck Dernoncourt, Seunghyun Yoon, Hanieh Deilamsalehy, Hao Tan, Ryan Rossi, Quan Hung Tran, Trung Bui, Thien Huu Nguyen
We introduce an approach to identifying speaker names in dialogue transcripts, a crucial task for enhancing content accessibility and searchability in digital media archives.
1 code implementation • 5 Jul 2024 • Mihir Parmar, Hanieh Deilamsalehy, Franck Dernoncourt, Seunghyun Yoon, Ryan A. Rossi, Trung Bui
Motivated by this, we propose a systematically created human-annotated dataset consisting of coherent summaries for five publicly available datasets and natural language user feedback, offering valuable insights into how to improve coherence in extractive summaries.
1 code implementation • 13 Jun 2024 • Kang-il Lee, Minbeom Kim, Seunghyun Yoon, Minsung Kim, Dongryeol Lee, Hyukhun Koh, Kyomin Jung
To this end, we propose a new benchmark called VLind-Bench, which is the first benchmark specifically designed to measure the language priors, or blindness, of LVLMs.
1 code implementation • 17 Apr 2024 • Joonho Yang, Seunghyun Yoon, Byeongjeong Kim, Hwanhee Lee
These atomic facts represent a more fine-grained unit of information, facilitating detailed understanding and interpretability of the summary's factual inconsistency.
no code implementations • CVPR 2024 • Dawit Mureja Argaw, Seunghyun Yoon, Fabian Caba Heilbron, Hanieh Deilamsalehy, Trung Bui, Zhaowen Wang, Franck Dernoncourt, Joon Son Chung
Long-form video content constitutes a significant portion of internet traffic, making automated video summarization an essential research problem.
1 code implementation • 8 Mar 2024 • Thang M. Pham, Peijie Chen, Tin Nguyen, Seunghyun Yoon, Trung Bui, Anh Totti Nguyen
CLIP-based classifiers rely on the prompt containing a {class name} that is known to the text encoder.
no code implementations • 23 Feb 2024 • Hyunjae Kim, Seunghyun Yoon, Trung Bui, Handong Zhao, Quan Tran, Franck Dernoncourt, Jaewoo Kang
Contrastive language-image pre-training (CLIP) models have demonstrated considerable success across various vision-language tasks, such as text-to-image retrieval, where the model is required to effectively process natural language input to produce an accurate visual output.
1 code implementation • 17 Feb 2024 • Yejun Yoon, Seunghyun Yoon, Kunwoo Park
This paper addresses the critical challenge of assessing the representativeness of news thumbnail images, which often serve as the first visual engagement for readers when an article is disseminated on social media.
no code implementations • 30 Nov 2023 • Linzi Xing, Quan Tran, Fabian Caba, Franck Dernoncourt, Seunghyun Yoon, Zhaowen Wang, Trung Bui, Giuseppe Carenini
Video topic segmentation unveils the coarse-grained semantic structure underlying videos and is essential for other video understanding tasks.
no code implementations • 7 Nov 2023 • Zhongfen Deng, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Quan Hung Tran, Shuaiqi Liu, Wenting Zhao, Tao Zhang, Yibo Wang, Philip S. Yu
Then we merge the sentences selected for a specific aspect as the input for the summarizer to produce the aspect-based summary.
no code implementations • 15 Sep 2023 • Meryem M'hamdi, Jonathan May, Franck Dernoncourt, Trung Bui, Seunghyun Yoon
Our approach leverages meta-distillation learning based on MAML, an optimization-based Model-Agnostic Meta-Learner.
1 code implementation • 15 Aug 2023 • Nakyeong Yang, Minsung Kim, Seunghyun Yoon, Joongbo Shin, Kyomin Jung
However, the existing VMR framework evaluates video moment retrieval performance, assuming that a video is given, which may not reveal whether the models exhibit overconfidence in the falsely given video.
no code implementations • 24 Jul 2023 • Viet Dac Lai, Abel Salinas, Hao Tan, Trung Bui, Quan Tran, Seunghyun Yoon, Hanieh Deilamsalehy, Franck Dernoncourt, Thien Huu Nguyen
Punctuation restoration is an important task in automatic speech recognition (ASR) which aim to restore the syntactic structure of generated ASR texts to improve readability.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +4
no code implementations • 15 Mar 2023 • Yongil Kim, Yerin Hwang, Hyeongu Yun, Seunghyun Yoon, Trung Bui, Kyomin Jung
Vulnerability to lexical perturbation is a critical weakness of automatic evaluation metrics for image captioning.
1 code implementation • ICCV 2023 • Ioana Croitoru, Simion-Vlad Bogolin, Samuel Albanie, Yang Liu, Zhaowen Wang, Seunghyun Yoon, Franck Dernoncourt, Hailin Jin, Trung Bui
To study this problem, we propose the first dataset of untrimmed, long-form tutorial videos for the task of Moment Detection called the Behance Moment Detection (BMD) dataset.
no code implementations • 14 Oct 2022 • Hyunjae Kim, Jaehyo Yoo, Seunghyun Yoon, Jaewoo Kang
Most weakly supervised named entity recognition (NER) models rely on domain-specific dictionaries provided by experts.
1 code implementation • COLING 2022 • Khalil Mrini, Harpreet Singh, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole
The system first matches the summarized user question with an FAQ from a trusted medical knowledge base, and then retrieves a fixed number of relevant sentences from the corresponding answer document.
no code implementations • 26 Jul 2022 • Yoonhyung Lee, Seunghyun Yoon, Kyomin Jung
Then, the attention weights of each modality are applied directly to the other modality in a crossed way, so that the CAN gathers the audio and text information from the same time steps based on each modality.
1 code implementation • 19 Jul 2022 • Thang M. Pham, Seunghyun Yoon, Trung Bui, Anh Nguyen
While contextualized word embeddings have been a de-facto standard, learning contextualized phrase embeddings is less explored and being hindered by the lack of a human-annotated benchmark that tests machine understanding of phrase semantics given a context sentence or paragraph (instead of phrases alone).
1 code implementation • Findings (NAACL) 2022 • Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, Franck Dernoncourt, Trung Bui, Mohit Bansal
Toward more descriptive and distinctive caption generation, we propose using CLIP, a multimodal encoder trained on huge image-text pairs from web, to calculate multimodal similarity and use it as a reward function.
Ranked #26 on Image Captioning on COCO Captions
no code implementations • 18 Apr 2022 • Hwanhee Lee, Cheoneum Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Juae Kim, Kyomin Jung
In this paper, we propose an efficient factual error correction system RFEC based on entities retrieval post-editing process.
1 code implementation • CONSTRAINT (ACL) 2022 • Hyewon Choi, Yejun Yoon, Seunghyun Yoon, Kunwoo Park
This study investigates how fake news uses a thumbnail for a news article with a focus on whether a news article's thumbnail represents the news content correctly.
1 code implementation • 24 Feb 2022 • Hyounghun Kim, Doo Soon Kim, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, Mohit Bansal
To our knowledge, this is the first dataset that provides conversational image search and editing annotations, where the agent holds a grounded conversation with users and helps them to search and edit images according to their requests.
no code implementations • COLING 2022 • Amir Pouran Ben Veyseh, Nicole Meister, Seunghyun Yoon, Rajiv Jain, Franck Dernoncourt, Thien Huu Nguyen
Acronym extraction is the task of identifying acronyms and their expanded forms in texts that is necessary for various NLP applications.
1 code implementation • 16 Dec 2021 • Hyunjae Kim, Jaehyo Yoo, Seunghyun Yoon, Jinhyuk Lee, Jaewoo Kang
Recent named entity recognition (NER) models often rely on human-annotated datasets, requiring the significant engagement of professional knowledge on the target domain and entities.
2 code implementations • EMNLP 2021 • JianGuo Zhang, Trung Bui, Seunghyun Yoon, Xiang Chen, Zhiwei Liu, Congying Xia, Quan Hung Tran, Walter Chang, Philip Yu
In this work, we focus on a more challenging few-shot intent detection scenario where many intents are fine-grained and semantically similar.
1 code implementation • Findings (EMNLP) 2021 • Hwanhee Lee, Thomas Scialom, Seunghyun Yoon, Franck Dernoncourt, Kyomin Jung
A Visual-QA system is necessary for QACE-Img.
1 code implementation • ACL 2021 • Khalil Mrini, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole
Users of medical question answering systems often submit long and detailed questions, making it hard to achieve high recall in answer retrieval.
1 code implementation • ACL 2021 • Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, Kyomin Jung
Also, we observe critical problems of the previous benchmark dataset (i. e., human annotations) on image captioning metric, and introduce a new collection of human annotations on the generated captions.
no code implementations • 16 Oct 2020 • Yanghoon Kim, Seungpil Won, Seunghyun Yoon, Kyomin Jung
Applying generative adversarial networks (GANs) to text-related tasks is challenging due to the discrete nature of language.
1 code implementation • NAACL 2021 • Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Joongbo Shin, Kyomin Jung
To evaluate our metric, we create high-quality human judgments of correctness on two GenQA datasets.
1 code implementation • ACL 2020 • Joongbo Shin, Yoonhyung Lee, Seunghyun Yoon, Kyomin Jung
Even though BERT achieves successful performance improvements in various supervised learning tasks, applying BERT for unsupervised tasks still holds a limitation that it requires repetitive inference for computing contextual language representations.
no code implementations • 1 Apr 2020 • Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Kyomin Jung
Audio Visual Scene-aware Dialog (AVSD) is the task of generating a response for a question with a given scene, video, audio, and the history of previous turns in the dialog.
no code implementations • 1 Apr 2020 • Heeyoung Kwak, Minwoo Lee, Seunghyun Yoon, Jooyoung Chang, Sangmin Park, Kyomin Jung
In this study, we develop a novel graph-based framework for ADR signal detection using healthcare claims data.
no code implementations • 23 Mar 2020 • Kunwoo Park, Taegyun Kim, Seunghyun Yoon, Meeyoung Cha, Kyomin Jung
In digital environments where substantial amounts of information are shared online, news headlines play essential roles in the selection and diffusion of news articles.
1 code implementation • 29 Nov 2019 • Seunghyun Yoon, Subhadeep Dey, Hwanhee Lee, Kyomin Jung
In this work, we explore the impact of visual modality in addition to speech and text for improving the accuracy of the emotion detection system.
1 code implementation • LREC 2020 • Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Kyomin Jung
In this study, we propose a novel graph neural network called propagate-selector (PS), which propagates information over sentences to understand information that cannot be inferred when considering sentences in isolation.
no code implementations • WS 2019 • Jiin Nam, Seunghyun Yoon, Kyomin Jung
While deep learning techniques have shown promising results in many natural language processing (NLP) tasks, it has not been widely applied to the clinical domain.
no code implementations • 30 May 2019 • Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Kyomin Jung
In this paper, we propose a novel method for a sentence-level answer-selection task that is a fundamental problem in natural language processing.
Ranked #8 on Question Answering on TrecQA
2 code implementations • 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2019 • Seunghyun Yoon, Seokhyun Byun, Subhadeep Dey, Kyomin Jung
As opposed to using knowledge from both the modalities separately, we propose a framework to exploit acoustic information in tandem with lexical data.
2 code implementations • 17 Nov 2018 • Seunghyun Yoon, Kunwoo Park, Joongbo Shin, Hongjun Lim, Seungpil Won, Meeyoung Cha, Kyomin Jung
Some news headlines mislead readers with overrated or false information, and identifying them in advance will better assist readers in choosing proper news stories to consume.
4 code implementations • 10 Oct 2018 • Seunghyun Yoon, Seokhyun Byun, Kyomin Jung
Speech emotion recognition is a challenging task, and extensive reliance has been placed on models that use audio features in building well-performing classifiers.
4 code implementations • WS 2018 • Younghun Lee, Seunghyun Yoon, Kyomin Jung
However, this dataset has not been comprehensively studied to its potential.
3 code implementations • NAACL 2018 • Seunghyun Yoon, Joongbo Shin, Kyomin Jung
In this paper, we propose a novel end-to-end neural architecture for ranking candidate answers, that adapts a hierarchical recurrent neural network and a latent topic clustering module.
Ranked #1 on Answer Selection on Ubuntu Dialogue (v1, Ranking)
no code implementations • 29 Sep 2017 • Seunghyun Yoon, Pablo Estrada, Kyomin Jung
We test our model in the Chinese and Sino-Korean vocabularies.
no code implementations • 13 Jan 2017 • Seunghyun Yoon, Hyeongu Yun, Yuna Kim, Gyu-tae Park, Kyomin Jung
In this paper, we propose an efficient transfer leaning methods for training a personalized language model using a recurrent neural network with long short-term memory architecture.