Search Results for author: Minjoon Seo

Found 55 papers, 42 papers with code

Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders

1 code implementation16 Nov 2023 Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, Kyle Lo

Prevailing research practice today often relies on training dense retrievers on existing large datasets such as MSMARCO and then experimenting with ways to improve zero-shot generalization capabilities to unseen domains.

Data Augmentation Domain Generalization +1

How Well Do Large Language Models Truly Ground?

1 code implementation15 Nov 2023 Hyunji Lee, Sejune Joo, Chaeeun Kim, Joel Jang, Doyoung Kim, Kyoung-Woon On, Minjoon Seo

Reliance on the inherent knowledge of Large Language Models (LLMs) can cause issues such as hallucinations, lack of control, and difficulties in integrating variable knowledge.

KTRL+F: Knowledge-Augmented In-Document Search

1 code implementation14 Nov 2023 Hanseok Oh, Haebin Shin, Miyoung Ko, Hyunji Lee, Minjoon Seo

We introduce a new problem KTRL+F, a knowledge-augmented in-document search task that necessitates real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query.


Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision

1 code implementation13 Nov 2023 Seongyun Lee, Sue Hyun Park, Yongrae Jo, Minjoon Seo

Through a qualitative analysis, we show that Volcano's feedback is properly grounded on the image than the initial response.

Visual Question Answering

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

1 code implementation12 Oct 2023 Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, Minjoon Seo

We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.

Language Modelling Large Language Model

FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets

1 code implementation20 Jul 2023 Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, Minjoon Seo

Evaluation of Large Language Models (LLMs) is challenging because instruction-following necessitates alignment with human values and the required set of skills varies depending on the instruction.

Instruction Following Language Modelling

Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment

no code implementations5 Jul 2023 Yongrae Jo, Seongyun Lee, Aiden SJ Lee, Hyunji Lee, Hanseok Oh, Minjoon Seo

This is accomplished by introducing a soft moment mask that represents a temporal segment in the video and jointly optimizing it with the prefix parameters of a language model.

Language Modelling Text Generation +1

Gradient Ascent Post-training Enhances Language Model Generalization

1 code implementation12 Jun 2023 Dongkeun Yoon, Joel Jang, Sungdong Kim, Minjoon Seo

In this work, we empirically show that updating pretrained LMs (350M, 1. 3B, 2. 7B) with just a few steps of Gradient Ascent Post-training (GAP) on random, unlabeled text corpora enhances its zero-shot generalization capabilities across diverse NLP tasks.

Language Modelling

Exploring the Practicality of Generative Retrieval on Dynamic Corpora

no code implementations27 May 2023 Soyoung Yoon, Chaeeun Kim, Hyunji Lee, Joel Jang, Sohee Yang, Minjoon Seo

Benchmarking the performance of information retrieval (IR) methods are mostly conducted with a fixed set of documents (static corpora); in realistic scenarios, this is rarely the case and the document to be retrieved are constantly updated and added.

Benchmarking Information Retrieval +1

Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis

no code implementations24 May 2023 Sohee Yang, Jonghyeon Kim, Joel Jang, Seonghyeon Ye, Hyunji Lee, Minjoon Seo

Using the finding, we develop several variants of MI and increases the effectiveness of the best prompt selection method from 87. 79% to 94. 98%, measured as the ratio of the performance of the selected prompt to that of the optimal oracle prompt.

Aligning Large Language Models through Synthetic Feedback

1 code implementation23 May 2023 Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Min Yoo, Minjoon Seo

In human evaluation, our model is preferred to Alpaca and Dolly-v2, 55. 0% and 58. 5% of the time, respectively.

Language Modelling

The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning

2 code implementations23 May 2023 Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, Minjoon Seo

Furthermore, we show that instruction tuning with CoT Collection allows LMs to possess stronger few-shot learning capabilities on 4 domain-specific tasks, resulting in an improvement of +2. 24% (Flan-T5 3B) and +2. 37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations until the max length by a +13. 98% margin.

Common Sense Reasoning (Zero-Shot) Few-Shot Learning +2

In-Context Instruction Learning

1 code implementation28 Feb 2023 Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim, Minjoon Seo

Instruction learning of Large Language Models (LLMs) has enabled zero-shot task generalization.

Exploring the Benefits of Training Expert Language Models over Instruction Tuning

2 code implementations7 Feb 2023 Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, Minjoon Seo

Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known as multitask-prompted fine-tuning (MT), have shown the capability to generalize to unseen tasks.

REPLUG: Retrieval-Augmented Black-Box Language Models

no code implementations30 Jan 2023 Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih

We introduce REPLUG, a retrieval-augmented language modeling framework that treats the language model (LM) as a black box and augments it with a tuneable retrieval model.

Language Modelling Multi-task Language Understanding +2

Semi-Parametric Video-Grounded Text Generation

no code implementations27 Jan 2023 Sungdong Kim, Jin-Hwa Kim, Jiyoung Lee, Minjoon Seo

Efficient video-language modeling should consider the computational cost because of a large, sometimes intractable, number of video frames.

Language Modelling Text Generation +2

EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records

1 code implementation NeurIPS 2022 Datasets and Benchmarks 2022 Gyubok Lee, Hyeonji Hwang, Seongsu Bae, Yeonsu Kwon, Woncheol Shin, Seongjun Yang, Minjoon Seo, Jong-Yeup Kim, Edward Choi

We then manually linked these questions to two open-source EHR databases, MIMIC-III and eICU, and included various time expressions and held-out unanswerable questions in the dataset, which were also collected from the poll.

Retrieval Text-To-SQL

Data-efficient End-to-end Information Extraction for Statistical Legal Analysis

1 code implementation3 Nov 2022 Wonseok Hwang, Saehee Eom, Hanuhl Lee, Hai Jin Park, Minjoon Seo

Lawyers, for instance, search for appropriate precedents favorable to their clients, while the number of legal precedents is ever-growing.

Instance Search

Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation

1 code implementation25 Oct 2022 Soyoung Yoon, Sungjoon Park, Gyuwan Kim, Junhee Cho, Kihyo Park, Gyutae Kim, Minjoon Seo, Alice Oh

We show that the model trained with our datasets significantly outperforms the currently used statistical Korean GEC system (Hanspell) on a wider range of error types, demonstrating the diversity and usefulness of the datasets.

Grammatical Error Correction

Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners

1 code implementation6 Oct 2022 Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, Minjoon Seo

Meta-training, which fine-tunes the language model (LM) on various downstream tasks by maximizing the likelihood of the target label given the task instruction and input instance, has improved the zero-shot task generalization performance.

Language Modelling

Efficiently Enhancing Zero-Shot Performance of Instruction Following Model via Retrieval of Soft Prompt

1 code implementation6 Oct 2022 Seonghyeon Ye, Joel Jang, Doyoung Kim, Yongrae Jo, Minjoon Seo

Enhancing the zero-shot performance of instruction-following models requires heavy computation, either by scaling the total number of training datasets or the model size.

Instruction Following Retrieval

Nonparametric Decoding for Generative Retrieval

1 code implementation5 Oct 2022 Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vlad Karpukhin, Yi Lu, Minjoon Seo

The generative retrieval model depends solely on the information encoded in its model parameters without external memory, its information capacity is limited and fixed.

Language Modelling Retrieval +1

Knowledge Unlearning for Mitigating Privacy Risks in Language Models

1 code implementation4 Oct 2022 Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, Minjoon Seo

Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities.

Language Modelling

Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts

1 code implementation26 Sep 2022 Joel Jang, Seonghyeon Ye, Minjoon Seo

Previous work has shown that there exists a scaling law between the size of Language Models (LMs) and their zero-shot performance on different downstream NLP tasks.

A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction

1 code implementation10 Jun 2022 Wonseok Hwang, Dongjun Lee, Kyoungyeon Cho, Hanuhl Lee, Minjoon Seo

Here we present the first large-scale benchmark of Korean legal AI datasets, LBOX OPEN, that consists of one legal corpus, two classification tasks, two legal judgement prediction (LJP) tasks, and one summarization task.

Language Modelling

Prompt Injection: Parameterization of Fixed Inputs

2 code implementations31 May 2022 Eunbi Choi, Yongrae Jo, Joel Jang, Minjoon Seo

Through these explorations, we show that PI can be a promising direction for conditioning language models, especially in scenarios with long and fixed prompts.

Semantic Parsing Zero-Shot Learning

ClaimDiff: Comparing and Contrasting Claims on Contentious Issues

1 code implementation24 May 2022 Miyoung Ko, Ingyu Seong, Hwaran Lee, Joonsuk Park, Minsuk Chang, Minjoon Seo

With the growing importance of detecting misinformation, many studies have focused on verifying factual claims by retrieving evidence.

Fact Verification Misinformation

TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models

1 code implementation29 Apr 2022 Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Minjoon Seo

Language Models (LMs) become outdated as the world changes; they often fail to perform tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment.

Continual Learning

Generative Multi-hop Retrieval

1 code implementation27 Apr 2022 Hyunji Lee, Sohee Yang, Hanseok Oh, Minjoon Seo

A common practice for text retrieval is to use an encoder to map the documents and the query to a common vector space and perform a nearest neighbor search (NNS); multi-hop retrieval also often adopts the same paradigm, usually with a modification of iteratively reformulating the query vector so that it can retrieve different documents at each hop.

Retrieval Text Retrieval

Towards Continual Knowledge Learning of Language Models

2 code implementations ICLR 2022 Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Jungkyu Choi, Minjoon Seo

By highlighting the critical causes of knowledge forgetting, we show that CKL is a challenging and important problem that helps us better understand and train ever-changing LMs.

Continual Learning Fact Checking +2

Cost-effective End-to-end Information Extraction for Semi-structured Document Images

no code implementations EMNLP 2021 Wonseok Hwang, Hyunji Lee, Jinyeong Yim, Geewook Kim, Minjoon Seo

A real-world information extraction (IE) system for semi-structured document images often involves a long pipeline of multiple modules, whose complexity dramatically increases its development and maintenance cost.

Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering

1 code implementation NAACL 2021 Sohee Yang, Minjoon Seo

In open-domain question answering (QA), retrieve-and-read mechanism has the inherent benefit of interpretability and the easiness of adding, removing, or editing knowledge compared to the parametric approaches of closed-book QA models.

Open-Domain Question Answering

Is Retriever Merely an Approximator of Reader?

no code implementations21 Oct 2020 Sohee Yang, Minjoon Seo

The state of the art in open-domain question answering (QA) relies on an efficient retriever that drastically reduces the search space for the expensive reader.

Open-Domain Question Answering

Spatial Dependency Parsing for Semi-Structured Document Information Extraction

1 code implementation Findings (ACL) 2021 Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Sohee Yang, Minjoon Seo

Information Extraction (IE) for semi-structured document images is often approached as a sequence tagging problem by classifying each recognized input token into one of the IOB (Inside, Outside, and Beginning) categories.

Dependency Parsing

Syntactic Question Abstraction and Retrieval for Data-Scarce Semantic Parsing

no code implementations AKBC 2020 Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Minjoon Seo

Deep learning approaches to semantic parsing require a large amount of labeled data, but annotating complex logical forms is costly.

Retrieval Semantic Parsing

Contextualized Sparse Representations for Real-Time Open-Domain Question Answering

3 code implementations ACL 2020 Jinhyuk Lee, Minjoon Seo, Hannaneh Hajishirzi, Jaewoo Kang

Open-domain question answering can be formulated as a phrase retrieval problem, in which we can expect huge scalability and speed benefit but often suffer from low accuracy due to the limitation of existing phrase representation models.

Information Retrieval Open-Domain Question Answering +1

MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension

1 code implementation WS 2019 Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, Danqi Chen

We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems.

Multi-Task Learning Question Answering +1

Mixture Content Selection for Diverse Sequence Generation

1 code implementation IJCNLP 2019 Jaemin Cho, Minjoon Seo, Hannaneh Hajishirzi

The diversification stage uses a mixture of experts to sample different binary masks on the source sequence for diverse content selection.

Abstractive Text Summarization Document Summarization +2

Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index

1 code implementation ACL 2019 Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, Hannaneh Hajishirzi

Existing open-domain question answering (QA) models are not suitable for real-time usage because they need to process several long documents on-demand for every input query.

Open-Domain Question Answering

A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization

5 code implementations4 Feb 2019 Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Minjoon Seo

We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to achieve human performance in WikiSQL dataset.

Semantic Parsing

Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension

1 code implementation EMNLP 2018 Minjoon Seo, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, Hannaneh Hajishirzi

We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder.

Question Answering Reading Comprehension +1

Neural Speed Reading via Skim-RNN

1 code implementation ICLR 2018 Minjoon Seo, Sewon Min, Ali Farhadi, Hannaneh Hajishirzi

Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens.

Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension

no code implementations CVPR 2017 Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, Hannaneh Hajishirzi

Our analysis shows that a significant portion of questions require complex parsing of the text and the diagrams and reasoning, indicating that our dataset is more complex compared to previous machine comprehension and visual question answering datasets.

Question Answering Reading Comprehension +1

Zero-Shot Relation Extraction via Reading Comprehension

2 code implementations CONLL 2017 Omer Levy, Minjoon Seo, Eunsol Choi, Luke Zettlemoyer

We show that relation extraction can be reduced to answering simple reading comprehension questions, by associating one or more natural-language questions with each relation slot.

Reading Comprehension Relation Extraction +4

Question Answering through Transfer Learning from Large Fine-grained Supervision Data

1 code implementation ACL 2017 Sewon Min, Minjoon Seo, Hannaneh Hajishirzi

We show that the task of question answering (QA) can significantly benefit from the transfer learning of models trained on a different large, fine-grained QA dataset.

Question Answering Transfer Learning

Bidirectional Attention Flow for Machine Comprehension

24 code implementations5 Nov 2016 Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, Hannaneh Hajishirzi

Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query.

Cloze Test Open-Domain Question Answering +2

A Diagram Is Worth A Dozen Images

1 code implementation24 Mar 2016 Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi

We define syntactic parsing of diagrams as learning to infer DPGs for diagrams and study semantic interpretation and reasoning of diagrams in the context of diagram question answering.

Visual Question Answering (VQA)

Cannot find the paper you are looking for? You can Submit a new open access paper.