1 code implementation • 3 Mar 2024 • Heegon Jin, Seonil Son, Jemin Park, Youngseok Kim, Hyungjong Noh, Yeonsoo Lee
The Attention Alignment Module in A2D performs a dense head-by-head comparison between student and teacher attention heads across layers, turning the combinatorial mapping heuristics into a learning problem.
no code implementations • 14 Feb 2024 • Minho Lee, Junghyun Min, Woochul Lee, Yeonsoo Lee
Previous work in structured prediction (e. g. NER, information extraction) using single model make use of explicit dataset information, which helps boost in-distribution performance but is orthogonal to robust generalization in real-world situations.
no code implementations • 13 Feb 2024 • Junghyun Min, Minho Lee, Woochul Lee, Yeonsoo Lee
Unsupervised learning objectives like language modeling and de-noising constitute a significant part in producing pre-trained models that perform various downstream applications from natural language understanding to conversational tasks.
2 code implementations • 22 Nov 2022 • Seonil Son, Junsoo Park, Jeong-in Hwang, Junghwa Lee, Hyungjong Noh, Yeonsoo Lee
One of the challenges of developing a summarization model arises from the difficulty in measuring the factual inconsistency of the generated text.
no code implementations • arXiv 2021 • Sanghyuk Choi, Jeong-in Hwang, Hyungjong Noh, Yeonsoo Lee
Recent neural sequence-to-sequence models with a copy mechanism have achieved remarkable progress in various text generation tasks.
2 code implementations • 16 Dec 2021 • Yoonna Jang, Jungwoo Lim, Yuna Hur, Dongsuk Oh, Suhyune Son, Yeonsoo Lee, Donghoon Shin, Seungryong Kim, Heuiseok Lim
Humans usually have conversations by making use of prior knowledge about a topic and background information of the people whom they are talking to.
no code implementations • 13 Sep 2019 • Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, Jaegul Choo
Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications.
no code implementations • 20 Jul 2018 • Minjeong Kim, David Keetae Park, Hyungjong Noh, Yeonsoo Lee, Jaegul Choo
Machine comprehension question answering, which finds an answer to the question given a passage, involves high-level reasoning processes of understanding and tracking the relevant contents across various semantic units such as words, phrases, and sentences in a document.