1 code implementation • 6 Jul 2022 • Jinwoo Kim, Tien Dat Nguyen, Seonwoo Min, Sungjun Cho, Moontae Lee, Honglak Lee, Seunghoon Hong
We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice.
Ranked #15 on Graph Regression on PCQM4Mv2-LSC
2 code implementations • 7 Feb 2023 • Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, Minjoon Seo
Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known as multitask-prompted fine-tuning (MT), have shown the capability to generalize to unseen tasks.
Ranked #9 on Question Answering on StoryCloze
1 code implementation • 4 Oct 2022 • Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, Minjoon Seo
Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities.
Ranked #3 on Language Modelling on The Pile (Test perplexity metric)
1 code implementation • 24 May 2023 • Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang
To address this issue, we propose Guiding chain-of-thought ReAsoning with a CorrectnEss Discriminator (GRACE), a stepwise decoding approach that steers the decoding process towards producing correct reasoning steps.
2 code implementations • 25 May 2022 • Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang
To alleviate the need for a large number of labeled question-document pairs for retriever training, we propose PromptRank, which relies on large language models prompting for multi-hop path reranking.
1 code implementation • 22 Oct 2023 • Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang
Open-domain question answering (QA) systems are often built with retrieval modules.
1 code implementation • 27 Oct 2022 • Sungjun Cho, Seonwoo Min, Jinwoo Kim, Moontae Lee, Honglak Lee, Seunghoon Hong
The forward and backward cost are thus linear to the number of edges, which each attention head can also choose flexibly based on the input.
1 code implementation • 30 Jan 2024 • Saelyne Yang, Sunghyun Park, Yunseok Jang, Moontae Lee
Experiments with answerability classification tasks demonstrate the complexity of YTCommentQA and emphasize the need to comprehend the combined role of visual and script information in video reasoning.
1 code implementation • 17 Aug 2023 • Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang
The standard approach for ICL is to prompt the LM with concatenated demonstrations followed by the test input.
1 code implementation • 24 Oct 2023 • Zheyuan Zhang, Shane Storks, Fengyuan Hu, Sungryull Sohn, Moontae Lee, Honglak Lee, Joyce Chai
We incorporate these interlinked dual processes in fine-tuning and in-context learning with PLMs, applying them to two language understanding tasks that require coherent physical commonsense reasoning.
no code implementations • 19 Nov 2017 • Moontae Lee, David Bindel, David Mimno
Spectral topic modeling algorithms operate on matrices/tensors of word co-occurrence statistics to learn topic-specific word distributions.
no code implementations • EMNLP 2014 • Moontae Lee, David Mimno
The anchor words algorithm performs provably efficient topic model inference by finding an approximate convex hull in a high-dimensional word co-occurrence space.
no code implementations • NeurIPS 2015 • Moontae Lee, David Bindel, David Mimno
Spectral inference provides fast algorithms and provable optimality for latent topic analysis.
no code implementations • NeurIPS 2016 • Moontae Lee, Seok Hyun Jin, David Mimno
Many online communities present user-contributed responses such as reviews of products and answers to questions.
no code implementations • 19 Nov 2015 • Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng, Paul Smolensky
Question answering tasks have shown remarkable progress with distributed vector representation.
no code implementations • 12 Jan 2016 • Paul Smolensky, Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng
In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)).
no code implementations • IJCNLP 2019 • Moontae Lee, Sungjun Cho, David Bindel, David Mimno
Despite great scalability on large data and their ability to understand correlations between topics, spectral topic models have not been widely used due to the absence of reliability in real data and lack of practical implementations.
no code implementations • 12 Nov 2021 • Moontae Lee, Sungjun Cho, Kun Dong, David Mimno, David Bindel
Across many data domains, co-occurrence statistics about the joint appearance of objects are powerfully informative.
no code implementations • CVPR 2023 • Sungmin Cha, Sungjun Cho, Dasol Hwang, Sunwon Hong, Moontae Lee, Taesup Moon
The main reason for the ineffectiveness of their method lies in not fully addressing the data imbalance issue, especially in computing the gradients for learning the affine transformation parameters of BN.
no code implementations • NAACL 2022 • Lajanugen Logeswaran, Yao Fu, Moontae Lee, Honglak Lee
Pre-trained large language models have shown successful progress in many language understanding benchmarks.
no code implementations • 16 Jun 2022 • Sungmin Cha, Jihwan Kwak, Dongsub Shim, Hyunwoo Kim, Moontae Lee, Honglak Lee, Taesup Moon
While the common method for evaluating CIL algorithms is based on average test accuracy for all learned classes, we argue that maximizing accuracy alone does not necessarily lead to effective CIL algorithms.
no code implementations • 7 Sep 2022 • Sung Moon Ko, Sungjun Cho, Dae-Woong Jeong, Sehui Han, Moontae Lee, Honglak Lee
Conventional methods ask users to specify an appropriate number of clusters as a hyperparameter, then assume that all input graphs share the same number of clusters.
no code implementations • 7 Jan 2023 • Byoungjip Kim, Sungik Choi, Dasol Hwang, Moontae Lee, Honglak Lee
Despite surprising performance on zero-shot transfer, pre-training a large-scale multimodal model is often prohibitive as it requires a huge amount of data and computing resources.
no code implementations • 27 Jan 2023 • Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, Moontae Lee
Since the recent advent of regulations for data protection (e. g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sensitive data in pre-trained models without retraining from scratch.
no code implementations • 17 Feb 2023 • Yunseok Jang, Sungryull Sohn, Lajanugen Logeswaran, Tiange Luo, Moontae Lee, Honglak Lee
Real-world tasks consist of multiple inter-dependent subtasks (e. g., a dirty pan needs to be washed before it can be used for cooking).
no code implementations • 17 Feb 2023 • Lajanugen Logeswaran, Sungryull Sohn, Yunseok Jang, Moontae Lee, Honglak Lee
This work explores the problem of generating task graphs of real-world activities.
no code implementations • 7 Jun 2023 • Kyungjae Lee, Sang-eun Han, Seung-won Hwang, Moontae Lee
This paper studies the problem of open-domain question answering, with the aim of answering a diverse range of questions leveraging knowledge resources.
no code implementations • 8 Sep 2023 • Sungjun Cho, Seunghyuk Cho, Sungwoo Park, Hankook Lee, Honglak Lee, Moontae Lee
Real-world graphs naturally exhibit hierarchical or cyclical structures that are unfit for the typical Euclidean space.
no code implementations • 8 Sep 2023 • Sungjun Cho, Dae-Woong Jeong, Sung Moon Ko, Jinwoo Kim, Sehui Han, Seunghoon Hong, Honglak Lee, Moontae Lee
Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels.
no code implementations • 16 Nov 2023 • Lajanugen Logeswaran, Sungryull Sohn, Yiwei Lyu, Anthony Zhe Liu, Dong-Ki Kim, Dongsub Shim, Moontae Lee, Honglak Lee
One of the fundamental skills required for an agent acting in an environment to complete tasks is the ability to understand what actions are plausible at any given point.
no code implementations • NeurIPS 2023 • Sungik Choi, Hankook Lee, Honglak Lee, Moontae Lee
Based on our observation that diffusion models can \emph{project} any sample to an in-distribution sample with similar background information, we propose \emph{Projection Regret (PR)}, an efficient novelty detection method that mitigates the bias of non-semantic information.
no code implementations • 2nd Annual Topology, Algebra, and Geometry in Machine Learning Workshop 2023 • Sungjun Cho, Seunghyuk Cho, Sungwoo Park, Hankook Lee, Honglak Lee, Moontae Lee
Real-world graphs naturally exhibit hierarchical or cyclical structures that are unfit for the typical Euclidean space.
no code implementations • 21 Mar 2024 • Kyungjae Lee, Dasol Hwang, Sunghyun Park, Youngsoo Jang, Moontae Lee
Despite the promise of RLHF in aligning LLMs with human preferences, it often leads to superficial alignment, prioritizing stylistic changes over improving downstream performance of LLMs.