Search Results for author:
Found 79 papers, 64 papers with code
DSAI: Unbiased and Interpretable Latent Feature Extraction for Data-Centric AI
Large language models (LLMs) often struggle to objectively identify latent characteristics in large datasets due to their reliance on pre-trained knowledge rather than actual data patterns.
Generative Context Distillation
To address this challenge, we propose Generative Context Distillation (GCD), a lightweight prompt internalization method that employs a joint training approach.
Latent Action Pretraining from Videos
We introduce Latent Action Pretraining for general Action models (LAPA), an unsupervised method for pretraining Vision-Language-Action (VLA) models without ground-truth robot action labels.
How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?
Vision-Language adaptation (VL adaptation) transforms Large Language Models (LLMs) into Large Vision-Language Models (LVLMs) for multimodal tasks, but this process often compromises the inherent safety capabilities embedded in the original LLMs.
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
In this work, we investigate how a model's tendency to broadly integrate its parametric knowledge evolves throughout pretraining, and how this behavior affects overall performance, particularly in terms of knowledge acquisition and forgetting.
RouterRetriever: Exploring the Benefits of Routing over Multiple Expert Embedding Models
To our knowledge, RouterRetriever is the first work to demonstrate the advantages of using multiple domain-specific expert embedding models with effective routing over a single, general-purpose embedding model in retrieval tasks.
TWLV-I: Analysis and Insights from Holistic Evaluation on Video Foundation Models
In this work, we discuss evaluating video foundation models in a fair and robust manner.
Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization
Based on a hierarchical graph, we quantify forward discrepancy, a discrepancy in LLM performance on simpler sub-problems versus complex questions.
How language models extrapolate outside the training data: A case study in Textualized Gridworld
Language models' ability to extrapolate learned behaviors to novel, more complex environments beyond their training scope is highly unknown.
How Do Large Language Models Acquire Factual Knowledge During Pretraining?
First, counterintuitively, we observe that pretraining on more data shows no significant improvement in the model's capability to acquire and maintain factual knowledge.
On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning
Recent advancements in language and vision assistants have showcased impressive capabilities but suffer from a lack of transparency, limiting broader research and reproducibility.
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
To overcome these limitations, we introduce the BiGGen Bench, a principled generation benchmark designed to thoroughly evaluate nine distinct capabilities of LMs across 77 diverse tasks.
Aligning to Thousands of Preferences via System Message Generalization
Using this dataset, we train a 7B LLM called Janus and test it on 921 prompts from 5 benchmarks (AlpacaEval 2. 0, FLASK, Koala, MT-Bench, and Self-Instruct) by adding various unseen system messages that reflect user preferences.
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Proprietary LMs such as GPT-4 are often employed to assess the quality of responses from various LMs.
Pegasus-v1 Technical Report
This technical report introduces Pegasus-1, a multimodal language model specialized in video content understanding and interaction through natural language.
SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs
While incorporating new information with the retrieval of relevant passages is a promising way to improve QA with LLMs, the existing methods often require additional fine-tuning which becomes infeasible with recent LLMs.
Semiparametric Token-Sequence Co-Supervision
Especially, the robustness of parametric token space which is established during the pretraining step tends to effectively enhance the stability of nonparametric sequence embedding space, a new space established by another language model.
INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Models
Enhancing the capability of retrievers to understand intentions and preferences of users, akin to language model instructions, has the potential to yield more aligned search targets.
Aligning Large Language Models by On-Policy Self-Judgment
Existing approaches for aligning large language models with human preferences face a trade-off that requires a separate reward model (RM) for on-policy learning.
Rethinking the Role of Proxy Rewards in Language Model Alignment
We highlight its potential usage as a simple but strong reward baseline for the LLM alignment, not requiring explicit human feedback dataset and RM training.
LangBridge: Multilingual Reasoning Without Multilingual Supervision
We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision.
Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation
Assessing long-form responses generated by Vision-Language Models (VLMs) is challenging.
Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders
Prevailing research practice today often relies on training dense retrievers on existing large datasets such as MSMARCO and then experimenting with ways to improve zero-shot generalization capabilities to unseen domains.
How Well Do Large Language Models Truly Ground?
To reduce issues like hallucinations and lack of control in Large Language Models (LLMs), a common method is to generate responses by grounding on external contexts given as input, known as knowledge-augmented models.
KTRL+F: Knowledge-Augmented In-Document Search
We introduce a new problem KTRL+F, a knowledge-augmented in-document search task that necessitates real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query.
Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision
Building on this approach, we introduce Volcano, a multimodal self-feedback guided revision model.
Ranked #123 on
Visual Question Answering
on MM-Vet
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.
FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
Evaluation of Large Language Models (LLMs) is challenging because instruction-following necessitates alignment with human values and the required set of skills varies depending on the instruction.
Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment
This is accomplished by introducing a soft moment mask that represents a temporal segment in the video and jointly optimizing it with the prefix parameters of a language model.
Gradient Ascent Post-training Enhances Language Model Generalization
In this work, we empirically show that updating pretrained LMs (350M, 1. 3B, 2. 7B) with just a few steps of Gradient Ascent Post-training (GAP) on random, unlabeled text corpora enhances its zero-shot generalization capabilities across diverse NLP tasks.
A Bayesian Approach To Analysing Training Data Attribution In Deep Learning
We recommend that future researchers and practitioners trust TDA estimates only in such cases.
Exploring the Practicality of Generative Retrieval on Dynamic Corpora
Benchmarking the performance of information retrieval (IR) is mostly conducted with a fixed set of documents (static corpora).
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis
Previous works in prompt engineering for large language models have introduced different gradient-free probability-based prompt selection methods that aim to choose the optimal prompt among the candidates for a given task but have failed to provide a comprehensive and fair comparison between each other.
Effortless Integration of Memory Management into Open-Domain Conversation Systems
Open-domain conversation systems integrate multiple conversation skills into a single system through a modular approach.
Aligning Large Language Models through Synthetic Feedback
In human evaluation, our model is preferred to Alpaca and Dolly-v2, 55. 0% and 58. 5% of the time, respectively.
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
Furthermore, we show that instruction tuning with CoT Collection allows LMs to possess stronger few-shot learning capabilities on 4 domain-specific tasks, resulting in an improvement of +2. 24% (Flan-T5 3B) and +2. 37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations until the max length by a +13. 98% margin.
Ranked #1 on
on BIG-bench (SNARKS)
Common Sense Reasoning
Common Sense Reasoning (Zero-Shot)
+7
Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following
In this paper, we present our finding that prepending a Task-Agnostic Prefix Prompt (TAPP) to the input improves the instruction-following ability of various Large Language Models (LLMs) during inference.
Exploring the Benefits of Training Expert Language Models over Instruction Tuning
Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known as multitask-prompted fine-tuning (MT), have shown the capability to generalize to unseen tasks.
Ranked #9 on
Question Answering
on StoryCloze
REPLUG: Retrieval-Augmented Black-Box Language Models
We introduce REPLUG, a retrieval-augmented language modeling framework that treats the language model (LM) as a black box and augments it with a tuneable retrieval model.
Ranked #15 on
Question Answering
on Natural Questions
Semi-Parametric Video-Grounded Text Generation
Efficient video-language modeling should consider the computational cost because of a large, sometimes intractable, number of video frames.
Ranked #31 on
Video Question Answering
on NExT-QA
EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records
We then manually linked these questions to two open-source EHR databases, MIMIC-III and eICU, and included various time expressions and held-out unanswerable questions in the dataset, which were also collected from the poll.
Data-efficient End-to-end Information Extraction for Statistical Legal Analysis
Lawyers, for instance, search for appropriate precedents favorable to their clients, while the number of legal precedents is ever-growing.
Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation
We show that the model trained with our datasets significantly outperforms the currently used statistical Korean GEC system (Hanspell) on a wider range of error types, demonstrating the diversity and usefulness of the datasets.
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
Meta-training, which fine-tunes the language model (LM) on various downstream tasks by maximizing the likelihood of the target label given the task instruction and input instance, has improved the zero-shot task generalization performance.
Ranked #2 on
Question Answering
on StoryCloze
Efficiently Enhancing Zero-Shot Performance of Instruction Following Model via Retrieval of Soft Prompt
Enhancing the zero-shot performance of instruction-following models requires heavy computation, either by scaling the total number of training datasets or the model size.
Nonparametric Decoding for Generative Retrieval
The generative retrieval model depends solely on the information encoded in its model parameters without external memory, its information capacity is limited and fixed.
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities.
Ranked #3 on
Language Modelling
on The Pile
(Test perplexity metric)
Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts
Previous work has shown that there exists a scaling law between the size of Language Models (LMs) and their zero-shot performance on different downstream NLP tasks.
A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction
Here we present the first large-scale benchmark of Korean legal AI datasets, LBOX OPEN, that consists of one legal corpus, two classification tasks, two legal judgement prediction (LJP) tasks, and one summarization task.
Prompt Injection: Parameterization of Fixed Inputs
Through these explorations, we show that PI can be a promising direction for conditioning language models, especially in scenarios with long and fixed prompts.
ClaimDiff: Comparing and Contrasting Claims on Contentious Issues
With the growing importance of detecting misinformation, many studies have focused on verifying factual claims by retrieving evidence.
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models
Language Models (LMs) become outdated as the world changes; they often fail to perform tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment.
Generative Multi-hop Retrieval
A common practice for text retrieval is to use an encoder to map the documents and the query to a common vector space and perform a nearest neighbor search (NNS); multi-hop retrieval also often adopts the same paradigm, usually with a modification of iteratively reformulating the query vector so that it can retrieve different documents at each hop.
Semi-Structured Query Grounding for Document-Oriented Databases with Deep Retrieval and Its Application to Receipt and POI Matching
Semi-structured query systems for document-oriented databases have many real applications.
Optical Character Recognition
Optical Character Recognition (OCR)
+1
ViSeRet: A simple yet effective approach to moment retrieval via fine-grained video segmentation
Video-text retrieval has many real-world applications such as media analytics, surveillance, and robotics.
Towards Continual Knowledge Learning of Language Models
By highlighting the critical causes of knowledge forgetting, we show that CKL is a challenging and important problem that helps us better understand and train ever-changing LMs.
Cost-effective End-to-end Information Extraction for Semi-structured Document Images
A real-world information extraction (IE) system for semi-structured document images often involves a long pipeline of multiple modules, whose complexity dramatically increases its development and maintenance cost.
Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering
In open-domain question answering (QA), retrieve-and-read mechanism has the inherent benefit of interpretability and the easiness of adding, removing, or editing knowledge compared to the parametric approaches of closed-book QA models.
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020.
Is Retriever Merely an Approximator of Reader?
The state of the art in open-domain question answering (QA) relies on an efficient retriever that drastically reduces the search space for the expensive reader.
Spatial Dependency Parsing for Semi-Structured Document Information Extraction
Information Extraction (IE) for semi-structured document images is often approached as a sequence tagging problem by classifying each recognized input token into one of the IOB (Inside, Outside, and Beginning) categories.
Syntactic Question Abstraction and Retrieval for Data-Scarce Semantic Parsing
Deep learning approaches to semantic parsing require a large amount of labeled data, but annotating complex logical forms is costly.
Contextualized Sparse Representations for Real-Time Open-Domain Question Answering
Open-domain question answering can be formulated as a phrase retrieval problem, in which we can expect huge scalability and speed benefit but often suffer from low accuracy due to the limitation of existing phrase representation models.
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems.
Post-OCR parsing: building simple and robust parser via BIO tagging
Parsing textual information embedded in images is important for various down- stream tasks.
Optical Character Recognition
Optical Character Recognition (OCR)
CORD: A Consolidated Receipt Dataset for Post-OCR Parsing
OCR is inevitably linked to NLP since its final output is in text.
Mixture Content Selection for Diverse Sequence Generation
The diversification stage uses a mixture of experts to sample different binary masks on the source sequence for diverse content selection.
Ranked #10 on
Question Generation
on SQuAD1.1
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index
Existing open-domain question answering (QA) models are not suitable for real-time usage because they need to process several long documents on-demand for every input query.
A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization
We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to achieve human performance in WikiSQL dataset.
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder.
Neural Speed Reading via Skim-RNN
Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens.
Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension
Our analysis shows that a significant portion of questions require complex parsing of the text and the diagrams and reasoning, indicating that our dataset is more complex compared to previous machine comprehension and visual question answering datasets.
Zero-Shot Relation Extraction via Reading Comprehension
We show that relation extraction can be reduced to answering simple reading comprehension questions, by associating one or more natural-language questions with each relation slot.
Question Answering through Transfer Learning from Large Fine-grained Supervision Data
We show that the task of question answering (QA) can significantly benefit from the transfer learning of models trained on a different large, fine-grained QA dataset.
Bidirectional Attention Flow for Machine Comprehension
Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query.
Ranked #4 on
Question Answering
on MS MARCO
Query-Reduction Networks for Question Answering
In this paper, we study the problem of question answering when reasoning over multiple facts is required.
Ranked #2 on
Question Answering
on bAbi
A Diagram Is Worth A Dozen Images
We define syntactic parsing of diagrams as learning to infer DPGs for diagrams and study semantic interpretation and reasoning of diagrams in the context of diagram question answering.
