Search Results for author:
Found 68 papers, 51 papers with code
Knowledge Transfer between Structured and Unstructured Sources for Complex Question Answering
More fine-grained analyses on transfer behaviors reveal the types of transferred knowledge and transfer patterns.
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
In this work, we narrow this gap by conducting the first study on the privacy risks of generalist web agents in adversarial environments.
MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark
This paper introduces MMMU-Pro, a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark.
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
The levels of generalization also vary across reasoning types: when faced with out-of-distribution examples, transformers fail to systematically generalize for composition but succeed for comparison.
AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs
Moreover, we utilize those successful suffixes as training data to learn a generative model, named AmpleGCG, which captures the distribution of adversarial suffixes given a harmful query and enables the rapid generation of hundreds of suffixes for any harmful queries in seconds.
Cleared for Takeoff? Compositional & Conditional Reasoning may be the Achilles Heel to (Flight-Booking) Language Agents
The rapid progress of large language models (LLMs) has seen them excel and frequently surpass human performance on standard benchmarks.
AttributionBench: How Hard is Automatic Attribution Evaluation?
Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence.
A Multi-Aspect Framework for Counter Narrative Evaluation using Large Language Models
Counter narratives - informed responses to hate speech contexts designed to refute hateful claims and de-escalate encounters - have emerged as an effective hate speech intervention strategy.
When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
In this paper, we examine how large language models (LLMs) solve multi-step problems under a language agent framework with three components: a generator, a discriminator, and a planning method.
A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents
There is a surprisingly large gap between the speed and scale of their development and deployment and our understanding of their safety risks.
LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset
Using SMolInstruct, we fine-tune a set of open-source LLMs, among which, we find that Mistral serves as the best base model for chemistry tasks.
eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data
Both the ECInstruct dataset and the eCeLLM models show great potential in empowering versatile and effective LLMs for e-commerce.
GPT-4V(ision) is a Generalist Web Agent, if Grounded
The recent development on large multimodal models (LMMs), especially GPT-4V(ision) and Gemini, has been quickly expanding the capability boundaries of multimodal models beyond traditional tasks like image captioning and visual question answering.
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.
TableLlama: Towards Open Large Generalist Models for Tables
Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs.
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
The MAmmoTH models are trained on MathInstruct, our meticulously curated instruction tuning dataset.
AgentBench: Evaluating LLMs as Agents
We present AgentBench, a multi-dimensional evolving benchmark that currently consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities in a multi-turn open-ended generation setting.
Roll Up Your Sleeves: Working with a Collaborative and Engaging Task-Oriented Dialogue System
We introduce TacoBot, a user-centered task-oriented digital assistant designed to guide users through complex real-world tasks with multiple steps.
Biomedical Language Models are Robust to Sub-optimal Tokenization
As opposed to general English, many concepts in biomedical terminology have been designed in recent history by biomedical professionals with the goal of being precise and concise.
MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing
To address this issue, we introduce MagicBrush (https://osu-nlp-group. github. io/MagicBrush/), the first large-scale, manually annotated dataset for instruction-guided real image editing that covers diverse scenarios: single-turn, multi-turn, mask-provided, and mask-free editing.
Mind2Web: Towards a Generalist Agent for the Web
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website.
Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms
By leveraging data from multiple clients, the FL paradigm can be especially beneficial for clients that have little training data to develop a data-hungry neural semantic parser on their own.
Exploring Chain-of-Thought Style Prompting for Text-to-SQL
Thus, we systematically study how to enhance LLMs' reasoning ability through chain of thought (CoT) style prompting, including the original chain-of-thought prompting (Wei et al., 2022b) and least-to-most prompting (Zhou et al., 2023).
Error Detection for Text-to-SQL Semantic Parsing
Despite remarkable progress in text-to-SQL semantic parsing in recent years, the performance of existing parsers is still far from perfect.
Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate
Large language models (LLMs) such as ChatGPT and GPT-4 have shown impressive performance in complex reasoning tasks.
Text-to-SQL Error Correction with Language Models of Code
Thus, we propose a novel representation for SQL queries and their edits that adheres more closely to the pre-training corpora of language models of code.
Automatic Evaluation of Attribution by Large Language Models
We manually curate a set of test examples covering 12 domains from a generative search engine, New Bing.
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks.
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs).
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data.
Bootstrapping a User-Centered Task-Oriented Dialogue System
We present TacoBot, a task-oriented dialogue system built for the inaugural Alexa Prize TaskBot Challenge, which assists users in completing multi-step cooking and home improvement tasks.
$\mathsf{G^2Retro}$ as a Two-Step Graph Generative Models for Retrosynthesis Prediction
It first predicts the reaction centers in the target molecules (products), identifies the synthons needed to assemble the products, and transforms these synthons into reactants.
Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again
In this paper, we present the first systematic and comprehensive study to compare the few-shot performance of GPT-3 in-context learning with fine-tuning smaller (i. e., BERT-sized) PLMs on two highly representative biomedical information extraction tasks, named entity recognition and relation extraction.
Iteratively Prompt Pre-trained Language Models for Chain of Thought
While Pre-trained Language Models (PLMs) internalize a great amount of world knowledge, they have been shown incapable of recalling these knowledge to solve tasks requiring complex & multi-step reasoning.
Synthetic Question Value Estimation for Domain Adaptation of Question Answering
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models.
DOM-LM: Learning Generalizable Representations for HTML Documents
We argue that the text and HTML structure together convey important semantics of the content and therefore warrant a special treatment for their representation learning.
Ranked #2 on
Attribute Extraction
on SWDE
TopNet: Learning from Neural Topic Model to Generate Long Stories
In particular, instead of directly generating a story, we first learn to map the short text input to a low-dimensional topic distribution (which is pre-assigned by a topic model).
Towards Transparent Interactive Semantic Parsing via Step-by-Step Correction
In this work, we investigate an interactive semantic parsing framework that explains the predicted logical form step by step in natural language and enables the user to make corrections through natural-language feedback for individual steps.
ReasonBERT: Pre-trained to Reason with Distant Supervision
We present ReasonBert, a pre-training method that augments language models with the ability to reason over long-range relations and multiple, possibly hybrid contexts.
Ranked #1 on
Semantic Parsing
on GraphQuestions
Differential Privacy for Text Analytics via Natural Text Sanitization
The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy-preserving natural language processing over the BERT language model with promising utility.
Learning Structural Edits via Incremental Tree Transformations
To show the unique benefits of modeling tree edits directly, we further propose a novel edit encoder for learning to represent edits, as well as an imitation learning method that allows the editor to be more robust.
CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering
Clinical question answering (QA) aims to automatically answer questions from medical professionals based on clinical texts.
Structure-Grounded Pretraining for Text-to-SQL
Additionally, to evaluate different methods under more realistic text-table alignment settings, we create a new evaluation set Spider-Realistic based on Spider dev set with explicit mentions of column names removed, and adopt eight existing text-to-SQL datasets for cross-database evaluation.
COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrieval
For evaluation, we introduce Query Bank and Relevance Set, where the former contains 1, 236 human-paraphrased queries while the latter contains ~32 human-annotated FAQ items for each query.
Adversarial Training for Code Retrieval with Question-Description Relevance Regularization
Code retrieval is a key task aiming to match natural and programming languages.
Learning a Cost-Effective Annotation Policy for Question Answering
State-of-the-art question answering (QA) relies upon large amounts of training data for which labeling is time consuming and thus expensive.
Energy Efficiency Optimization in IRS-Enhanced mmWave Systems with Lens Antenna Array
To address this problem, we propose an intelligent reflect surface (IRS) enhanced multi-user mmWave communication system with lens antenna array.
Joint Passive Beamforming and User Association Optimization for IRS-assisted mmWave Systems
Considering the impact of IRS on user association, we formulate a sum rate maximization problem by jointly optimizing the passive beamforming at IRS and user association, which is an intractable non-convex problem.
TURL: Table Understanding through Representation Learning
In this paper, we present TURL, a novel framework that introduces the pre-training/fine-tuning paradigm to relational Web tables.
Ranked #1 on
Column Type Annotation
on WikipediaGS-CTA
Rationalizing Medical Relation Prediction from Corpus-level Statistics
Nowadays, the interpretability of machine learning models is becoming increasingly important, especially in the medical domain.
An Imitation Game for Learning Semantic Parsers from User Interaction
Despite the widely successful applications, bootstrapping and fine-tuning semantic parsers are still a tedious process with challenges such as costly data annotation and privacy risks.
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
In this paper, we provide an in-depth analysis of this dataset and the clinical reading comprehension (CliniRC) task.
Practical Annotation Strategies for Question Answering Datasets
Here, remarkably, annotating a stratified subset with only 1. 2% of the original training set achieves 97. 7% of the performance as if the complete dataset was annotated.
Easy-to-Hard: Leveraging Simple Questions for Complex Question Generation
This paper makes one of the first efforts toward automatically generating complex questions from knowledge graphs.
An End-to-End Framework for Cold Question Routing in Community Question Answering Services
Routing newly posted questions (a. k. a cold questions) to potential answerers with the suitable expertise in Community Question Answering sites (CQAs) is an important and challenging task.
Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study
As a promising paradigm, interactive semantic parsing has shown to improve both semantic parsing accuracy and user confidence in the results.
Automatic Table completion using Knowledge Base
In this work, we propose to resolve a new type of heterogeneous query viz: tabular query, which contains a natural language query description, column names of the desired table, and an example row.
Leveraging 2-hop Distant Supervision from Table Entity Pairs for Relation Extraction
Given two entities, distant supervision exploits sentences that directly mention them for predicting their semantic relation.
Reinforced Dynamic Reasoning for Conversational Question Generation
This paper investigates a new task named Conversational Question Generation (CQG) which is to generate a question based on a passage and a conversation history (i. e., previous turns of question-answer pairs).
SurfCon: Synonym Discovery on Privacy-Aware Clinical Data
To solve the problem, we propose a new framework SurfCon that leverages two important types of information in the privacy-aware clinical data, i. e., the surface form information, and the global context information for synonym discovery.
Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations
Our experimental results demonstrate that the recent graph embedding methods achieve promising results and deserve more attention in the future biomedical graph analysis.
CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning
In this work, we investigate a novel perspective of Code annotation for Code retrieval (hence called `CoaCor'), where a code annotation model is trained to generate a natural language annotation that can represent the semantic meaning of a given code snippet and can be leveraged by a code retrieval model to better distinguish relevant code snippets from others.
Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
Given a text description, most existing semantic parsers synthesize a program in one shot.
Hierarchical Reinforcement Learning
reinforcement-learning
+3
StaQC: A Systematically Mined Question-Code Dataset from Stack Overflow
In this paper, we investigate a new problem of systematically mining question-code pairs from Stack Overflow (in contrast to heuristically collecting them).
An End-to-End Deep Framework for Answer Triggering with a Novel Group-Level Objective
Given a question and a set of answer candidates, answer triggering determines whether the candidate set contains any correct answers.
Global Relation Embedding for Relation Extraction
We study the problem of textual relation embedding with distant supervision.
