Search Results for author:
Found 34 papers, 29 papers with code
Gated Slot Attention for Efficient Linear-Time Sequence Modeling
Linear attention Transformers and their gated variants, celebrated for enabling parallel training and efficient recurrent inference, still fall short in recall-intensive tasks compared to traditional Transformers and demand significant resources for training from scratch.
MR-BEN: A Comprehensive Meta-Reasoning Benchmark for Large Language Models
Large language models (LLMs) have shown increasing capability in problem-solving and decision-making, largely based on the step-by-step chain-of-thought reasoning processes.
Parallelizing Linear Transformers with the Delta Rule over Sequence Length
While more expressive variants of linear transformers which replace the additive outer-product update in linear transformers with the delta rule have been found to be more effective at associative recall, existing algorithms for training such models do not parallelize over sequence length and are thus inefficient to train on modern hardware.
Language Model Evolution: An Iterated Learning Perspective
With the widespread adoption of Large Language Models (LLMs), the prevalence of iterative interactions among these models is anticipated to increase.
Learning to Decode Collaboratively with Multiple Language Models
We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level.
In-Context Language Learning: Architectures and Algorithms
Finally, we show that hard-wiring these heads into neural models improves performance not just on ICLL, but natural language modeling -- improving the perplexity of 340M-parameter models by up to 1. 14 points (6. 7%) on the SlimPajama dataset.
Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models
In this work, we explore data-efficient adaptation of pre-trained code models by further pre-training and fine-tuning them with program structures.
Gated Linear Attention Transformers with Hardware-Efficient Training
When used as a replacement for the standard attention layer in Transformers, the resulting gated linear attention (GLA) Transformer is found to perform competitively against the LLaMA-architecture Transformer (Touvron et al., 2023) as well recent linear-time-inference baselines such as RetNet (Sun et al., 2023a) and Mamba (Gu & Dao, 2023) on moderate-scale language modeling experiments.
Explain-then-Translate: An Analysis on Improving Program Translation with Self-generated Explanations
This work explores the use of self-generated natural language explanations as an intermediate step for code-to-code translation with language models.
Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
The ability to derive underlying principles from a handful of observations and then generalize to novel situations -- known as inductive reasoning -- is central to human intelligence.
Lemur: Harmonizing Natural Language and Code for Language Agents
We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents.
An Investigation of LLMs' Inefficacy in Understanding Converse Relations
Our ConvRE features two tasks, Re2Text and Text2Re, which are formulated as multi-choice question answering to evaluate LLMs' ability to determine the matching between relations and associated text.
GenSim: Generating Robotic Simulation Tasks via Large Language Models
Collecting large amounts of real-world interaction data to train general robotic policies is often prohibitively expensive, thus motivating the use of simulation data.
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills.
Grammar Prompting for Domain-Specific Language Generation with Large Language Models
Large language models (LLMs) can learn to perform a wide range of natural language tasks from just a handful of in-context examples.
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques
Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs).
Ranked #6 on
Text-To-SQL
on spider
Iterative Forward Tuning Boosts In-Context Learning in Language Models
Despite the advancements in in-context learning (ICL) for large language models (LLMs), current research centers on specific prompt engineering, such as demonstration selection, with the expectation that a single iteration of demonstrations processing can generalize effectively to a given test sample.
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs
Our emphasis on database values highlights the new challenges of dirty database contents, external knowledge between NL questions and database contents, and SQL efficiency, particularly in the context of massive databases.
Ranked #1 on
Text-To-SQL
on BIRD (BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation)
(Execution Accurarcy (Human) metric)
Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract)
While large language models (LLMs) have demonstrated strong capability in structured prediction tasks such as semantic parsing, few amounts of research have explored the underlying mechanisms of their success.
Hierarchical Phrase-based Sequence-to-Sequence Learning
We describe a neural transducer that maintains the flexibility of standard sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference.
Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing
The importance of building text-to-SQL parsers which can be applied to new databases has long been acknowledged, and a critical step to achieve this goal is schema linking, i. e., properly recognizing mentions of unseen columns or tables when generating SQLs.
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.
Ranked #1 on
Task-Oriented Dialogue Systems
on KVRET
Meta-Learning to Compositionally Generalize
Natural language is compositional; the meaning of a sentence is a function of the meaning of its parts.
Structured Reordering for Modeling Latent Alignments in Sequence Transduction
Despite success in many domains, neural models struggle in settings where train and test examples are drawn from different distributions.
Learning from Executions for Semantic Parsing
Based on the observation that programs which correspond to NL utterances must be always executable, we propose to encourage a parser to generate executable programs for unlabeled utterances.
Learning to Synthesize Data for Semantic Parsing
Moreover, explicitly modeling compositions using PCFG leads to a better exploration of unseen programs, thus generate more diverse data.
Meta-Learning for Domain Generalization in Semantic Parsing
The importance of building semantic parsers which can be applied to new domains and generate programs unseen at training has long been acknowledged, and datasets testing out-of-domain performance are becoming increasingly available.
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing
We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data.
Ranked #8 on
Semantic Parsing
on spider
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
The generalization challenge lies in (a) encoding the database relations in an accessible way for the semantic parser, and (b) modeling alignment between database columns and their mentions in a given query.
Ranked #9 on
Semantic Parsing
on spider
Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs
Semantic parsing aims to map natural language utterances onto machine interpretable meaning representations, aka programs whose execution against a real-world environment produces a denotation.
Ranked #14 on
Semantic Parsing
on WikiTableQuestions
Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities
In medical documents, it is possible that an entity of interest not only contains a discontiguous sequence of words but also overlaps with another entity.
A Neural Transition-based Model for Nested Mention Recognition
It is common that entity mentions can contain other mentions recursively.
Ranked #6 on
Nested Named Entity Recognition
on NNE
Nested Mention Recognition
Nested Named Entity Recognition
+1
Neural Segmental Hypergraphs for Overlapping Mention Recognition
In this work, we propose a novel segmental hypergraph representation to model overlapping entity mentions that are prevalent in many practical datasets.
Ranked #5 on
Nested Named Entity Recognition
on NNE
Nested Mention Recognition
Nested Named Entity Recognition
+1
Learning Latent Opinions for Aspect-Level Sentiment Classification
Aspect-level sentiment classification aims at detecting the sentiment expressed towards a particular target in a sentence.
