Search Results for author:
Found 24 papers, 10 papers with code
Repository Structure-Aware Training Makes SLMs Better Issue Resolver
Large Language Models (LLMs) outperform Small Language Models (SLMs) in complex tasks like repository-level issue resolving, but raise concerns about privacy and cost.
Dehallucinating Parallel Context Extension for Retrieval-Augmented Generation
Specifically, (1) for fact fabrication, we apply the context-aware negative training that fine-tunes the LLMs with negative supervisions, thus explicitly guiding the LLMs to refuse to answer when contexts are not related to questions; (2) for fact omission, we propose the information-calibrated aggregation which prioritizes context windows with higher information increment from their contexts.
STAND-Guard: A Small Task-Adaptive Content Moderation Model
Content moderation, the process of reviewing and monitoring the safety of generated content, is important for development of welcoming online platforms and responsible large language models.
Self-Evolved Reward Learning for LLMs
Reinforcement Learning from Human Feedback (RLHF) is a crucial technique for aligning language models with human preferences, playing a pivotal role in the success of conversational models like GPT-4, ChatGPT, and Llama 2.
Make Your LLM Fully Utilize the Context
While many contemporary large language models (LLMs) can process lengthy input, they still struggle to fully utilize information within the long context, known as the lost-in-the-middle challenge.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Ranked #5 on
MMR total
on MRR-Benchmark
(using extra training data)
Compositional API Recommendation for Library-Oriented Code Generation
However, the performance remains unsatisfactory in generating library-oriented code, especially for the libraries not present in the training data of LLMs.
Skill-Based Few-Shot Selection for In-Context Learning
Few-shot selection -- selecting appropriate examples for each test instance separately -- is important for in-context learning.
How Do In-Context Examples Affect Compositional Generalization?
Compositional generalization--understanding unseen combinations of seen primitives--is an essential reasoning capability in human intelligence.
Does Deep Learning Learn to Abstract? A Systematic Probing Framework
Abstraction is a desirable capability for deep learning models, which means to induce abstract concepts from concrete instances and flexibly apply them beyond the learning context.
When Language Model Meets Private Library
In this paper, we investigate how to equip pre-trained language models with the ability of code generation for private libraries.
CodeT: Code Generation with Generated Tests
A natural way to evaluate the quality and correctness of a code solution is to run it against a set of test cases, but the manual creation of such test cases is often costly and time-consuming.
Ranked #4 on
Code Generation
on APPS
CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation
Usually, expensive text-code paired data is essential for training a code generation model.
Making Large Language Models Better Reasoners with Step-Aware Verifier
Few-shot learning is a challenging task that requires language models to generalize from limited examples.
Ranked #54 on
Arithmetic Reasoning
on GSM8K
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models
This motivates us to propose input-tuning, which fine-tunes both the continuous prompts and the input representations, leading to a more effective way to adapt unfamiliar inputs to frozen PLMs.
Reasoning Like Program Executors
Reasoning over natural language is a long-standing goal for the research community.
Ranked #2 on
Question Answering
on DROP Test
(using extra training data)
TAPEX: Table Pre-training via Learning a Neural SQL Executor
TAPEX addresses the data scarcity challenge via guiding the language model to mimic a SQL executor on the diverse, large-scale and high-quality synthetic corpus.
Ranked #1 on
Semantic Parsing
on WikiSQL
(Denotation accuracy (test) metric)
Learning Algebraic Recombination for Compositional Generalization
In this paper, we propose LeAR, an end-to-end neural model to learn algebraic recombination for compositional generalization.
Ranked #2 on
Semantic Parsing
on CFQ
Iterative Utterance Segmentation for Neural Semantic Parsing
Experiments on Geo, ComplexWebQuestions, and Formulas show that our framework can consistently improve performances of neural semantic parsers in different domains.
Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization
Human intelligence exhibits compositional generalization (i. e., the capacity to understand and produce unseen combinations of seen components), but current neural seq2seq models lack such ability.
Hierarchical Poset Decoding for Compositional Generalization in Language
We formalize human language understanding as a structured prediction task where the output is a partially ordered set (poset).
Ranked #4 on
Semantic Parsing
on CFQ
Compositional Generalization by Learning Analytical Expressions
Compositional generalization is a basic and essential intellective capability of human beings, which allows us to recombine known parts readily.
