Search Results for author: Tianyu Gao

Found 29 papers, 22 papers with code

Few-shot Relation Extraction via Bayesian Meta-learning on Task Graphs

no code implementations ICML 2020 Meng Qu, Tianyu Gao, Louis-Pascal Xhonneux, Jian Tang

This paper studies few-shot relation extraction, which aims at predicting the relation for a pair of entities in a sentence by training with a few labeled examples in each relation.

Meta-Learning Relation +2

Long-Context Language Modeling with Parallel Context Encoding

1 code implementation26 Feb 2024 Howard Yen, Tianyu Gao, Danqi Chen

We further introduce a CEPE variant that can extend the context window of instruction-tuned models with only unlabeled data, and showcase its effectiveness on LLAMA-2-CHAT, leading to a strong instruction-following model that can leverage very long context on downstream tasks.

In-Context Learning Instruction Following +1

Improving Language Understanding from Screenshots

1 code implementation21 Feb 2024 Tianyu Gao, ZiRui Wang, Adithya Bhaskar, Danqi Chen

An emerging family of language models (LMs), capable of processing both text and images within a single visual view, has the promise to unlock complex tasks such as chart understanding and UI navigation.

Harmonizing Covariance and Expressiveness for Deep Hamiltonian Regression in Crystalline Material Research: a Hybrid Cascaded Regression Framework

no code implementations1 Jan 2024 Shi Yin, Xinyang Pan, XUDONG ZHU, Tianyu Gao, Haochong Zhang, Feng Wu, Lixin He

Deep learning for Hamiltonian regression of quantum systems in material research necessitates satisfying the covariance laws, among which achieving SO(3)-equivariance without sacrificing the expressiveness capability of networks remains unsolved due to the restriction on non-linear mappings in assuring theoretical equivariance.

regression

Evaluating Large Language Models at Evaluating Instruction Following

1 code implementation11 Oct 2023 Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, Danqi Chen

As research in large language models (LLMs) continues to accelerate, LLM-based evaluation has emerged as a scalable and cost-effective alternative to human evaluations for comparing the ever increasing list of models.

Instruction Following

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

1 code implementation10 Oct 2023 Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen

In this work, we study structured pruning as an effective means to develop smaller LLMs from pre-trained, larger models.

Language Modelling Question Answering +1

Fine-Tuning Language Models with Just Forward Passes

2 code implementations NeurIPS 2023 Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, Sanjeev Arora

Fine-tuning language models (LMs) has yielded success on diverse downstream tasks, but as LMs grow in size, backpropagation requires a prohibitively large amount of memory.

In-Context Learning Multiple-choice

What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning

1 code implementation16 May 2023 Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen

Large language models (LLMs) exploit in-context learning (ICL) to solve tasks with only a few demonstrations, but its mechanisms are not yet well-understood.

In-Context Learning

The CRINGE Loss: Learning what language not to model

no code implementations10 Nov 2022 Leonard Adolphs, Tianyu Gao, Jing Xu, Kurt Shuster, Sainbayar Sukhbaatar, Jason Weston

Standard language model training employs gold human documents or human-human interaction data, and treats all training data as positive examples.

Language Modelling

Transformer-based dimensionality reduction

no code implementations15 Oct 2022 Ruisheng Ran, Tianyu Gao, Bin Fang

Recently, Transformer is much popular and plays an important role in the fields of Machine Learning (ML), Natural Language Processing (NLP), and Computer Vision (CV), etc.

Data Visualization Dimensionality Reduction +2

Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models

1 code implementation COLING 2022 Zichun Yu, Tianyu Gao, Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Maosong Sun, Jie zhou

Prompting, which casts downstream applications as language modeling tasks, has shown to be sample efficient compared to standard fine-tuning with pre-trained models.

Few-Shot Learning Language Modelling +1

Recovering Private Text in Federated Learning of Language Models

1 code implementation17 May 2022 Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen

For the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences.

Federated Learning Word Embeddings

Ditch the Gold Standard: Re-evaluating Conversational Question Answering

2 code implementations ACL 2022 Huihan Li, Tianyu Gao, Manan Goenka, Danqi Chen

In this work, we conduct the first large-scale human evaluation of state-of-the-art conversational QA systems, where human evaluators converse with models and judge the correctness of their answers.

Question Rewriting

SimCSE: Simple Contrastive Learning of Sentence Embeddings

21 code implementations EMNLP 2021 Tianyu Gao, Xingcheng Yao, Danqi Chen

This paper presents SimCSE, a simple contrastive learning framework that greatly advances state-of-the-art sentence embeddings.

Contrastive Learning Data Augmentation +6

Making Pre-trained Language Models Better Few-shot Learners

9 code implementations ACL 2021 Tianyu Gao, Adam Fisch, Danqi Chen

We present LM-BFF--better few-shot fine-tuning of language models--a suite of simple and complementary techniques for fine-tuning language models on a small number of annotated examples.

Few-Shot Learning Zero-Shot Text Classification

TADO: Time-varying Attention with Dual-Optimizer Model

1 code implementation8 Dec 2020 Yuexin Wu, Tianyu Gao, Sihao Wang, Zhongmin Xiong

As the first attempt in this field to address this problem, we propose a flexible dual-optimizer model to gain robustness from both regression loss and classification loss.

Recommendation Systems

Learning from Context or Names? An Empirical Study on Neural Relation Extraction

1 code implementation EMNLP 2020 Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie zhou

We find that (i) while context is the main source to support the predictions, RE models also heavily rely on the information from entity mentions, most of which is type information, and (ii) existing datasets may leak shallow heuristics via entity mentions and thus contribute to the high performance on RE benchmarks.

Memorization Relation +1

Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs

1 code implementation5 Jul 2020 Meng Qu, Tianyu Gao, Louis-Pascal A. C. Xhonneux, Jian Tang

To more effectively generalize to new relations, in this paper we study the relationships between different relations and propose to leverage a global relation graph.

Meta-Learning Relation +2

Continual Relation Learning via Episodic Memory Activation and Reconsolidation

no code implementations ACL 2020 Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie zhou

Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations.

Continual Learning Relation

FewRel 2.0: Towards More Challenging Few-Shot Relation Classification

1 code implementation IJCNLP 2019 Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie zhou

We present FewRel 2. 0, a more challenging task to investigate two aspects of few-shot relation classification models: (1) Can they adapt to a new domain with only a handful of instances?

Classification Domain Adaptation +3

OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction

1 code implementation IJCNLP 2019 Xu Han, Tianyu Gao, Yuan YAO, Demin Ye, Zhiyuan Liu, Maosong Sun

OpenNRE is an open-source and extensible toolkit that provides a unified framework to implement neural models for relation extraction (RE).

Information Retrieval Question Answering +3

Neural Snowball for Few-Shot Relation Learning

1 code implementation29 Aug 2019 Tianyu Gao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations.

Knowledge Graphs Relation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.