Search Results for author: Chi Han

Found 17 papers, 8 papers with code

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning

no code implementations26 Mar 2024 Rui Pan, Xiang Liu, Shizhe Diao, Renjie Pi, Jipeng Zhang, Chi Han, Tong Zhang

Attempting to complement this deficiency, we investigate layerwise properties of LoRA on fine-tuning tasks and observe an uncommon skewness of weight norms across different layers.

GSM8K Language Modelling +1

Large Language Models on Graphs: A Comprehensive Survey

1 code implementation5 Dec 2023 Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, Jiawei Han

Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i. e., graph-based reasoning).

Language Modelling

InfoPattern: Unveiling Information Propagation Patterns in Social Media

no code implementations27 Nov 2023 Chi Han, Jialiang Xu, Manling Li, Hanning Zhang, Tarek Abdelzaher, Heng Ji

Social media play a significant role in shaping public opinion and influencing ideological communities through information propagation.

Stance Detection

Defining a New NLP Playground

no code implementations31 Oct 2023 Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi R. Fung, Charles Yu, Joel R. Tetreault, Eduard H. Hovy, Heng Ji

The recent explosion of performance of large language models (LLMs) has changed the field of Natural Language Processing (NLP) more abruptly and seismically than any other shift in the field's 80-year history.

LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models

1 code implementation30 Aug 2023 Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang

As a result, their performance suffers drastically on inputs longer than those encountered during training, substantially limiting their applications in real-world tasks involving long contexts such as encoding scientific articles, code repositories, or long dialogues.

Text Generation

CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models

2 code implementations23 May 2023 Cheng Qian, Chi Han, Yi R. Fung, Yujia Qin, Zhiyuan Liu, Heng Ji

Additionally, we introduce the Creation Challenge dataset, featuring 2K diverse questions, to emphasize the necessity and benefits of LLMs' tool creation ability.

Math Transfer Learning

LM-Switch: Lightweight Language Model Conditioning in Word Embedding Space

no code implementations22 May 2023 Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, Nan Jiang, Tarek Abdelzaher, Heng Ji

As pre-training and fine-tuning are costly and might negatively impact model performance, it is desired to efficiently adapt an existing model to different conditions such as styles, sentiments or narratives, when facing different audiences or scenarios.

Language Modelling Word Embeddings

Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learning

1 code implementation22 May 2023 Chi Han, Qizheng He, Charles Yu, Xinya Du, Hanghang Tong, Heng Ji

A LERP is designed as a vector of probabilistic logical functions on the entity's neighboring sub-graph.

Link Prediction

Understanding the Effect of Data Augmentation on Knowledge Distillation

no code implementations21 May 2023 Ziqi Wang, Chi Han, Wenxuan Bao, Heng Ji

However, such data augmentation methods are sub-optimal for knowledge distillation since the teacher model could provide label distributions and is more tolerant to semantic shifts.

Data Augmentation Knowledge Distillation

Zero-Shot Classification by Logical Reasoning on Natural Language Explanations

1 code implementation7 Nov 2022 Chi Han, Hengzhi Pei, Xinya Du, Heng Ji

To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations).

Classification Logical Reasoning +1

Learning Homophilic Incentives in Sequential Social Dilemmas

no code implementations29 Sep 2021 Heng Dong, Tonghan Wang, Jiayuan Liu, Chi Han, Chongjie Zhang

Promoting cooperation among self-interested agents is a long-standing and interdisciplinary problem, but receives less attention in multi-agent reinforcement learning (MARL).

Multi-agent Reinforcement Learning

Learning Shared Semantic Space for Speech-to-Text Translation

2 code implementations Findings (ACL) 2021 Chi Han, Mingxuan Wang, Heng Ji, Lei LI

By projecting audio and text features to a common semantic representation, Chimera unifies MT and ST tasks and boosts the performance on ST benchmarks, MuST-C and Augmented Librispeech, to a new state-of-the-art.

Machine Translation Speech-to-Text Translation +1

Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL

no code implementations23 Apr 2021 Heng Dong, Tonghan Wang, Jiayuan Liu, Chi Han, Chongjie Zhang

We propose a novel learning framework to encourage homophilic incentives and show that it achieves stable cooperation in both SSDs of public goods and tragedy of the commons.

Multi-agent Reinforcement Learning

Visual Concept-Metaconcept Learning

1 code implementation NeurIPS 2019 Chi Han, Jiayuan Mao, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu

Humans reason with concepts and metaconcepts: we recognize red and green from visual input; we also understand that they describe the same property of objects (i. e., the color).

Cannot find the paper you are looking for? You can Submit a new open access paper.