Search Results for author: Guangtao Zeng

Found 7 papers, 7 papers with code

Unsupervised Non-transferable Text Classification

1 code implementation23 Oct 2022 Guangtao Zeng, Wei Lu

Training a good deep learning model requires substantial data and computing resources, which makes the resulting neural model a valuable intellectual property.

text-classification Text Classification

One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning

1 code implementation28 May 2023 Guangtao Zeng, Peiyuan Zhang, Wei Lu

Fine-tuning pre-trained language models for multiple tasks tends to be expensive in terms of storage.

Transfer Learning

Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models

1 code implementation23 Oct 2023 Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan

We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases.

TinyLlama: An Open-Source Small Language Model

4 code implementations4 Jan 2024 Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, Wei Lu

We present TinyLlama, a compact 1. 1B language model pretrained on around 1 trillion tokens for approximately 3 epochs.

Computational Efficiency Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.