Search Results for author: Haotian Ye

Found 12 papers, 5 papers with code

Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective

1 code implementation24 May 2023 Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, LiWei Wang

We start by giving an impossibility result showing that bounded-depth Transformers are unable to directly produce correct answers for basic arithmetic/equation tasks unless the model size grows super-polynomially with respect to the input length.

Decision Making

A study of conceptual language similarity: comparison and evaluation

no code implementations22 May 2023 Haotian Ye, Yihong Liu, Hinrich Schütze

An interesting line of research in natural language processing (NLP) aims to incorporate linguistic typology to bridge linguistic diversity and assist the research of low-resource languages.

Binary Classification

A Crosslingual Investigation of Conceptualization in 1335 Languages

2 code implementations15 May 2023 Yihong Liu, Haotian Ye, Leonie Weissweiler, Philipp Wicke, Renhao Pei, Robert Zangenfeind, Hinrich Schütze

The resulting measure for the conceptual similarity of two languages is complementary to standard genealogical, typological, and surface similarity measures.

Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages

no code implementations15 May 2023 Chunlan Ma, Ayyoob ImaniGooghari, Haotian Ye, Ehsaneddin Asgari, Hinrich Schütze

While natural language processing tools have been developed extensively for some of the world's languages, a significant portion of the world's over 7000 languages are still neglected.

text-classification Text Classification

Discovering Latent Knowledge in Language Models Without Supervision

1 code implementation7 Dec 2022 Collin Burns, Haotian Ye, Dan Klein, Jacob Steinhardt

Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect.

Imitation Learning Language Modelling +1

Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise

1 code implementation20 Oct 2022 Haotian Ye, James Zou, Linjun Zhang

This opens a promising strategy to first train a feature learner rather than a classifier, and then perform linear probing (last layer retraining) in the test environment.

Representation Learning

On the Power of Pre-training for Generalization in RL: Provable Benefits and Hardness

no code implementations19 Oct 2022 Haotian Ye, Xiaoyu Chen, LiWei Wang, Simon S. Du

Generalization in Reinforcement Learning (RL) aims to learn an agent during training that generalizes to the target environment.

Reinforcement Learning (RL)

Towards a Theoretical Framework of Out-of-Distribution Generalization

no code implementations NeurIPS 2021 Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, LiWei Wang

We also introduce a new concept of expansion function, which characterizes to what extent the variance is amplified in the test domains over the training domains, and therefore give a quantitative meaning of invariant features.

Domain Generalization Model Selection +1

Risk Variance Penalization

no code implementations13 Jun 2020 Chuanlong Xie, Haotian Ye, Fei Chen, Yue Liu, Rui Sun, Zhenguo Li

The key of the out-of-distribution (OOD) generalization is to generalize invariance from training domains to target domains.

Cannot find the paper you are looking for? You can Submit a new open access paper.