Search Results for author: Yian Zhang

Found 4 papers, 4 papers with code

When Do You Need Billions of Words of Pretraining Data?

1 code implementation ACL 2021 Yian Zhang, Alex Warstadt, Haau-Sing Li, Samuel R. Bowman

We adopt four probing methods---classifier probing, information-theoretic probing, unsupervised relative acceptability judgment, and fine-tuning on NLU tasks---and draw learning curves that track the growth of these different measures of linguistic ability with respect to pretraining data volume using the MiniBERTas, a group of RoBERTa models pretrained on 1M, 10M, 100M and 1B words.

Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)

1 code implementation EMNLP 2020 Alex Warstadt, Yian Zhang, Haau-Sing Li, Haokun Liu, Samuel R. Bowman

One reason pretraining on self-supervised linguistic tasks is effective is that it teaches models features that are helpful for language understanding.

Binary Classification

Latent Tree Learning with Ordered Neurons: What Parses Does It Produce?

1 code implementation EMNLP (BlackboxNLP) 2020 Yian Zhang

Recent latent tree learning models can learn constituency parsing without any exposure to human-annotated tree structures.

Constituency Parsing Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.