Search Results for author: Jason Wei

Found 21 papers, 10 papers with code

Frequency Effects on Syntactic Rule Learning in Transformers

1 code implementation14 Sep 2021 Jason Wei, Dan Garrette, Tal Linzen, Ellie Pavlick

Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract symbols and rules.

Good-Enough Example Extrapolation

no code implementations12 Sep 2021 Jason Wei

This paper asks whether extrapolating the hidden space distribution of text examples from one class onto another is a valid inductive bias for data augmentation.

Data Augmentation Text Classification

Finetuned Language Models Are Zero-Shot Learners

1 code implementation3 Sep 2021 Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, Quoc V. Le

We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially boosts zero-shot performance on unseen tasks.

Common Sense Reasoning Language Modelling +5

Language Model Augmented Relevance Score

no code implementations ACL 2021 Ruibo Liu, Jason Wei, Soroush Vosoughi

Although automated metrics are commonly used to evaluate NLG systems, they often correlate poorly with human judgements.

Language Modelling

Modulating Language Models with Emotions

no code implementations17 Aug 2021 Ruibo Liu, Jason Wei, Chenyan Jia, Soroush Vosoughi

Generating context-aware language that embodies diverse emotions is an important step towards building empathetic NLP systems.

The MultiBERTs: BERT Reproductions for Robustness Analysis

1 code implementation30 Jun 2021 Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D'Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, Ian Tenney, Ellie Pavlick

To address this question, we introduce MultiBERTs: a set of 25 BERT-base checkpoints, trained with similar hyper-parameters as the original BERT model but differing in random initialization and data shuffling.

Linguistic Complexity Loss in Text-Based Therapy

no code implementations NAACL 2021 Jason Wei, Kelly Finn, Emma Templeton, Thalia Wheatley, Soroush Vosoughi

The recent advent of online text-based therapy presents a new opportunity to analyze the complexity loss paradox in a novel operationalization: linguistic complexity loss in text-based therapy conversations.

A Cognitive Regularizer for Language Modeling

no code implementations ACL 2021 Jason Wei, Clara Meister, Ryan Cotterell

The uniform information density (UID) hypothesis, which posits that speakers behaving optimally tend to distribute information uniformly across a linguistic signal, has gained traction in psycholinguistics as an explanation for certain syntactic, morphological, and prosodic choices.

Language Modelling

A Survey of Data Augmentation Approaches for NLP

1 code implementation7 May 2021 Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, Eduard Hovy

In this paper, we present a comprehensive and unifying survey of data augmentation for NLP by summarizing the literature in a structured manner.

Data Augmentation

Mitigating Political Bias in Language Models Through Reinforced Calibration

no code implementations30 Apr 2021 Ruibo Liu, Chenyan Jia, Jason Wei, Guangxuan Xu, Lili Wang, Soroush Vosoughi

Current large-scale language models can be politically biased as a result of the data they are trained on, potentially causing serious problems when they are deployed in real-world settings.

Word Embeddings

Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning

1 code implementation NAACL 2021 Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, Shiqi Xu

Few-shot text classification is a fundamental NLP task in which a model aims to classify text into a large number of categories, given only a few training examples per category.

Classification Curriculum Learning +4

A Petri Dish for Histopathology Image Analysis

no code implementations29 Jan 2021 Jerry Wei, Arief Suriawinata, Bing Ren, Xiaoying Liu, Mikhail Lisovsky, Louis Vaickus, Charles Brown, Michael Baker, Naofumi Tomita, Lorenzo Torresani, Jason Wei, Saeed Hassanpour

With the rise of deep learning, there has been increased interest in using neural networks for histopathology image analysis, a field that investigates the properties of biopsy or resected specimens traditionally manually examined under a microscope by pathologists.

Transfer Learning

Text Augmentation in a Multi-Task View

no code implementations EACL 2021 Jason Wei, Chengyu Huang, Shiqi Xu, Soroush Vosoughi

Traditional data augmentation aims to increase the coverage of the input distribution by generating augmented examples that strongly resemble original samples in an online fashion where augmented examples dominate training.

Text Augmentation Text Classification

Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification

no code implementations29 Sep 2020 Jerry Wei, Arief Suriawinata, Bing Ren, Xiaoying Liu, Mikhail Lisovsky, Louis Vaickus, Charles Brown, Michael Baker, Mustafa Nasir-Moin, Naofumi Tomita, Lorenzo Torresani, Jason Wei, Saeed Hassanpour

Based on the nature of histopathology images, a range of difficulty inherently exists among examples, and, since medical datasets are often labeled by multiple annotators, annotator agreement can be used as a natural proxy for the difficulty of a given example.

Curriculum Learning General Classification +1

What Are People Asking About COVID-19? A Question Classification Dataset

1 code implementation ACL 2020 Jerry Wei, Chengyu Huang, Soroush Vosoughi, Jason Wei

We present COVID-Q, a set of 1, 690 questions about COVID-19 from 13 sources, which we annotate into 15 question categories and 207 question clusters.

General Classification

Difficulty Translation in Histopathology Images

1 code implementation27 Apr 2020 Jerry Wei, Arief Suriawinata, Xiaoying Liu, Bing Ren, Mustafa Nasir-Moin, Naofumi Tomita, Jason Wei, Saeed Hassanpour

Our model comprises a scorer, which provides an output confidence to measure the difficulty of images, and an image translator, which learns to translate images from easy-to-classify to hard-to-classify using a training set defined by the scorer.


Generative Image Translation for Data Augmentation in Colorectal Histopathology Images

1 code implementation13 Oct 2019 Jerry Wei, Arief Suriawinata, Louis Vaickus, Bing Ren, Xiaoying Liu, Jason Wei, Saeed Hassanpour

We present an image translation approach to generate augmented data for mitigating data imbalances in a dataset of histopathology images of colorectal polyps, adenomatous tumors that can lead to colorectal cancer if left untreated.

Data Augmentation Image Classification +1

Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slides

1 code implementation20 Nov 2018 Naofumi Tomita, Behnaz Abdollahi, Jason Wei, Bing Ren, Arief Suriawinata, Saeed Hassanpour

Deep learning-based methods, such as the sliding window approach for cropped-image classification and heuristic aggregation for whole-slide inference, for analyzing histological patterns in high-resolution microscopy images have shown promising results.

Crop Classification General Classification +2

Cannot find the paper you are looking for? You can Submit a new open access paper.