Search Results for author:
Found 13 papers, 5 papers with code
Active Learning Helps Pretrained Models Learn the Intended Task
Models can fail in unpredictable ways during deployment due to task ambiguity, when multiple behaviors are consistent with the provided training data.
Oolong: Investigating What Makes Crosslingual Transfer Hard with Controlled Studies
Little is known about what makes cross-lingual transfer hard, since factors like tokenization, morphology, and syntax all change at once between languages.
Tradeoffs Between Contrastive and Supervised Learning: An Empirical Study
Contrastive learning has made considerable progress in computer vision, outperforming supervised pretraining on a range of downstream datasets.
DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning
Self-supervised learning algorithms, including BERT and SimCLR, have enabled significant strides in fields like natural language processing, computer vision, and speech processing.
Ranked #1 on
Self-Supervised Learning
on DABS
Pretrained models are active learners
An important barrier to the safe deployment of machine learning systems is the risk of \emph{task ambiguity}, where multiple behaviors are consistent with the provided examples.
C5T5: Controllable Generation of Organic Molecules with Transformers
Methods for designing organic materials with desired properties have high potential impact across fields such as medicine, renewable energy, petrochemical engineering, and agriculture.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
On October 14th, 2020, researchers from OpenAI, the Stanford Institute for Human-Centered Artificial Intelligence, and other universities convened to discuss open research questions surrounding GPT-3, the largest publicly-disclosed dense language model at the time.
Language Through a Prism: A Spectral Approach for Multiscale Language Representations
Language exhibits structure at different scales, ranging from subwords to words, sentences, paragraphs, and documents.
Viewmaker Networks: Learning Views for Unsupervised Representation Learning
However, designing these views requires considerable trial and error by human experts, hindering widespread adoption of unsupervised representation learning methods across domains and modalities.
Investigating Transferability in Pretrained Language Models
How does language model pretraining help transfer learning?
Being Optimistic to Be Conservative: Quickly Learning a CVaR Policy
While maximizing expected return is the goal in most reinforcement learning approaches, risk-sensitive objectives such as conditional value at risk (CVaR) are more suitable for many high-stakes applications.
Recursive Routing Networks: Learning to Compose Modules for Language Understanding
The model jointly optimizes the parameters of the functions and the meta-learner{'}s policy for routing inputs through those functions.
