Search Results for author:
Found 12 papers, 6 papers with code
Emergent Abilities of Large Language Models
Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks.
Flamingo: a Visual Language Model for Few-Shot Learning
Building models that can be rapidly adapted to numerous tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research.
Ranked #1 on
Zero-Shot Learning
on iVQA
Training Compute-Optimal Large Language Models
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
Ranked #1 on
Multi-task Language Understanding
on MMLU
General-purpose, long-context autoregressive modeling with Perceiver AR
Real-world data is high-dimensional: a book, image, or musical performance can easily contain hundreds of thousands of elements even after compression.
Unified Scaling Laws for Routed Language Models
The performance of a language model has been shown to be effectively modeled as a power-law in its parameter count.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
Analogical Similarity
on BIG-bench
Improving language models by retrieving from trillions of tokens
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens.
Ranked #4 on
Question Answering
on Natural Questions (short)
Perceiver IO: A General Architecture for Structured Inputs & Outputs
A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible.
Ranked #1 on
Optical Flow Estimation
on KITTI 2015
(Average End-Point Error metric)
Human-Agent Cooperation in Bridge Bidding
We introduce a human-compatible reinforcement-learning approach to a cooperative game, making use of a third-party hand-coded human-compatible bot to generate initial training data and to perform initial evaluation.
OpenSpiel: A Framework for Reinforcement Learning in Games
OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.
Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting
We propose a method for natural language generation, choosing the most representative output rather than the most likely output.
Unsupervised Learning of Object Keypoints for Perception and Control
In this work we aim to learn object representations that are useful for control and reinforcement learning (RL).
