Search Results for author:
Found 15 papers, 8 papers with code
Position Paper: Agent AI Towards a Holistic Intelligence
Recent advancements in large foundation models have remarkably enhanced our understanding of sensory information in open-world environments.
An Interactive Agent Foundation Model
We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents across a wide range of domains, datasets, and tasks.
Agent AI: Surveying the Horizons of Multimodal Interaction
To accelerate research on agent-based multimodal intelligence, we define "Agent AI" as a class of interactive systems that can perceive visual stimuli, language inputs, and other environmentally-grounded data, and can produce meaningful embodied actions.
Benchmarking Multi-Domain Active Learning on Image Classification
Active learning aims to enhance model performance by strategically labeling informative data points.
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use
These descriptions enable 1) collecting human-verified reference outputs for each instance; and 2) automatic evaluation of candidate multimodal generations using a text-only LLM, aligning with human judgment.
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
As a demonstration of the research possible in AlpacaFarm, we find that methods that use a reward model can substantially improve over supervised fine-tuning and that our reference PPO implementation leads to a +10% improvement in win-rate against Davinci003.
Data Feedback Loops: Model-driven Amplification of Dataset Biases
Datasets scraped from the internet have been critical to the successes of large-scale machine learning.
Is a Caption Worth a Thousand Images? A Controlled Study for Representation Learning
The development of CLIP [Radford et al., 2021] has sparked a debate on whether language supervision can result in vision models with more transferable representations than traditional image-only methods.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization
For machine learning systems to be reliable, we must understand their performance in unseen, out-of-distribution environments.
Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
Specifically, we directly treat the whole encoded feature map of the input texture as transposed convolution filters and the features' self-similarity map, which captures the auto-correlation information, as input to the transposed convolution.
Measuring Robustness to Natural Distribution Shifts in Image Classification
We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets.
Ranked #41 on
Domain Generalization
on VizWiz-Classification
Autoregressive Models: What Are They Good For?
Autoregressive (AR) models have become a popular tool for unsupervised learning, achieving state-of-the-art log likelihood estimates.
When Robustness Doesn’t Promote Robustness: Synthetic vs. Natural Distribution Shifts on ImageNet
We conduct a large experimental comparison of various robustness metrics for image classification.
Targeted Adversarial Examples for Black Box Audio Systems
The application of deep recurrent networks to audio transcription has led to impressive gains in automatic speech recognition (ASR) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
