Search Results for author:
Found 15 papers, 7 papers with code
Brain Mapping with Dense Features: Grounding Cortical Semantic Selectivity in Natural Images With Vision Transformers
To overcome the challenge presented by the co-occurrence of multiple categories in natural images, we introduce BrainSAIL (Semantic Attribution and Image Localization), a method for isolating specific neurally-activating visual concepts in images.
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought
In TEACh, combining fine-tuning and retrieval on ICAL examples outperforms raw human demonstrations and expert examples, achieving a 17. 5% increase in goal-condition success.
StableSemantics: A Synthetic Language-Vision Dataset of Semantic Representations in Naturalistic Images
To the best of our knowledge, we are the first to release a diffusion dataset with semantic attributions.
Reanimating Images using Neural Representations of Dynamic Stimuli
Our approach leverages state-of-the-art video diffusion models to decouple static image representation from motion generation, enabling us to utilize fMRI brain activity for a deeper understanding of human responses to dynamic visual stimuli.
HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models
Recent research on instructable agents has used memory-augmented Large Language Models (LLMs) as task planners, a technique that retrieves language-program examples relevant to the input instruction and uses them as in-context examples in the LLM prompt to improve the performance of the LLM in inferring the correct action and task plans.
Divergences between Language Models and Human Brains
Do machines and humans process language in similar ways?
Open-Ended Instructable Embodied Agents with Memory-Augmented Large Language Models
Pre-trained and frozen large language models (LLMs) can effectively map simple scene rearrangement instructions to programs over a robot's visuomotor functions through appropriate few-shot example prompting.
BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity
Our results show that BrainSCUBA is a promising means for understanding functional preferences in the brain, and provides motivation for further hypothesis-driven investigation of visual cortex.
Thinking Like an Annotator: Generation of Dataset Labeling Instructions
In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples.
Quantifying the Roles of Visual, Linguistic, and Visual-Linguistic Complexity in Verb Acquisition
Based on these results, we conclude that verb acquisition is influenced by all three sources of complexity, but that the variability of visual structure poses the most significant challenge for verb learning.
TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priors
We introduce TIDEE, an embodied agent that tidies up a disordered scene based on learned commonsense object placement and room arrangement priors.
Learning Neural Acoustic Fields
By modeling acoustic propagation in a scene as a linear time-invariant system, NAFs learn to continuously map all emitter and listener location pairs to a neural impulse response function that can then be applied to arbitrary sounds.
AlphaNet: Improving Long-Tail Classification By Combining Classifiers
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes.
Learning Intermediate Features of Object Affordances with a Convolutional Neural Network
Our ability to interact with the world around us relies on being able to infer what actions objects afford -- often referred to as affordances.
BOLD5000: A public fMRI dataset of 5000 images
Vision science, particularly machine vision, has been revolutionized by introducing large-scale image datasets and statistical learning approaches.
