Search Results for author:
Found 30 papers, 10 papers with code
Revealing Vision-Language Integration in the Brain with Multimodal Networks
We operationalize sites of multimodal integration as regions where a multimodal vision-language model predicts recordings better than unimodal language, unimodal vision, or linearly-integrated language-vision models.
Population Transformer: Learning Population-level Representations of Neural Activity
We present a self-supervised framework that learns population-level codes for arbitrary ensembles of neural recordings at scale.
SecureLLM: Using Compositionality to Build Provably Secure Language Models for Private, Sensitive, and Secret Data
We reflect the compositional nature of such security mechanisms back into the structure of LLMs to build a provably secure LLM; that we term SecureLLM.
BrainBERT: Self-supervised representation learning for intracranial recordings
We create a reusable Transformer, BrainBERT, for intracranial recordings bringing modern representation learning approaches to neuroscience.
Query The Agent: Improving sample efficiency through epistemic uncertainty estimation
Curricula for goal-conditioned reinforcement learning agents typically rely on poor estimates of the agent's epistemic uncertainty or fail to consider the agents' epistemic uncertainty altogether, resulting in poor sample efficiency.
Developing a Series of AI Challenges for the United States Department of the Air Force
Through a series of federal initiatives and orders, the U. S. Government has been making a concerted effort to ensure American leadership in AI.
Neural Regression, Representational Similarity, Model Zoology & Neural Taskonomy at Scale in Rodent Visual Cortex
Using our benchmark as an atlas, we offer preliminary answers to overarching questions about levels of analysis (e. g. do models that better predict the representations of individual neurons also predict representational similarity across neural populations?
Trajectory Prediction with Linguistic Representations
Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions.
Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset
These results show that models trained on other datasets and then evaluated on Spoken ObjectNet tend to perform poorly due to biases in other datasets that the models have learned.
PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
The ability to perceive and reason about social interactions in the context of physical environments is core to human social intelligence and human-machine cooperation.
Multi-resolution modeling of a discrete stochastic process identifies cusses of cancer
Detection of cancer-causing mutations within the vast and mostly unexploredhuman genome is a major challenge.
Migratable AI: Personalizing Dialog Conversations with migration context
The migration of conversational AI agents across different embodiments in order to maintain the continuity of the task has been recently explored to further improve user experience.
Migratable AI : Investigating users' affect on identity and information migration of a conversational AI agent
Conversational AI agents are becoming ubiquitous and provide assistance to us in our everyday activities.
Learning a natural-language to LTL executable semantic parser for grounded robotics
We take a step toward robots that can do the same by training a grounded semantic parser, which discovers latent linguistic representations that can be used for the execution of natural-language commands.
Compositional Networks Enable Systematic Generalization for Grounded Language Understanding
Recent work has shown that while deep networks can mimic some human language abilities when presented with novel sentences, systematic variation uncovers the limitations in the language-understanding abilities of networks.
Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas
We demonstrate a reinforcement learning agent which uses a compositional recurrent neural network that takes as input an LTL formula and determines satisfying actions.
Investigating the Decoders of Maximum Likelihood Sequence Models: A Look-ahead Approach
We evaluate our look-ahead module on three datasets of varying difficulties: IM2LATEX-100k OCR image to LaTeX, WMT16 multimodal machine translation, and WMT14 machine translation.
Multimodal Machine Translation
Optical Character Recognition (OCR)
+2
Measuring Social Biases in Grounded Vision and Language Embeddings
We generalize the notion of social biases from language embeddings to grounded vision and language embeddings.
Deep compositional robotic planners that follow natural language commands
We demonstrate how a sampling-based robotic planner can be augmented to learn to understand a sequence of natural language commands in a continuous configuration space to move and manipulate objects.
ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models
Although we focus on object recognition here, data with controls can be gathered at scale using automated tools throughout machine learning to generate datasets that exercise models in new ways thus providing valuable feedback to researchers.
Ranked #51 on
Image Classification
on ObjectNet
(using extra training data)
Grounding language acquisition by training semantic parsers using captioned videos
We develop a semantic parser that is trained in a grounded setting using pairs of videos captioned with sentences.
Complex Relations in a Deep Structured Prediction Model for Fine Image Segmentation
Many deep learning architectures for semantic segmentation involve a Fully Convolutional Neural Network (FCN) followed by a Conditional Random Field (CRF) to carry out inference over an image.
Assessing Language Proficiency from Eye Movements in Reading
We present a novel approach for determining learners' second language proficiency which utilizes behavioral traces of eye movements during reading.
Predicting Native Language from Gaze
A fundamental question in language learning concerns the role of a speaker's first language in second language acquisition.
Anchoring and Agreement in Syntactic Annotations
Our agreement results control for parser bias, and are consequential in that they are on par with state of the art parsing performance for English newswire.
Universal Dependencies for Learner English
We introduce the Treebank of Learner English (TLE), the first publicly available syntactic treebank for English as a Second Language (ESL).
Do You See What I Mean? Visual Resolution of Linguistic Ambiguities
Understanding language goes hand in hand with the ability to integrate complex contextual information obtained via perception.
Contrastive Analysis with Predictive Power: Typology Driven Estimation of Grammatical Error Distributions in ESL
This work examines the impact of cross-linguistic transfer on grammatical errors in English as Second Language (ESL) texts.
Reconstructing Native Language Typology from Foreign Language Usage
Linguists and psychologists have long been studying cross-linguistic transfer, the influence of native language properties on linguistic performance in a foreign language.
