Search Results for author:
Found 11 papers, 3 papers with code
Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP
Furthermore, we release our BLIP causal tracing tool as open source to enable further experimentation in vision-language mechanistic interpretability by the community.
Athena 2.0: Discourse and User Modeling in Open Domain Dialogue
Conversational agents are consistently growing in popularity and many people interact with them every day.
Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications
We propose two lower bounds based on the amount of shared information between modalities and the disagreement between separately trained unimodal classifiers, and derive an upper bound through connections to approximate algorithms for min-entropy couplings.
Semantic Composition in Visually Grounded Language Models
What is sentence meaning and its ideal representation?
Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings
Past work probing compositionality in sentence embedding models faces issues determining the causal impact of implicit syntax representations.
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
To tackle this problem, we show that relation alignment can be enforced by encouraging the directed language attention from 'mug' to 'grass' (capturing the semantic relation 'in') to match the directed visual attention from the mug to the grass.
Does Structural Attention Improve Compositional Representations in Vision-Language Models?
Although scaling self-supervised approaches has gained widespread success in Vision-Language pre-training, a number of works providing structural knowledge of visually-grounded semantics have recently shown incremental performance gains.
Ranked #27 on
Visual Reasoning
on Winoground
Athena 2.0: Contextualized Dialogue Management for an Alexa Prize SocialBot
Athena 2. 0 is an Alexa Prize SocialBot that has been a finalist in the last two Alexa Prize Grand Challenges.
(Un)Masked COVID-19 Trends from Social Media
Wearing masks is a useful protection method against COVID-19, which has caused widespread economic and social impact worldwide.
A Cross-lingual Natural Language Processing Framework for Infodemic Management
In this work, we present a novel Cross-lingual Natural Language Processing framework to provide relevant information by matching daily news with trusted guidelines from the World Health Organization.
A Machine Learning Application for Raising WASH Awareness in the Times of COVID-19 Pandemic
Conclusion: We conclude that a multi-pronged machine learning application delivering vernacular bite-sized audios and conversational AI is an effective approach to mitigate health misinformation.
