PropTest: Automatic Property Testing for Improved Visual Programming

no code implementations • 25 Mar 2024 • Jaywon Koo, Ziyan Yang, Paola Cascante-Bonilla, Baishakhi Ray, Vicente Ordonez

Visual Programming has emerged as an alternative to end-to-end black-box visual reasoning models.

Question Answering Referring Expression +3

Multi-Modality Multi-Loss Fusion Network

no code implementations • 1 Aug 2023 • Zehui Wu, Ziwei Gong, Jaywon Koo, Julia Hirschberg

In this work we investigate the optimal selection and fusion of features across multiple modalities and combine these in a neural network to improve emotion detection.

feature selection Multimodal Sentiment Analysis

Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities

no code implementations • 14 Jun 2022 • Hammad A. Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Long Chen, Yulei Niu, Xudong Lin, Xuande Feng, Jaywon Koo, Sounak Ray, Shih-Fu Chang

To support research on this task, we introduce the Multimodal Hierarchical Events (MultiHiEve) dataset.