Discriminative and Generative Transformer-based Models For Situation Entity Classification

no code implementations • 15 Sep 2021 • Mehdi Rezaee, Kasra Darvish, Gaoussou Youssouf Kebe, Francis Ferraro

We re-examine the situation entity (SE) classification task with varying amounts of available training data.

Practical Cross-modal Manifold Alignment for Grounded Language

no code implementations • 1 Sep 2020 • Andre T. Nguyen, Luke E. Richards, Gaoussou Youssouf Kebe, Edward Raff, Kasra Darvish, Frank Ferraro, Cynthia Matuszek

We propose a cross-modality manifold alignment procedure that leverages triplet loss to jointly learn consistent, multi-modal embeddings of language-based concepts of real-world items.

Grounded language learning

Presentation and Analysis of a Multimodal Dataset for Grounded Language Learning

no code implementations • 29 Jul 2020 • Patrick Jenkins, Rishabh Sachdeva, Gaoussou Youssouf Kebe, Padraig Higgins, Kasra Darvish, Edward Raff, Don Engel, John Winder, Francis Ferraro, Cynthia Matuszek

Grounded language acquisition -- learning how language-based interactions refer to the world around them -- is amajor area of research in robotics, NLP, and HCI.

Grounded language learning