Search Results for author:
Found 17 papers, 4 papers with code
The VoxWorld Platform for Multimodal Embodied Agents
We present a five-year retrospective on the development of the VoxWorld platform, first introduced as a multimodal platform for modeling motion language, that has evolved into a platform for rapidly building and deploying embodied agents with contextual and situational awareness, capable of interacting with humans in multiple modalities, and exploring their environments.
Multimodal Cross-Document Event Coreference Resolution Using Linear Semantic Transfer and Mixed-Modality Ensembles
We establish three methods that incorporate images and text for coreference: 1) a standard fused model with finetuning, 2) a novel linear mapping method without finetuning and 3) an ensembling approach based on splitting mention pairs by semantic and discourse-level difficulty.
Common Ground Tracking in Multimodal Dialogue
Within Dialogue Modeling research in AI and NLP, considerable attention has been spent on ``dialogue state tracking'' (DST), which is the ability to update the representations of the speaker's needs at each turn in the dialogue by taking into account the past dialogue moves and history.
How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid?
We (1) manually transcribe utterances in a dataset of triads collaboratively solving a problem involving dialogue and physical object manipulation, (2) annotate collaborative moves according to these gold-standard transcripts, and then (3) apply these annotations to utterances that have been automatically segmented using toolkits from Google and OpenAI's Whisper.
Hamming Similarity and Graph Laplacians for Class Partitioning and Adversarial Image Detection
Here, we investigate the potential for ReLU activation patterns (encoded as bit vectors) to aid in understanding and interpreting the behavior of neural networks.
A Computer Vision Method for Estimating Velocity from Jumps
Athletes routinely undergo fitness evaluations to evaluate their training progress.
Dual Graphs of Polyhedral Decompositions for the Detection of Adversarial Attacks
This paper illustrates how one can utilize the dual graph to detect and analyze adversarial attacks in the context of digital images.
APE-V: Athlete Performance Evaluation using Video
However, athlete error calls into question 46. 2% of movements — in these cases, an expert assessor would have the athlete redo the movement to eliminate the error.
Canonical Face Embeddings
When instead evaluating embeddings generated from two CNNs, where one CNN's embeddings are mapped with a linear transformation, the mean true accept rate drops to 0. 95 using the same verification paradigm.
Motion Prediction Performance Analysis for Autonomous Driving Systems and the Effects of Tracking Noise
In this work, we systematically explore the importance of the tracking module for the motion prediction task and ultimately conclude that the overall motion prediction performance is highly sensitive to the tracking module's imperfections.
Exploring the Interchangeability of CNN Embedding Spaces
When image embeddings generated by one CNN are transformed into embeddings corresponding to the feature space of a second CNN trained on the same task, their respective image classification or face verification performance is largely preserved.
A Pose Proposal and Refinement Network for Better Object Pose Estimation
Our approach is composed of 2 main components: the first component classifies the objects in the input image and proposes an initial 6D pose estimate through a multi-task, CNN-based encoder/multi-decoder module.
Utilizing Network Properties to Detect Erroneous Inputs
Neural networks are vulnerable to a wide range of erroneous inputs such as adversarial, corrupted, out-of-distribution, and misclassified examples.
Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction
Our end-to-end model consists of two stages: the first stage is an encoder/decoder network that learns to predict future video frames.
Getting the subtext without the text: Scalable multimodal sentiment classification from visual and acoustic modalities
We discard traditional transcription features in order to minimize human intervention and to maximize the deployability of our model on at-scale real-world data.
A Neurobiological Evaluation Metric for Neural Network Model Search
In this paper we introduce a human-model similarity (HMS) metric, which quantifies the similarity of human fMRI and network activation behavior.
