Multimodal Deep Learning

37 papers with code • 1 benchmarks • 8 datasets

This task has no description! Would you like to contribute one?

Most implemented papers

ShapeWorld - A new test methodology for multimodal language understanding

AlexKuhnle/ShapeWorld 14 Apr 2017

We introduce a novel framework for evaluating multimodal deep learning models with respect to their language understanding and generalization abilities.

Multimodal deep networks for text and image-based document classification

Quicksign/ocrized-text-dataset 15 Jul 2019

Classification of document images is a critical step for archival of old manuscripts, online subscription and administrative procedures.

aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception

aimotive/aimotive_dataset 17 Nov 2022

The dataset consists of 176 scenes with synchronized and calibrated LiDAR, camera, and radar sensors covering a 360-degree field of view.

Multimodal Deep Learning for Robust RGB-D Object Recognition

isrugeek/robotics_recognition 24 Jul 2015

Robust object recognition is a crucial ingredient of many, if not all, real-world robotics applications.

Are These Birds Similar: Learning Branched Networks for Fine-grained Representations

nicolalandro/ntsnet-cub200 16 Jan 2020

In recent years, natural language descriptions are used to obtain information on discriminative parts of the object.

Supervised Video Summarization via Multiple Feature Sets with Parallel Attention

TIBHannover/MSVA 23 Apr 2021

The proposed architecture utilizes an attention mechanism before fusing motion features and features representing the (static) visual content, i. e., derived from an image classification model.

A multimodal deep learning framework for scalable content based visual media retrieval

ambareeshravi/media_retrieval 18 May 2021

We propose a novel, efficient, modular and scalable framework for content based visual media retrieval systems by leveraging the power of Deep Learning which is flexible to work both for images and videos conjointly and we also introduce an efficient comparison and filtering metric for retrieval.

Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

declare-lab/multimodal-deep-learning 28 Jul 2021

Multimodal sentiment analysis aims to extract and integrate semantic information collected from multiple modalities to recognize the expressed emotions and sentiment in multimodal data.

XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification

catalina17/XFlow 2 Sep 2017

Our work improves on existing multimodal deep learning algorithms in two essential ways: (1) it presents a novel method for performing cross-modality (before features are learned from individual modalities) and (2) extends the previously proposed cross-connections which only transfer information between streams that process compatible data.

Learn to Combine Modalities in Multimodal Deep Learning

skywaLKer518/MultiplicativeMultimodal 29 May 2018

Combining complementary information from multiple modalities is intuitively appealing for improving the performance of learning-based approaches.