Search Results for author:
Found 22 papers, 6 papers with code
An empirical study on the effectiveness of images in Multimodal Neural Machine Translation
In state-of-the-art Neural Machine Translation (NMT), an attention mechanism is used during decoding to enhance the translation.
Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation
In Multimodal Neural Machine Translation (MNMT), a neural model generates a translated sentence that describes an image, given the image itself and one source descriptions in English.
Modulating and attending the source image during encoding improves Multimodal Translation
We propose a new and fully end-to-end approach for multimodal translation where the source text encoder modulates the entire visual input processing using conditional batch normalization, in order to compute the most informative image features for our task.
Proceedings of eNTERFACE 2015 Workshop on Intelligent Interfaces
The 11th Summer Workshop on Multimodal Interfaces eNTERFACE 2015 was hosted by the Numediart Institute of Creative Technologies of the University of Mons from August 10th to September 2015.
UMONS Submission for WMT18 Multimodal Translation Task
This paper describes the UMONS solution for the Multimodal Machine Translation Task presented at the third conference on machine translation (WMT18).
Bringing back simplicity and lightliness into neural image captioning
So far, the goal has been to maximize scores on automated metric and to do so, one has to come up with a plurality of new modules and techniques.
Adversarial reconstruction for Multi-modal Machine Translation
Even with the growing interest in problems at the intersection of Computer Vision and Natural Language, grounding (i. e. identifying) the components of a structured description in an image still remains a challenging task.
Modulated Self-attention Convolutional Network for VQA
As new data-sets for real-world visual reasoning and compositional question answering are emerging, it might be needed to use the visual feature extraction as a end-to-end process during training.
Can adversarial training learn image captioning ?
Recently, generative adversarial networks (GAN) have gathered a lot of interest.
A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis
Understanding expressed sentiment and emotions are two crucial factors in human multimodal language.
Ranked #5 on
Multimodal Sentiment Analysis
on CMU-MOSEI
(using extra training data)
AVECL-UMONS database for audio-visual event classification and localization
We introduce the AVECL-UMons dataset for audio-visual event classification and localization in the context of office environments.
Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition
This paper aims to bring a new lightweight yet powerful solution for the task of Emotion Recognition and Sentiment Analysis.
Ranked #6 on
Multimodal Sentiment Analysis
on CMU-MOSEI
(using extra training data)
Improved Soccer Action Spotting using both Audio and Video Streams
Action spotting and classification are the tasks that consist in finding the temporal anchors of events in a video and determine which event they are.
Ranked #5 on
Action Spotting
on SoccerNet
Multi-level Attention Fusion Network for Audio-visual Event Recognition
Event classification is inherently sequential and multimodal.
Deep soccer captioning with transformer: dataset, semantics-related losses, and multi-level evaluation
This work aims at generating captions for soccer videos using deep learning.
Analysis of Co-Laughter Gesture Relationship on RGB videos in Dyadic Conversation Contex
Laughter is not just an audio signal, but an intrinsic relationship of multimodal non-verbal communication, in addition to audio, it includes facial expressions and body movements.
Transformers and CNNs both Beat Humans on SBIR
Sketch-based image retrieval (SBIR) is the task of retrieving natural images (photos) that match the semantics and the spatial configuration of hand-drawn sketch queries.
A Recipe for Efficient SBIR Models: Combining Relative Triplet Loss with Batch Normalization and Knowledge Distillation
Then, we introduce a Relative Triplet Loss (RTL), an adapted triplet loss to overcome those limitations through loss weighting based on anchors similarity.
Deep learning in medical image registration: introduction and survey
Image registration (IR) is a process that deforms images to align them with respect to a reference space, making it easier for medical practitioners to examine various medical images in a standardized reference frame, such as having the same rotation and scale.
Analysis of Co-Laughter Gesture Relationship on RGB Videos in Dyadic Conversation Context
Laughter is not just an audio signal, but an intrinsic relationship of multimodal non-verbal communication, in addition to audio, it includes facial expressions and body movements.
Are There Any Body-movement Differences between Women and Men When They Laugh?
Women smile more than men although the expressiveness of women is not universally more across all facial actions.
