Search Results for author: Louis-Philippe Morency

Found 100 papers, 55 papers with code

Diverse and Admissible Trajectory Prediction through Multimodal Context Understanding

1 code implementation ECCV 2020 Seong Hyeon Park, Gyubok Lee, Jimin Seo, Manoj Bhat, Minseok Kang, Jonathan Francis, Ashwin Jadhav, Paul Pu Liang, Louis-Philippe Morency

Multi-agent trajectory forecasting in autonomous driving requires an agent to accurately anticipate the behaviors of the surrounding vehicles and pedestrians, for safe and reliable decision-making.

Autonomous Driving Decision Making +1

HOLM: Hallucinating Objects with Language Models for Referring Expression Recognition in Partially-Observed Scenes

no code implementations ACL 2022 Volkan Cirik, Louis-Philippe Morency, Taylor Berg-Kirkpatrick

AI systems embodied in the physical world face a fundamental challenge of partial observability; operating with only a limited view and knowledge of the environment.

Referring Expression

CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French

no code implementations EMNLP 2020 AmirAli Bagher Zadeh, Yansheng Cao, Simon Hessner, Paul Pu Liang, Soujanya Poria, Louis-Philippe Morency

It covers a diverse set topics and speakers, and carries supervision of 20 labels including sentiment (and subjectivity), emotions, and attributes.

DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations

1 code implementation3 Mar 2022 Yiwei Lyu, Paul Pu Liang, Zihao Deng, Ruslan Salakhutdinov, Louis-Philippe Morency

The ability for a human to understand an Artificial Intelligence (AI) model's decision-making process is critical in enabling stakeholders to visualize model behavior, perform model debugging, promote trust in AI models, and assist in collaborative human-AI decision-making.

Decision Making Disentanglement +1

HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning

1 code implementation2 Mar 2022 Paul Pu Liang, Yiwei Lyu, Xiang Fan, Shentong Mo, Dani Yogatama, Louis-Philippe Morency, Ruslan Salakhutdinov

Learning multimodal representations involves discovering correspondences and integrating information from multiple heterogeneous sources of data.

Representation Learning Time Series +1

Conditional Contrastive Learning with Kernel

1 code implementation ICLR 2022 Yao-Hung Hubert Tsai, Tianqin Li, Martin Q. Ma, Han Zhao, Kun Zhang, Louis-Philippe Morency, Ruslan Salakhutdinov

Conditional contrastive learning frameworks consider the conditional sampling procedure that constructs positive or negative data pairs conditioned on specific variables.

Contrastive Learning

Relay Variational Inference: A Method for Accelerated Encoderless VI

no code implementations26 Oct 2021 Amir Zadeh, Santiago Benoit, Louis-Philippe Morency

We find RVI to be a unique tool, often superior in both performance and convergence speed to previously proposed encoderless as well as amortized VI models (e. g. VAE).

Imputation Variational Inference

Learning Visual-Linguistic Adequacy, Fidelity, and Fluency for Novel Object Captioning

no code implementations29 Sep 2021 Cheng-Fu Yang, Yao-Hung Hubert Tsai, Wan-Cyuan Fan, Yu-Chiang Frank Wang, Louis-Philippe Morency, Ruslan Salakhutdinov

Novel object captioning (NOC) learns image captioning models for describing objects or visual concepts which are unseen (i. e., novel) in the training captions.

Image Captioning

M2H2: A Multimodal Multiparty Hindi Dataset For Humor Recognition in Conversations

1 code implementation3 Aug 2021 Dushyant Singh Chauhan, Gopendra Vikram Singh, Navonil Majumder, Amir Zadeh, Asif Ekbal, Pushpak Bhattacharyya, Louis-Philippe Morency, Soujanya Poria

We propose several strong multimodal baselines and show the importance of contextual and multimodal information for humor recognition in conversations.

Dialogue Understanding

Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

2 code implementations28 Jul 2021 Wei Han, Hui Chen, Alexander Gelbukh, Amir Zadeh, Louis-Philippe Morency, Soujanya Poria

Multimodal sentiment analysis aims to extract and integrate semantic information collected from multiple modalities to recognize the expressed emotions and sentiment in multimodal data.

Multimodal Deep Learning Multimodal Sentiment Analysis

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

2 code implementations15 Jul 2021 Paul Pu Liang, Yiwei Lyu, Xiang Fan, Zetian Wu, Yun Cheng, Jason Wu, Leslie Chen, Peter Wu, Michelle A. Lee, Yuke Zhu, Ruslan Salakhutdinov, Louis-Philippe Morency

In order to accelerate progress towards understudied modalities and tasks while ensuring real-world robustness, we release MultiBench, a systematic and unified large-scale benchmark spanning 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas.

Representation Learning

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

no code implementations ACL 2021 Paul Pu Liang, Terrance Liu, Anna Cai, Michal Muszynski, Ryo Ishii, Nicholas Allen, Randy Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency

Using computational models, we find that language and multimodal representations of mobile typed text (spanning typed characters, words, keystroke timings, and app usage) are predictive of daily mood.

Towards Understanding and Mitigating Social Biases in Language Models

1 code implementation24 Jun 2021 Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, Ruslan Salakhutdinov

As machine learning methods are deployed in real-world settings such as healthcare, legal systems, and social science, it is crucial to recognize how they shape social biases and stereotypes in these sensitive decision-making processes.

Decision Making Fairness +2

Crossmodal clustered contrastive learning: Grounding of spoken language to gesture

1 code implementation ACM ICMI Workshop GENEA 2021 Dong Won Lee, Chaitanya Ahuja, Louis-Philippe Morency

Crossmodal grounding is a key challenge for the task of generating relevant and well-timed gestures from just spoken language as an input.

Contrastive Learning

Integrating Auxiliary Information in Self-supervised Learning

no code implementations5 Jun 2021 Yao-Hung Hubert Tsai, Tianqin Li, Weixin Liu, Peiyuan Liao, Ruslan Salakhutdinov, Louis-Philippe Morency

Our approach contributes as follows: 1) Comparing to conventional self-supervised representations, the auxiliary-information-infused self-supervised representations bring the performance closer to the supervised representations; 2) The presented Cl-InfoNCE can also work with unsupervised constructed clusters (e. g., k-means clusters) and outperform strong clustering-based self-supervised learning approaches, such as the Prototypical Contrastive Learning (PCL) method; 3) We show that Cl-InfoNCE may be a better approach to leverage the data clustering information, by comparing it to the baseline approach - learning to predict the clustering assignments with cross-entropy loss.

Contrastive Learning Self-Supervised Learning

A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning

2 code implementations28 Apr 2021 Yao-Hung Hubert Tsai, Shaojie Bai, Louis-Philippe Morency, Ruslan Salakhutdinov

In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approach that is free of negative samples.

Contrastive Learning Self-Supervised Learning

StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer

2 code implementations NAACL 2021 Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, Louis-Philippe Morency

Many of the existing style transfer benchmarks primarily focus on individual high-level semantic changes (e. g. positive to negative), which enable controllability at a high level but do not offer fine-grained control involving sentence structure, emphasis, and content of the sentence.

Style Transfer Text Style Transfer

Self-supervised Representation Learning with Relative Predictive Coding

1 code implementation ICLR 2021 Yao-Hung Hubert Tsai, Martin Q. Ma, Muqiao Yang, Han Zhao, Louis-Philippe Morency, Ruslan Salakhutdinov

This paper introduces Relative Predictive Coding (RPC), a new contrastive representation learning objective that maintains a good balance among training stability, minibatch size sensitivity, and downstream task performance.

Representation Learning Self-Supervised Learning

StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations

no code implementations3 Jan 2021 Amir Zadeh, Santiago Benoit, Louis-Philippe Morency

In this paper we present an approach for training deep generative models solely based on solving determined systems of linear equations.

Importance-based Multimodal Autoencoder

no code implementations1 Jan 2021 Sayan Ghosh, Eugene Laksana, Louis-Philippe Morency, Stefan Scherer

In this paper we propose the IMA (Importance-based Multimodal Autoencoder) model, a scalable model that learns modality importances and robust multimodal representations through a novel cross-covariance based loss function.

Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

1 code implementation4 Dec 2020 Paul Pu Liang, Peter Wu, Liu Ziyin, Louis-Philippe Morency, Ruslan Salakhutdinov

In this work, we propose algorithms for cross-modal generalization: a learning paradigm to train a model that can (1) quickly perform new tasks in a target modality (i. e. meta-learning) and (2) doing so while being trained on a different source modality.

Meta-Learning

MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences

1 code implementation NAACL 2021 Jianing Yang, Yongxin Wang, Ruitao Yi, Yuying Zhu, Azaan Rehman, Amir Zadeh, Soujanya Poria, Louis-Philippe Morency

Human communication is multimodal in nature; it is through multiple modalities such as language, voice, and facial expressions, that opinions and emotions are expressed.

Emotion Recognition Multimodal Sentiment Analysis

Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach

1 code implementation ECCV 2020 Chaitanya Ahuja, Dong Won Lee, Yukiko I. Nakano, Louis-Philippe Morency

A key challenge, called gesture style transfer, is to learn a model that generates these gestures for a speaking agent 'A' in the gesturing style of a target speaker 'B'.

Gesture Generation Style Transfer

Towards Debiasing Sentence Representations

1 code implementation ACL 2020 Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, Louis-Philippe Morency

As natural language processing methods are increasingly deployed in real-world scenarios such as healthcare, legal systems, and social science, it becomes necessary to recognize the role they potentially play in shaping social biases and stereotypes.

Linguistic Acceptability Natural Language Understanding +2

What Gives the Answer Away? Question Answering Bias Analysis on Video QA Datasets

no code implementations7 Jul 2020 Jianing Yang, Yuying Zhu, Yongxin Wang, Ruitao Yi, Amir Zadeh, Louis-Philippe Morency

In this paper, we analyze QA biases in popular video question answering datasets and discover pretrained language models can answer 37-48% questions correctly without using any multimodal context information, far exceeding the 20% random guess baseline for 5-choose-1 multiple-choice questions.

Multiple-choice Pretrained Language Models +2

Language to Network: Conditional Parameter Adaptation with Natural Language Descriptions

no code implementations ACL 2020 Tian Jin, Zhun Liu, Shengjia Yan, Alex Eichenberger, re, Louis-Philippe Morency

In this paper, we propose \textbf{N3} (\textbf{N}eural \textbf{N}etworks from \textbf{N}atural Language) - a new paradigm of synthesizing task-specific neural networks from language descriptions and a generic pre-trained model.

General Classification Image Classification +3

Refer360$^\circ$: A Referring Expression Recognition Dataset in 360$^\circ$ Images

1 code implementation ACL 2020 Volkan Cirik, Taylor Berg-Kirkpatrick, Louis-Philippe Morency

We propose a novel large-scale referring expression recognition dataset, Refer360{\mbox{$^\circ$}}, consisting of 17, 137 instruction sequences and ground-truth actions for completing these instructions in 360{\mbox{$^\circ$}} scenes.

Referring Expression

Self-supervised Learning from a Multi-view Perspective

1 code implementation ICLR 2021 Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, Louis-Philippe Morency

In particular, we propose a composite objective that bridges the gap between prior contrastive and predictive learning objectives, and introduce an additional objective term to discard task-irrelevant information.

Image Captioning Language Modelling +3

Neural Methods for Point-wise Dependency Estimation

1 code implementation NeurIPS 2020 Yao-Hung Hubert Tsai, Han Zhao, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov

Since its inception, the neural estimation of mutual information (MI) has demonstrated the empirical success of modeling expected dependency between high-dimensional random variables.

Cross-Modal Retrieval Representation Learning

Improving Aspect-Level Sentiment Analysis with Aspect Extraction

no code implementations3 May 2020 Navonil Majumder, Rishabh Bhardwaj, Soujanya Poria, Amir Zadeh, Alexander Gelbukh, Amir Hussain, Louis-Philippe Morency

Aspect-based sentiment analysis (ABSA), a popular research area in NLP has two distinct parts -- aspect extraction (AE) and labeling the aspects with sentiment polarity (ALSA).

Aspect Extraction Word Embeddings

Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding

1 code implementation6 Mar 2020 Seong Hyeon Park, Gyubok Lee, Manoj Bhat, Jimin Seo, Minseok Kang, Jonathan Francis, Ashwin R. Jadhav, Paul Pu Liang, Louis-Philippe Morency

Multi-agent trajectory forecasting in autonomous driving requires an agent to accurately anticipate the behaviors of the surrounding vehicles and pedestrians, for safe and reliable decision-making.

Autonomous Driving Decision Making +1

On Emergent Communication in Competitive Multi-Agent Teams

1 code implementation4 Mar 2020 Paul Pu Liang, Jeffrey Chen, Ruslan Salakhutdinov, Louis-Philippe Morency, Satwik Kottur

Several recent works have found the emergence of grounded compositional language in the communication protocols developed by mostly cooperative multi-agent systems when learned end-to-end to maximize performance on a downstream task.

Learning Not to Learn in the Presence of Noisy Labels

no code implementations16 Feb 2020 Liu Ziyin, Blair Chen, Ru Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda

Learning in the presence of label noise is a challenging yet important task: it is crucial to design models that are robust in the presence of mislabeled datasets.

Text Classification

Think Locally, Act Globally: Federated Learning with Local and Global Representations

1 code implementation6 Jan 2020 Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency

To this end, we propose a new federated learning algorithm that jointly learns compact local representations on each device and a global model across all devices.

Federated Learning Representation Learning +1

Context-Dependent Models for Predicting and Characterizing Facial Expressiveness

no code implementations10 Dec 2019 Victoria Lin, Jeffrey M. Girard, Louis-Philippe Morency

In recent years, extensive research has emerged in affective computing on topics like automatic emotion recognition and determining the signals that characterize individual emotions.

Emotion Recognition

Factorized Multimodal Transformer for Multimodal Sequential Learning

1 code implementation22 Nov 2019 Amir Zadeh, Chengfeng Mao, Kelly Shi, Yiwei Zhang, Paul Pu Liang, Soujanya Poria, Louis-Philippe Morency

As machine learning leaps towards better generalization to real world, multimodal sequential learning becomes a fundamental research area.

Transformer Dissection: An Unified Understanding for Transformer's Attention via the Lens of Kernel

no code implementations IJCNLP 2019 Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov

This new formulation gives us a better way to understand individual components of the Transformer{'}s attention, such as the better way to integrate the positional embedding.

Machine Translation Translation

To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversations

3 code implementations5 Oct 2019 Chaitanya Ahuja, Shugao Ma, Louis-Philippe Morency, Yaser Sheikh

In this paper, we introduce a neural architecture named Dyadic Residual-Attention Model (DRAM), which integrates intrapersonal (monadic) and interpersonal (dyadic) dynamics using selective attention to generate sequences of body pose conditioned on audio and body pose of the interlocutor and audio of the human operating the avatar.

Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel

1 code implementation EMNLP 2019 Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov

This new formulation gives us a better way to understand individual components of the Transformer's attention, such as the better way to integrate the positional embedding.

Machine Translation Translation

Language2Pose: Natural Language Grounded Pose Forecasting

2 code implementations2 Jul 2019 Chaitanya Ahuja, Louis-Philippe Morency

In this paper, we address this multimodal problem by introducing a neural architecture called Joint Language to Pose (or JL2P), which learns a joint embedding of language and pose.

Motion Planning

Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization

no code implementations ACL 2019 Paul Pu Liang, Zhun Liu, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhutdinov, Louis-Philippe Morency

Our method is based on the observation that high-dimensional multimodal time series data often exhibit correlations across time and modalities which leads to low-rank tensor representations.

Question Answering Sentiment Analysis +2

Deep Gamblers: Learning to Abstain with Portfolio Theory

2 code implementations NeurIPS 2019 Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda

We deal with the \textit{selective classification} problem (supervised-learning problem with a rejection option), where we want to achieve the best performance at a certain level of coverage of the data.

Classification General Classification

Strong and Simple Baselines for Multimodal Utterance Embeddings

1 code implementation NAACL 2019 Paul Pu Liang, Yao Chong Lim, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Louis-Philippe Morency

Human language is a rich multimodal signal consisting of spoken words, facial expressions, body gestures, and vocal intonations.

Video Relationship Reasoning using Gated Spatio-Temporal Energy Graph

1 code implementation CVPR 2019 Yao-Hung Hubert Tsai, Santosh Divvala, Louis-Philippe Morency, Ruslan Salakhutdinov, Ali Farhadi

Visual relationship reasoning is a crucial yet challenging task for understanding rich interactions across visual concepts.

Variational Auto-Decoder: A Method for Neural Generative Modeling from Incomplete Data

1 code implementation3 Mar 2019 Amir Zadeh, Yao-Chong Lim, Paul Pu Liang, Louis-Philippe Morency

We study a specific implementation of the Auto-Encoding Variational Bayes (AEVB) algorithm, named in this paper as a Variational Auto-Decoder (VAD).

Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities

2 code implementations19 Dec 2018 Hai Pham, Paul Pu Liang, Thomas Manzini, Louis-Philippe Morency, Barnabas Poczos

Our method is based on the key insight that translation from a source to a target modality provides a method of learning joint representations using only the source modality as input.

Machine Translation Multimodal Sentiment Analysis +1

Multimodal Language Analysis with Recurrent Multistage Fusion

1 code implementation EMNLP 2018 Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency

In this paper, we propose the Recurrent Multistage Fusion Network (RMFN) which decomposes the fusion problem into multiple stages, each of them focused on a subset of multimodal signals for specialized, effective fusion.

Emotion Recognition Multimodal Sentiment Analysis

Learning Factorized Multimodal Representations

2 code implementations ICLR 2019 Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov

Multimodal discriminative factors are shared across all modalities and contain joint multimodal features required for discriminative tasks such as sentiment prediction.

Representation Learning

Speaker-Follower Models for Vision-and-Language Navigation

1 code implementation NeurIPS 2018 Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, Trevor Darrell

We use this speaker model to (1) synthesize new instructions for data augmentation and to (2) implement pragmatic reasoning, which evaluates how well candidate action sequences explain an instruction.

Data Augmentation Vision and Language Navigation

Visual Referring Expression Recognition: What Do Systems Actually Learn?

1 code implementation NAACL 2018 Volkan Cirik, Louis-Philippe Morency, Taylor Berg-Kirkpatrick

We present an empirical analysis of the state-of-the-art systems for referring expression recognition -- the task of identifying the object in an image referred to by a natural language expression -- with the goal of gaining insight into how these systems reason about language and vision.

Referring Expression

Using Syntax to Ground Referring Expressions in Natural Images

1 code implementation26 May 2018 Volkan Cirik, Taylor Berg-Kirkpatrick, Louis-Philippe Morency

We introduce GroundNet, a neural network for referring expression recognition -- the task of localizing (or grounding) in an image the object referred to by a natural language expression.

Referring Expression

Memory Fusion Network for Multi-view Sequential Learning

2 code implementations3 Feb 2018 Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, Louis-Philippe Morency

In this paper, we present a new neural architecture for multi-view sequential learning called the Memory Fusion Network (MFN) that explicitly accounts for both interactions in a neural architecture and continuously models them through time.

Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling

1 code implementation6 Oct 2017 Chaitanya Ahuja, Louis-Philippe Morency

We evaluate this family on new LRU models on computational convergence rates and statistical efficiency.

Hand2Face: Automatic Synthesis and Recognition of Hand Over Face Occlusions

no code implementations1 Aug 2017 Behnaz Nojavanasghari, Charles. E. Hughes, Tadas Baltrusaitis, Louis-Philippe Morency

We then propose a model for facial occlusion type recognition to differentiate between hand over face occlusions and other types of occlusions such as scarves, hair, glasses and objects.

Tensor Fusion Network for Multimodal Sentiment Analysis

no code implementations EMNLP 2017 Amir Zadeh, Minghai Chen, Soujanya Poria, Erik Cambria, Louis-Philippe Morency

Multimodal sentiment analysis is an increasingly popular research area, which extends the conventional language-based definition of sentiment analysis to a multimodal setup where other relevant modalities accompany language.

Multimodal Sentiment Analysis

Multimodal Machine Learning: Integrating Language, Vision and Speech

no code implementations ACL 2017 Louis-Philippe Morency, Tadas Baltru{\v{s}}aitis

Multimodal machine learning is a vibrant multi-disciplinary research field which addresses some of the original goals of artificial intelligence by integrating and modeling multiple communicative modalities, including linguistic, acoustic and visual messages.

Audio-Visual Speech Recognition Image Captioning +7

Preserving Intermediate Objectives: One Simple Trick to Improve Learning for Hierarchical Models

no code implementations23 Jun 2017 Abhilasha Ravichander, Shruti Rijhwani, Rajat Kulshreshtha, Chirag Nagpal, Tadas Baltrušaitis, Louis-Philippe Morency

In this work, we focus on improving learning for such hierarchical models and demonstrate our method on the task of speaker trait prediction.

Multimodal Machine Learning: A Survey and Taxonomy

no code implementations26 May 2017 Tadas Baltrušaitis, Chaitanya Ahuja, Louis-Philippe Morency

Our experience of the world is multimodal - we see objects, hear sounds, feel texture, smell odors, and taste flavors.

Translation

Combating Human Trafficking with Deep Multimodal Models

no code implementations8 May 2017 Edmund Tong, Amir Zadeh, Cara Jones, Louis-Philippe Morency

Human trafficking is a global epidemic affecting millions of people across the planet.

Temporal Attention-Gated Model for Robust Sequence Classification

1 code implementation CVPR 2017 Wenjie Pei, Tadas Baltrušaitis, David M. J. Tax, Louis-Philippe Morency

An important advantage of our approach is interpretability since the temporal attention weights provide a meaningful value for the salience of each time step in the sequence.

Classification General Classification +1

Convolutional Experts Constrained Local Model for Facial Landmark Detection

2 code implementations26 Nov 2016 Amir Zadeh, Tadas Baltrušaitis, Louis-Philippe Morency

In our work, we present a novel local detector -- Convolutional Experts Network (CEN) -- that brings together the advantages of neural architectures and mixtures of experts in an end-to-end framework.

Facial Landmark Detection

Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis

1 code implementation16 Sep 2016 Haohan Wang, Aaksha Meghawat, Louis-Philippe Morency, Eric P. Xing

In this paper, we propose a Select-Additive Learning (SAL) procedure that improves the generalizability of trained neural networks for multimodal sentiment analysis.

Multimodal Sentiment Analysis

MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

4 code implementations20 Jun 2016 Amir Zadeh, Rowan Zellers, Eli Pincus, Louis-Philippe Morency

This paper introduces to the scientific community the first opinion-level annotated corpus of sentiment and subjectivity analysis in online videos called Multimodal Opinion-level Sentiment Intensity dataset (MOSI).

Frame Sentiment Analysis +1

A Multimodal Corpus for the Assessment of Public Speaking Ability and Anxiety

no code implementations LREC 2016 Mathieu Chollet, Torsten W{\"o}rtwein, Louis-Philippe Morency, Stefan Scherer

As such, tools enabling the improvement of public speaking performance and the assessment and mitigation of anxiety related to public speaking would be very useful.

Learning Representations of Affect from Speech

no code implementations15 Nov 2015 Sayan Ghosh, Eugene Laksana, Louis-Philippe Morency, Stefan Scherer

Experiments on a well-established real-life speech dataset (IEMOCAP) show that the learnt representations are comparable to state of the art feature extractors (such as voice quality features and MFCCs) and are competitive with state-of-the-art approaches at emotion and dimensional affect recognition.

Denoising Emotion Classification +2

The Distress Analysis Interview Corpus of human and computer interviews

no code implementations LREC 2014 Jonathan Gratch, Ron artstein, Gale Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, David Traum, Skip Rizzo, Louis-Philippe Morency

The Distress Analysis Interview Corpus (DAIC) contains clinical interviews designed to support the diagnosis of psychological distress conditions such as anxiety, depression, and post traumatic stress disorder.

Relative Facial Action Unit Detection

no code implementations1 May 2014 Mahmoud Khademi, Louis-Philippe Morency

This paper presents a subject-independent facial action unit (AU) detection method by introducing the concept of relative AU detection, for scenarios where the neutral face is not provided.

Action Unit Detection Facial Action Unit Detection +2

Action Recognition by Hierarchical Sequence Summarization

no code implementations CVPR 2013 Yale Song, Louis-Philippe Morency, Randall Davis

We develop an efficient learning method to train our model and show that its complexity grows sublinearly with the size of the hierarchy.

Action Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.