PatchBlender: A Motion Prior for Video Transformers

no code implementations • 11 Nov 2022 • Gabriele Prato, Yale Song, Janarthanan Rajendran, R Devon Hjelm, Neel Joshi, Sarath Chandar

We show that our method is successful at enabling vision transformers to encode the temporal component of video data.

Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes

1 code implementation • 7 Sep 2022 • Alex Fedorov, Eloy Geenjaar, Lei Wu, Tristan Sylvain, Thomas P. DeRamus, Margaux Luck, Maria Misiura, R Devon Hjelm, Sergey M. Plis, Vince D. Calhoun

Coarse labels do not capture the long-tailed spectrum of brain disorder phenotypes, which leads to a loss of generalizability of the model that makes them less useful in diagnostic settings.

Self-Supervised Learning

The Sandbox Environment for Generalizable Agent Research (SEGAR)

1 code implementation • 19 Mar 2022 • R Devon Hjelm, Bogdan Mazoure, Florian Golemo, Felipe Frujeri, Mihai Jalobeanu, Andrey Kolobov

A broad challenge of research on generalization for sequential decision-making tasks in interactive environments is designing benchmarks that clearly landmark progress.

Decision Making

Robust Contrastive Learning against Noisy Views

1 code implementation • CVPR 2022 • Ching-Yao Chuang, R Devon Hjelm, Xin Wang, Vibhav Vineet, Neel Joshi, Antonio Torralba, Stefanie Jegelka, Yale Song

Contrastive learning relies on an assumption that positive pairs contain related views, e. g., patches of an image or co-occurring multimodal signals of a video, that share certain underlying information about an instance.

Binary Classification Contrastive Learning

Cross-Trajectory Representation Learning for Zero-Shot Generalization in RL

1 code implementation • ICLR 2022 • Bogdan Mazoure, Ahmed M. Ahmed, Patrick MacAlpine, R Devon Hjelm, Andrey Kolobov

A highly desirable property of a reinforcement learning (RL) agent -- and a major difficulty for deep RL approaches -- is the ability to generalize policies learned on a few tasks over a high-dimensional observation space to similar tasks not seen during training.

Reinforcement Learning (RL) Representation Learning +1

Understanding by Understanding Not: Modeling Negation in Language Models

1 code implementation • NAACL 2021 • Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, Aaron Courville

To improve language models in this regard, we propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus.

Language Modelling Negation

Pretraining Reward-Free Representations for Data-Efficient Reinforcement Learning

no code implementations • ICLR Workshop SSL-RL 2021 • Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, R Devon Hjelm, Philip Bachman, Aaron Courville

Data efficiency poses a major challenge for deep reinforcement learning.

reinforcement-learning Reinforcement Learning (RL) +1

Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations

no code implementations • 22 Oct 2020 • Tristan Sylvain, Linda Petrini, R Devon Hjelm

Zero-shot classification is a generalization task where no instance from the target classes is seen during training.

General Classification Representation Learning +1

Adversarial Mixup Resynthesizers

1 code implementation • ICLR Workshop DeepGenStruct 2019 • Christopher Beckham, Sina Honari, Alex Lamb, Vikas Verma, Farnoosh Ghadiri, R Devon Hjelm, Christopher Pal

In this paper, we explore new approaches to combining information encoded within the learned representations of autoencoders.