Search Results for author: Mengye Ren

Found 40 papers, 15 papers with code

BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling

no code implementations28 Nov 2023 YiXuan Luo, Mengye Ren, Sai Qian Zhang

This approach significantly reduces computational costs in comparison with training each DNN backbone individually.

Contrastive Learning Language Modelling +2

Rethinking Closed-loop Training for Autonomous Driving

no code implementations27 Jun 2023 Chris Zhang, Runsheng Guo, Wenyuan Zeng, Yuwen Xiong, Binbin Dai, Rui Hu, Mengye Ren, Raquel Urtasun

Recent advances in high-fidelity simulators have enabled closed-loop training of autonomous driving agents, potentially solving the distribution shift in training v. s.

Autonomous Driving

Gaussian-Bernoulli RBMs Without Tears

1 code implementation19 Oct 2022 Renjie Liao, Simon Kornblith, Mengye Ren, David J. Fleet, Geoffrey Hinton

We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations.

Scaling Forward Gradient With Local Losses

1 code implementation7 Oct 2022 Mengye Ren, Simon Kornblith, Renjie Liao, Geoffrey Hinton

Forward gradient learning computes a noisy directional gradient and is a biologically plausible alternative to backprop for learning deep neural networks.

Learning to Reason With Relational Abstractions

no code implementations6 Oct 2022 Andrew J. Nam, Mengye Ren, Chelsea Finn, James L. McClelland

Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.

Mathematical Reasoning

Online Unsupervised Learning of Visual Representations and Categories

1 code implementation13 Sep 2021 Mengye Ren, Tyler R. Scott, Michael L. Iuzzolino, Michael C. Mozer, Richard Zemel

Real world learning scenarios involve a nonstationary distribution of classes with sequential dependencies among the samples, in contrast to the standard machine learning formulation of drawing samples independently from a fixed, typically uniform distribution.

Few-Shot Learning Representation Learning +1

Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes

no code implementations8 Apr 2021 Sean Segal, Nishanth Kumar, Sergio Casas, Wenyuan Zeng, Mengye Ren, Jingkang Wang, Raquel Urtasun

As data collection is often significantly cheaper than labeling in this domain, the decision of which subset of examples to label can have a profound impact on model performance.

Active Learning

Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving

no code implementations17 Jan 2021 James Tu, Huichen Li, Xinchen Yan, Mengye Ren, Yun Chen, Ming Liang, Eilyan Bitar, Ersin Yumer, Raquel Urtasun

Yet, there have been limited studies on the adversarial robustness of multi-modal models that fuse LiDAR features with image features.

Adversarial Robustness Denoising +1

Cost-Efficient Online Hyperparameter Optimization

no code implementations17 Jan 2021 Jingkang Wang, Mengye Ren, Ilija Bogunovic, Yuwen Xiong, Raquel Urtasun

Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters.

Bayesian Optimization Hyperparameter Optimization

Adversarial Attacks On Multi-Agent Communication

no code implementations ICCV 2021 James Tu, TsunHsuan Wang, Jingkang Wang, Sivabalan Manivasagam, Mengye Ren, Raquel Urtasun

Growing at a fast pace, modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems.

Domain Adaptation

Self-Supervised Representation Learning from Flow Equivariance

no code implementations ICCV 2021 Yuwen Xiong, Mengye Ren, Wenyuan Zeng, Raquel Urtasun

Motivated by this ability, we present a new self-supervised learning representation framework that can be directly deployed on a video stream of complex scenes with many moving objects.

Instance Segmentation object-detection +5

SceneGen: Learning to Generate Realistic Traffic Scenes

no code implementations CVPR 2021 Shuhan Tan, Kelvin Wong, Shenlong Wang, Sivabalan Manivasagam, Mengye Ren, Raquel Urtasun

Existing methods typically insert actors into the scene according to a set of hand-crafted heuristics and are limited in their ability to model the true complexity and diversity of real traffic scenes, thus inducing a content gap between synthesized traffic scenes versus real ones.

AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles

no code implementations CVPR 2021 Jingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam, Abbas Sadat, Sergio Casas, Mengye Ren, Raquel Urtasun

Importantly, by simulating directly from sensor data, we obtain adversarial scenarios that are safety-critical for the full autonomy stack.

Exploring representation learning for flexible few-shot tasks

no code implementations1 Jan 2021 Mengye Ren, Eleni Triantafillou, Kuan-Chieh Wang, James Lucas, Jake Snell, Xaq Pitkow, Andreas S. Tolias, Richard Zemel

In this work, we consider a realistic setting where the relationship between examples can change from episode to episode depending on the task context, which is not given to the learner.

Few-Shot Learning Representation Learning

Probing Few-Shot Generalization with Attributes

no code implementations10 Dec 2020 Mengye Ren, Eleni Triantafillou, Kuan-Chieh Wang, James Lucas, Jake Snell, Xaq Pitkow, Andreas S. Tolias, Richard Zemel

Despite impressive progress in deep learning, generalizing far beyond the training distribution is an important open challenge.

Few-Shot Learning Zero-Shot Learning

Learning to Communicate and Correct Pose Errors

no code implementations10 Nov 2020 Nicholas Vadivelu, Mengye Ren, James Tu, Jingkang Wang, Raquel Urtasun

Learned communication makes multi-agent systems more effective by aggregating distributed information.

Motion Forecasting object-detection +1

Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Driving

no code implementations2 Nov 2020 Bob Wei, Mengye Ren, Wenyuan Zeng, Ming Liang, Bin Yang, Raquel Urtasun

In this paper, we propose an end-to-end self-driving network featuring a sparse attention module that learns to automatically attend to important regions of the input.

Motion Planning

Theoretical bounds on estimation error for meta-learning

no code implementations ICLR 2021 James Lucas, Mengye Ren, Irene Kameni, Toniann Pitassi, Richard Zemel

Machine learning models have traditionally been developed under the assumption that the training and test distributions match exactly.

Few-Shot Learning Test

Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations

no code implementations ECCV 2020 Abbas Sadat, Sergio Casas, Mengye Ren, Xinyu Wu, Pranaab Dhawan, Raquel Urtasun

In this paper we propose a novel end-to-end learnable network that performs joint perception, prediction and motion planning for self-driving vehicles and produces interpretable intermediate representations.

Motion Planning

LoCo: Local Contrastive Representation Learning

no code implementations NeurIPS 2020 Yuwen Xiong, Mengye Ren, Raquel Urtasun

Deep neural nets typically perform end-to-end backpropagation to learn the weights, a procedure that creates synchronization constraints in the weight update step across layers and is not biologically plausible.

Contrastive Learning Instance Segmentation +4

Wandering Within a World: Online Contextualized Few-Shot Learning

1 code implementation ICLR 2021 Mengye Ren, Michael L. Iuzzolino, Michael C. Mozer, Richard S. Zemel

We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting.

Few-Shot Learning

Physically Realizable Adversarial Examples for LiDAR Object Detection

no code implementations CVPR 2020 James Tu, Mengye Ren, Siva Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, Raquel Urtasun

Modern autonomous driving systems rely heavily on deep learning models to process point cloud sensory data; meanwhile, deep models have been shown to be susceptible to adversarial attacks with visually imperceptible perturbations.

Adversarial Defense Autonomous Driving +3

Identifying Unknown Instances for Autonomous Driving

no code implementations24 Oct 2019 Kelvin Wong, Shenlong Wang, Mengye Ren, Ming Liang, Raquel Urtasun

In the past few years, we have seen great progress in perception algorithms, particular through the use of deep learning.

Autonomous Driving Instance Segmentation +1

Learning to Remember from a Multi-Task Teacher

no code implementations10 Oct 2019 Yuwen Xiong, Mengye Ren, Raquel Urtasun

Recent studies on catastrophic forgetting during sequential learning typically focus on fixing the accuracy of the predictions for a previously learned task.


Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles

no code implementations10 Oct 2019 Abbas Sadat, Mengye Ren, Andrei Pokrovsky, Yen-Chen Lin, Ersin Yumer, Raquel Urtasun

The motion planners used in self-driving vehicles need to generate trajectories that are safe, comfortable, and obey the traffic rules.

Trajectory Planning

Incremental Few-Shot Learning with Attention Attractor Networks

1 code implementation NeurIPS 2019 Mengye Ren, Renjie Liao, Ethan Fetaya, Richard S. Zemel

This paper addresses this problem, incremental few-shot learning, where a regular classification network has already been trained to recognize a set of base classes, and several extra novel classes are being considered, each with only a few labeled examples.

Few-Shot Learning General Classification

Graph HyperNetworks for Neural Architecture Search

1 code implementation ICLR 2019 Chris Zhang, Mengye Ren, Raquel Urtasun

Neural architecture search (NAS) automatically finds the best task-specific neural network topology, outperforming many manual architecture designs.

Neural Architecture Search

Learning to Reweight Examples for Robust Deep Learning

9 code implementations ICML 2018 Mengye Ren, Wenyuan Zeng, Bin Yang, Raquel Urtasun

Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns.


Understanding Short-Horizon Bias in Stochastic Meta-Optimization

1 code implementation ICLR 2018 Yuhuai Wu, Mengye Ren, Renjie Liao, Roger Grosse

Careful tuning of the learning rate, or even schedules thereof, can be crucial to effective neural net training.

Meta-Learning for Semi-Supervised Few-Shot Classification

8 code implementations ICLR 2018 Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, Richard S. Zemel

To address this paradigm, we propose novel extensions of Prototypical Networks (Snell et al., 2017) that are augmented with the ability to use unlabeled examples when producing prototypes.

General Classification Meta-Learning +1

SBNet: Sparse Blocks Network for Fast Inference

2 code implementations CVPR 2018 Mengye Ren, Andrei Pokrovsky, Bin Yang, Raquel Urtasun

Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications.

3D Object Detection object-detection +1

The Reversible Residual Network: Backpropagation Without Storing Activations

9 code implementations NeurIPS 2017 Aidan N. Gomez, Mengye Ren, Raquel Urtasun, Roger B. Grosse

Deep residual networks (ResNets) have significantly pushed forward the state-of-the-art on image classification, increasing in performance as networks grow both deeper and wider.

General Classification Image Classification

End-to-End Instance Segmentation with Recurrent Attention

1 code implementation CVPR 2017 Mengye Ren, Richard S. Zemel

While convolutional neural networks have gained impressive success recently in solving structured prediction problems such as semantic segmentation, it remains a challenge to differentiate individual object instances in the scene.

Autonomous Driving Image Captioning +6

Cannot find the paper you are looking for? You can Submit a new open access paper.