Search Results for author: Bolei Zhou

Found 123 papers, 64 papers with code

A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation

no code implementations11 Mar 2024 Pan He, Quanyi Li, Xiaoyong Yuan, Bolei Zhou

Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions.


FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition

no code implementations12 Dec 2023 Sicheng Mo, Fangzhou Mu, Kuan Heng Lin, Yanli Liu, Bochen Guan, Yin Li, Bolei Zhou

Recent approaches such as ControlNet offer users fine-grained spatial control over text-to-image (T2I) diffusion models.

BerfScene: Bev-conditioned Equivariant Radiance Fields for Infinite 3D Scene Generation

no code implementations4 Dec 2023 Qihang Zhang, Yinghao Xu, Yujun Shen, Bo Dai, Bolei Zhou, Ceyuan Yang

Generating large-scale 3D scenes cannot simply apply existing 3D object synthesis technique since 3D scenes usually hold complex spatial configurations and consist of a number of objects at varying scales.

Scene Generation

In-Domain GAN Inversion for Faithful Reconstruction and Editability

no code implementations25 Sep 2023 Jiapeng Zhu, Yujun Shen, Yinghao Xu, Deli Zhao, Qifeng Chen, Bolei Zhou

This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model.

Image Generation Image Reconstruction

Efficient 3D Articulated Human Generation with Layered Surface Volumes

no code implementations11 Jul 2023 Yinghao Xu, Wang Yifan, Alexander W. Bergman, Menglei Chai, Bolei Zhou, Gordon Wetzstein

These layers are rendered using alpha compositing with fast differentiable rasterization, and they can be interpreted as a volumetric representation that allocates its capacity to a manifold of finite thickness around the template.

Guarded Policy Optimization with Imperfect Online Demonstrations

no code implementations3 Mar 2023 Zhenghai Xue, Zhenghao Peng, Quanyi Li, Zhihan Liu, Bolei Zhou

Assuming optimal, the teacher policy has the perfect timing and capability to intervene in the learning process of the student agent, providing safety guarantee and exploration guidance.

Continuous Control Efficient Exploration +2

Spatial Steerability of GANs via Self-Supervision from Discriminator

no code implementations20 Jan 2023 Jianyuan Wang, Lalit Bhagat, Ceyuan Yang, Yinghao Xu, Yujun Shen, Hongdong Li, Bolei Zhou

In this work, we propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space or requiring extra annotations.

Image Generation Inductive Bias +1

GH-Feat: Learning Versatile Generative Hierarchical Features from GANs

no code implementations12 Jan 2023 Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, Bolei Zhou

In this work we investigate that such a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones.

Face Verification Image Harmonization +3

Street-View Image Generation from a Bird's-Eye View Layout

1 code implementation11 Jan 2023 Alexander Swerdlow, Runsheng Xu, Bolei Zhou

Instead of using perception data from real-life scenarios, an ideal model for simulation would generate realistic street-view images that align with a given HD map and traffic layout, a task that is critical for visualizing complex traffic scenarios and developing robust perception models for autonomous driving.

Autonomous Driving Image Generation

Towards Smooth Video Composition

1 code implementation14 Dec 2022 Qihang Zhang, Ceyuan Yang, Yujun Shen, Yinghao Xu, Bolei Zhou

Video generation requires synthesizing consistent and persistent frames with dynamic content over time.

Image Generation single-image-generation +2

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

1 code implementation27 Sep 2022 Hao Xiang, Runsheng Xu, Xin Xia, Zhaoliang Zheng, Bolei Zhou, Jiaqi Ma

Recent advancements in Vehicle-to-Everything communication technology have enabled autonomous vehicles to share sensory information to obtain better perception performance.

Autonomous Vehicles

Improving GANs with A Dynamic Discriminator

no code implementations20 Sep 2022 Ceyuan Yang, Yujun Shen, Yinghao Xu, Deli Zhao, Bo Dai, Bolei Zhou

Two capacity adjusting schemes are developed for training GANs under different data regimes: i) given a sufficient amount of training data, the discriminator benefits from a progressively increased learning capacity, and ii) when the training data is limited, gradually decreasing the layer width mitigates the over-fitting issue of the discriminator.

3D-Aware Image Synthesis Data Augmentation

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping

1 code implementation15 Sep 2022 Hao Sun, Lei Han, Rui Yang, Xiaoteng Ma, Jian Guo, Bolei Zhou

We validate our insight on a range of RL tasks and show its improvement over baselines: (1) In offline RL, the conservative exploitation leads to improved performance based on off-the-shelf algorithms; (2) In online continuous control, multiple value functions with different shifting constants can be used to tackle the exploration-exploitation dilemma for better sample efficiency; (3) In discrete control tasks, a negative reward shifting yields an improvement over the curiosity-based exploration method.

Continuous Control Offline RL

CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers

2 code implementations5 Jul 2022 Runsheng Xu, Zhengzhong Tu, Hao Xiang, Wei Shao, Bolei Zhou, Jiaqi Ma

The extensive experiments on the V2V perception dataset, OPV2V, demonstrate that CoBEVT achieves state-of-the-art performance for cooperative BEV semantic segmentation.

3D Object Detection Autonomous Driving +2

Human-AI Shared Control via Policy Dissection

1 code implementation31 May 2022 Quanyi Li, Zhenghao Peng, Haibin Wu, Lan Feng, Bolei Zhou

Inspired by the neuroscience approach to investigate the motor cortex in primates, we develop a simple yet effective frequency-based approach called \textit{Policy Dissection} to align the intermediate representation of the learned neural controller with the kinematic attributes of the agent behavior.

Autonomous Driving Reinforcement Learning (RL)

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

1 code implementation5 Apr 2022 Qihang Zhang, Zhenghao Peng, Bolei Zhou

Specifically, we train an inverse dynamic model with a small amount of labeled data and use it to predict action labels for all the YouTube video frames.

Autonomous Driving Imitation Learning

LocATe: End-to-end Localization of Actions in 3D with Transformers

no code implementations21 Mar 2022 Jiankai Sun, Bolei Zhou, Michael J. Black, Arjun Chandrasekaran

An important component of this problem is 3D Temporal Action Localization (3D-TAL), which involves recognizing what actions a person is performing, and when.

Action Recognition object-detection +2

Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization

no code implementations ICLR 2022 Quanyi Li, Zhenghao Peng, Bolei Zhou

HACO can train agents to drive in unseen traffic scenarios with a handful of human intervention budget and achieve high safety and generalizability, outperforming both reinforcement learning and imitation learning baselines with a large margin.

Imitation Learning reinforcement-learning +1

Visual Sound Localization in the Wild by Cross-Modal Interference Erasing

1 code implementation13 Feb 2022 Xian Liu, Rui Qian, Hang Zhou, Di Hu, Weiyao Lin, Ziwei Liu, Bolei Zhou, Xiaowei Zhou

Specifically, we observe that the previous practice of learning only a single audio representation is insufficient due to the additive nature of audio signals.

Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation

no code implementations19 Jan 2022 Xian Liu, Yinghao Xu, Qianyi Wu, Hang Zhou, Wayne Wu, Bolei Zhou

Moreover, to enable portrait rendering in one unified neural radiance field, a Torso Deformation module is designed to stabilize the large-scale non-rigid torso motions.

3D-aware Image Synthesis via Learning Structural and Textural Representations

1 code implementation CVPR 2022 Yinghao Xu, Sida Peng, Ceyuan Yang, Yujun Shen, Bolei Zhou

The feature field is further accumulated into a 2D feature map as the textural representation, followed by a neural renderer for appearance synthesis.

3D-Aware Image Synthesis Generative Adversarial Network

Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition

no code implementations CVPR 2022 Yinghao Xu, Fangyun Wei, Xiao Sun, Ceyuan Yang, Yujun Shen, Bo Dai, Bolei Zhou, Stephen Lin

Typically in recent work, the pseudo-labels are obtained by training a model on the labeled data, and then using confident predictions from the model to teach itself.

Action Recognition

SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

1 code implementation9 Dec 2021 Zhenyu Li, Zehui Chen, Ang Li, Liangji Fang, Qinhong Jiang, Xianming Liu, Junjun Jiang, Bolei Zhou, Hang Zhao

To bridge this gap, we aim to learn a spatial-aware visual representation that can describe the three-dimensional space and is more suitable and effective for these tasks.

Contrastive Learning Unsupervised Pre-training

Improving GAN Equilibrium by Raising Spatial Awareness

1 code implementation CVPR 2022 Jianyuan Wang, Ceyuan Yang, Yinghao Xu, Yujun Shen, Hongdong Li, Bolei Zhou

We further propose to align the spatial awareness of G with the attention map induced from D. Through this way we effectively lessen the information gap between D and G. Extensive results show that our method pushes the two-player game in GANs closer to the equilibrium, leading to a better synthesis performance.

Attribute Inductive Bias

One-Shot Generative Domain Adaptation

no code implementations ICCV 2023 Ceyuan Yang, Yujun Shen, Zhiyi Zhang, Yinghao Xu, Jiapeng Zhu, Zhirong Wu, Bolei Zhou

We then equip the well-learned discriminator backbone with an attribute classifier to ensure that the generator captures the appropriate characters from the reference.

Attribute Domain Adaptation +1

Safe Driving via Expert Guided Policy Optimization

1 code implementation13 Oct 2021 Zhenghao Peng, Quanyi Li, Chunxiao Liu, Bolei Zhou

Offline RL technique is further used to learn from the partial demonstration generated by the expert.

Offline RL reinforcement-learning +1

Improving Out-of-Distribution Robustness of Classifiers Through Interpolated Generative Models

no code implementations29 Sep 2021 Haoyue Bai, Ceyuan Yang, Yinghao Xu, S.-H. Gary Chan, Bolei Zhou

In this paper, we employ interpolated generative models to generate OoD samples at training time via data augmentation.

Data Augmentation

Interpreting Molecule Generative Models for Interactive Molecule Discovery

no code implementations29 Sep 2021 Yuanqi Du, Xian Liu, Shengchao Liu, Bolei Zhou

In this work, we develop a simple yet effective method to interpret the latent space of the learned generative models with various molecular properties for more interactive molecule generation and discovery.

Drug Discovery

SPLID: Self-Imitation Policy Learning through Iterative Distillation

no code implementations29 Sep 2021 Zhihan Liu, Hao Sun, Bolei Zhou

To this end, we propose a novel meta-algorithm Self-Imitation Policy Learning through Iterative Distillation (SPLID) which relies on the concept of $\delta$-distilled policy to iteratively level up the quality of the target data and agent mimics from the relabeled target data.

Continuous Control

Reward Shifting for Optimistic Exploration and Conservative Exploitation

no code implementations29 Sep 2021 Hao Sun, Lei Han, Jian Guo, Bolei Zhou

We verify our insight on a range of tasks: (1) In offline RL, the conservative exploitation leads to improved learning performance based on off-the-shelf algorithms; (2) In online continuous control, multiple value functions with different shifting constants can be used to trade-off between exploration and exploitation thus improving learning efficiency; (3) In online RL with discrete action space, a negative reward shifting brings an improvement over the previous curiosity-based exploration method.

Continuous Control Offline RL

MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning

2 code implementations26 Sep 2021 Quanyi Li, Zhenghao Peng, Lan Feng, Qihang Zhang, Zhenghai Xue, Bolei Zhou

Based on MetaDrive, we construct a variety of RL tasks and baselines in both single-agent and multi-agent settings, including benchmarking generalizability across unseen scenes, safe exploration, and learning multi-agent traffic.

Benchmarking Decision Making +5

Interpreting Generative Adversarial Networks for Interactive Image Generation

no code implementations10 Aug 2021 Bolei Zhou

Significant progress has been made by the advances in Generative Adversarial Networks (GANs) for image generation.

Image Generation

Data-Efficient Instance Generation from Instance Discrimination

1 code implementation NeurIPS 2021 Ceyuan Yang, Yujun Shen, Yinghao Xu, Bolei Zhou

Meanwhile, the learned instance discrimination capability from the discriminator is in turn exploited to encourage the generator for diverse generation.

2k Data Augmentation +1

Multimodal Motion Prediction with Stacked Transformers

1 code implementation CVPR 2021 Yicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang, Bolei Zhou

Predicting multiple plausible future trajectories of the nearby vehicles is crucial for the safety of autonomous driving.

Autonomous Driving motion prediction

Unsupervised Image Transformation Learning via Generative Adversarial Networks

no code implementations13 Mar 2021 Kaiwen Zha, Yujun Shen, Bolei Zhou

In this work, we study the image transformation problem, which targets at learning the underlying transformations (e. g., the transition of seasons) from a collection of unlabeled images.

Image Generation valid

Instance Localization for Self-supervised Detection Pretraining

1 code implementation CVPR 2021 Ceyuan Yang, Zhirong Wu, Bolei Zhou, Stephen Lin

The pretext task is to predict the instance category given the composited images as well as the foreground bounding boxes.

Classification General Classification +6

Deep Learning for Scene Classification: A Survey

no code implementations26 Jan 2021 Delu Zeng, Minyu Liao, Mohammad Tavakolian, Yulan Guo, Bolei Zhou, Dewen Hu, Matti Pietikäinen, Li Liu

Scene classification, aiming at classifying a scene image to one of the predefined scene categories by comprehending the entire image, is a longstanding, fundamental and challenging problem in computer vision.

Classification General Classification +1

GAN Inversion: A Survey

1 code implementation14 Jan 2021 Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang

GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model, for the image to be faithfully reconstructed from the inverted code by the generator.

Image Manipulation Image Restoration

Self-Supervised Continuous Control without Policy Gradient

no code implementations1 Jan 2021 Hao Sun, Ziping Xu, Meng Fang, Yuhang Song, Jiechao Xiong, Bo Dai, Zhengyou Zhang, Bolei Zhou

Despite the remarkable progress made by the policy gradient algorithms in reinforcement learning (RL), sub-optimal policies usually result from the local exploration property of the policy gradient update.

Continuous Control Policy Gradient Methods +3

Improving the Generalization of End-to-End Driving through Procedural Generation

2 code implementations26 Dec 2020 Quanyi Li, Zhenghao Peng, Qihang Zhang, Chunxiao Liu, Bolei Zhou

We validate that training with the increasing number of procedurally generated scenes significantly improves the generalization of the agent across scenarios of different traffic densities and road networks.

Autonomous Driving

Positional Encoding as Spatial Inductive Bias in GANs

no code implementations CVPR 2021 Rui Xu, Xintao Wang, Kai Chen, Bolei Zhou, Chen Change Loy

In this work, taking SinGAN and StyleGAN2 as examples, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.

Image Manipulation Inductive Bias +1

Improving the Fairness of Deep Generative Models without Retraining

1 code implementation9 Dec 2020 Shuhan Tan, Yujun Shen, Bolei Zhou

Generative Adversarial Networks (GANs) advance face synthesis through learning the underlying distribution of observed data.

Attribute Face Generation +3

Texture Memory-Augmented Deep Patch-Based Image Inpainting

1 code implementation28 Sep 2020 Rui Xu, Minghao Guo, Jiaqi Wang, Xiaoxiao Li, Bolei Zhou, Chen Change Loy

By bringing together the best of both paradigms, we propose a new deep inpainting framework where texture generation is guided by a texture memory of patch samples extracted from unmasked regions.

Image Inpainting Retrieval +1

Generative Hierarchical Features from Synthesizing Images

1 code implementation CVPR 2021 Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, Bolei Zhou

Generative Adversarial Networks (GANs) have recently advanced image synthesis by learning the underlying distribution of the observed data.

Face Verification Image Classification +2

Closed-Form Factorization of Latent Semantics in GANs

11 code implementations CVPR 2021 Yujun Shen, Bolei Zhou

A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images.

Attribute Image Generation +1

Unsupervised Landmark Learning from Unpaired Data

1 code implementation29 Jun 2020 Yinghao Xu, Ceyuan Yang, Ziwei Liu, Bo Dai, Bolei Zhou

Recent attempts for unsupervised landmark learning leverage synthesized image pairs that are similar in appearance but different in poses.

Video Representation Learning with Visual Tempo Consistency

1 code implementation28 Jun 2020 Ceyuan Yang, Yinghao Xu, Bo Dai, Bolei Zhou

Visual tempo, which describes how fast an action goes, has shown its potential in supervised action recognition.

Action Anticipation Action Detection +3

Non-local Policy Optimization via Diversity-regularized Collaborative Exploration

no code implementations14 Jun 2020 Zhenghao Peng, Hao Sun, Bolei Zhou

Conventional Reinforcement Learning (RL) algorithms usually have one single agent learning to solve the task independently.

Reinforcement Learning (RL)

Zeroth-Order Supervised Policy Improvement

no code implementations11 Jun 2020 Hao Sun, Ziping Xu, Yuhang Song, Meng Fang, Jiechao Xiong, Bo Dai, Bolei Zhou

However, PG algorithms rely on exploiting the value function being learned with the first-order update locally, which results in limited sample efficiency.

Continuous Control Policy Gradient Methods +2

InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs

2 code implementations18 May 2020 Yujun Shen, Ceyuan Yang, Xiaoou Tang, Bolei Zhou

In this work, we propose a framework called InterFaceGAN to interpret the disentangled face representation learned by the state-of-the-art GAN models and study the properties of the facial semantics encoded in the latent space.

Attribute Face Generation

Evolutionary Stochastic Policy Distillation

1 code implementation27 Apr 2020 Hao Sun, Xinyu Pan, Bo Dai, Dahua Lin, Bolei Zhou

Solving the Goal-Conditioned Reward Sparse (GCRS) task is a challenging reinforcement learning problem due to the sparsity of reward signals.

TPNet: Trajectory Proposal Network for Motion Prediction

no code implementations CVPR 2020 Liangji Fang, Qinhong Jiang, Jianping Shi, Bolei Zhou

However, it remains difficult for these methods to provide multimodal predictions as well as integrate physical constraints such as traffic rules and movable areas.

Autonomous Driving motion prediction +1

Temporal Pyramid Network for Action Recognition

3 code implementations CVPR 2020 Ceyuan Yang, Yinghao Xu, Jianping Shi, Bo Dai, Bolei Zhou

Previous works often capture the visual tempo through sampling raw videos at multiple rates and constructing an input-level frame pyramid, which usually requires a costly multi-branch network to handle.

Action Recognition

A Local-to-Global Approach to Multi-modal Movie Scene Segmentation

4 code implementations CVPR 2020 Anyi Rao, Linning Xu, Yu Xiong, Guodong Xu, Qingqiu Huang, Bolei Zhou, Dahua Lin

Scene, as the crucial unit of storytelling in movies, contains complex activities of actors and their interactions in a physical environment.

Action Recognition Scene Segmentation +1

In-Domain GAN Inversion for Real Image Editing

2 code implementations ECCV 2020 Jiapeng Zhu, Yujun Shen, Deli Zhao, Bolei Zhou

A common practice of feeding a real image to a trained GAN generator is to invert it back to a latent code.

Image Reconstruction

TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting

no code implementations CVPR 2020 Zhuoqian Yang, Wentao Zhu, Wayne Wu, Chen Qian, Qiang Zhou, Bolei Zhou, Chen Change Loy

We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.

motion retargeting

Image Processing Using Multi-Code GAN Prior

1 code implementation CVPR 2020 Jinjin Gu, Yujun Shen, Bolei Zhou

Such an over-parameterization of the latent space significantly improves the image reconstruction quality, outperforming existing competitors.

Blind Face Restoration Colorization +6

Learning a Decision Module by Imitating Driver's Control Behaviors

no code implementations30 Nov 2019 Junning Huang, Sirui Xie, Jiankai Sun, Qiurui Ma, Chunxiao Liu, Jianping Shi, Dahua Lin, Bolei Zhou

In this work, we propose a hybrid framework to learn neural decisions in the classical modular pipeline through end-to-end imitation learning.

Autonomous Driving Imitation Learning

Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

no code implementations28 Nov 2019 Mingyu Ding, Zhe Wang, Bolei Zhou, Jianping Shi, Zhiwu Lu, Ping Luo

Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference.

Optical Flow Estimation Segmentation +3

Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis

2 code implementations21 Nov 2019 Ceyuan Yang, Yujun Shen, Bolei Zhou

Despite the success of Generative Adversarial Networks (GANs) in image synthesis, there lacks enough understanding on what generative models have learned inside the deep generative representations and how photo-realistic images are able to be composed of the layer-wise stochasticity introduced in recent GANs.

Image Generation

Policy Continuation with Hindsight Inverse Dynamics

1 code implementation NeurIPS 2019 Hao Sun, Zhizhong Li, Xiaotong Liu, Dahua Lin, Bolei Zhou

This approach learns from Hindsight Inverse Dynamics based on Hindsight Experience Replay, enabling the learning process in a self-imitated manner and thus can be trained with supervised learning.

Reinforcement Learning (RL)

A Graph-Based Framework to Bridge Movies and Synopses

no code implementations ICCV 2019 Yu Xiong, Qingqiu Huang, Lingfeng Guo, Hang Zhou, Bolei Zhou, Dahua Lin

On top of this dataset, we develop a framework to perform matching between movie segments and synopsis paragraphs.

Semantic Hierarchy Emerges in the Deep Generative Representations for Scene Synthesis

no code implementations25 Sep 2019 Ceyuan Yang, Yujun Shen, Bolei Zhou

Despite the success of Generative Adversarial Networks (GANs) in image synthesis, there lacks enough understanding on what networks have learned inside the deep generative representations and how photo-realistic images are able to be composed from random noises.

Image Generation

LIA: Latently Invertible Autoencoder with Adversarial Learning

no code implementations25 Sep 2019 Jiapeng Zhu, Deli Zhao, Bolei Zhou, Bo Zhang

A two-stage stochasticity-free training scheme is designed to train LIA via adversarial learning, in the sense that the decoder of LIA is first trained as a standard GAN with the invertible network and then the partial encoder is learned from an autoencoder by detaching the invertible network from LIA.

Decoder Generative Adversarial Network +1

Learning with Social Influence through Interior Policy Differentiation

no code implementations25 Sep 2019 Hao Sun, Bo Dai, Jiankai Sun, Zhenghao Peng, Guodong Xu, Dahua Lin, Bolei Zhou

In this work we model the social influence into the scheme of reinforcement learning, enabling the agents to learn both from the environment and from their peers.

Reinforcement Learning (RL)

Reasoning About Human-Object Interactions Through Dual Attention Networks

no code implementations ICCV 2019 Tete Xiao, Quanfu Fan, Dan Gutfreund, Mathew Monfort, Aude Oliva, Bolei Zhou

The model not only finds when an action is happening and which object is being manipulated, but also identifies which part of the object is being interacted with.

Human-Object Interaction Detection Object

Interpreting the Latent Space of GANs for Semantic Face Editing

4 code implementations CVPR 2020 Yujun Shen, Jinjin Gu, Xiaoou Tang, Bolei Zhou

In this work, we propose a novel framework, called InterFaceGAN, for semantic face editing by interpreting the latent semantics learned by GANs.

Attribute Disentanglement +2

Disentangled Inference for GANs with Latently Invertible Autoencoder

3 code implementations19 Jun 2019 Jiapeng Zhu, Deli Zhao, Bo Zhang, Bolei Zhou

In this paper, we show that the entanglement of the latent space for the VAE/GAN framework poses the main challenge for encoder learning.


Cross-view Semantic Segmentation for Sensing Surroundings

1 code implementation9 Jun 2019 Bowen Pan, Jiankai Sun, Ho Yin Tiga Leung, Alex Andonian, Bolei Zhou

Our further experiment on a LoCoBot robot shows that our model enables the surrounding sensing capability from 2D image input.

Domain Adaptation Semantic Segmentation

Deep Flow-Guided Video Inpainting

2 code implementations CVPR 2019 Rui Xu, Xiaoxiao Li, Bolei Zhou, Chen Change Loy

Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video.

One-shot visual object segmentation Optical Flow Estimation +2

On the Units of GANs (Extended Abstract)

no code implementations29 Jan 2019 David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, Antonio Torralba

We quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output.

Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

no code implementations25 Jan 2019 Quanshi Zhang, Lixin Fan, Bolei Zhou

This is the Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

FaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face Synthesis

no code implementations4 Dec 2018 Yujun Shen, Bolei Zhou, Ping Luo, Xiaoou Tang

In the second stage, they compete in the image domain to render photo-realistic images that contain high diversity but preserve identity.

Face Generation Vocal Bursts Valence Prediction

Interpretable Basis Decomposition for Visual Explanation

1 code implementation ECCV 2018 Bolei Zhou, Yiyou Sun, David Bau, Antonio Torralba

Explanations of the decisions made by a deep neural network are important for human end-users to be able to understand and diagnose the trustworthiness of the system.

Single Image Intrinsic Decomposition without a Single Intrinsic Image

no code implementations ECCV 2018 Wei-Chiu Ma, Hang Chu, Bolei Zhou, Raquel Urtasun, Antonio Torralba

At inference time, our model can be easily reduced to a single stream module that performs intrinsic decomposition on a single input image.

Intrinsic Image Decomposition

Unified Perceptual Parsing for Scene Understanding

18 code implementations ECCV 2018 Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, Jian Sun

In this paper, we study a new task called Unified Perceptual Parsing, which requires the machine vision systems to recognize as many visual concepts as possible from a given image.

Scene Understanding Semantic Segmentation

Revisiting the Importance of Individual Units in CNNs via Ablation

no code implementations7 Jun 2018 Bolei Zhou, Yiyou Sun, David Bau, Antonio Torralba

We confirm that unit attributes such as class selectivity are a poor predictor for impact on overall accuracy as found previously in recent work \cite{morcos2018importance}.

General Classification

DeepMiner: Discovering Interpretable Representations for Mammogram Classification and Explanation

1 code implementation31 May 2018 Jimmy Wu, Bolei Zhou, Diondra Peck, Scott Hsieh, Vandana Dialani, Lester Mackey, Genevieve Patterson

We propose DeepMiner, a framework to discover interpretable representations in deep neural networks and to build explanations for medical predictions.

Classification General Classification +1

Recurrent Residual Module for Fast Inference in Videos

no code implementations CVPR 2018 Bowen Pan, Wuwei Lin, Xiaolin Fang, Chaoqin Huang, Bolei Zhou, Cewu Lu

Deep convolutional neural networks (CNNs) have made impressive progress in many video recognition tasks such as video pose estimation and video object detection.

object-detection Pose Estimation +2

Temporal Relational Reasoning in Videos

5 code implementations ECCV 2018 Bolei Zhou, Alex Andonian, Aude Oliva, Antonio Torralba

Temporal relational reasoning, the ability to link meaningful transformations of objects or entities over time, is a fundamental property of intelligent species.

Action Classification Action Recognition In Videos +4

Interpreting Deep Visual Representations via Network Dissection

2 code implementations15 Nov 2017 Bolei Zhou, David Bau, Aude Oliva, Antonio Torralba

In this work, we describe Network Dissection, a method that interprets networks by providing labels for the units of their deep visual representations.

Visual Question Generation as Dual Task of Visual Question Answering

no code implementations CVPR 2018 Yikang Li, Nan Duan, Bolei Zhou, Xiao Chu, Wanli Ouyang, Xiaogang Wang

Recently visual question answering (VQA) and visual question generation (VQG) are two trending topics in the computer vision, which have been explored separately.

Question Answering Question Generation +2

Scene Graph Generation from Objects, Phrases and Region Captions

1 code implementation ICCV 2017 Yikang Li, Wanli Ouyang, Bolei Zhou, Kun Wang, Xiaogang Wang

Object detection, scene graph generation and region captioning, which are three scene understanding tasks at different semantic levels, are tied together: scene graphs are generated on top of objects detected in an image with their pairwise relationship predicted, while region captioning gives a language description of the objects, their attributes, relations, and other context information.

Graph Generation object-detection +3

Scene Parsing Through ADE20K Dataset

no code implementations CVPR 2017 Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, Antonio Torralba

A novel network design called Cascade Segmentation Module is proposed to parse a scene into stuff, objects, and object parts in a cascade and improve over the baselines.

Object Scene Parsing +1

Network Dissection: Quantifying Interpretability of Deep Visual Representations

1 code implementation CVPR 2017 David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, Antonio Torralba

Given any CNN model, the proposed method draws on a broad data set of visual concepts to score the semantics of hidden units at each intermediate convolutional layer.

Open Vocabulary Scene Parsing

no code implementations ICCV 2017 Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler, Antonio Torralba

Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets.

General Classification Scene Parsing

SegICP: Integrated Deep Semantic Segmentation and Pose Estimation

2 code implementations5 Mar 2017 Jay M. Wong, Vincent Kee, Tiffany Le, Syler Wagner, Gian-Luca Mariottini, Abraham Schneider, Lei Hamilton, Rahul Chipalkatty, Mitchell Hebert, David M. S. Johnson, Jimmy Wu, Bolei Zhou, Antonio Torralba

Recent robotic manipulation competitions have highlighted that sophisticated robots still struggle to achieve fast and reliable perception of task-relevant objects in complex, realistic scenarios.

Object Recognition Point Cloud Registration +3

Person Search with Natural Language Description

1 code implementation CVPR 2017 Shuang Li, Tong Xiao, Hongsheng Li, Bolei Zhou, Dayu Yue, Xiaogang Wang

Searching persons in large-scale image databases with the query of natural language description has important applications in video surveillance.

Attribute Person Search +1

Places: An Image Database for Deep Scene Understanding

no code implementations6 Oct 2016 Bolei Zhou, Aditya Khosla, Agata Lapedriza, Antonio Torralba, Aude Oliva

The rise of multi-million-item dataset initiatives has enabled data-hungry machine learning algorithms to reach near-human semantic classification at tasks such as object and scene recognition.

BIG-bench Machine Learning Classification +4

Semantic Understanding of Scenes through the ADE20K Dataset

21 code implementations18 Aug 2016 Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, Antonio Torralba

Scene parsing, or recognizing and segmenting objects and stuff in an image, is one of the key problems in computer vision.

Scene Parsing Segmentation +1

Learning Deep Features for Discriminative Localization

33 code implementations CVPR 2016 Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba

In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network to have remarkable localization ability despite being trained on image-level labels.

Weakly-Supervised Object Localization

Optimization as Estimation with Gaussian Processes in Bandit Settings

1 code implementation21 Oct 2015 Zi Wang, Bolei Zhou, Stefanie Jegelka

Recently, there has been rising interest in Bayesian optimization -- the optimization of an unknown function with assumptions usually expressed by a Gaussian Process (GP) prior.

Bayesian Optimization Gaussian Processes

Understanding Intra-Class Knowledge Inside CNN

no code implementations9 Jul 2015 Donglai Wei, Bolei Zhou, Antonio Torrabla, William Freeman

Convolutional Neural Network (CNN) has been successful in image recognition tasks, and recent works shed lights on how CNN separates different classes with the learned inter-class knowledge through visualization.

Image Retrieval Object +1

Object Detectors Emerge in Deep Scene CNNs

1 code implementation22 Dec 2014 Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba

With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e. g., ImageNet, Places), the state of the art in computer vision is advancing rapidly.

General Classification Object +3

Learning Deep Features for Scene Recognition using Places Database

no code implementations NeurIPS 2014 Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, Aude Oliva

Whereas the tremendous recent progress in object recognition tasks is due to the availability of large datasets like ImageNet and the rise of Convolutional Neural Networks (CNNs) for learning high-level features, performance at scene recognition has not attained the same level of success.

Object Object Recognition +1

ConceptLearner: Discovering Visual Concepts from Weakly Labeled Image Collections

no code implementations CVPR 2015 Bolei Zhou, Vignesh Jagadeesh, Robinson Piramuthu

Discovering visual knowledge from weakly labeled data is crucial to scale up computer vision recognition system, since it is expensive to obtain fully labeled data for a large number of concept categories.

object-detection Object Detection +1

Measuring Crowd Collectiveness

no code implementations CVPR 2013 Bolei Zhou, Xiaoou Tang, Xiaogang Wang

Collective motions are common in crowd systems and have attracted a great deal of attention in a variety of multidisciplinary fields.

Cannot find the paper you are looking for? You can Submit a new open access paper.