Search Results for author: Chenliang Xu

Found 49 papers, 22 papers with code

Monocular 3D Object Detection via Feature Domain Adaptation

no code implementations ECCV 2020 Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, Chenliang Xu

Monocular 3D object detection is a challenging task due to unreliable depth, resulting in a distinct performance gap between monocular and LiDAR-based approaches.

Domain Adaptation Monocular 3D Object Detection

Learning to Generate Scene Graph from Natural Language Supervision

1 code implementation6 Sep 2021 Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, Yin Li

To bridge the gap between images and texts, we leverage an off-the-shelf object detector to identify and localize object instances, match labels of detected regions to concepts parsed from captions, and thus create "pseudo" labels for learning scene graph.

Graph Generation Scene Graph Generation

Learning by Planning: Language-Guided Global Image Editing

1 code implementation CVPR 2021 Jing Shi, Ning Xu, Yihang Xu, Trung Bui, Franck Dernoncourt, Chenliang Xu

Recently, language-guided global image editing draws increasing attention with growing application potentials.

Discover the Unknown Biased Attribute of an Image Classifier

1 code implementation29 Apr 2021 Zhiheng Li, Chenliang Xu

To help human experts better find the AI algorithms' biases, we study a new problem in this work -- for a classifier that predicts a target attribute of the input image, discover its unknown biased attribute.

High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation

no code implementations CVPR 2021 Lele Chen, Chen Cao, Fernando de la Torre, Jason Saragih, Chenliang Xu, Yaser Sheikh

This paper addresses previous limitations by learning a deep learning lighting model, that in combination with a high-quality 3D face tracking algorithm, provides a method for subtle and robust facial motion transfer from a regular video to a 3D photo-realistic avatar.

A Benchmark and Baseline for Language-Driven Image Editing

no code implementations5 Oct 2020 Jing Shi, Ning Xu, Trung Bui, Franck Dernoncourt, Zheng Wen, Chenliang Xu

To solve this new task, we first present a new language-driven image editing dataset that supports both local and global editing with editing operation and mask annotations.

Cubic Spline Smoothing Compensation for Irregularly Sampled Sequences

no code implementations3 Oct 2020 Jing Shi, Jing Bi, Yingru Liu, Chenliang Xu

The marriage of recurrent neural networks and neural ordinary differential networks (ODE-RNN) is effective in modeling irregularly-observed sequences.

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

1 code implementation ECCV 2020 Yapeng Tian, DIngzeyu Li, Chenliang Xu

In this paper, we introduce a new problem, named audio-visual video parsing, which aims to parse a video into temporal event segments and label them as either audible, visible, or both.

Multiple Instance Learning

Talking-head Generation with Rhythmic Head Motion

1 code implementation16 Jul 2020 Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, Chenliang Xu

When people deliver a speech, they naturally move heads, and this rhythmic head motion conveys prosodic information.

Talking Head Generation

Explaining Local, Global, And Higher-Order Interactions In Deep Learning

1 code implementation12 Jun 2020 Samuel Lerman, Chenliang Xu, Charles Venuto, Henry Kautz

We present a simple yet highly generalizable method for explaining interacting parts within a neural network's reasoning process.

Explainable artificial intelligence Object Detection +1

What comprises a good talking-head video generation?: A Survey and Benchmark

1 code implementation7 May 2020 Lele Chen, Guofeng Cui, Ziyi Kou, Haitian Zheng, Chenliang Xu

In this work, we present a carefully-designed benchmark for evaluating talking-head video generation with standardized dataset pre-processing strategies.

Talking Head Generation Video Generation

Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection

no code implementations CVPR 2020 Jie Chen, Zhiheng Li, Jiebo Luo, Chenliang Xu

Instead of blindly trusting quality-inconsistent PAs, WS^2 employs a learning-based selection to select effective PAs and a novel region integrity criterion as a stopping condition for weakly-supervised training.

Action Segmentation Semantic Segmentation +2

Deep Grouping Model for Unified Perceptual Parsing

no code implementations CVPR 2020 Zhiheng Li, Wenxuan Bao, Jiayang Zheng, Chenliang Xu

The perceptual-based grouping process produces a hierarchical and compositional image representation that helps both human and machine vision systems recognize heterogeneous visual concepts.

Semantic Segmentation

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

3 code implementations CVPR 2020 Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P. Allebach, Chenliang Xu

Rather than synthesizing missing LR video frames as VFI networks do, we firstly temporally interpolate LR frame features in missing LR video frames capturing local temporal contexts by the proposed feature temporal interpolation network.

Space-time Video Super-resolution Video Frame Interpolation +1

TailorGAN: Making User-Defined Fashion Designs

2 code implementations17 Jan 2020 Lele Chen, Justin Tian, Guo Li, Cheng-Haw Wu, Erh-Kan King, Kuan-Ting Chen, Shao-Hang Hsieh, Chenliang Xu

To overcome those limitations, we propose a novel self-supervised model to synthesize garment images with disentangled attributes (e. g., collar and sleeves) without paired data.

Deep Audio Prior

1 code implementation21 Dec 2019 Yapeng Tian, Chenliang Xu, DIngzeyu Li

We are interested in applying deep networks in the absence of training dataset.

Texture Synthesis

Learning from Interventions using Hierarchical Policies for Safe Learning

no code implementations4 Dec 2019 Jing Bi, Vikas Dhiman, Tianyou Xiao, Chenliang Xu

The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer.

Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation

no code implementations17 Nov 2019 Ziyi Kou, Guofeng Cui, Shaojie Wang, Wentian Zhao, Chenliang Xu

In this paper, we propose a confidence segmentation (ConfSeg) module that builds confidence score for each pixel in CAM without introducing additional hyper-parameters.

Weakly-Supervised Object Localization

Unsupervised Pose Flow Learning for Pose Guided Synthesis

no code implementations30 Sep 2019 Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo

Pose guided synthesis aims to generate a new image in an arbitrary target pose while preserving the appearance details from the source image.

Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss

1 code implementation9 May 2019 Lele Chen, Ross K. Maddox, Zhiyao Duan, Chenliang Xu

We devise a cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions.

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

2 code implementations7 Dec 2018 Yapeng Tian, Yulun Zhang, Yun Fu, Chenliang Xu

Video super-resolution (VSR) aims to restore a photo-realistic high-resolution (HR) video frame from both its corresponding low-resolution (LR) frame (reference frame) and multiple neighboring frames (supporting frames).

Optical Flow Estimation Video Super-Resolution

An Attempt towards Interpretable Audio-Visual Video Captioning

no code implementations7 Dec 2018 Yapeng Tian, Chenxiao Guan, Justin Goodman, Marc Moore, Chenliang Xu

To achieve this, we propose a multimodal convolutional neural network-based audio-visual video captioning framework and introduce a modality-aware module for exploring modality selection during sentence generation.

Audio captioning Audio-Visual Video Captioning +2

How to Make a BLT Sandwich? Learning to Reason towards Understanding Web Instructional Videos

no code implementations2 Dec 2018 Shaojie Wang, Wentian Zhao, Ziyi Kou, Chenliang Xu

Furthermore, we study multiple modalities including description and transcripts for the purpose of boosting video understanding.

Question Answering Video Understanding

GAN-EM: GAN based EM learning framework

no code implementations2 Dec 2018 Wentian Zhao, Shaojie Wang, Zhihuai Xie, Jing Shi, Chenliang Xu

To overcome such limitation, we propose a GAN based EM learning framework that can maximize the likelihood of images and estimate the latent variables with only the constraint of L-Lipschitz continuity.

Dimensionality Reduction General Classification +1

Navigation by Imitation in a Pedestrian-Rich Environment

no code implementations1 Nov 2018 Jing Bi, Tianyou Xiao, Qiuyue Sun, Chenliang Xu

Deep neural networks trained on demonstrations of human actions give robot the ability to perform self-driving on the road.

Imitation Learning

Lip Movements Generation at a Glance

1 code implementation ECCV 2018 Lele Chen, Zhiheng Li, Ross K. Maddox, Zhiyao Duan, Chenliang Xu

In this paper, we consider a task of such: given an arbitrary audio speech and one lip image of arbitrary target identity, generate synthesized lip movements of the target identity saying the speech.

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment

1 code implementation CVPR 2018 Li Ding, Chenliang Xu

In this work, we address the task of weakly-supervised human action segmentation in long, untrimmed videos.

Action Segmentation

Generating Talking Face Landmarks from Speech

no code implementations26 Mar 2018 Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, Zhiyao Duan

In this paper, we present a system that can generate landmark points of a talking face from an acoustic speech in real time.

MRI Tumor Segmentation with Densely Connected 3D CNN

2 code implementations18 Jan 2018 Lele Chen, Yue Wu, Adora M. DSouza, Anas Z. Abidin, Axel Wismuller, Chenliang Xu

The major difficulty of our segmentation model comes with the fact that the location, structure, and shape of gliomas vary significantly among different patients.

Tumor Segmentation

Video Action Segmentation with Hybrid Temporal Networks

no code implementations ICLR 2018 Li Ding, Chenliang Xu

Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years.

Action Segmentation

Weakly Supervised Actor-Action Segmentation via Robust Multi-Task Ranking

no code implementations CVPR 2017 Yan Yan, Chenliang Xu, Dawen Cai, Jason J. Corso

However, current methods for detailed understanding of actor and action have significant limitations: they require large amounts of finely labeled data, and they fail to capture any internal relationship among actors and actions.

Action Classification Action Segmentation +1

TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation

no code implementations22 May 2017 Li Ding, Chenliang Xu

Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years.

Action Segmentation

Action Understanding with Multiple Classes of Actors

no code implementations27 Apr 2017 Chenliang Xu, Caiming Xiong, Jason J. Corso

Despite the rapid progress, existing works on action understanding focus strictly on one type of action agent, which we call actor---a human adult, ignoring the diversity of actions performed by other actors.

Action Recognition Action Segmentation +2

Deep Cross-Modal Audio-Visual Generation

no code implementations26 Apr 2017 Lele Chen, Sudhanshu Srivastava, Zhiyao Duan, Chenliang Xu

Being the first to explore this new problem, we compose two new datasets with pairs of images and sounds of musical performances of different instruments.

Towards Automatic Learning of Procedures from Web Instructional Videos

1 code implementation28 Mar 2017 Luowei Zhou, Chenliang Xu, Jason J. Corso

To answer this question, we introduce the problem of procedure segmentation--to segment a video procedure into category-independent procedure segments.

Dense Video Captioning

Watch What You Just Said: Image Captioning with Text-Conditional Attention

1 code implementation15 Jun 2016 Luowei Zhou, Chenliang Xu, Parker Koch, Jason J. Corso

Attention mechanisms have attracted considerable interest in image captioning due to its powerful performance.

Image Captioning Language Modelling

LIBSVX: A Supervoxel Library and Benchmark for Early Video Processing

no code implementations30 Dec 2015 Chenliang Xu, Jason J. Corso

Supervoxel segmentation has strong potential to be incorporated into early video analysis as superpixel segmentation has in image analysis.

Boundary Detection

Actor-Action Semantic Segmentation with Grouping Process Models

no code implementations CVPR 2016 Chenliang Xu, Jason J. Corso

Actor-action semantic segmentation made an important step toward advanced video understanding problems: what action is happening; who is performing the action; and where is the action in space-time.

Semantic Segmentation Video Understanding

Can Humans Fly? Action Understanding With Multiple Classes of Actors

no code implementations CVPR 2015 Chenliang Xu, Shao-Hang Hsieh, Caiming Xiong, Jason J. Corso

There is no work we know of on simultaneously inferring actors and actions in the video, not to mention a dataset to experiment with.

Action Recognition Action Understanding +1

A Study of Actor and Action Semantic Retention in Video Supervoxel Segmentation

no code implementations13 Nov 2013 Chenliang Xu, Richard F. Doell, Stephen José Hanson, Catherine Hanson, Jason J. Corso

In this paper, we conduct a systematic study of how well the actor and action semantics are retained in video supervoxel segmentation.

Object Detection

Cannot find the paper you are looking for? You can Submit a new open access paper.