Search Results for author: Chenliang Xu

Found 67 papers, 28 papers with code

Monocular 3D Object Detection via Feature Domain Adaptation

no code implementations ECCV 2020 Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, Chenliang Xu

Monocular 3D object detection is a challenging task due to unreliable depth, resulting in a distinct performance gap between monocular and LiDAR-based approaches.

Domain Adaptation Foreground Segmentation +2

Unveiling Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA

no code implementations31 May 2023 Ali Vosoughi, Shijian Deng, Songyang Zhang, Yapeng Tian, Chenliang Xu, Jiebo Luo

In this paper, we first model a confounding effect that causes language and vision bias simultaneously, then propose a counterfactual inference to remove the influence of this effect.

Counterfactual Inference Question Answering +1

Training Neural Networks without Backpropagation: A Deeper Dive into the Likelihood Ratio Method

no code implementations15 May 2023 Jinyang Jiang, Zeliang Zhang, Chenliang Xu, Zhaofei Yu, Yijie Peng

We develop the likelihood ratio (LR) method, a new gradient estimation method, for training a broad range of neural network architectures, including convolutional neural networks, recurrent neural networks, graph neural networks, and spiking neural networks, without recursive gradient computation.

Egocentric Audio-Visual Object Localization

1 code implementation CVPR 2023 Chao Huang, Yapeng Tian, Anurag Kumar, Chenliang Xu

In this paper, we explore the challenging egocentric audio-visual object localization task and observe that 1) egomotion commonly exists in first-person recordings, even within a short duration; 2) The out-of-view sound components can be created while wearers shift their attention.

Object Localization

AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis

no code implementations4 Feb 2023 Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, Chenliang Xu

Human perception of the complex world relies on a comprehensive analysis of multi-modal signals, and the co-occurrences of audio and video signals provide humans with rich cues.

Audio Generation Data Augmentation

Improving Adversarial Transferability with Scheduled Step Size and Dual Example

no code implementations30 Jan 2023 Zeliang Zhang, Peihan Liu, Xiaosen Wang, Chenliang Xu

Motivated by this finding, we argue that the information of adversarial perturbations near the benign sample, especially the direction, benefits more on the transferability.

Adversarial Attack

A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others

1 code implementation CVPR 2023 Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, Mark Ibrahim

Key to advancing the reliability of vision systems is understanding whether existing methods can overcome multiple shortcuts or struggle in a Whac-A-Mole game, i. e., where mitigating one shortcut amplifies reliance on others.

Domain Generalization Image Classification +1

Discover and Mitigate Unknown Biases with Debiasing Alternate Networks

1 code implementation20 Jul 2022 Zhiheng Li, Anthony Hoogs, Chenliang Xu

By training in an alternate manner, the discoverer tries to find multiple unknown biases of the classifier without any annotations of biases, and the classifier aims at unlearning the biases identified by the discoverer.

Action Recognition Facial Attribute Classification +1

StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis

1 code implementation CVPR 2022 Zhiheng Li, Martin Renqiang Min, Kai Li, Chenliang Xu

Based on the identified latent directions of attributes, we propose Compositional Attribute Adjustment to adjust the latent code, resulting in better compositionality of image synthesis.

Fairness Image Generation +1

Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution

1 code implementation CVPR 2022 Guangyuan Li, Jun Lv, Yapeng Tian, Qi Dou, Chengyan Wang, Chenliang Xu, Jing Qin

However, existing methods still have two shortcomings: (1) they neglect that the multi-contrast features at different scales contain different anatomical details and hence lack effective mechanisms to match and fuse these features for better reconstruction; and (2) they are still deficient in capturing long-range dependencies, which are essential for the regions with complicated anatomical structures.


Learning to Answer Questions in Dynamic Audio-Visual Scenarios

1 code implementation CVPR 2022 Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, Di Hu

In this paper, we focus on the Audio-Visual Question Answering (AVQA) task, which aims to answer questions regarding different visual objects, sounds, and their associations in videos.

audio-visual learning Audio-visual Question Answering +4

Cross-modal Contrastive Distillation for Instructional Activity Anticipation

no code implementations18 Jan 2022 Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo

In this study, we aim to predict the plausible future action steps given an observation of the past and study the task of instructional activity anticipation.

Knowledge Distillation

SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Color Editing

no code implementations CVPR 2022 Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu

Recently, large pretrained models (e. g., BERT, StyleGAN, CLIP) show great knowledge transfer and generalization capability on various downstream tasks within their domains.

Image-to-Image Translation Retrieval +1

SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing

no code implementations30 Nov 2021 Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu

Recently, large pretrained models (e. g., BERT, StyleGAN, CLIP) have shown great knowledge transfer and generalization capability on various downstream tasks within their domains.

Image-to-Image Translation Retrieval +1

Space-Time Memory Network for Sounding Object Localization in Videos

no code implementations10 Nov 2021 Sizhe Li, Yapeng Tian, Chenliang Xu

Leveraging temporal synchronization and association within sight and sound is an essential step towards robust localization of sounding objects.

Object Localization

Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning

no code implementations ICCV 2021 Jing Bi, Jiebo Luo, Chenliang Xu

In this work, we leverage instructional videos to study humans' decision-making processes, focusing on learning a model to plan goal-directed actions in real-life videos.

Action Recognition Bayesian Inference +1

rQdia: Regularizing Q-Value Distributions With Image Augmentation

no code implementations29 Sep 2021 Samuel Lerman, Jing Bi, Chenliang Xu

rQdia (pronounced “Arcadia”) regularizes Q-value distributions with augmented images in pixel-based deep reinforcement learning.

Continuous Control Image Augmentation +2

Learning to Generate Scene Graph from Natural Language Supervision

1 code implementation ICCV 2021 Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, Yin Li

To bridge the gap between images and texts, we leverage an off-the-shelf object detector to identify and localize object instances, match labels of detected regions to concepts parsed from captions, and thus create "pseudo" labels for learning scene graph.

Graph Generation Scene Graph Generation

Learning by Planning: Language-Guided Global Image Editing

1 code implementation CVPR 2021 Jing Shi, Ning Xu, Yihang Xu, Trung Bui, Franck Dernoncourt, Chenliang Xu

Recently, language-guided global image editing draws increasing attention with growing application potentials.

Discover the Unknown Biased Attribute of an Image Classifier

1 code implementation ICCV 2021 Zhiheng Li, Chenliang Xu

To help human experts better find the AI algorithms' biases, we study a new problem in this work -- for a classifier that predicts a target attribute of the input image, discover its unknown biased attribute.


High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation

no code implementations CVPR 2021 Lele Chen, Chen Cao, Fernando de la Torre, Jason Saragih, Chenliang Xu, Yaser Sheikh

This paper addresses previous limitations by learning a deep learning lighting model, that in combination with a high-quality 3D face tracking algorithm, provides a method for subtle and robust facial motion transfer from a regular video to a 3D photo-realistic avatar.

Vocal Bursts Intensity Prediction

A Simple Baseline for Weakly-Supervised Scene Graph Generation

no code implementations ICCV 2021 Jing Shi, Yiwu Zhong, Ning Xu, Yin Li, Chenliang Xu

We investigate the weakly-supervised scene graph generation, which is a challenging task since no correspondence of label and object is provided.

Contrastive Learning Graph Generation +2

A Benchmark and Baseline for Language-Driven Image Editing

no code implementations5 Oct 2020 Jing Shi, Ning Xu, Trung Bui, Franck Dernoncourt, Zheng Wen, Chenliang Xu

To solve this new task, we first present a new language-driven image editing dataset that supports both local and global editing with editing operation and mask annotations.

Cubic Spline Smoothing Compensation for Irregularly Sampled Sequences

no code implementations3 Oct 2020 Jing Shi, Jing Bi, Yingru Liu, Chenliang Xu

The marriage of recurrent neural networks and neural ordinary differential networks (ODE-RNN) is effective in modeling irregularly-observed sequences.

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

1 code implementation ECCV 2020 Yapeng Tian, DIngzeyu Li, Chenliang Xu

In this paper, we introduce a new problem, named audio-visual video parsing, which aims to parse a video into temporal event segments and label them as either audible, visible, or both.

Multiple Instance Learning

Talking-head Generation with Rhythmic Head Motion

1 code implementation16 Jul 2020 Lele Chen, Guofeng Cui, Celong Liu, Zhong Li, Ziyi Kou, Yi Xu, Chenliang Xu

When people deliver a speech, they naturally move heads, and this rhythmic head motion conveys prosodic information.

Talking Head Generation

What comprises a good talking-head video generation?: A Survey and Benchmark

1 code implementation7 May 2020 Lele Chen, Guofeng Cui, Ziyi Kou, Haitian Zheng, Chenliang Xu

In this work, we present a carefully-designed benchmark for evaluating talking-head video generation with standardized dataset pre-processing strategies.

Talking Head Generation Video Generation

Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection

no code implementations CVPR 2020 Jie Chen, Zhiheng Li, Jiebo Luo, Chenliang Xu

Instead of blindly trusting quality-inconsistent PAs, WS^2 employs a learning-based selection to select effective PAs and a novel region integrity criterion as a stopping condition for weakly-supervised training.

Action Segmentation Semantic Segmentation +2

Deep Grouping Model for Unified Perceptual Parsing

no code implementations CVPR 2020 Zhiheng Li, Wenxuan Bao, Jiayang Zheng, Chenliang Xu

The perceptual-based grouping process produces a hierarchical and compositional image representation that helps both human and machine vision systems recognize heterogeneous visual concepts.

Image Segmentation Semantic Segmentation

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

3 code implementations CVPR 2020 Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P. Allebach, Chenliang Xu

Rather than synthesizing missing LR video frames as VFI networks do, we firstly temporally interpolate LR frame features in missing LR video frames capturing local temporal contexts by the proposed feature temporal interpolation network.

Space-time Video Super-resolution Video Frame Interpolation +1

TailorGAN: Making User-Defined Fashion Designs

2 code implementations17 Jan 2020 Lele Chen, Justin Tian, Guo Li, Cheng-Haw Wu, Erh-Kan King, Kuan-Ting Chen, Shao-Hang Hsieh, Chenliang Xu

To overcome those limitations, we propose a novel self-supervised model to synthesize garment images with disentangled attributes (e. g., collar and sleeves) without paired data.

Deep Audio Prior

1 code implementation21 Dec 2019 Yapeng Tian, Chenliang Xu, DIngzeyu Li

We are interested in applying deep networks in the absence of training dataset.

Texture Synthesis

Learning from Interventions using Hierarchical Policies for Safe Learning

no code implementations4 Dec 2019 Jing Bi, Vikas Dhiman, Tianyou Xiao, Chenliang Xu

The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer.

Improve CAM with Auto-adapted Segmentation and Co-supervised Augmentation

no code implementations17 Nov 2019 Ziyi Kou, Guofeng Cui, Shaojie Wang, Wentian Zhao, Chenliang Xu

In this paper, we propose a confidence segmentation (ConfSeg) module that builds confidence score for each pixel in CAM without introducing additional hyper-parameters.

Weakly-Supervised Object Localization

Unsupervised Pose Flow Learning for Pose Guided Synthesis

no code implementations30 Sep 2019 Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo

Pose guided synthesis aims to generate a new image in an arbitrary target pose while preserving the appearance details from the source image.

Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss

1 code implementation9 May 2019 Lele Chen, Ross K. Maddox, Zhiyao Duan, Chenliang Xu

We devise a cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions.

An Attempt towards Interpretable Audio-Visual Video Captioning

no code implementations7 Dec 2018 Yapeng Tian, Chenxiao Guan, Justin Goodman, Marc Moore, Chenliang Xu

To achieve this, we propose a multimodal convolutional neural network-based audio-visual video captioning framework and introduce a modality-aware module for exploring modality selection during sentence generation.

Audio captioning Audio-Visual Video Captioning +2

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

2 code implementations7 Dec 2018 Yapeng Tian, Yulun Zhang, Yun Fu, Chenliang Xu

Video super-resolution (VSR) aims to restore a photo-realistic high-resolution (HR) video frame from both its corresponding low-resolution (LR) frame (reference frame) and multiple neighboring frames (supporting frames).

Optical Flow Estimation Video Super-Resolution

How to Make a BLT Sandwich? Learning to Reason towards Understanding Web Instructional Videos

no code implementations2 Dec 2018 Shaojie Wang, Wentian Zhao, Ziyi Kou, Chenliang Xu

Furthermore, we study multiple modalities including description and transcripts for the purpose of boosting video understanding.

Logical Reasoning Question Answering +1

GAN-EM: GAN based EM learning framework

no code implementations2 Dec 2018 Wentian Zhao, Shaojie Wang, Zhihuai Xie, Jing Shi, Chenliang Xu

To overcome such limitation, we propose a GAN based EM learning framework that can maximize the likelihood of images and estimate the latent variables with only the constraint of L-Lipschitz continuity.

Dimensionality Reduction General Classification +1

Navigation by Imitation in a Pedestrian-Rich Environment

no code implementations1 Nov 2018 Jing Bi, Tianyou Xiao, Qiuyue Sun, Chenliang Xu

Deep neural networks trained on demonstrations of human actions give robot the ability to perform self-driving on the road.

Imitation Learning Navigate

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment

1 code implementation CVPR 2018 Li Ding, Chenliang Xu

In this work, we address the task of weakly-supervised human action segmentation in long, untrimmed videos.

Action Segmentation

Lip Movements Generation at a Glance

1 code implementation ECCV 2018 Lele Chen, Zhiheng Li, Ross K. Maddox, Zhiyao Duan, Chenliang Xu

In this paper, we consider a task of such: given an arbitrary audio speech and one lip image of arbitrary target identity, generate synthesized lip movements of the target identity saying the speech.

Generating Talking Face Landmarks from Speech

no code implementations26 Mar 2018 Sefik Emre Eskimez, Ross K. Maddox, Chenliang Xu, Zhiyao Duan

In this paper, we present a system that can generate landmark points of a talking face from an acoustic speech in real time.

MRI Tumor Segmentation with Densely Connected 3D CNN

3 code implementations18 Jan 2018 Lele Chen, Yue Wu, Adora M. DSouza, Anas Z. Abidin, Axel Wismuller, Chenliang Xu

The major difficulty of our segmentation model comes with the fact that the location, structure, and shape of gliomas vary significantly among different patients.

Tumor Segmentation

Video Action Segmentation with Hybrid Temporal Networks

no code implementations ICLR 2018 Li Ding, Chenliang Xu

Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years.

Action Segmentation

Weakly Supervised Actor-Action Segmentation via Robust Multi-Task Ranking

no code implementations CVPR 2017 Yan Yan, Chenliang Xu, Dawen Cai, Jason J. Corso

However, current methods for detailed understanding of actor and action have significant limitations: they require large amounts of finely labeled data, and they fail to capture any internal relationship among actors and actions.

Action Classification Action Segmentation +1

TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation

no code implementations22 May 2017 Li Ding, Chenliang Xu

Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years.

Action Segmentation

Action Understanding with Multiple Classes of Actors

no code implementations27 Apr 2017 Chenliang Xu, Caiming Xiong, Jason J. Corso

Despite the rapid progress, existing works on action understanding focus strictly on one type of action agent, which we call actor---a human adult, ignoring the diversity of actions performed by other actors.

Action Recognition Action Segmentation +3

Deep Cross-Modal Audio-Visual Generation

no code implementations26 Apr 2017 Lele Chen, Sudhanshu Srivastava, Zhiyao Duan, Chenliang Xu

Being the first to explore this new problem, we compose two new datasets with pairs of images and sounds of musical performances of different instruments.

Towards Automatic Learning of Procedures from Web Instructional Videos

1 code implementation28 Mar 2017 Luowei Zhou, Chenliang Xu, Jason J. Corso

To answer this question, we introduce the problem of procedure segmentation--to segment a video procedure into category-independent procedure segments.

Dense Video Captioning Procedure Learning

Watch What You Just Said: Image Captioning with Text-Conditional Attention

1 code implementation15 Jun 2016 Luowei Zhou, Chenliang Xu, Parker Koch, Jason J. Corso

Attention mechanisms have attracted considerable interest in image captioning due to its powerful performance.

Image Captioning Language Modelling

LIBSVX: A Supervoxel Library and Benchmark for Early Video Processing

no code implementations30 Dec 2015 Chenliang Xu, Jason J. Corso

Supervoxel segmentation has strong potential to be incorporated into early video analysis as superpixel segmentation has in image analysis.

Boundary Detection Superpixels

Actor-Action Semantic Segmentation with Grouping Process Models

no code implementations CVPR 2016 Chenliang Xu, Jason J. Corso

Actor-action semantic segmentation made an important step toward advanced video understanding problems: what action is happening; who is performing the action; and where is the action in space-time.

Semantic Segmentation Video Understanding

Can Humans Fly? Action Understanding With Multiple Classes of Actors

no code implementations CVPR 2015 Chenliang Xu, Shao-Hang Hsieh, Caiming Xiong, Jason J. Corso

There is no work we know of on simultaneously inferring actors and actions in the video, not to mention a dataset to experiment with.

Action Recognition Action Understanding +2

A Study of Actor and Action Semantic Retention in Video Supervoxel Segmentation

no code implementations13 Nov 2013 Chenliang Xu, Richard F. Doell, Stephen José Hanson, Catherine Hanson, Jason J. Corso

In this paper, we conduct a systematic study of how well the actor and action semantics are retained in video supervoxel segmentation.

object-detection Object Detection

Cannot find the paper you are looking for? You can Submit a new open access paper.