Search Results for author: Jue Wang

Found 133 papers, 63 papers with code

SkipBERT: Efficient Inference with Shallow Layer Skipping

1 code implementation ACL 2022 Jue Wang, Ke Chen, Gang Chen, Lidan Shou, Julian McAuley

In this paper, we propose SkipBERT to accelerate BERT inference by skipping the computation of shallow layers.

Invisible Gas Detection: An RGB-Thermal Cross Attention Network and A New Benchmark

no code implementations26 Mar 2024 Jue Wang, Yuxiang Lin, Qi Zhao, Dong Luo, Shuaibao Chen, Wei Chen, Xiaojiang Peng

The widespread use of various chemical gases in industrial processes necessitates effective measures to prevent their leakage during transportation and storage, given their high toxicity.

Infinite forecast combinations based on Dirichlet process

no code implementations21 Nov 2023 Yinuo Ren, Feng Li, Yanfei Kang, Jue Wang

Forecast combination integrates information from various sources by consolidating multiple forecast results from the target time series.

Time Series

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time

1 code implementation26 Oct 2023 Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen

We show that contextual sparsity exists, that it can be accurately predicted, and that we can exploit it to speed up LLM inference in wall-clock time without compromising LLM's quality or in-context learning ability.

In-Context Learning

Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers

1 code implementation19 Oct 2023 Yuanduo Hong, Jue Wang, Weichao Sun, Huihui Pan

Building upon the original motivations of plain ViTs, which are simplicity and generality, we explore high-performance `minimalist' systems to this end.

Segmentation Semantic Segmentation

Revisiting the Temporal Modeling in Spatio-Temporal Predictive Learning under A Unified View

no code implementations9 Oct 2023 Cheng Tan, Jue Wang, Zhangyang Gao, Siyuan Li, Lirong Wu, Jun Xia, Stan Z. Li

In this paper, we re-examine the two dominant temporal modeling approaches within the realm of spatio-temporal predictive learning, offering a unified perspective.

Self-Supervised Learning

Learning Anchor Transformations for 3D Garment Animation

no code implementations CVPR 2023 Fang Zhao, Zekun Li, Shaoli Huang, Junwu Weng, Tianfei Zhou, Guo-Sen Xie, Jue Wang, Ying Shan

Once the anchor transformations are found, per-vertex nonlinear displacements of the garment template can be regressed in a canonical space, which reduces the complexity of deformation space learning.


Improving Fast Adversarial Training with Prior-Guided Knowledge

no code implementations1 Apr 2023 Xiaojun Jia, Yong Zhang, Xingxing Wei, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao

This initialization is generated by using high-quality adversarial perturbations from the historical training process.

Selective Structured State-Spaces for Long-Form Video Understanding

no code implementations CVPR 2023 Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, Raffay Hamid

To address this limitation, we present a novel Selective S4 (i. e., S5) model that employs a lightweight mask generator to adaptively select informative image tokens resulting in more efficient and accurate modeling of long-term spatiotemporal dependencies in videos.

Contrastive Learning Token Reduction +2

CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying

1 code implementation15 Mar 2023 Weihuang Liu, Xiaodong Cun, Chi-Man Pun, Menghan Xia, Yong Zhang, Jue Wang

Thanks to the proposed structure, we only encode the high-resolution image in a relatively low resolution for larger reception field capturing.

Image Inpainting Vocal Bursts Intensity Prediction

Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry

1 code implementation CVPR 2023 Jiaxu Zhang, Junwu Weng, Di Kang, Fang Zhao, Shaoli Huang, Xuefei Zhe, Linchao Bao, Ying Shan, Jue Wang, Zhigang Tu

Driven by our explored distance-based losses that explicitly model the motion semantics and geometry, these two modules can learn residual motion modifications on the source motion to generate plausible retargeted motion in a single inference without post-processing.

motion retargeting

ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction

1 code implementation CVPR 2023 Zhengdi Yu, Shaoli Huang, Chen Fang, Toby P. Breckon, Jue Wang

Our method significantly outperforms the best interacting-hand approaches on the InterHand2. 6M dataset while yielding comparable performance with the state-of-the-art single-hand methods on the FreiHand dataset.

3D Interacting Hand Pose Estimation 3D Reconstruction +1

CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior

1 code implementation CVPR 2023 Jinbo Xing, Menghan Xia, Yuechen Zhang, Xiaodong Cun, Jue Wang, Tien-Tsin Wong

In this paper, we propose to cast speech-driven facial animation as a code query task in a finite proxy space of the learned codebook, which effectively promotes the vividness of the generated motions by reducing the cross-modal mapping uncertainty.

3D Face Animation regression

High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction

no code implementations3 Jan 2023 Boyu Zhang, Hongliang Yuan, Mingyan Zhu, Ligang Liu, Jue Wang

Generating high-quality, realistic rendering images for real-time applications generally requires tracing a few samples-per-pixel (spp) and using deep learning-based approaches to denoise the resulting low-spp images.

2k Denoising

Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos

no code implementations27 Dec 2022 Zixiao Wang, Junwu Weng, Chun Yuan, Jue Wang

Thanks to Noise Contrastive Learning, the average classification accuracy improvement on Mini-Kinetics and Sth-Sth-V1 is over 1. 6\%.

Contrastive Learning Video Classification

Disentangled Image Colorization via Global Anchors

1 code implementation SIGGRAPH 2022 Menghan Xia, WenBo Hu, Tien-Tsin Wong, Jue Wang

Our key insight is that several carefully located anchors could approximately represent the color distribution of an image, and conditioned on the anchor colors, we can predict the image color in a deterministic manner by utilizing internal correlation.

Colorization Image Colorization

VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild

1 code implementation27 Nov 2022 Kun Cheng, Xiaodong Cun, Yong Zhang, Menghan Xia, Fei Yin, Mingrui Zhu, Xuan Wang, Jue Wang, Nannan Wang

Our system disentangles this objective into three sequential tasks: (1) face video generation with a canonical expression; (2) audio-driven lip-sync; and (3) face enhancement for improving photo-realism.

Video Editing Video Generation

Fine-Grained Face Swapping via Regional GAN Inversion

no code implementations CVPR 2023 Zhian Liu, Maomao Li, Yong Zhang, Cairong Wang, Qi Zhang, Jue Wang, Yongwei Nie

We rethink face swapping from the perspective of fine-grained face editing, \textit{i. e., ``editing for swapping'' (E4S)}, and propose a framework that is based on the explicit disentanglement of the shape and texture of facial components.

Disentanglement Face Swapping

One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations

1 code implementation14 Oct 2022 Yiming Zhu, Hongyu Liu, Yibing Song, Ziyang Yuan, Xintong Han, Chun Yuan, Qifeng Chen, Jue Wang

Based on the visual latent space of StyleGAN[21] and text embedding space of CLIP[34], studies focus on how to map these two latent spaces for text-driven attribute manipulations.

Attribute Image Manipulation

Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation

3 code implementations12 Oct 2022 Zeyu Qin, Yanbo Fan, Yi Liu, Li Shen, Yong Zhang, Jue Wang, Baoyuan Wu

Furthermore, RAP can be naturally combined with many existing black-box attack techniques, to further boost the transferability.

Adversarial Attack

Stability Analysis and Generalization Bounds of Adversarial Training

1 code implementation3 Oct 2022 Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Jue Wang, Zhi-Quan Luo

In adversarial machine learning, deep neural networks can fit the adversarial examples on the training dataset but have poor generalization ability on the test set.

Generalization Bounds

Understanding Adversarial Robustness Against On-manifold Adversarial Examples

1 code implementation2 Oct 2022 Jiancong Xiao, Liusha Yang, Yanbo Fan, Jue Wang, Zhi-Quan Luo

On synthetic datasets, theoretically, We prove that on-manifold adversarial examples are powerful, yet adversarial training focuses on off-manifold directions and ignores the on-manifold adversarial examples.

Adversarial Robustness

Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis

1 code implementation2 Oct 2022 Jiancong Xiao, Zeyu Qin, Yanbo Fan, Baoyuan Wu, Jue Wang, Zhi-Quan Luo

Therefore, adversarial training for multiple perturbations (ATMP) is proposed to generalize the adversarial robustness over different perturbation types (in $\ell_1$, $\ell_2$, and $\ell_\infty$ norm-bounded perturbations).

Adversarial Robustness

Control-Oriented Power Allocation for Integrated Satellite-UAV Networks

no code implementations31 Aug 2022 Chengleyang Lei, Wei Feng, Jue Wang, Shi Jin, Ning Ge

This letter presents a sensing-communication-computing-control (SC3) integrated satellite unmanned aerial vehicle (UAV) network, where the UAV is equipped with on-board sensors, mobile edge computing (MEC) servers, base stations and satellite communication module.


Towards Real-World Video Deblurring by Exploring Blur Formation Process

1 code implementation28 Aug 2022 Mingdeng Cao, Zhihang Zhong, Yanbo Fan, Jiahao Wang, Yong Zhang, Jue Wang, Yujiu Yang, Yinqiang Zheng

We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.


HyP$^2$ Loss: Beyond Hypersphere Metric Space for Multi-label Image Retrieval

1 code implementation14 Aug 2022 Chengyin Xu, Zenghao Chai, Zhengzhuo Xu, Chun Yuan, Yanbo Fan, Jue Wang

Image retrieval has become an increasingly appealing technique with broad multimedia application prospects, where deep hashing serves as the dominant branch towards low storage and efficient retrieval.

Deep Hashing Metric Learning +1

LocVTP: Video-Text Pre-training for Temporal Localization

1 code implementation21 Jul 2022 Meng Cao, Tianyu Yang, Junwu Weng, Can Zhang, Jue Wang, Yuexian Zou

To further enhance the temporal reasoning ability of the learned feature, we propose a context projection head and a temporal aware contrastive loss to perceive the contextual relationships.

Retrieval Temporal Localization +1

Prior-Guided Adversarial Initialization for Fast Adversarial Training

1 code implementation18 Jul 2022 Xiaojun Jia, Yong Zhang, Xingxing Wei, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao

Based on the observation, we propose a prior-guided FGSM initialization method to avoid overfitting after investigating several initialization strategies, improving the quality of the AEs during the whole training process.

Adversarial Attack Adversarial Attack on Video Classification

Neural Parameterization for Dynamic Human Head Editing

no code implementations1 Jul 2022 Li Ma, Xiaoyu Li, Jing Liao, Xuan Wang, Qi Zhang, Jue Wang, Pedro Sander

Implicit radiance functions emerged as a powerful scene representation for reconstructing and rendering photo-realistic views of a 3D scene.

Fast Adversarial Training with Adaptive Step Size

no code implementations6 Jun 2022 Zhichao Huang, Yanbo Fan, Chen Liu, Weizhong Zhang, Yong Zhang, Mathieu Salzmann, Sabine Süsstrunk, Jue Wang

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet.

Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees

1 code implementation2 Jun 2022 Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Re, Ce Zhang

Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks.

IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis

1 code implementation31 May 2022 Jingxiang Sun, Xuan Wang, Yichun Shi, Lizhen Wang, Jue Wang, Yebin Liu

Existing 3D-aware facial generation methods face a dilemma in quality versus editability: they either generate editable results in low resolution or high-quality ones with no editing flexibility.

3D-Aware Image Synthesis

AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition

2 code implementations26 May 2022 Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, Ping Luo

To address this challenge, we propose an effective adaptation approach for Transformer, namely AdaptFormer, which can adapt the pre-trained ViTs into many different image and video tasks efficiently.

Action Recognition Video Recognition

Improving the Latent Space of Image Style Transfer

no code implementations24 May 2022 Yunpeng Bai, Cairong Wang, Chun Yuan, Yanbo Fan, Jue Wang

The content contrastive loss enables the encoder to retain more available details.

Style Transfer

VDTR: Video Deblurring with Transformer

1 code implementation17 Apr 2022 Mingdeng Cao, Yanbo Fan, Yong Zhang, Jue Wang, Yujiu Yang

For multi-frame temporal modeling, we adapt Transformer to fuse multiple spatial features efficiently.

Deblurring Video Restoration

Deformable Video Transformer

no code implementations CVPR 2022 Jue Wang, Lorenzo Torresani

Video transformers have recently emerged as an effective alternative to convolutional networks for action classification.

Action Classification

Multi-Robot Active Mapping via Neural Bipartite Graph Matching

no code implementations CVPR 2022 Kai Ye, Siyan Dong, Qingnan Fan, He Wang, Li Yi, Fei Xia, Jue Wang, Baoquan Chen

Previous approaches either choose the frontier as the goal position via a myopic solution that hinders the time efficiency, or maximize the long-term value via reinforcement learning to directly regress the goal position, but does not guarantee the complete map construction.

Graph Matching Position +2

UV Volumes for Real-time Rendering of Editable Free-view Human Performance

1 code implementation CVPR 2023 Yue Chen, Xuan Wang, Xingyu Chen, Qi Zhang, Xiaoyu Li, Yu Guo, Jue Wang, Fei Wang

Neural volume rendering enables photo-realistic renderings of a human performer in free-view, a critical task in immersive VR/AR applications.

Unsupervised Pre-training for Temporal Action Localization Tasks

1 code implementation CVPR 2022 Can Zhang, Tianyu Yang, Junwu Weng, Meng Cao, Jue Wang, Yuexian Zou

These pre-trained models can be sub-optimal for temporal localization tasks due to the inherent discrepancy between video-level classification and clip-level localization.

Contrastive Learning Representation Learning +4

VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

4 code implementations23 Mar 2022 Zhan Tong, Yibing Song, Jue Wang, LiMin Wang

Pre-training video transformers on extra large-scale datasets is generally required to achieve premier performance on relatively small datasets.

4k Action Classification +3

Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection

1 code implementation CVPR 2022 Liang Chen, Yong Zhang, Yibing Song, Lingqiao Liu, Jue Wang

Following this principle, we propose to enrich the "diversity" of forgeries by synthesizing augmented forgeries with a pool of forgery configurations and strengthen the "sensitivity" to the forgeries by enforcing the model to predict the forgery configurations.

DeepFake Detection Face Swapping +1

LAS-AT: Adversarial Training with Learnable Attack Strategy

1 code implementation CVPR 2022 Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao

In this paper, we propose a novel framework for adversarial training by introducing the concept of "learnable attack strategy", dubbed LAS-AT, which learns to automatically produce attack strategies to improve the model robustness.

StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN

1 code implementation8 Mar 2022 Fei Yin, Yong Zhang, Xiaodong Cun, Mingdeng Cao, Yanbo Fan, Xuan Wang, Qingyan Bai, Baoyuan Wu, Jue Wang, Yujiu Yang

Our framework elevates the resolution of the synthesized talking face to 1024*1024 for the first time, even though the training dataset has a lower resolution.

Facial Editing Talking Face Generation +1

Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations

1 code implementation16 Feb 2022 Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, Pengtao Xie

Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.

Efficient ViTs

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

no code implementations27 Jan 2022 Peng Wei, Kun Guo, Ye Li, Jue Wang, Wei Feng, Shi Jin, Ning Ge, Ying-Chang Liang

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond.

Edge-computing reinforcement-learning +1

Exploring Denoised Cross-Video Contrast for Weakly-Supervised Temporal Action Localization

no code implementations CVPR 2022 Jingjing Li, Tianyu Yang, Wei Ji, Jue Wang, Li Cheng

Inspired by recent success in unsupervised contrastive representation learning, we propose a novel denoised cross-video contrastive algorithm, aiming to enhance the feature discrimination ability of video snippets for accurate temporal action localization in the weakly-supervised setting.

Contrastive Learning Denoising +4

PONet: Robust 3D Human Pose Estimation via Learning Orientations Only

no code implementations21 Dec 2021 Jue Wang, Shaoli Huang, Xinchao Wang, DaCheng Tao

Conventional 3D human pose estimation relies on first detecting 2D body keypoints and then solving the 2D to 3D correspondence problem. Despite the promising results, this learning paradigm is highly dependent on the quality of the 2D keypoint detector, which is inevitably fragile to occlusions and out-of-image absences. In this paper, we propose a novel Pose Orientation Net (PONet) that is able to robustly estimate 3D pose by learning orientations only, hence bypassing the error-prone keypoint detector in the absence of image evidence.

3D Human Pose Estimation

DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings

1 code implementation10 Dec 2021 Chaochen Gao, Xing Wu, Peng Wang, Jue Wang, Liangjun Zang, Zhongyuan Wang, Songlin Hu

To tackle that, we propose an effective knowledge distillation framework for contrastive sentence embeddings, termed DistilCSE.

Contrastive Learning Knowledge Distillation +5

Revitalizing CNN Attention via Transformers in Self-Supervised Visual Representation Learning

1 code implementation NeurIPS 2021 Chongjian Ge, Youwei Liang, Yibing Song, Jianbo Jiao, Jue Wang, Ping Luo

Motivated by the transformers that explore visual attention effectively in recognition scenarios, we propose a CNN Attention REvitalization (CARE) framework to train attentive CNN encoders guided by transformers in SSL.

Image Classification object-detection +3

Hallucinated Neural Radiance Fields in the Wild

no code implementations CVPR 2022 Xingyu Chen, Qi Zhang, Xiaoyu Li, Yue Chen, Ying Feng, Xuan Wang, Jue Wang

This paper studies the problem of hallucinated NeRF: i. e., recovering a realistic NeRF at a different time of day from a group of tourism images.

Hallucination Novel View Synthesis

Deblur-NeRF: Neural Radiance Fields from Blurry Images

1 code implementation CVPR 2022 Li Ma, Xiaoyu Li, Jing Liao, Qi Zhang, Xuan Wang, Jue Wang, Pedro V. Sander

We demonstrate that our method can be used on both camera motion blur and defocus blur: the two most common types of blur in real scenes.

3D Scene Reconstruction Novel View Synthesis

TransAug: Translate as Augmentation for Sentence Embeddings

no code implementations30 Oct 2021 Jue Wang, Haofan Wang, Xing Wu, Chaochen Gao, Debing Zhang

In this paper, we present TransAug (Translate as Augmentation), which provide the first exploration of utilizing translated sentence pairs as data augmentation for text, and introduce a two-stage paradigm to advances the state-of-the-art sentence embeddings.

Contrastive Learning Data Augmentation +4

Boosting Fast Adversarial Training with Learnable Adversarial Initialization

no code implementations11 Oct 2021 Xiaojun Jia, Yong Zhang, Baoyuan Wu, Jue Wang, Xiaochun Cao

Adversarial training (AT) has been demonstrated to be effective in improving model robustness by leveraging adversarial examples for training.

Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning

1 code implementation11 Oct 2021 Chongjian Ge, Youwei Liang, Yibing Song, Jianbo Jiao, Jue Wang, Ping Luo

Motivated by the transformers that explore visual attention effectively in recognition scenarios, we propose a CNN Attention REvitalization (CARE) framework to train attentive CNN encoders guided by transformers in SSL.

Image Classification object-detection +3

Motion-aware Contrastive Video Representation Learning via Foreground-background Merging

1 code implementation CVPR 2022 Shuangrui Ding, Maomao Li, Tianyu Yang, Rui Qian, Haohang Xu, Qingyi Chen, Jue Wang, Hongkai Xiong

To alleviate such bias, we propose \textbf{F}oreground-b\textbf{a}ckground \textbf{Me}rging (FAME) to deliberately compose the moving foreground region of the selected video onto the static background of others.

Action Recognition Contrastive Learning +1

EViT: Expediting Vision Transformers via Token Reorganizations

1 code implementation ICLR 2022 Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, Pengtao Xie

Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.

Robust Physical-World Attacks on Face Recognition

no code implementations20 Sep 2021 Xin Zheng, Yanbo Fan, Baoyuan Wu, Yong Zhang, Jue Wang, Shirui Pan

Face recognition has been greatly facilitated by the development of deep neural networks (DNNs) and has been widely applied to many safety-critical applications.

Adversarial Attack Adversarial Robustness +1

High-Fidelity GAN Inversion for Image Attribute Editing

1 code implementation CVPR 2022 Tengfei Wang, Yong Zhang, Yanbo Fan, Jue Wang, Qifeng Chen

With a low bit-rate latent code, previous works have difficulties in preserving high-fidelity details in reconstructed and edited images.

Attribute Generative Adversarial Network +2

Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization

2 code implementations13 Sep 2021 Jingtang Liang, Xiaodong Cun, Chi-Man Pun, Jue Wang

To this end, we propose a novel spatial-separated curve rendering network(S$^2$CRNet) for efficient and high-resolution image harmonization for the first time.

Image Harmonization Image-to-Image Translation +2

Joint Multi-User Communication and Sensing Exploiting Both Signal and Environment Sparsity

no code implementations6 Sep 2021 Xin Tong, Zhaoyang Zhang, Jue Wang, Chongwen Huang, Merouane Debbah

As a potential technology feature for 6G wireless networks, the idea of sensing-communication integration requires the system not only to complete reliable multi-user communication but also to achieve accurate environment sensing.

object-detection Object Detection

End-to-End Adaptive Monte Carlo Denoising and Super-Resolution

no code implementations16 Aug 2021 Xinyue Wei, HaoZhi Huang, Yujin Shi, Hongliang Yuan, Li Shen, Jue Wang

We show in this work that Monte Carlo path tracing can be further accelerated by joint super-resolution and denoising (SRD) in post-processing.

Denoising Super-Resolution

Generalized One-Class Learning Using Pairs of Complementary Classifiers

no code implementations24 Jun 2021 Anoop Cherian, Jue Wang

One-class learning is the classic problem of fitting a model to the data for which annotations are available only for a single class.

Anomaly Detection

Long-Short Temporal Contrastive Learning of Video Transformers

no code implementations CVPR 2022 Jue Wang, Gedas Bertasius, Du Tran, Lorenzo Torresani

Our approach, named Long-Short Temporal Contrastive Learning (LSTCL), enables video transformers to learn an effective clip-level representation by predicting temporal context captured from a longer temporal extent.

Action Recognition Contrastive Learning +1

Effective Slot Filling via Weakly-Supervised Dual-Model Learning

1 code implementation AAAI 2021 Jue Wang, Ke Chen, Lidan Shou, Sai Wu, Gang Chen

By using some particular weakly-labeled data, namely the plain phrases included in sentences, we propose a weaklysupervised slot filling approach.

slot-filling Slot Filling +1

DeepOIS: Gyroscope-Guided Deep Optical Image Stabilizer Compensation

1 code implementation27 Jan 2021 Haipeng Li, Shuaicheng Liu, Jue Wang

In this work, we propose a deep network that compensates the motions caused by the OIS, such that the gyroscopes can be used for image alignment on the OIS cameras.

Towards Accurate Active Camera Localization

1 code implementation8 Dec 2020 Qihang Fang, Yingda Yin, Qingnan Fan, Fei Xia, Siyan Dong, Sheng Wang, Jue Wang, Leonidas Guibas, Baoquan Chen

These approaches localize the camera in the discrete pose space and are agnostic to the localization-driven scene property, which restricts the camera pose accuracy in the coarse scale.

Camera Localization Pose Estimation +1

UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learning

2 code implementations CVPR 2021 Kunming Luo, Chuan Wang, Shuaicheng Liu, Haoqiang Fan, Jue Wang, Jian Sun

By integrating these two components together, our method achieves the best performance for unsupervised optical flow learning on multiple leading benchmarks, including MPI-SIntel, KITTI 2012 and KITTI 2015.

Optical Flow Estimation

Practical Deep Raw Image Denoising on Mobile Devices

1 code implementation ECCV 2020 Yuzhi Wang, Haibin Huang, Qin Xu, Jiaming Liu, Yiqun Liu, Jue Wang

Deep learning-based image denoising approaches have been extensively studied in recent years, prevailing in many public benchmark datasets.

Efficient Neural Network Image Denoising

High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies

2 code implementations12 Oct 2020 Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang, Sheng Wang, Xuefei Zhe, Di Kang, HaoZhi Huang, Xinwei Jiang, Jue Wang, Dong Yu, Zhengyou Zhang

We present a fully automatic system that can produce high-fidelity, photo-realistic 3D digital human heads with a consumer RGB-D selfie camera.

Vocal Bursts Intensity Prediction

Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders

2 code implementations EMNLP 2020 Jue Wang, Wei Lu

In this work, we propose the novel {\em table-sequence encoders} where two different encoders -- a table encoder and a sequence encoder are designed to help each other in the representation learning process.

Joint Entity and Relation Extraction named-entity-recognition +5

Distributed ADMM with Synergetic Communication and Computation

no code implementations29 Sep 2020 Zhuojun Tian, Zhaoyang Zhang, Jue Wang, Xiaoming Chen, Wei Wang, Huaiyu Dai

In this paper, we propose a novel distributed alternating direction method of multipliers (ADMM) algorithm with synergetic communication and computation, called SCCD-ADMM, to reduce the total communication and computation cost of the system.

Learning Color Compatibility in Fashion Outfits

no code implementations5 Jul 2020 Heming Zhang, Xuewen Yang, Jianchao Tan, Chi-Hao Wu, Jue Wang, C. -C. Jay Kuo

Color compatibility is important for evaluating the compatibility of a fashion outfit, yet it was neglected in previous studies.

graph construction

Pyramid: A Layered Model for Nested Named Entity Recognition

2 code implementations ACL 2020 Jue Wang, Lidan Shou, Ke Chen, Gang Chen

Its hidden state at layer l represents an l-gram in the input text, which is labeled only if its corresponding text region represents a complete entity mention.

named-entity-recognition Named Entity Recognition +2

Enabling 5G on the Ocean: A Hybrid Satellite-UAV-Terrestrial Network Solution

no code implementations1 Jun 2020 Xiangling Li, Wei Feng, Jue Wang, Yunfei Chen, Ning Ge, Cheng-Xiang Wang

In this paper, we study the integration of UAVs with existing MCNs, and investigate the potential gains of hybrid satellite-UAV-terrestrial networks for maritime coverage.


Spatio-Temporal Ranked-Attention Networks for Video Captioning

no code implementations17 Jan 2020 Anoop Cherian, Jue Wang, Chiori Hori, Tim K. Marks

To this end, we propose a Spatio-Temporal and Temporo-Spatial (STaTS) attention model which, conditioned on the language state, hierarchically combines spatial and temporal attention to videos in two different orders: (i) a spatio-temporal (ST) sub-model, which first attends to regions that have temporal evolution, then temporally pools the features from these regions; and (ii) a temporo-spatial (TS) sub-model, which first decides a single frame to attend to, then applies spatial attention within that frame.

Video Captioning

DeepMeshFlow: Content Adaptive Mesh Deformation for Robust Image Registration

no code implementations11 Dec 2019 Nianjin Ye, Chuan Wang, Shuaicheng Liu, Lanpeng Jia, Jue Wang, Yongqing Cui

Deep homography methods, on the other hand, are free from such problem by learning deep features for robust performance.

Denoising Homography Estimation +2

Not All Parts Are Created Equal: 3D Pose Estimation by Modeling Bi-Directional Dependencies of Body Parts

no code implementations ICCV 2019 Jue Wang, Shaoli Huang, Xinchao Wang, Dacheng Tao

We model parts with higher DOFs like the elbows, as dependent components of the corresponding parts with lower DOFs like the torso, of which the 3D locations can be more reliably estimated.

3D Human Pose Estimation 3D Pose Estimation

Disentangled Image Matting

no code implementations ICCV 2019 Shaofan Cai, Xiaoshuai Zhang, Haoqiang Fan, Haibin Huang, Jiangyu Liu, Jiaming Liu, Jiaying Liu, Jue Wang, Jian Sun

Most previous image matting methods require a roughly-specificed trimap as input, and estimate fractional alpha values for all pixels that are in the unknown region of the trimap.

Image Matting

Discriminative Video Representation Learning Using Support Vector Classifiers

no code implementations5 Sep 2019 Jue Wang, Anoop Cherian

With the features from the video as a positive bag and the irrelevant features as the negative bag, we cast an objective to learn a (nonlinear) hyperplane that separates the unknown useful features from the rest in a multiple instance learning formulation within a support vector machine setup.

Action Recognition In Videos Multiple Instance Learning +1

GODS: Generalized One-class Discriminative Subspaces for Anomaly Detection

no code implementations ICCV 2019 Jue Wang, Anoop Cherian

One-class learning is the classic problem of fitting a model to data for which annotations are available only for a single class.

Anomaly Detection Novelty Detection +1

Semi-supervised Skin Detection by Network with Mutual Guidance

no code implementations ICCV 2019 Yi He, Jiayuan Shi, Chuan Wang, Haibin Huang, Jiaming Liu, Guanbin Li, Risheng Liu, Jue Wang

In this paper we present a new data-driven method for robust skin detection from a single human portrait image.

Not All Parts Are Created Equal: 3D Pose Estimation by Modelling Bi-directional Dependencies of Body Parts

no code implementations20 May 2019 Jue Wang, Shaoli Huang, Xinchao Wang, DaCheng Tao

We model parts with higher DOFs like the elbows, as dependent components of the corresponding parts with lower DOFs like the torso, of which the 3D locations can be more reliably estimated.

3D Pose Estimation Attribute

Frame-Recurrent Video Inpainting by Robust Optical Flow Inference

no code implementations8 May 2019 Yifan Ding, Chuan Wang, Haibin Huang, Jiaming Liu, Jue Wang, Liqiang Wang

Compared with image inpainting, performing this task on video presents new challenges such as how to preserving temporal consistency and spatial details, as well as how to handle arbitrary input video size and length fast and efficiently.

Image Inpainting Optical Flow Estimation +1

Semi-Supervised Few-Shot Learning for Dual Question-Answer Extraction

no code implementations8 Apr 2019 Jue Wang, Ke Chen, Lidan Shou, Sai Wu, Sharad Mehrotra

In this paper, we redefine the problem as question-answer extraction, and present SAMIE: Self-Asking Model for Information Ixtraction, a semi-supervised model which dually learns to ask and to answer questions by itself.

Clustering Few-Shot Learning +1

When AWGN-based Denoiser Meets Real Noises

2 code implementations6 Apr 2019 Yuqian Zhou, Jianbo Jiao, Haibin Huang, Yang Wang, Jue Wang, Honghui Shi, Thomas Huang

In this paper, we propose a novel approach to boost the performance of a real image denoiser which is trained only with synthetic pixel-independent noise data dominated by AWGN.


GIF2Video: Color Dequantization and Temporal Interpolation of GIF images

no code implementations CVPR 2019 Yang Wang, Haibin Huang, Chuan Wang, Tong He, Jue Wang, Minh Hoai

In this paper, we propose GIF2Video, the first learning-based method for enhancing the visual quality of GIFs in the wild.


Learning Discriminative Video Representations Using Adversarial Perturbations

no code implementations ECCV 2018 Jue Wang, Anoop Cherian

As the perturbed features belong to data classes that are likely to be confused with the original features, the discriminative subspace will characterize parts of the feature space that are more representative of the original data, and thus may provide robust video representations.

Binary Classification Riemannian optimization +1

Deep Portrait Image Completion and Extrapolation

no code implementations23 Aug 2018 Xian Wu, Rui-Long Li, Fang-Lue Zhang, Jian-Cheng Liu, Jue Wang, Ariel Shamir, Shi-Min Hu

We evaluate our method on public portrait image datasets, and show that it outperforms other state-of-art general image completion methods.


Contrastive Video Representation Learning via Adversarial Perturbations

no code implementations ECCV 2018 Jue Wang, Anoop Cherian

In this paper, we propose to use such perturbations within a novel contrastive learning setup to build negative samples, which are then used to produce improved video representations.

Action Recognition Binary Classification +4

Video Inpainting by Jointly Learning Temporal Structure and Spatial Details

no code implementations22 Jun 2018 Chuan Wang, Haibin Huang, Xiaoguang Han, Jue Wang

We present a new data-driven video inpainting method for recovering missing regions of video frames.

Video Inpainting

DocUNet: Document Image Unwarping via a Stacked U-Net

1 code implementation CVPR 2018 Ke Ma, Zhixin Shu, Xue Bai, Jue Wang, Dimitris Samaras

The network is trained on this dataset with various data augmentations to improve its generalization ability.

Ranked #4 on SSIM on DocUNet (using extra training data)

Local Distortion MS-SSIM +1

Video Representation Learning Using Discriminative Pooling

no code implementations CVPR 2018 Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould

In an attempt to tackle this problem, we propose discriminative pooling, based on the notion that among the deep features generated on all short clips, there is at least one that characterizes the action.

Action Recognition In Videos Multiple Instance Learning +2

Scale-recurrent Network for Deep Image Deblurring

4 code implementations CVPR 2018 Xin Tao, Hongyun Gao, Yi Wang, Xiaoyong Shen, Jue Wang, Jiaya Jia

In single image deblurring, the "coarse-to-fine" scheme, i. e. gradually restoring the sharp image on different resolutions in a pyramid, is very successful in both traditional optimization-based methods and recent neural-network-based approaches.

Ranked #3 on Image Deblurring on GoPro (Params (M) metric, using extra training data)

Deblurring Image Deblurring +1

Human Action Forecasting by Learning Task Grammars

no code implementations19 Sep 2017 Tengda Han, Jue Wang, Anoop Cherian, Stephen Gould

For effective human-robot interaction, it is important that a robotic assistant can forecast the next action a human will consider in a given task.

Action Recognition Temporal Action Localization

Deep Video Deblurring for Hand-Held Cameras

1 code implementation CVPR 2017 Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, Oliver Wang

We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.

Deblurring Image Deblurring +1

Zero-order Reverse Filtering

1 code implementation ICCV 2017 Xin Tao, Chao Zhou, Xiaoyong Shen, Jue Wang, Jiaya Jia

In this paper, we study an unconventional but practically meaningful reversibility problem of commonly used image filters.

Action Representation Using Classifier Decision Boundaries

no code implementations6 Apr 2017 Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould

Applying multiple instance learning in an SVM setup, we use the parameters of this separating hyperplane as a descriptor for the video.

Action Recognition Multiple Instance Learning +1

Removing Shadows from Images of Documents

2 code implementations ACCV 2017 Steve Bako, Soheil Darabi, Eli Shechtman, Jue Wang, Kalyan Sunkavalli, Pradeep Sen

In this work, we automatically detect and remove distracting shadows from photographs of documents and other text-based items.

Document Shadow Removal

Ordered Pooling of Optical Flow Sequences for Action Recognition

no code implementations12 Jan 2017 Jue Wang, Anoop Cherian, Fatih Porikli

Training of Convolutional Neural Networks (CNNs) on long video sequences is computationally expensive due to the substantial memory requirements and the massive number of parameters that deep architectures demand.

Action Recognition Optical Flow Estimation +1

Deep Video Deblurring

1 code implementation25 Nov 2016 Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, Oliver Wang

We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.

Deblurring Image Deblurring +1

Automatic Fence Segmentation in Videos of Dynamic Scenes

no code implementations CVPR 2016 Renjiao Yi, Jue Wang, Ping Tan

We present a fully automatic approach to detect and segment fence-like occluders from a video clip.

Appearance Harmonization for Single Image Shadow Removal

no code implementations21 Mar 2016 Liqian Ma, Jue Wang, Eli Shechtman, Kalyan Sunkavalli, Shi-Min Hu

In this work we propose a fully automatic shadow region harmonization approach that improves the appearance compatibility of the de-shadowed region as typically produced by previous methods.

Image Generation Image Shadow Removal +1

Segmentation Rectification for Video Cutout via One-Class Structured Learning

no code implementations16 Feb 2016 Junyan Wang, Sai-Kit Yeung, Jue Wang, Kun Zhou

Comprehensive experiments on both RGB and RGB-D data demonstrate that our simple and effective method significantly outperforms the segmentation propagation methods adopted in the state-of-the-art video cutout systems, and the results also suggest the potential usefulness of our method in image cutout system.


Blind Optical Aberration Correction by Exploring Geometric and Visual Priors

no code implementations CVPR 2015 Tao Yue, Jinli Suo, Jue Wang, Xun Cao, Qionghai Dai

Furthermore, by investigating the visual artifacts of aberration degenerated images captured by consumer-level cameras, the non-uniform distribution of sharpness across color channels and the image lattice is exploited as visual priors, resulting in a novel strategy to utilize the guidance from the sharpest channel and local image regions to improve the overall performance and robustness.

Supervised Semantic Gradient Extraction Using Linear-Time Optimization

no code implementations CVPR 2013 Shulin Yang, Jue Wang, Linda Shapiro

This paper proposes a new supervised semantic edge and gradient extraction approach, which allows the user to roughly scribble over the desired region to extract semantically-dominant and coherent edges in it.

Edge Detection

Handling Noise in Single Image Deblurring Using Directional Filters

no code implementations CVPR 2013 Lin Zhong, Sunghyun Cho, Dimitris Metaxas, Sylvain Paris, Jue Wang

Based on this observation, our method applies a series of directional filters at different orientations to the input image, and estimates an accurate Radon transform of the blur kernel from each filtered image.

Deblurring Image Deblurring +2

Unsupervised Template Learning for Fine-Grained Object Recognition

no code implementations NeurIPS 2012 Shulin Yang, Liefeng Bo, Jue Wang, Linda G. Shapiro

It differs from recognition of basic categories, such as humans, tables, and computers, in that there are global similarities in shape or structure shared within a category, and the differences are in the details of the object parts.

Object Object Recognition

Avoiding False Positive in Multi-Instance Learning

no code implementations NeurIPS 2010 Yanjun Han, Qing Tao, Jue Wang

In multi-instance learning, there are two kinds of prediction failure, i. e., false negative and false positive.

Cannot find the paper you are looking for? You can Submit a new open access paper.