Search Results for author: Yun Fu

Found 140 papers, 77 papers with code

LatticeNet: Towards Lightweight Image Super-resolution with Lattice Block

2 code implementations ECCV 2020 Xiaotong Luo, Yuan Xie, Yulun Zhang, Yanyun Qu, Cuihua Li, Yun Fu

Drawing lessons from lattice filter bank, we design the lattice block (LB) in which two butterfly structures are applied to combine two RBs.

Image Super-Resolution

Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering

no code implementations16 Apr 2024 Zaid Khan, Yun Fu

We find that neighborhood consistency can be used to identify model responses to visual questions that are likely unreliable, even in adversarial settings or settings that are out-of-distribution to the proxy model.

Language Modelling Question Answering +1

Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement

no code implementations6 Apr 2024 Zaid Khan, Vijay Kumar BG, Samuel Schulter, Yun Fu, Manmohan Chandraker

We propose a method where we exploit existing annotations for a vision-language task to improvise a coarse reward signal for that task, treat the LLM as a policy, and apply reinforced self-training to improve the visual program synthesis ability of the LLM for that task.

object-detection Object Detection +4

OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising

1 code implementation2 Apr 2024 Haichao Zhang, Yi Xu, HongSheng Lu, Takayuki Shimizu, Yun Fu

By enhancing trajectory prediction accuracy and addressing the challenges of out-of-sight objects, our work significantly contributes to improving the safety and reliability of autonomous driving in complex environments.

Autonomous Driving Decision Making +2

Adapting to Length Shift: FlexiLength Network for Trajectory Prediction

no code implementations31 Mar 2024 Yi Xu, Yun Fu

Trajectory prediction plays an important role in various applications, including autonomous driving, robotics, and scene understanding.

Autonomous Driving Scene Understanding +1

Efficient Modulation for Vision Networks

1 code implementation29 Mar 2024 Xu Ma, Xiyang Dai, Jianwei Yang, Bin Xiao, Yinpeng Chen, Yun Fu, Lu Yuan

We demonstrate that the modulation mechanism is particularly well suited for efficient networks and further tailor the modulation design by proposing the efficient modulation (EfficientMod) block, which is considered the essential building block for our networks.

Rewrite the Stars

2 code implementations29 Mar 2024 Xu Ma, Xiyang Dai, Yue Bai, Yizhou Wang, Yun Fu

Recent studies have drawn attention to the untapped potential of the "star operation" (element-wise multiplication) in network design.

Don't Judge by the Look: Towards Motion Coherent Video Representation

1 code implementation14 Mar 2024 Yitian Zhang, Yue Bai, Huan Wang, Yizhou Wang, Yun Fu

Current training pipelines in object recognition neglect Hue Jittering when doing data augmentation as it not only brings appearance changes that are detrimental to classification, but also the implementation is inefficient in practice.

Data Augmentation Object Recognition +2

VaQuitA: Enhancing Alignment in LLM-Assisted Video Understanding

no code implementations4 Dec 2023 Yizhou Wang, Ruiyi Zhang, Haoliang Wang, Uttaran Bhattacharya, Yun Fu, Gang Wu

Recent advancements in language-model-based video understanding have been progressing at a remarkable pace, spurred by the introduction of Large Language Models (LLMs).

Language Modelling Question Answering +2

Layout Sequence Prediction From Noisy Mobile Modality

no code implementations9 Oct 2023 Haichao Zhang, Yi Xu, HongSheng Lu, Takayuki Shimizu, Yun Fu

In summary, our approach offers a promising solution to the challenges faced by layout sequence and trajectory prediction models in real-world settings, paving the way for utilizing sensor data from mobile phones to accurately predict pedestrian bounding box trajectories.

Autonomous Driving Denoising +1

Latent Graph Inference with Limited Supervision

no code implementations NeurIPS 2023 Jianglin Lu, Yi Xu, Huan Wang, Yue Bai, Yun Fu

We begin by defining the pivotal nodes as $k$-hop starved nodes, which can be identified based on a given adjacency matrix.

Camouflaged Image Synthesis Is All You Need to Boost Camouflaged Detection

no code implementations13 Aug 2023 Haichao Zhang, Can Qin, Yu Yin, Yun Fu

This approach can serve as a plug-and-play data generation and augmentation module for existing camouflaged object detection tasks and provides a novel way to introduce more diversity and distributions into current camouflage datasets.

Image Generation Object +2

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

1 code implementation CVPR 2023 Yi Xu, Armin Bazarjani, Hyung-gun Chi, Chiho Choi, Yun Fu

As far as we know, this is the first work to address the lack of benchmarks and techniques for trajectory imputation and prediction in a unified manner.

Imputation Trajectory Prediction

Frame Flexible Network

2 code implementations CVPR 2023 Yitian Zhang, Yue Bai, Chang Liu, Huan Wang, Sheng Li, Yun Fu

To fix this issue, we propose a general framework, named Frame Flexible Network (FFN), which not only enables the model to be evaluated at different frames to adjust its computation, but also reduces the memory costs of storing multiple models significantly.

Video Recognition

Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning

1 code implementation21 Mar 2023 Zaid Khan, Yun Fu

We find that a minimal set of parameter updates ($<$7%) can achieve the same performance as full-model training, and updating specific components ($<$1% of parameters) can match 75% of full-model training.

Language Modelling Transfer Learning

GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation

1 code implementation ICCV 2023 Can Qin, Ning Yu, Chen Xing, Shu Zhang, Zeyuan Chen, Stefano Ermon, Yun Fu, Caiming Xiong, ran Xu

Empirical results show that GlueNet can be trained efficiently and enables various capabilities beyond previous state-of-the-art models: 1) multilingual language models such as XLM-Roberta can be aligned with existing T2I models, allowing for the generation of high-quality images from captions beyond English; 2) GlueNet can align multi-modal encoders such as AudioCLIP with the Stable Diffusion model, enabling sound-to-image generation; 3) it can also upgrade the current text encoder of the latent diffusion model for challenging case generation.

Image Generation

Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution

2 code implementations ICCV 2023 Jiamian Wang, Huan Wang, Yulun Zhang, Yun Fu, Zhiqiang Tao

Second, existing pruning methods generally operate upon a pre-trained network for the sparse structure determination, hard to get rid of dense model training in the traditional SR paradigm.

Image Super-Resolution Network Pruning

Image as Set of Points

2 code implementations2 Mar 2023 Xu Ma, Yuqian Zhou, Huan Wang, Can Qin, Bin Sun, Chang Liu, Yun Fu

Context clusters (CoCs) view an image as a set of unorganized points and extract features via simplified clustering algorithm.


Explainable Anomaly Detection in Images and Videos: A Survey

1 code implementation13 Feb 2023 Yizhou Wang, Dongliang Guo, Sheng Li, Octavia Camps, Yun Fu

This paper provides the first survey concentrated on explainable visual anomaly detection methods.

Anomaly Detection

Making Reconstruction-based Method Great Again for Video Anomaly Detection

1 code implementation28 Jan 2023 Yizhou Wang, Can Qin, Yue Bai, Yi Xu, Xu Ma, Yun Fu

With the same perturbation magnitude, the testing reconstruction error of the normal frames lowers more than that of the abnormal frames, which contributes to mitigating the overfitting problem of reconstruction.

Anomaly Detection Optical Flow Estimation +1

Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning

2 code implementations12 Jan 2023 Huan Wang, Can Qin, Yue Bai, Yun Fu

The state of neural network pruning has been noticed to be unclear and even confusing for a while, largely due to "a lack of standardized benchmarks and metrics" [3].

Fairness Network Pruning

Efficient Converted Spiking Neural Network for 3D and 2D Classification

no code implementations ICCV 2023 Yuxiang Lan, Yachao Zhang, Xu Ma, Yanyun Qu, Yun Fu

Spiking Neural Networks (SNNs) have attracted enormous research interest due to their low-power and biologically plausible nature.

Image Classification Point Cloud Classification

A Close Look at Spatial Modeling: From Attention to Convolution

1 code implementation23 Dec 2022 Xu Ma, Huan Wang, Can Qin, Kunpeng Li, Xingchen Zhao, Jie Fu, Yun Fu

Vision Transformers have shown great promise recently for many vision tasks due to the insightful architecture design and attention mechanism.

Instance Segmentation object-detection +2

Real-Time Neural Light Field on Mobile Devices

1 code implementation CVPR 2023 Junli Cao, Huan Wang, Pavlo Chemerys, Vladislav Shakhrai, Ju Hu, Yun Fu, Denys Makoviichuk, Sergey Tulyakov, Jian Ren

Nevertheless, to reach a similar rendering quality as NeRF, the network in NeLF is designed with intensive computation, which is not mobile-friendly.

Neural Rendering Novel View Synthesis

NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation

no code implementations CVPR 2023 Yu Yin, Kamran Ghasedi, HsiangTao Wu, Jiaolong Yang, Xin Tong, Yun Fu

Furthermore, our method leverages explicit and implicit 3D regularizations using the in-domain neighborhood samples around the optimized latent code to remove geometrical and visual artifacts.

Image Animation

Look More but Care Less in Video Recognition

1 code implementation18 Nov 2022 Yitian Zhang, Yue Bai, Huan Wang, Yi Xu, Yun Fu

To tackle this problem, we propose Ample and Focal Network (AFNet), which is composed of two branches to utilize more frames but with less computation.

Action Recognition Video Recognition

Parameter-Efficient Masking Networks

1 code implementation13 Oct 2022 Yue Bai, Huan Wang, Xu Ma, Yitian Zhang, Zhiqiang Tao, Yun Fu

We validate the potential of PEMN learning masks on random weights with limited unique values and test its effectiveness for a new compression paradigm based on different network architectures.

Model Compression

Trainability Preserving Neural Pruning

1 code implementation25 Jul 2022 Huan Wang, Yun Fu

Moreover, results on ImageNet-1K with ResNets suggest that TPP consistently performs more favorably against other top-performing structured pruning approaches.

Network Pruning

Towards Layer-wise Image Vectorization

1 code implementation CVPR 2022 Xu Ma, Yuqian Zhou, Xingqian Xu, Bin Sun, Valerii Filev, Nikita Orlov, Yun Fu, Humphrey Shi

Image rasterization is a mature technique in computer graphics, while image vectorization, the reverse path of rasterization, remains a major challenge.

Test-time Fourier Style Calibration for Domain Generalization

1 code implementation13 May 2022 Xingchen Zhao, Chang Liu, Anthony Sicilia, Seong Jae Hwang, Yun Fu

Thus, it is still possible that those methods can overfit to source domains and perform poorly on target domains.

Domain Generalization

R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis

1 code implementation31 Mar 2022 Huan Wang, Jian Ren, Zeng Huang, Kyle Olszewski, Menglei Chai, Yun Fu, Sergey Tulyakov

On the other hand, Neural Light Field (NeLF) presents a more straightforward representation over NeRF in novel view synthesis -- the rendering of a pixel amounts to one single forward pass without ray-marching.

Novel View Synthesis

Single-Stream Multi-Level Alignment for Vision-Language Pretraining

1 code implementation27 Mar 2022 Zaid Khan, Vijay Kumar BG, Xiang Yu, Samuel Schulter, Manmohan Chandraker, Yun Fu

Self-supervised vision-language pretraining from pure images and text with a contrastive loss is effective, but ignores fine-grained alignment due to a dual-stream architecture that aligns image and text representations only on a global level.

Question Answering Referring Expression +4

Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution

1 code implementation16 Mar 2022 Bin Sun, Yulun Zhang, Songyao Jiang, Yun Fu

In this paper, we propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task.

Image Super-Resolution

Adaptive Trajectory Prediction via Transferable GNN

no code implementations CVPR 2022 Yi Xu, Lichen Wang, Yizhou Wang, Yun Fu

To the best of our knowledge, our work is the pioneer which fills the gap in benchmarks and techniques for practical pedestrian trajectory prediction across different domains.

Autonomous Driving Pedestrian Trajectory Prediction +2

Dual Lottery Ticket Hypothesis

1 code implementation ICLR 2022 Yue Bai, Huan Wang, Zhiqiang Tao, Kunpeng Li, Yun Fu

In this work, we regard the winning ticket from LTH as the subnetwork which is in trainable condition and its performance as our benchmark, then go from a complementary direction to articulate the Dual Lottery Ticket Hypothesis (DLTH): Randomly selected subnetworks from a randomly initialized dense network can be transformed into a trainable condition and achieve admirable performance compared with LTH -- random tickets in a given lottery pool can be transformed into winning tickets.

Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework

1 code implementation ICLR 2022 Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, Yun Fu

We notice that detailed local geometrical information probably is not the key to point cloud analysis -- we introduce a pure residual MLP network, called PointMLP, which integrates no sophisticated local geometrical extractors but still performs very competitively.

3D Point Cloud Classification Point Cloud Segmentation

Semi-supervised Domain Adaptive Structure Learning

1 code implementation12 Dec 2021 Can Qin, Lichen Wang, Qianqian Ma, Yu Yin, Huan Wang, Yun Fu

Semi-supervised domain adaptation (SSDA) is quite a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains.

Domain Adaptation Representation Learning +1

Slow Learning and Fast Inference: Efficient Graph Similarity Computation via Knowledge Distillation

1 code implementation NeurIPS 2021 Can Qin, Handong Zhao, Lichen Wang, Huan Wang, Yulun Zhang, Yun Fu

For slow learning of graph similarity, this paper proposes a novel early-fusion approach by designing a co-attention-based feature fusion network on multilevel GNN features.

Anomaly Detection Graph Similarity +3

Aligned Structured Sparsity Learning for Efficient Image Super-Resolution

1 code implementation NeurIPS 2021 Yulun Zhang, Huan Wang, Can Qin, Yun Fu

To address the above issues, we propose aligned structured sparsity learning (ASSL), which introduces a weight normalization layer and applies $L_2$ regularization to the scale parameters for sparsity.

Image Super-Resolution Knowledge Distillation +3

SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation

1 code implementation25 Nov 2021 Yizhou Wang, Can Qin, Rongzhe Wei, Yi Xu, Yue Bai, Yun Fu

Next we add adversarial perturbation to the transformed features to decrease their softmax scores of the predicted labels and design anomaly scores based on the predictive uncertainties of the classifier on these perturbed features.

Pseudo Label Self-Supervised Anomaly Detection +3

The 5th Recognizing Families in the Wild Data Challenge: Predicting Kinship from Faces

1 code implementation31 Oct 2021 Joseph P. Robinson, Can Qin, Ming Shao, Matthew A. Turk, Rama Chellappa, Yun Fu

Recognizing Families In the Wild (RFIW), held as a data challenge in conjunction with the 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG), is a large-scale, multi-track visual kinship recognition evaluation.

Gesture Recognition Kinship Verification +1

Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble

2 code implementations12 Oct 2021 Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, Yun Fu

Current Sign Language Recognition (SLR) methods usually extract features via deep neural networks and suffer overfitting due to limited and noisy data.

Action Recognition Sign Language Recognition +1

Rethinking Again the Value of Network Pruning -- A Dynamical Isometry Perspective

no code implementations29 Sep 2021 Huan Wang, Can Qin, Yue Bai, Yun Fu

Several recent works questioned the value of inheriting weight in structured neural network pruning because they empirically found training from scratch can match or even outperform finetuning a pruned model.

Network Pruning

Structured Pruning Meets Orthogonality

no code implementations29 Sep 2021 Huan Wang, Yun Fu

In this paper, we present \emph{orthogonality preserving pruning} (OPP), a regularization-based structured pruning method that maintains the dynamical isometry during pruning.

Network Pruning

Understanding the Success of Knowledge Distillation -- A Data Augmentation Perspective

no code implementations29 Sep 2021 Huan Wang, Suhas Lohit, Michael Jeffrey Jones, Yun Fu

We achieve new state-of-the-art accuracy by using the original KD loss armed with stronger augmentation schemes, compared to existing state-of-the-art methods that employ more advanced distillation losses.

Active Learning Data Augmentation +1


no code implementations29 Sep 2021 Jiamian Wang, Yulun Zhang, Xin Yuan, Yun Fu, Zhiqiang Tao

As the inverse process of snapshot compressive imaging, the hyperspectral image (HSI) reconstruction takes the 2D measurement as input and posteriorly retrieves the captured 3D spatial-spectral signal.

Computational Efficiency Image Reconstruction

Where is the bottleneck in long-tailed classification?

no code implementations29 Sep 2021 Zaid Khan, Yun Fu

A commonly held belief in deep-learning based long-tailed classification is that the representations learned from long-tailed data are ”good enough” and the performance bottleneck is the classification head atop the representation learner.

Classification Data Augmentation

MemREIN: Rein the Domain Shift for Cross-Domain Few-Shot Learning

no code implementations29 Sep 2021 Yi Xu, Lichen Wang, Yizhou Wang, Can Qin, Yulun Zhang, Yun Fu

In this paper, we propose a novel framework, MemREIN, which considers Memorized, Restitution, and Instance Normalization for cross-domain few-shot learning.

Contrastive Learning cross-domain few-shot learning

A Simple and Efficient Reconstruction Backbone for Snapshot Compressive Imaging

1 code implementation17 Aug 2021 Jiamian Wang, Yulun Zhang, Xin Yuan, Yun Fu, Zhiqiang Tao

The emerging technology of snapshot compressive imaging (SCI) enables capturing high dimensional (HD) data in an efficient way.

Compressive Sensing Computational Efficiency +4

Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation

1 code implementation3 Aug 2021 Zaid Khan, Yun Fu

Our approach increases the amount of text available to the language model and distills the object-level information in complex images.

Language Modelling Multimodal Sentiment Analysis +4

Rethinking Adam: A Twofold Exponential Moving Average Approach

1 code implementation22 Jun 2021 Yizhou Wang, Yue Kang, Can Qin, Huan Wang, Yi Xu, Yulun Zhang, Yun Fu

The intuition is that gradient with momentum contains more accurate directional information and therefore its second moment estimation is a more favorable option for learning rate scaling than that of the raw gradient.

MR Image Super-Resolution With Squeeze and Excitation Reasoning Attention Network

no code implementations CVPR 2021 Yulun Zhang, Kai Li, Kunpeng Li, Yun Fu

They also fail to sense the entire space of the input, which is critical for high-quality MR image SR. To address those problems, we propose squeeze and excitation reasoning attention networks (SERAN) for accurate MR image SR. We propose to squeeze attention from global spatial information of the input and obtain global descriptors.

Image Super-Resolution

Dynamical Isometry: The Missing Ingredient for Neural Network Pruning

no code implementations12 May 2021 Huan Wang, Can Qin, Yue Bai, Yun Fu

This paper is meant to explain it through the lens of dynamical isometry [42].

Network Pruning

ECACL: A Holistic Framework for Semi-Supervised Domain Adaptation

1 code implementation ICCV 2021 Kai Li, Chang Liu, Handong Zhao, Yulun Zhang, Yun Fu

This paper studies Semi-Supervised Domain Adaptation (SSDA), a practical yet under-investigated research topic that aims to learn a model of good performance using unlabeled samples and a few labeled samples in the target domain, with the help of labeled samples from a source domain.

Data Augmentation Domain Adaptation +1

RPCL: A Framework for Improving Cross-Domain Detection with Auxiliary Tasks

no code implementations18 Apr 2021 Kai Li, Curtis Wigington, Chris Tensmeyer, Vlad I. Morariu, Handong Zhao, Varun Manjunatha, Nikolaos Barmpalios, Yun Fu

Contrasted with prior work, this paper provides a complementary solution to align domains by learning the same auxiliary tasks in both domains simultaneously.

Balancing Biases and Preserving Privacy on Balanced Faces in the Wild

1 code implementation16 Mar 2021 Joseph P Robinson, Can Qin, Yann Henon, Samson Timoner, Yun Fu

This scheme boosts the average performance and preserves identity information while removing demographic knowledge.

Decision Making Domain Adaptation

Skeleton Aware Multi-modal Sign Language Recognition

3 code implementations16 Mar 2021 Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, Yun Fu

Sign language is commonly used by deaf or speech impaired people to communicate but requires significant effort to master.

Sign Language Recognition Skeleton Based Action Recognition

Recent Advances on Neural Network Pruning at Initialization

2 code implementations11 Mar 2021 Huan Wang, Can Qin, Yue Bai, Yulun Zhang, Yun Fu

Neural network pruning typically removes connections or neurons from a pretrained converged model; while a new pruning paradigm, pruning at initialization (PaI), attempts to prune a randomly initialized network.

Benchmarking Network Pruning

VINS: Visual Search for Mobile User Interface Design

1 code implementation10 Feb 2021 Sara Bunian, Kai Li, Chaima Jemmali, Casper Harteveld, Yun Fu, Magy Seif El-Nasr

By utilizing this dataset, we propose an object-detection based image retrieval framework that models the UI context and hierarchical structure.

Image Retrieval object-detection +2

One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision

no code implementations3 Feb 2021 Zaid Khan, Yun Fu

Using the insight that a classifier can learn the racial system encoded by a dataset, we conduct an empirical study of computer vision datasets supplying categorical race labels for face images to determine the cross-dataset consistency and generalization of racial categories.

Benchmarking Fairness

Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts

no code implementations11 Jan 2021 Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Zhichao Lu, Yun Fu, Tomas Pfister

To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos.

Action Recognition Pseudo Label +1

Dynamic High-Pass Filtering and Multi-Spectral Attention for Image Super-Resolution

no code implementations ICCV 2021 Salma Abdel Magid, Yulun Zhang, Donglai Wei, Won-Dong Jang, Zudi Lin, Yun Fu, Hanspeter Pfister

Specifically, we propose a dynamic high-pass filtering (HPF) module that locally applies adaptive filter weights for each spatial location and channel group to preserve high-frequency signals.

Image Super-Resolution

Aspect-based Sentiment Classification via Reinforcement Learning

no code implementations1 Jan 2021 Lichen Wang, Bo Zong, Yunyu Liu, Can Qin, Wei Cheng, Wenchao Yu, Xuchao Zhang, Haifeng Chen, Yun Fu

As texts always contain a large proportion of task-irrelevant words, accurate alignment between aspects and their sentimental descriptions is the most crucial and challenging step.

Classification General Classification +4

Neighbor Class Consistency on Unsupervised Domain Adaptation

no code implementations1 Jan 2021 Chang Liu, Kai Li, Yun Fu

Unsupervised domain adaptation (UDA) is to make predictions for unlabeled data in a target domain with labeled data from source domain available.

Clustering Image Classification +1

Exploring Sub-Pseudo Labels for Learning from Weakly-Labeled Web Videos

no code implementations1 Jan 2021 Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Yun Fu, Tomas Pfister

To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos.

Action Recognition Pseudo Label +1

Context Reasoning Attention Network for Image Super-Resolution

no code implementations ICCV 2021 Yulun Zhang, Donglai Wei, Can Qin, Huan Wang, Hanspeter Pfister, Yun Fu

However, the basic convolutional layer in CNNs is designed to extract local patterns, lacking the ability to model global context.

Image Super-Resolution

Multimodal In-bed Pose and Shape Estimation under the Blankets

no code implementations12 Dec 2020 Yu Yin, Joseph P. Robinson, Yun Fu

Typically, humans are covered by a blanket when resting, for which we propose a multimodal approach to uncover the subjects so their bodies at rest can be viewed without the occlusion of the blankets above.

SuperFront: From Low-resolution to High-resolution Frontal Face Synthesis

no code implementations7 Dec 2020 Yu Yin, Joseph P. Robinson, Songyao Jiang, Yue Bai, Can Qin, Yun Fu

Even as impressive milestones are achieved in synthesizing faces, the importance of preserving identity is needed in practice and should not be overlooked.

Face Generation Generative Adversarial Network +2

Multi-head Knowledge Distillation for Model Compression

no code implementations5 Dec 2020 Huan Wang, Suhas Lohit, Michael Jones, Yun Fu

We add loss terms for training the student that measure the dissimilarity between student and teacher outputs of the auxiliary classifiers.

Image Classification Knowledge Distillation +1

Learning to Mutate with Hypergradient Guided Population

no code implementations NeurIPS 2020 Zhiqiang Tao, Yaliang Li, Bolin Ding, Ce Zhang, Jingren Zhou, Yun Fu

Computing the gradient of model hyperparameters, i. e., hypergradient, enables a promising and natural way to solve the hyperparameter optimization task.

Hyperparameter Optimization

Collaborative Attention Mechanism for Multi-View Action Recognition

no code implementations14 Sep 2020 Yue Bai, Zhiqiang Tao, Lichen Wang, Sheng Li, Yu Yin, Yun Fu

Extensive experiments on four action datasets illustrate the proposed CAM achieves better results for each view and also boosts multi-view performance.

Action Recognition Representation Learning

Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection

1 code implementation ECCV 2020 Shuhan Chen, Yun Fu

In this paper, we aim to develop an efficient and compact deep network for RGB-D salient object detection, where the depth image provides complementary information to boost performance in complex scenarios.

object-detection RGB-D Salient Object Detection +2

Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship

no code implementations28 Jul 2020 Joseph P. Robinson, Zaid Khan, Yu Yin, Ming Shao, Yun Fu

Thus, to narrow the gap between research and reality and enhance the power of kinship recognition systems, we extend FIW with multimedia (MM) data (i. e., video, audio, and text captions).

Survey on the Analysis and Modeling of Visual Kinship: A Decade in the Making

1 code implementation29 Jun 2020 Joseph P. Robinson, Ming Shao, Yun Fu

We review the public resources and data challenges that enabled and inspired many to hone-in on the views of automatic kinship recognition in the visual domain.

Gesture Recognition

Neural Sparse Representation for Image Restoration

1 code implementation NeurIPS 2020 Yuchen Fan, Jiahui Yu, Yiqun Mei, Yulun Zhang, Yun Fu, Ding Liu, Thomas S. Huang

Inspired by the robustness and efficiency of sparse representation in sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks.

Image Compression Image Denoising +2

HyperSTAR: Task-Aware Hyperparameters for Deep Networks

no code implementations CVPR 2020 Gaurav Mittal, Chang Liu, Nikolaos Karianakis, Victor Fragoso, Mei Chen, Yun Fu

To reduce HPO time, we present HyperSTAR (System for Task Aware Hyperparameter Recommendation), a task-aware method to warm-start HPO for deep neural networks.

Hyperparameter Optimization Image Classification

Inductive and Unsupervised Representation Learning on Graph Structured Objects

no code implementations ICLR 2020 Lichen Wang, Bo Zong, Qianqian Ma, Wei Cheng, Jingchao Ni, Wenchao Yu, Yanchi Liu, Dongjin Song, Haifeng Chen, Yun Fu

Inductive and unsupervised graph learning is a critical technique for predictive or information retrieval tasks where label information is difficult to obtain.

Graph Learning Graph Similarity +3

Pyramid Attention Networks for Image Restoration

2 code implementations28 Apr 2020 Yiqun Mei, Yuchen Fan, Yulun Zhang, Jiahui Yu, Yuqian Zhou, Ding Liu, Yun Fu, Thomas S. Huang, Humphrey Shi

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales.

Demosaicking Image Denoising +1

Learnable Subspace Clustering

1 code implementation9 Apr 2020 Jun Li, Hongfu Liu, Zhiqiang Tao, Handong Zhao, Yun Fu

This paper studies the large-scale subspace clustering (LSSC) problem with million data points.


Adversarial Feature Hallucination Networks for Few-Shot Learning

1 code implementation CVPR 2020 Kai Li, Yulun Zhang, Kunpeng Li, Yun Fu

The recent flourish of deep learning in various tasks is largely accredited to the rich and accessible labeled data.

Data Augmentation Few-Shot Learning +1

Cross-Domain Document Object Detection: Benchmark Suite and Method

1 code implementation CVPR 2020 Kai Li, Curtis Wigington, Chris Tensmeyer, Handong Zhao, Nikolaos Barmpalios, Vlad I. Morariu, Varun Manjunatha, Tong Sun, Yun Fu

We establish a benchmark suite consisting of different types of PDF document datasets that can be utilized for cross-domain DOD model training and evaluation.

object-detection Object Detection

Generative Partial Multi-View Clustering

no code implementations29 Mar 2020 Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, Yun Fu

Nowadays, with the rapid development of data collection sources and feature extraction methods, multi-view data are getting easy to obtain and have received increasing research attention in recent years, among which, multi-view clustering (MVC) forms a mainstream research direction and is widely used in data analysis.

Clustering Imputation

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

3 code implementations CVPR 2020 Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P. Allebach, Chenliang Xu

Rather than synthesizing missing LR video frames as VFI networks do, we firstly temporally interpolate LR frame features in missing LR video frames capturing local temporal contexts by the proposed feature temporal interpolation network.

Space-time Video Super-resolution Video Frame Interpolation +1

Dual-Attention GAN for Large-Pose Face Frontalization

1 code implementation17 Feb 2020 Yu Yin, Songyao Jiang, Joseph P. Robinson, Yun Fu

Face frontalization provides an effective and efficient way for face data augmentation and further improves the face recognition performance in extreme pose scenario.

Data Augmentation Face Generation +3

Face Recognition: Too Bias, or Not Too Bias?

1 code implementation16 Feb 2020 Joseph P. Robinson, Gennady Livitz, Yann Henon, Can Qin, Yun Fu, Samson Timoner

Thus, the conventional approach of learning a global threshold for all pairs resulting in performance gaps among subgroups.

Face Recognition

Recognizing Families In the Wild: White Paper for the 4th Edition Data Challenge

2 code implementations15 Feb 2020 Joseph P. Robinson, Yu Yin, Zaid Khan, Ming Shao, Siyu Xia, Michael Stopa, Samson Timoner, Matthew A. Turk, Rama Chellappa, Yun Fu

Recognizing Families In the Wild (RFIW): an annual large-scale, multi-track automatic kinship recognition evaluation that supports various visual kin-based problems on scales much higher than ever before.

Gesture Recognition Kinship Verification +1

Contradictory Structure Learning for Semi-supervised Domain Adaptation

1 code implementation6 Feb 2020 Can Qin, Lichen Wang, Qianqian Ma, Yu Yin, Huan Wang, Yun Fu

Current adversarial adaptation methods attempt to align the cross-domain features, whereas two challenges remain unsolved: 1) the conditional distribution mismatch and 2) the bias of the decision boundary towards the source domain.

Clustering Domain Adaptation +1

Self-Directed Online Machine Learning for Topology Optimization

1 code implementation4 Feb 2020 Changyu Deng, Yizhou Wang, Can Qin, Yun Fu, Wei Lu

A small number of training data is generated dynamically based on the DNN's prediction of the optimum.

BIG-bench Machine Learning Stochastic Optimization

Reverse Attention-Based Residual Network for Salient Object Detection

6 code implementations IEEE Transactions on Image Processing 2020 Shuhan Chen, Xiuli Tan, Ben Wang, Huchuan Lu, Xuelong Hu, Yun Fu

Benefiting from the quick development of deep convolutional neural networks, especially fully convolutional neural networks (FCNs), remarkable progresses have been achieved on salient object detection recently.

Object object-detection +2

Texture Hallucination for Large-Factor Painting Super-Resolution

no code implementations ECCV 2020 Yulun Zhang, Zhifei Zhang, Stephen DiVerdi, Zhaowen Wang, Jose Echevarria, Yun Fu

We aim to super-resolve digital paintings, synthesizing realistic details from high-resolution reference painting materials for very large scaling factors (e. g., 8X, 16X).

4k Hallucination +2

Lifelong Spectral Clustering

no code implementations27 Nov 2019 Gan Sun, Yang Cong, Qianqian Wang, Jun Li, Yun Fu

As a new spectral clustering task arrives, L2SC firstly transfers knowledge from both basis library and feature library to obtain encoding matrix, and further redefines the library base over time to maximize performance across all the clustering tasks.


Correlative Channel-Aware Fusion for Multi-View Time Series Classification

no code implementations24 Nov 2019 Yue Bai, Lichen Wang, Zhiqiang Tao, Sheng Li, Yun Fu

Multi-view time series classification (MVTSC) aims to improve the performance by fusing the distinctive temporal information from multiple views.

Classification General Classification +3

Joint Super-Resolution and Alignment of Tiny Faces

1 code implementation19 Nov 2019 Yu Yin, Joseph P. Robinson, Yulun Zhang, Yun Fu

As for SR, the proposed method recovers sharper edges and more details from LR face images than other state-of-the-art methods, which we demonstrate qualitatively and quantitatively.


What Will Your Child Look Like? DNA-Net: Age and Gender Aware Kin Face Synthesizer

no code implementations16 Nov 2019 Pengyu Gao, Siyu Xia, Joseph Robinson, Junkang Zhang, Chao Xia, Ming Shao, Yun Fu

Specifically, we propose a two-stage kin-face generation model to predict the appearance of a child given a pair of parents.

Face Generation Kinship Verification +1

PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation

2 code implementations NeurIPS 2019 Can Qin, Haoxuan You, Lichen Wang, C. -C. Jay Kuo, Yun Fu

Specifically, most general-purpose DA methods that struggle for global feature alignment and ignore local geometric information are not suitable for 3D domain alignment.

Unsupervised Domain Adaptation

Real-time Memory Efficient Large-pose Face Alignment via Deep Evolutionary Network

no code implementations25 Oct 2019 Bin Sun, Ming Shao, Siyu Xia, Yun Fu

To accelerate the model, we propose an efficient network structure to accelerate the evolutionary learning process through a factorization strategy.

Face Alignment Face Recognition

LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution

no code implementations25 Oct 2019 Bin Sun, Jun Li, Ming Shao, Yun Fu

To reduce the computation and memory costs, we propose a novel lightweight deep learning module by low-rank pointwise residual (LPR) convolution, called LPRNet.

Face Alignment Image Classification +1

Generative One-Shot Face Recognition

no code implementations28 Sep 2019 Zhengming Ding, Yandong Guo, Lei Zhang, Yun Fu

Specifically, we target at building a more effective general face classifier for both normal persons and one-shot persons.

Face Recognition One-Shot Learning +1

Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective

1 code implementation ICCV 2019 Kai Li, Martin Renqiang Min, Yun Fu

We instead reformulate ZSL as a conditioned visual classification problem, i. e., classifying visual features based on the classifiers learned from the semantic descriptions.

Classification General Classification +1

Visual Semantic Reasoning for Image-Text Matching

2 code implementations ICCV 2019 Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, Yun Fu

It outperforms the current best method by 6. 8% relatively for image retrieval and 4. 8% relatively for caption retrieval on MS-COCO (Recall@1 using 1K test set).

Image Retrieval Image-text matching +3

Large Scale Incremental Learning

4 code implementations CVPR 2019 Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, Yun Fu

We believe this is because of the combination of two factors: (a) the data imbalance between the old and new classes, and (b) the increasing number of visually similar classes.

Class Incremental Learning Incremental Learning

Spatially Constrained GAN for Face and Fashion Synthesis

1 code implementation7 May 2019 Songyao Jiang, Hongfu Liu, Yue Wu, Yun Fu

Besides, a segmentor network is constructed to impose spatial constraints on the generator.

Attribute Conditional Image Generation +3

EV-Action: Electromyography-Vision Multi-Modal Action Dataset

1 code implementation20 Apr 2019 Lichen Wang, Bin Sun, Joseph Robinson, Taotao Jing, Yun Fu

To make up this, we introduce a new, large-scale EV-Action dataset in this work, which consists of RGB, depth, electromyography (EMG), and two skeleton modalities.

Action Analysis Action Recognition +3

Rethinking Classification and Localization for Object Detection

2 code implementations CVPR 2020 Yue Wu, Yinpeng Chen, Lu Yuan, Zicheng Liu, Lijuan Wang, Hongzhi Li, Yun Fu

Two head structures (i. e. fully connected head and convolution head) have been widely used in R-CNN based detectors for classification and localization tasks.

Classification General Classification +3

Multimodal Style Transfer via Graph Cuts

2 code implementations ICCV 2019 Yulun Zhang, Chen Fang, Yilin Wang, Zhaowen Wang, Zhe Lin, Yun Fu, Jimei Yang

An assumption widely used in recent neural style transfer methods is that image styles can be described by global statics of deep features like Gram or covariance matrices.

Style Transfer

Laplace Landmark Localization

no code implementations ICCV 2019 Joseph P. Robinson, Yuncheng Li, Ning Zhang, Yun Fu, and Sergey Tulyakov

Our method claims state-of-the-art on all of the 300W benchmarks and ranks second-to-best on the Annotated Facial Landmarks in the Wild (AFLW) dataset.

Ranked #5 on Face Alignment on AFLW-19 (NME_box (%, Full) metric)

Face Alignment Facial Landmark Detection

Residual Non-local Attention Networks for Image Restoration

2 code implementations ICLR 2019 Yulun Zhang, Kunpeng Li, Kai Li, Bineng Zhong, Yun Fu

To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts.

Demosaicking Image Denoising +1

Representative Task Self-selection for Flexible Clustered Lifelong Learning

no code implementations6 Mar 2019 Gan Sun, Yang Cong, Qianqian Wang, Bineng Zhong, Yun Fu

Consider the lifelong machine learning paradigm whose objective is to learn a sequence of tasks depending on previous experiences, e. g., knowledge library or deep network weights.

Model Optimization Multi-Task Learning

Segmentation Guided Image-to-Image Translation with Adversarial Networks

1 code implementation6 Jan 2019 Songyao Jiang, Zhiqiang Tao, Yun Fu

Recently image-to-image translation has received increasing attention, which aims to map images in one domain to another specific one.

Image-to-Image Translation Segmentation +2

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

2 code implementations7 Dec 2018 Yapeng Tian, Yulun Zhang, Yun Fu, Chenliang Xu

Video super-resolution (VSR) aims to restore a photo-realistic high-resolution (HR) video frame from both its corresponding low-resolution (LR) frame (reference frame) and multiple neighboring frames (supporting frames).

Optical Flow Estimation Video Super-Resolution

Visual Font Pairing

no code implementations19 Nov 2018 Shuhui Jiang, Zhaowen Wang, Aaron Hertzmann, Hailin Jin, Yun Fu

Third, font pairing is an asymmetric problem in that the roles played by header and body fonts are not interchangeable.

Metric Learning

Arc-support Line Segments Revisited: An Efficient and High-quality Ellipse Detection

3 code implementations8 Oct 2018 Changsheng Lu, Siyu Xia, Ming Shao, Yun Fu

Over the years many ellipse detection algorithms spring up and are studied broadly, while the critical issue of detecting ellipses accurately and efficiently in real-world images remains a challenge.

Clustering valid

GraphSeq2Seq: Graph-Sequence-to-Sequence for Neural Machine Translation

no code implementations27 Sep 2018 Guoshuai Zhao, Jun Li, Lu Wang, Xueming Qian, Yun Fu

In this paper, we propose a Graph-Sequence-to-Sequence(GraphSeq2Seq) model to fuse the dependency graph among words into the traditional Seq2Seq framework.

Image Captioning Machine Translation +5

Support Neighbor Loss for Person Re-Identification

1 code implementation18 Aug 2018 Kai Li, Zhengming Ding, Kunpeng Li, Yulun Zhang, Yun Fu

To ensure scalability and separability, a softmax-like function is formulated to push apart the positive and negative support sets.

Person Re-Identification

Human Action Recognition and Prediction: A Survey

no code implementations28 Jun 2018 Yu Kong, Yun Fu

Derived from rapid advances in computer vision and machine learning, video analysis tasks have been moving from inferring the present state to predicting the future state.

Action Recognition Autonomous Driving +3

Predictive Local Smoothness for Stochastic Gradient Methods

no code implementations ICLR 2019 Jun Li, Hongfu Liu, Bineng Zhong, Yue Wu, Yun Fu

To address this problem, we propose a simple yet effective method for improving stochastic gradient methods named predictive local smoothness (PLS).

Tell Me Where to Look: Guided Attention Inference Network

2 code implementations CVPR 2018 Kunpeng Li, Ziyan Wu, Kuan-Chuan Peng, Jan Ernst, Yun Fu

Weakly supervised learning with only coarse labels can obtain visual explanations of deep neural network such as attention maps by back-propagating gradients.

Object Localization Semantic Segmentation +1

Residual Dense Network for Image Super-Resolution

16 code implementations CVPR 2018 Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, Yun Fu

In this paper, we propose a novel residual dense network (RDN) to address this problem in image SR. We fully exploit the hierarchical features from all the convolutional layers.

Color Image Denoising Image Super-Resolution

Incremental Classifier Learning with Generative Adversarial Networks

no code implementations2 Feb 2018 Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, Zhengyou Zhang, Yun Fu

To address these problems, we propose (a) a new loss function to combine the cross-entropy loss and distillation loss, (b) a simple way to estimate and remove the unbalance between the old and new classes , and (c) using Generative Adversarial Networks (GANs) to generate historical data and select representative exemplars during generation.

General Classification

Clustering with Outlier Removal

no code implementations5 Jan 2018 Hongfu Liu, Jun Li, Yue Wu, Yun Fu

Then an objective function based Holoentropy is designed to enhance the compactness of each cluster with a few outliers removed.

Clustering Outlier Detection

Deep Sequential Context Networks for Action Prediction

no code implementations CVPR 2017 Yu Kong, Zhiqiang Tao, Yun Fu

Different from after-the-fact action recognition, action prediction task requires action labels to be predicted from these partially observed videos.

Action Recognition Temporal Action Localization

Families in the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks

no code implementations7 Apr 2016 Joseph P. Robinson, Ming Shao, Yue Wu, Yun Fu

Motivated by the lack of a single, unified dataset for kinship recognition, we aim to provide a dataset that captivates the interest of the research community.

Kinship Verification Metric Learning

Bilinear Heterogeneous Information Machine for RGB-D Action Recognition

no code implementations CVPR 2015 Yu Kong, Yun Fu

Rich heterogeneous RGB and depth data are effectively compressed and projected to a learned shared space, in order to reduce noise and capture useful information for recognition.

Action Recognition Temporal Action Localization

Fast Resampling Weighted v-Statistics

no code implementations NeurIPS 2012 Chunxiao Zhou, Jiseong Park, Yun Fu

In this paper, a novel, computationally fast, and alternative algorithm for com- puting weighted v-statistics in resampling both univariate and multivariate data is proposed.

Cannot find the paper you are looking for? You can Submit a new open access paper.