Search Results for author: Yan Yan

Found 191 papers, 61 papers with code

Augmentation Matters: A Mix-Paste Method for X-Ray Prohibited Item Detection under Noisy Annotations

1 code implementation3 Jan 2025 Ruikang Chen, Yan Yan, Jing-Hao Xue, Yang Lu, Hanzi Wang

However, obtaining correct annotations is extremely hard if not impossible for large-scale X-ray images, where item overlapping is ubiquitous. As a result, X-ray images are easily contaminated with noisy annotations, leading to performance deterioration of existing methods. In this paper, we address the challenging problem of training a robust prohibited item detector under noisy annotations (including both category noise and bounding box noise) from a novel perspective of data augmentation, and propose an effective label-aware mixed patch paste augmentation method (Mix-Paste).

Data Augmentation

Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification

1 code implementation21 Dec 2024 Changchang Sun, Ren Wang, Yihua Zhang, Jinghan Jia, Jiancheng Liu, Gaowen Liu, Sijia Liu, Yan Yan

Machine unlearning (MU), which seeks to erase the influence of specific unwanted data from already-trained models, is becoming increasingly vital in model editing, particularly to comply with evolving data regulations like the ``right to be forgotten''.

Image Classification Machine Unlearning +1

E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling

no code implementations18 Dec 2024 Zhihang Yuan, Yuzhang Shang, Hanling Zhang, Tongcheng Fang, Rui Xie, Bingxin Xu, Yan Yan, Shengen Yan, Guohao Dai, Yu Wang

Our approach not only enhances computational efficiency but also aligns naturally with image generation principles by operating in continuous token space and following a hierarchical generation process from coarse to fine details.

Computational Efficiency Denoising +1

GALOT: Generative Active Learning via Optimizable Zero-shot Text-to-image Generation

no code implementations18 Dec 2024 Hanbin Hong, Shenao Yan, Shuya Feng, Yan Yan, Yuan Hong

Active Learning (AL) represents a crucial methodology within machine learning, emphasizing the identification and utilization of the most informative samples for efficient model training.

Active Learning Pseudo Label +1

Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need

no code implementations10 Dec 2024 Bo-Wen Zhang, Yan Yan, Boxiang Yang, Yifei Xue, Guang Liu

While scaling laws optimize training configurations for large language models (LLMs) through experiments on smaller or early-stage models, they fail to predict emergent abilities due to the absence of such capabilities in these models.

I$^2$OL-Net: Intra-Inter Objectness Learning Network for Point-Supervised X-Ray Prohibited Item Detection

no code implementations5 Dec 2024 Sanjoeng Wong, Yan Yan

Meanwhile, the inter-OL module introduces the wavelet decomposition-based adversarial learning block and the objectness block, effectively reducing the modality discrepancy and transferring the objectness knowledge learned from natural images with box annotations to X-ray images.

Long text outline generation: Chinese text outline based on unsupervised framework and large language mode

no code implementations1 Dec 2024 Yan Yan, Yuanchi Ma

Finally, we employ a large model to generate summaries of each plot segment and produce the overall outline.

Graph Attention

freePruner: A Training-free Approach for Large Multimodal Model Acceleration

no code implementations23 Nov 2024 Bingxin Xu, Yuzhang Shang, Yunhao Ge, Qian Lou, Yan Yan

Large Multimodal Models (LMMs) have demonstrated impressive capabilities in visual-language tasks but face significant deployment challenges due to their high computational demands.

Quantization Question Answering +2

Video-to-Task Learning via Motion-Guided Attention for Few-Shot Action Recognition

no code implementations18 Nov 2024 Hanyu Guo, Wanchuan Yu, Suzhou Que, Kaiwen Du, Yan Yan, Hanzi Wang

In this paper, we propose a novel Dual Motion-Guided Attention Learning method (called DMGAL) for few-shot action recognition, aiming to learn the spatio-temporal relationships from the video-specific to the task-specific level.

Few-Shot action recognition Few Shot Action Recognition

MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration

no code implementations3 Nov 2024 Kaiang Wen, Bin Xie, Bin Duan, Yan Yan

The Mamba-based architecture seamlessly integrates the local feature extraction power of convolutional layers with the long-range dependency modeling capabilities of Mamba.

Image Registration Mamba

Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning

1 code implementation30 Sep 2024 Weitai Kang, Haifeng Huang, Yuzhang Shang, Mubarak Shah, Yan Yan

RIG generates two key instruction data: 1) the Adversarial Instruction-following data, which features mixed negative and positive samples to enhance the model's discriminative understanding.

Instruction Following Language Modeling +2

Interpolating Video-LLMs: Toward Longer-sequence LMMs in a Training-free Manner

no code implementations19 Sep 2024 Yuzhang Shang, Bingxin Xu, Weitai Kang, Mu Cai, Yuheng Li, Zehao Wen, Zhen Dong, Kurt Keutzer, Yong Jae Lee, Yan Yan

In this paper, we first identify the primary challenges in interpolating Video-LLMs: (1) the video encoder and modality alignment projector are fixed, preventing the integration of additional frames into Video-LLMs, and (2) the LLM backbone is limited in its content length capabilities, which complicates the processing of an increased number of video tokens.

DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture

no code implementations5 Sep 2024 Qianlong Xiang, Miao Zhang, Yuzhang Shang, Jianlong Wu, Yan Yan, Liqiang Nie

Furthermore, considering that the source data is either unaccessible or too enormous to store for current generative models, we introduce a new paradigm for their distillation without source data, termed Data-Free Knowledge Distillation for Diffusion Models (DKDM).

Data-free Knowledge Distillation Denoising +1

Distilling Long-tailed Datasets

1 code implementation24 Aug 2024 Zhenghao Zhao, Haoxuan Wang, Yuzhang Shang, Kai Wang, Yan Yan

It reduces the distance between the student and the biased expert trajectories and prevents the tail class bias from being distilled to the synthetic dataset.

Dataset Distillation Efficient Neural Network

Dataset Quantization with Active Learning based Adaptive Sampling

1 code implementation9 Jul 2024 Zhenghao Zhao, Yuzhang Shang, Junyi Wu, Yan Yan

In addition, we introduce a novel pipeline for dataset quantization, utilizing feature space from the final stage of dataset quantization to generate more precise dataset bins.

Active Learning Dataset Distillation +1

ACTRESS: Active Retraining for Semi-supervised Visual Grounding

no code implementations3 Jul 2024 Weitai Kang, Mengxue Qu, Yunchao Wei, Yan Yan

Building upon this, ACTRESS consists of an active sampling strategy and a selective retraining strategy.

Binary Classification Visual Grounding

SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding

1 code implementation3 Jul 2024 Weitai Kang, Gaowen Liu, Mubarak Shah, Yan Yan

Specifically, we propose the Multi-layer Multi-task Encoder-Decoder as the target grounding stage, where we learn a regression query and multiple segmentation queries to ground the target by regression and segmentation of the box in each decoding layer, respectively.

object-detection Object Detection +3

Visual Grounding with Attention-Driven Constraint Balancing

no code implementations3 Jul 2024 Weitai Kang, Luowei Zhou, Junyi Wu, Changchang Sun, Yan Yan

Building upon this, we further propose a novel framework named Attention-Driven Constraint Balancing (AttBalance) to optimize the behavior of visual features within language-relevant regions.

Object object-detection +2

Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration

1 code implementation10 Jun 2024 Yuanjie Shi, Subhankar Ghosh, Taha Belkhouja, Janardhan Rao Doppa, Yan Yan

In contrast to the standard class-conditional CP (CCP) method that uniformly thresholds the class-wise conformity score for each class, the augmented label rank calibration step allows RC3P to selectively iterate this class-wise thresholding subroutine only for a subset of classes whose class-wise top-k error is small.

Conformal Prediction imbalanced classification +2

LMO-DP: Optimizing the Randomization Mechanism for Differentially Private Fine-Tuning (Large) Language Models

no code implementations29 May 2024 Qin Yang, Meisam Mohammad, Han Wang, Ali Payani, Ashish Kundu, Kai Shu, Yan Yan, Yuan Hong

To address such limitations, we propose a novel Language Model-based Optimal Differential Privacy (LMO-DP) mechanism, which takes the first step to enable the tight composition of accurately fine-tuning (large) language models with a sub-optimal DP mechanism, even in strong privacy regimes (e. g., $0. 1\leq \epsilon<3$).

Language Modelling SST-2 +1

Intent3D: 3D Object Detection in RGB-D Scans Based on Human Intention

no code implementations28 May 2024 Weitai Kang, Mengxue Qu, Jyoti Kini, Yunchao Wei, Mubarak Shah, Yan Yan

To achieve detection based on human intention, it relies on humans to observe the scene, reason out the target that aligns with their intention ("pillow" in this case), and finally provide a reference to the AI system, such as "A pillow on the couch".

3D Object Detection 3D visual grounding +2

PTQ4DiT: Post-training Quantization for Diffusion Transformers

1 code implementation25 May 2024 Junyi Wu, Haoxuan Wang, Yuzhang Shang, Mubarak Shah, Yan Yan

SSC extends this approach by dynamically adjusting the balanced salience to capture the temporal variations in activation.

Image Generation Quantization

Efficient Multitask Dense Predictor via Binarization

no code implementations CVPR 2024 Yuzhang Shang, Dan Xu, Gaowen Liu, Ramana Rao Kompella, Yan Yan

Moreover, we introduce a knowledge distillation mechanism to correct the direction of information flow in backward propagation.

Binarization Knowledge Distillation +1

The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks

1 code implementation14 May 2024 Ziquan Liu, Yufei Cui, Yan Yan, Yi Xu, Xiangyang Ji, Xue Liu, Antoni B. Chan

In safety-critical applications such as medical imaging and autonomous driving, where decisions have profound implications for patient health and road safety, it is imperative to maintain both high adversarial robustness to protect against potential adversarial attacks and reliable uncertainty quantification in decision-making.

Adversarial Defense Adversarial Robustness +4

Versatile Navigation under Partial Observability via Value-guided Diffusion Policy

no code implementations CVPR 2024 Gengyu Zhang, Hao Tang, Yan Yan

To address these deficiencies, we propose a versatile diffusion-based approach for both 2D and 3D route planning under partial observability.

Autonomous Driving Semantic Segmentation

On the Faithfulness of Vision Transformer Explanations

no code implementations CVPR 2024 Junyi Wu, Weitai Kang, Hao Tang, Yuan Hong, Yan Yan

In contrast, our proposed SaCo offers a reliable faithfulness measurement, establishing a robust metric for interpretations.

LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models

1 code implementation22 Mar 2024 Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, Yan Yan

In response, we propose PruMerge, a novel adaptive visual token reduction strategy that significantly reduces the number of visual tokens without compromising the performance of LMMs.

Language Modelling Large Language Model +4

MaskSAM: Towards Auto-prompt SAM with Mask Classification for Medical Image Segmentation

no code implementations21 Mar 2024 Bin Xie, Hao Tang, Bin Duan, Dawen Cai, Yan Yan

Each pair of auxiliary mask and box prompts, which can solve the requirements of extra prompts, is associated with class label predictions by the sum of the auxiliary classifier token and the learnable global classifier tokens in the mask decoder of SAM to solve the predictions of semantic labels.

Decoder Image Segmentation +3

Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer

no code implementations CVPR 2024 Junyi Wu, Bin Duan, Weitai Kang, Hao Tang, Yan Yan

To incorporate the influence of token transformation into interpretation, we propose TokenTM, a novel post-hoc explanation method that utilizes our introduced measurement of token transformation effects.

FBPT: A Fully Binary Point Transformer

no code implementations15 Mar 2024 Zhixing Hou, Yuzhang Shang, Yan Yan

This paper presents a novel Fully Binary Point Cloud Transformer (FBPT) model which has the potential to be widely applied and expanded in the fields of robotics and mobile devices.

Binarization Point Cloud Classification

Online Multi-spectral Neuron Tracing

no code implementations10 Mar 2024 Bin Duan, Yuzhang Shang, Dawen Cai, Yan Yan

In this paper, we propose an online multi-spectral neuron tracing method with uniquely designed modules, where no offline training are required.

LLM Inference Unveiled: Survey and Roofline Model Insights

2 code implementations26 Feb 2024 Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, Yan Yan, Beidi Chen, Guangyu Sun, Kurt Keutzer

Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques.

Knowledge Distillation Language Modelling +5

QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning

1 code implementation6 Feb 2024 Haoxuan Wang, Yuzhang Shang, Zhihang Yuan, Junyi Wu, Junchi Yan, Yan Yan

We empirically verify that our approach modifies the activation distribution and provides meaningful temporal information, facilitating easier and more accurate quantization.

Image Generation Model Compression +1

Frequency Domain Nuances Mining for Visible-Infrared Person Re-identification

no code implementations4 Jan 2024 Yukang Zhang, Yang Lu, Yan Yan, Hanzi Wang, Xuelong Li

Specifically, we propose a novel Frequency Domain Nuances Mining (FDNM) method to explore the cross-modality frequency domain information, which mainly includes an amplitude guided phase (AGP) module and an amplitude nuances mining (ANM) module.

Face Recognition Person Re-Identification

Enhancing Post-training Quantization Calibration through Contrastive Learning

no code implementations CVPR 2024 Yuzhang Shang, Gaowen Liu, Ramana Rao Kompella, Yan Yan

We aim to calibrate the quantized activations by maximizing the mutual information between the pre- and post-quantized activations.

Contrastive Learning Quantization

BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition

1 code implementation CVPR 2024 Yuxuan Zhou, Xudong Yan, Zhi-Qi Cheng, Yan Yan, Qi Dai, Xian-Sheng Hua

To remedy this we propose a two-fold strategy: (1) We introduce an innovative approach that encodes bone connectivity by harnessing the power of graph distances to describe the physical topology; we further incorporate action-specific topological representation via persistent homology analysis to depict systemic dynamics.

Action Recognition Skeleton Based Action Recognition

MIM4DD: Mutual Information Maximization for Dataset Distillation

1 code implementation NeurIPS 2023 Yuzhang Shang, Zhihang Yuan, Yan Yan

Thus, we introduce mutual information (MI) as the metric to quantify the shared information between the synthetic and the real datasets, and devise MIM4DD numerically maximizing the MI via a newly designed optimizable objective within a contrastive learning framework to update the synthetic dataset.

Contrastive Learning Dataset Distillation

High-Order Structure Based Middle-Feature Learning for Visible-Infrared Person Re-Identification

1 code implementation13 Dec 2023 Liuxiang Qiu, Si Chen, Yan Yan, Jing-Hao Xue, Da-Han Wang, Shunzhi Zhu

Existing VI-ReID methods ignore high-order structure information of features while being relatively difficult to learn a reasonable common feature space due to the large modality discrepancy between VIS and IR images.

Person Re-Identification

Spatial-Contextual Discrepancy Information Compensation for GAN Inversion

1 code implementation12 Dec 2023 Ziqiang Zhang, Yan Yan, Jing-Hao Xue, Hanzi Wang

SDIC follows a "compensate-and-edit" paradigm and successfully bridges the gap in image details between the original image and the reconstructed/edited image.

ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models

1 code implementation10 Dec 2023 Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, Guangyu Sun

Based on the success of the low-rank decomposition of projection matrices in the self-attention module, we further introduce ASVD to compress the KV cache.

Discovery and Expansion of New Domains within Diffusion Models

no code implementations13 Oct 2023 Ye Zhu, Yu Wu, Duo Xu, Zhiwei Deng, Yan Yan, Olga Russakovsky

In this work, we study the generalization properties of diffusion models in a few-shot setup, introduce a novel tuning-free paradigm to synthesize the target out-of-domain (OOD) data, and demonstrate its advantages compared to existing methods in data-sparse scenarios with large domain gaps.

Denoising Image Generation

Causal-DFQ: Causality Guided Data-free Network Quantization

1 code implementation ICCV 2023 Yuzhang Shang, Bingxin Xu, Gaowen Liu, Ramana Kompella, Yan Yan

Inspired by the causal understanding, we propose the Causality-guided Data-free Network Quantization method, Causal-DFQ, to eliminate the reliance on data via approaching an equilibrium of causality-driven intervened distributions.

Data Free Quantization Neural Network Compression

TSTTC: A Large-Scale Dataset for Time-to-Contact Estimation in Driving Scenarios

1 code implementation4 Sep 2023 Yuheng Shi, Zehao Huang, Yan Yan, Naiyan Wang, Xiaojie Guo

Time-to-Contact (TTC) estimation is a critical task for assessing collision risk and is widely used in various driver assistance and autonomous driving systems.

Autonomous Driving Neural Rendering

Probabilistically robust conformal prediction

no code implementations31 Jul 2023 Subhankar Ghosh, Yuanjie Shi, Taha Belkhouja, Yan Yan, Jana Doppa, Brian Jones

We propose a novel adaptive PRCP (aPRCP) algorithm to achieve probabilistically robust coverage.

Conformal Prediction

Towards Saner Deep Image Registration

1 code implementation ICCV 2023 Bin Duan, Ming Zhong, Yan Yan

Moreover, we derive a set of theoretical guarantees for our sanity-checked image registration method, with experimental results supporting our theoretical findings and their effectiveness in increasing the sanity of models without sacrificing any performance.

Image Registration

A Decision Making Framework for Recommended Maintenance of Road Segments

no code implementations19 Jul 2023 Haoyu Sun, Yan Yan

Due to limited budgets allocated for road maintenance projects in various countries, road management departments face difficulties in making scientific maintenance decisions.

Decision Making Deep Reinforcement Learning +1

MRCN: A Novel Modality Restitution and Compensation Network for Visible-Infrared Person Re-identification

no code implementations26 Mar 2023 Yukang Zhang, Yan Yan, Jie Li, Hanzi Wang

Furthermore, to better disentangle the modality-relevant features and the modality-irrelevant features, we propose a novel Center-Quadruplet Causal (CQC) loss to encourage the network to effectively learn the modality-relevant features and the modality-irrelevant features.

Person Re-Identification

BPT: Binary Point Cloud Transformer for Place Recognition

no code implementations2 Mar 2023 Zhixing Hou, Yuzhang Shang, Tian Gao, Yan Yan

To solve this issue, we propose a binary point cloud transformer for place recognition.

Boundary Guided Learning-Free Semantic Control with Diffusion Models

1 code implementation NeurIPS 2023 Ye Zhu, Yu Wu, Zhiwei Deng, Olga Russakovsky, Yan Yan

Applying pre-trained generative denoising diffusion models (DDMs) for downstream tasks such as image semantic editing usually requires either fine-tuning DDMs or learning auxiliary editing networks in the existing literature.

Denoising

Optical Flow Estimation in 360$^\circ$ Videos: Dataset, Model and Application

no code implementations27 Jan 2023 Bin Duan, Keshav Bhandari, Gaowen Liu, Yan Yan

Moreover, we present a novel Siamese representation Learning framework for Omnidirectional Flow (SLOF) estimation, which is trained in a contrastive manner via a hybrid loss that combines siamese contrastive and optical flow losses.

Egocentric Activity Recognition Optical Flow Estimation +1

Deep Stereo Video Inpainting

no code implementations CVPR 2023 Zhiliang Wu, Changchang Sun, Hanyu Xuan, Yan Yan

Stereo video inpainting aims to fill the missing regions on the left and right views of the stereo video with plausible content simultaneously.

Video Inpainting

Few-shot Medical Image Segmentation with Cycle-resemblance Attention

no code implementations7 Dec 2022 Hao Ding, Changchang Sun, Hao Tang, Dawen Cai, Yan Yan

Recently, due to the increasing requirements of medical imaging applications and the professional requirements of annotating medical images, few-shot learning has gained increasing attention in the medical image semantic segmentation field.

Few-Shot Learning Image Segmentation +4

Post-training Quantization on Diffusion Models

1 code implementation CVPR 2023 Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, Yan Yan

These approaches define a forward diffusion process for transforming data into noise and a backward denoising process for sampling data from noise.

Denoising Noise Estimation +1

Vision+X: A Survey on Multimodal Learning in the Light of Data

no code implementations5 Oct 2022 Ye Zhu, Yu Wu, Nicu Sebe, Yan Yan

We are perceiving and communicating with the world in a multisensory manner, where different information sources are sophisticatedly processed and interpreted by separate parts of the human brain to constitute a complex, yet harmonious and unified sensing system.

Representation Learning

3D Cross-Pseudo Supervision (3D-CPS): A semi-supervised nnU-Net architecture for abdominal organ segmentation

1 code implementation19 Sep 2022 Yongzhi Huang, Hanwen Zhang, Yan Yan, Haseeb Hassan

Large curated datasets are necessary, but annotating medical images is a time-consuming, laborious, and expensive process.

Organ Segmentation

DPTNet: A Dual-Path Transformer Architecture for Scene Text Detection

no code implementations21 Aug 2022 Jingyu Lin, Jie Jiang, Yan Yan, Chunchao Guo, Hongfa Wang, Wei Liu, Hanzi Wang

We further propose a parallel design that integrates the convolutional network with a powerful self-attention mechanism to provide complementary clues between the attention path and convolutional path.

Scene Text Detection Text Detection

Progressive Cross-modal Knowledge Distillation for Human Action Recognition

no code implementations17 Aug 2022 Jianyuan Ni, Anne H. H. Ngu, Yan Yan

However, the accuracy performance of wearable sensor-based HAR is still far behind the ones from the visual modalities-based system (i. e., RGB video, skeleton, and depth).

Action Recognition Knowledge Distillation +3

Semi-Supervised Video Inpainting with Cycle Consistency Constraints

no code implementations CVPR 2023 Zhiliang Wu, Hanyu Xuan, Changchang Sun, Kang Zhang, Yan Yan

Specifically, in this work, we propose an end-to-end trainable framework consisting of completion network and mask prediction network, which are designed to generate corrupted contents of the current frame using the known mask and decide the regions to be filled of the next frame, respectively.

Video Inpainting

Seeing your sleep stage: cross-modal distillation from EEG to infrared video

1 code implementation11 Aug 2022 Jianan Han, Shaoxing Zhang, Aidong Men, Yang Liu, Ziming Yao, Yan Yan, Qingchao Chen

$S^3VE$ is a large-scale dataset including synchronized infrared video and EEG signal for sleep stage classification, including 105 subjects and 154, 573 video clips that is more than 1100 hours long.

EEG

Learning Omnidirectional Flow in 360-degree Video via Siamese Representation

no code implementations7 Aug 2022 Keshav Bhandari, Bin Duan, Gaowen Liu, Hugo Latapie, Ziliang Zong, Yan Yan

Optical flow estimation in omnidirectional videos faces two significant issues: the lack of benchmark datasets and the challenge of adapting perspective video-based methods to accommodate the omnidirectional nature.

Diversity Optical Flow Estimation +1

Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem

no code implementations24 Jul 2022 Yudong Han, Liqiang Nie, Jianhua Yin, Jianlong Wu, Yan Yan

Several studies have recently pointed that existing Visual Question Answering (VQA) models heavily suffer from the language prior problem, which refers to capturing superficial statistical correlations between the question type and the answer whereas ignoring the image contents.

Question Answering Visual Question Answering

MLP-GAN for Brain Vessel Image Segmentation

no code implementations17 Jul 2022 Bin Xie, Hao Tang, Bin Duan, Dawen Cai, Yan Yan

Brain vessel image segmentation can be used as a promising biomarker for better prevention and treatment of different diseases.

Generative Adversarial Network Image Segmentation +2

Learn-to-Decompose: Cascaded Decomposition Network for Cross-Domain Few-Shot Facial Expression Recognition

1 code implementation16 Jul 2022 Xinyi Zou, Yan Yan, Jing-Hao Xue, Si Chen, Hanzi Wang

Extensive experiments on both in-the-lab and in-the-wild compound expression datasets demonstrate the superiority of our proposed CDNet against several state-of-the-art FSL methods.

cross-domain few-shot learning Facial Expression Recognition +1

Lipschitz Continuity Retained Binary Neural Network

1 code implementation13 Jul 2022 Yuzhang Shang, Dan Xu, Bin Duan, Ziliang Zong, Liqiang Nie, Yan Yan

Relying on the premise that the performance of a binary neural network can be largely restored with eliminated quantization error between full-precision weight vectors and their corresponding binary vectors, existing works of network binarization frequently adopt the idea of model robustness to reach the aforementioned objective.

Binarization Quantization

Out-of-Distribution Detection in Time-Series Domain: A Novel Seasonal Ratio Scoring Approach

1 code implementation9 Jul 2022 Taha Belkhouja, Yan Yan, Janardhan Rao Doppa

Experiments on diverse real-world benchmarks demonstrate that the SRS method is well-suited for time-series OOD detection when compared to baseline methods.

Out-of-Distribution Detection Out of Distribution (OOD) Detection +2

Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis

1 code implementation9 Jul 2022 Taha Belkhouja, Yan Yan, Janardhan Rao Doppa

Despite the success of deep neural networks (DNNs) for real-world applications over time-series data such as mobile health, little is known about how to train robust DNNs for time-series domain due to its unique characteristics compared to images and text data.

Data Augmentation Dynamic Time Warping +3

Dynamic Time Warping based Adversarial Framework for Time-Series Domain

1 code implementation9 Jul 2022 Taha Belkhouja, Yan Yan, Janardhan Rao Doppa

Despite the rapid progress on research in adversarial robustness of deep neural networks (DNNs), there is little principled work for the time-series domain.

Adversarial Robustness Dynamic Time Warping +2

Network Binarization via Contrastive Learning

1 code implementation6 Jul 2022 Yuzhang Shang, Dan Xu, Ziliang Zong, Liqiang Nie, Yan Yan

Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit.

Binarization Contrastive Learning +2

Unsupervised High-Resolution Portrait Gaze Correction and Animation

1 code implementation1 Jul 2022 Jichao Zhang, Jingjing Chen, Hao Tang, Enver Sangineto, Peng Wu, Yan Yan, Nicu Sebe, Wei Wang

Solving this problem using an unsupervised method remains an open problem, especially for high-resolution face images in the wild, which are not easy to annotate with gaze and head pose labels.

Image Inpainting Vocal Bursts Intensity Prediction

Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation

1 code implementation15 Jun 2022 Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, Yan Yan

Diffusion probabilistic models (DPMs) have become a popular approach to conditional generation, due to their promising results and support for cross-modal synthesis.

Contrastive Learning Denoising +2

Supplementing Missing Visions via Dialog for Scene Graph Generations

1 code implementation23 Apr 2022 Zhenghao Zhao, Ye Zhu, Xiaoguang Zhu, Yuzhang Shang, Yan Yan

Most current AI systems rely on the premise that the input visual data are sufficient to achieve competitive performance in various computer vision tasks.

Graph Generation Scene Graph Generation

Quantized GAN for Complex Music Generation from Dance Videos

1 code implementation1 Apr 2022 Ye Zhu, Kyle Olszewski, Yu Wu, Panos Achlioptas, Menglei Chai, Yan Yan, Sergey Tulyakov

We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates complex musical samples conditioned on dance videos.

Music Generation

Cross-View Panorama Image Synthesis

1 code implementation22 Mar 2022 Songsong Wu, Hao Tang, Xiao-Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, Yan Yan

In this paper, we tackle the problem of synthesizing a ground-view panorama image conditioned on a top-view aerial image, which is a challenging problem due to the large gap between the two image domains with different view-points.

Image Generation

Deep Transfer Learning with Graph Neural Network for Sensor-Based Human Activity Recognition

no code implementations14 Mar 2022 Yan Yan, Tianzheng Liao, Jinjin Zhao, Jiahong Wang, Liang Ma, Wei Lv, Jing Xiong, Lei Wang

Given this observation, we devised a graph-inspired deep learning approach toward the sensor-based HAR tasks, which was further used to build a deep transfer learning model toward giving a tentative solution for these two challenging problems.

Few-Shot Learning Graph Neural Network +2

Topological EEG Nonlinear Dynamics Analysis for Emotion Recognition

no code implementations14 Mar 2022 Yan Yan, Xuankun Wu, Chengdong Li, Yini He, Zhicheng Zhang, Huihui Li, Ang Li, Lei Wang

The proposed work is the first investigation in the emotion recognition oriented EEG topological feature analysis, which brought a novel insight into the brain neural system nonlinear dynamics analysis and feature extraction.

Arousal Estimation Dominance Estimation +6

Deep Multi-Branch Aggregation Network for Real-Time Semantic Segmentation in Street Scenes

no code implementations8 Mar 2022 Xi Weng, Yan Yan, Genshun Dong, Chang Shu, Biao Wang, Hanzi Wang, Ji Zhang

This shows that DMA-Net provides a good tradeoff between segmentation quality and speed for semantic segmentation in street scenes.

Decoder Real-Time Semantic Segmentation +1

Stage-Aware Feature Alignment Network for Real-Time Semantic Segmentation of Street Scenes

no code implementations8 Mar 2022 Xi Weng, Yan Yan, Si Chen, Jing-Hao Xue, Hanzi Wang

In this paper, we present a novel Stage-aware Feature Alignment Network (SFANet) based on the encoder-decoder structure for real-time semantic segmentation of street scenes.

Decoder Real-Time Semantic Segmentation +1

Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition

no code implementations23 Feb 2022 Xiaoguang Zhu, Ye Zhu, Haoyu Wang, Honglin Wen, Yan Yan, Peilin Liu

To solve the problem, we propose a multi-modality feature fusion network to combine the modalities of the skeleton sequence and RGB frame instead of the RGB video, as the key information contained by the combination of skeleton sequence and RGB frame is close to that of the skeleton sequence and RGB video.

Action Recognition

Win the Lottery Ticket via Fourier Analysis: Frequencies Guided Network Pruning

no code implementations30 Jan 2022 Yuzhang Shang, Bin Duan, Ziliang Zong, Liqiang Nie, Yan Yan

Extensive experiments on CIFAR-10 and CIFAR-100 demonstrate the superiority of our novel Fourier analysis based MBP compared to other traditional MBP algorithms.

Knowledge Distillation Network Pruning

When Facial Expression Recognition Meets Few-Shot Learning: A Joint and Alternate Learning Framework

no code implementations18 Jan 2022 Xinyi Zou, Yan Yan, Jing-Hao Xue, Si Chen, Hanzi Wang

To alleviate the problem of limited base classes in our FER task, we propose a novel Emotion Guided Similarity Network (EGS-Net), consisting of an emotion branch and a similarity branch, based on a two-stage learning framework.

cross-domain few-shot learning Facial Expression Recognition +1

Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos

1 code implementation12 Oct 2021 Zongmeng Zhang, Xianjing Han, Xuemeng Song, Yan Yan, Liqiang Nie

Towards this end, in this work, we propose a Multi-modal Interaction Graph Convolutional Network (MIGCN), which jointly explores the complex intra-modal relations and inter-modal interactions residing in the video and sentence query to facilitate the understanding and semantic correspondence capture of the video and sentence query.

Semantic correspondence Semantic Similarity +2

Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition

1 code implementation8 Oct 2021 Jianyuan Ni, Raunak Sarbajna, Yang Liu, Anne H. H. Ngu, Yan Yan

Human activity recognition (HAR) based on multi-modal approach has been recently shown to improve the accuracy performance of HAR.

Action Recognition Human Activity Recognition +3

Contrastive Mutual Information Maximization for Binary Neural Networks

no code implementations29 Sep 2021 Yuzhang Shang, Dan Xu, Ziliang Zong, Liqiang Nie, Yan Yan

Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit.

Binarization Contrastive Learning +2

Lipschitz Continuity Guided Knowledge Distillation

no code implementations ICCV 2021 Yuzhang Shang, Bin Duan, Ziliang Zong, Liqiang Nie, Yan Yan

Knowledge distillation has become one of the most important model compression techniques by distilling knowledge from larger teacher networks to smaller student ones.

Knowledge Distillation Model Compression +2

Cross-View Exocentric to Egocentric Video Synthesis

no code implementations7 Jul 2021 Gaowen Liu, Hao Tang, Hugo Latapie, Jason Corso, Yan Yan

Particularly, we propose a novel Bi-directional Spatial Temporal Attention Fusion Generative Adversarial Network (STA-GAN) to learn both spatial and temporal information to generate egocentric video sequences from the exocentric view.

Generative Adversarial Network Video Generation

Saying the Unseen: Video Descriptions via Dialog Agents

1 code implementation26 Jun 2021 Ye Zhu, Yu Wu, Yi Yang, Yan Yan

Current vision and language tasks usually take complete visual data (e. g., raw images or videos) as input, however, practical scenarios may often consist the situations where part of the visual information becomes inaccessible due to various reasons e. g., restricted view with fixed camera or intentional vision block for security concerns.

Transfer Learning

Learning Spatial-Semantic Relationship for Facial Attribute Recognition With Limited Labeled Data

no code implementations CVPR 2021 Ying Shu, Yan Yan, Si Chen, Jing-Hao Xue, Chunhua Shen, Hanzi Wang

First, three auxiliary tasks, consisting of a Patch Rotation Task (PRT), a Patch Segmentation Task (PST), and a Patch Classification Task (PCT), are jointly developed to learn the spatial-semantic relationship from large-scale unlabeled facial data.

Attribute Facial Attribute Classification +1

Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation

1 code implementation15 Jun 2021 Cody Blakeney, Nathaniel Huish, Yan Yan, Ziliang Zong

In recent years the ubiquitous deployment of AI has posed great concerns in regards to algorithmic bias, discrimination, and fairness.

Fairness Knowledge Distillation

Adversarial-Metric Learning for Audio-Visual Cross-Modal Matching

1 code implementation IEEE Transactions on Multimedia 2021 Aihua Zheng, Menglan Hu, Bo Jiang *, Yan Huang, Yan Yan, and Bin Luo

AML aims to generate a modality-independent representation for each person in each modality via adversarial learning, while simultaneously learns a robust similarity measure for cross-modality matching via metric learning.

audio-visual learning Metric Learning +1

Hierarchical Representation via Message Propagation for Robust Model Fitting

no code implementations29 Dec 2020 Shuyuan Lin, Xing Wang, Guobao Xiao, Yan Yan, Hanzi Wang

In this paper, we propose a novel hierarchical representation via message propagation (HRMP) method for robust model fitting, which simultaneously takes advantages of both the consensus analysis and the preference analysis to estimate the parameters of multiple model instances from data corrupted by outliers, for robust model fitting.

Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification

4 code implementations ICCV 2021 Zhuoning Yuan, Yan Yan, Milan Sonka, Tianbao Yang

Our studies demonstrate that the proposed DAM method improves the performance of optimizing cross-entropy loss by a large margin, and also achieves better performance than optimizing the existing AUC square loss on these medical image classification tasks.

General Classification Graph Property Prediction +3

Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression

1 code implementation5 Dec 2020 Cody Blakeney, Xiaomin Li, Yan Yan, Ziliang Zong

The experimental results running on an AMD server with four Geforce RTX 2080Ti GPUs show that our algorithm can achieve 3x speedup plus 19% energy savings on VGG distillation, and 3. 5x speedup plus 29% energy savings on ResNet distillation, both with negligible accuracy loss.

Knowledge Distillation Neural Network Compression +3

Robust Visual Tracking via Statistical Positive Sample Generation and Gradient Aware Learning

no code implementations9 Nov 2020 Lijian Lin, Haosheng Chen, Yanjie Liang, Yan Yan, Hanzi Wang

In this paper, we propose a robust tracking method via Statistical Positive sample generation and Gradient Aware learning (SPGA) to address the above two limitations.

Diversity Visual Tracking

Revisiting Optical Flow Estimation in 360 Videos

no code implementations15 Oct 2020 Keshav Bhandari, Ziliang Zong, Yan Yan

Second, we refine the network by training with augmented data in a supervised manner.

Data Augmentation Domain Adaptation +1

Egok360: A 360 Egocentric Kinetic Human Activity Video Dataset

no code implementations15 Oct 2020 Keshav Bhandari, Mario A. DeLaGarza, Ziliang Zong, Hugo Latapie, Yan Yan

To bridge this gap, in this paper we propose a novel Egocentric (first-person) 360{\deg} Kinetic human activity video dataset (EgoK360).

Egocentric Activity Recognition Video Understanding

Photometric and Spectroscopic Study of Flares on Ross 15

no code implementations15 Sep 2020 Jian-Ying Bai, Ali Esamdin, Xing Gao, Yan Yan, Juan-Juan Ren

We conducted photometric and spectroscopic observations for Ross 15 in order to further study the flare properties of this less observed flare star.

Solar and Stellar Astrophysics High Energy Astrophysical Phenomena

Describing Unseen Videos via Multi-Modal Cooperative Dialog Agents

1 code implementation ECCV 2020 Ye Zhu, Yu Wu, Yi Yang, Yan Yan

With the arising concerns for the AI systems provided with direct access to abundant sensitive information, researchers seek to develop more reliable AI with implicit information sources.

Video Description

Hierarchical HMM for Eye Movement Classification

no code implementations18 Aug 2020 Ye Zhu, Yan Yan, Oleg Komogortsev

In this work, we tackle the problem of ternary eye movement classification, which aims to separate fixations, saccades and smooth pursuits from the raw eye positional data.

Classification General Classification

Dual In-painting Model for Unsupervised Gaze Correction and Animation in the Wild

1 code implementation9 Aug 2020 Jichao Zhang, Jingjing Chen, Hao Tang, Wei Wang, Yan Yan, Enver Sangineto, Nicu Sebe

In this paper we address the problem of unsupervised gaze correction in the wild, presenting a solution that works without the need for precise annotations of the gaze angle and the head pose.

How does stock market reflect the change in economic demand? A study on the industry-specific volatility spillover networks of China's stock market during the outbreak of COVID-19

no code implementations15 Jul 2020 Fu Qiao, Yan Yan

At the beginning of the outbreak of COVID-19, in China's stock market, spillover effects from industry indices of sectors meeting the investment demand to those meeting the consumption demands rose significantly.

Correlation filter tracking with adaptive proposal selection for accurate scale estimation

no code implementations14 Jul 2020 Luo Xiong, Yanjie Liang, Yan Yan, Hanzi Wang

In this paper, we propose an adaptive proposal selection algorithm which can generate a small number of high-quality proposals to handle the problem of scale variations for visual object tracking.

Visual Object Tracking

Nearly Optimal Robust Method for Convex Compositional Problems with Heavy-Tailed Noise

no code implementations17 Jun 2020 Yan Yan, Xin Man, Tianbao Yang

In this paper, we propose robust stochastic algorithms for solving convex compositional problems of the form $f(\E_\xi g(\cdot; \xi)) + r(\cdot)$ by establishing {\bf sub-Gaussian confidence bounds} under weak assumptions about the tails of noise distribution, i. e., {\bf heavy-tailed noise} with bounded second-order moments.

Fast Objective & Duality Gap Convergence for Non-Convex Strongly-Concave Min-Max Problems with PL Condition

no code implementations12 Jun 2020 Zhishuai Guo, Yan Yan, Zhuoning Yuan, Tianbao Yang

However, most of the existing algorithms are slow in practice, and their analysis revolves around the convergence to a nearly stationary point. We consider leveraging the Polyak-Lojasiewicz (PL) condition to design faster stochastic algorithms with stronger convergence guarantee.

Multi-Level Generative Models for Partial Label Learning with Non-random Label Noise

no code implementations11 May 2020 Yan Yan, Yuhong Guo

Partial label (PL) learning tackles the problem where each training instance is associated with a set of candidate labels that include both the true label and irrelevant noise labels.

Denoising Partial Label Learning

Incorporating Multiple Cluster Centers for Multi-Label Learning

no code implementations17 Apr 2020 Senlin Shu, Fengmao Lv, Yan Yan, Li Li, Shuo He, Jun He

In this article, we propose to leverage the data augmentation technique to improve the performance of multi-label learning.

Clustering Data Augmentation +1

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

no code implementations11 Mar 2020 Genshun Dong, Yan Yan, Chunhua Shen, Hanzi Wang

Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate high-resolution feature maps preserving the detailed spatial information.

Image Segmentation Segmentation +2

Revisiting SGD with Increasingly Weighted Averaging: Optimization and Generalization Perspectives

no code implementations9 Mar 2020 Zhishuai Guo, Yan Yan, Tianbao Yang

It remains unclear how these averaging schemes affect the convergence of {\it both optimization error and generalization error} (two equally important components of testing error) for {\bf non-strongly convex objectives, including non-convex problems}.

Learning Object Scale With Click Supervision for Object Detection

no code implementations20 Feb 2020 Liao Zhang, Yan Yan, Lin Cheng, Hanzi Wang

Finally, we fuse these CAMs together to generate pseudoground-truths and train a fully-supervised object detector withthese ground-truths.

Object object-detection +1

Hypergraph Optimization for Multi-structural Geometric Model Fitting

no code implementations13 Feb 2020 Shuyuan Lin, Guobao Xiao, Yan Yan, David Suter, Hanzi Wang

Recently, some hypergraph-based methods have been proposed to deal with the problem of model fitting in computer vision, mainly due to the superior capability of hypergraph to represent the complex relationship between data points.

Clustering model

Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max Optimization

no code implementations NeurIPS 2020 Yan Yan, Yi Xu, Qihang Lin, Wei Liu, Tianbao Yang

In this paper, we bridge this gap by providing a sharp analysis of epoch-wise stochastic gradient descent ascent method (referred to as Epoch-GDA) for solving strongly convex strongly concave (SCSC) min-max problems, without imposing any additional assumption about smoothness or the function's structure.

LEMMA

Deep Multi-task Multi-label CNN for Effective Facial Attribute Classification

no code implementations10 Feb 2020 Longbiao Mao, Yan Yan, Jing-Hao Xue, Hanzi Wang

Two different network architectures are respectively designed to extract features for two groups of attributes, and a novel dynamic weighting scheme is proposed to automatically assign the loss weight to each facial attribute during training.

Attribute Face Detection +5

Adaptive Deep Metric Embeddings for Person Re-Identification under Occlusions

no code implementations7 Feb 2020 Wanxiang Yang, Yan Yan, Si Chen

In this paper, we propose a novel person ReID method, which learns the spatial dependencies between the local regions and extracts the discriminative feature representation of the pedestrian image based on Long Short-Term Memory (LSTM), dealing with the problem of occlusions.

Person Re-Identification

Object-Adaptive LSTM Network for Real-time Visual Tracking with Adversarial Data Augmentation

no code implementations7 Feb 2020 Yihan Du, Yan Yan, Si Chen, Yang Hua

This strategy efficiently filters out some irrelevant proposals and avoids the redundant computation for feature extraction, which enables our method to operate faster than conventional classification-based tracking methods.

Computational Efficiency Data Augmentation +3

Joint Deep Learning of Facial Expression Synthesis and Recognition

no code implementations6 Feb 2020 Yan Yan, Ying Huang, Si Chen, Chunhua Shen, Hanzi Wang

Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions.

Deep Learning Facial Expression Recognition +2

Adversarial Paritial Multi-label Learning

no code implementations ICLR 2020 Yan Yan, Yuhong Guo

Partial multi-label learning (PML), which tackles the problem of learning multi-label prediction models from instances with overcomplete noisy annotations, has recently started gaining attention from the research community.

Decoder Generative Adversarial Network +1

Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

2 code implementations CVPR 2020 Hao Tang, Dan Xu, Yan Yan, Philip H. S. Torr, Nicu Sebe

To tackle this issue, in this work we consider learning the scene generation in a local context, and correspondingly design a local class-specific generative network with semantic maps as a guidance, which separately constructs and learns sub-generators concentrating on the generation of different classes, and is able to provide more scene details.

Image Generation Scene Generation

A Simple and Effective Framework for Pairwise Deep Metric Learning

1 code implementation ECCV 2020 Qi Qi, Yan Yan, Xiaoyu Wang, Tianbao Yang

To tackle this issue, we propose a simple and effective framework to sample pairs in a batch of data for updating the model.

Binary Classification Metric Learning

Adversarial Partial Multi-Label Learning

no code implementations15 Sep 2019 Yan Yan, Yuhong Guo

Partial multi-label learning (PML), which tackles the problem of learning multi-label prediction models from instances with overcomplete noisy annotations, has recently started gaining attention from the research community.

Decoder Generative Adversarial Network +1

Time-weighted Attentional Session-Aware Recommender System

no code implementations12 Sep 2019 Mei Wang, Weizhi Li, Yan Yan

Session-based Recurrent Neural Networks (RNNs) are gaining increasing popularity for recommendation task, due to the high autocorrelation of user's behavior on the latest session and the effectiveness of RNN to capture the sequence order information.

Collaborative Filtering Recommendation Systems

Stochastic Optimization for Non-convex Inf-Projection Problems

no code implementations ICML 2020 Yan Yan, Yi Xu, Lijun Zhang, Xiaoyu Wang, Tianbao Yang

In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable.

Stochastic Optimization

Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation

1 code implementation2 Aug 2019 Hao Tang, Dan Xu, Gaowen Liu, Wei Wang, Nicu Sebe, Yan Yan

In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C$^2$GAN) for the task of keypoint-guided image generation.

Generative Adversarial Network Image Generation

Cascade Attention Guided Residue Learning GAN for Cross-Modal Translation

1 code implementation3 Jul 2019 Bin Duan, Wei Wang, Hao Tang, Hugo Latapie, Yan Yan

However, in machine learning, this cross-modal learning is a nontrivial task because different modalities have no homogeneous properties.

BIG-bench Machine Learning Translation

Hallucinated Adversarial Learning for Robust Visual Tracking

no code implementations17 Jun 2019 Qiangqiang Wu, Zhihui Chen, Lin Cheng, Yan Yan, Bo Li, Hanzi Wang

Incorporating such an ability to hallucinate diverse new samples of the tracked instance can help the trackers alleviate the over-fitting problem in the low-data tracking regime.

Visual Tracking

Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation

no code implementations CVPR 2019 Zhen-Yu Zhang, Zhen Cui, Chunyan Xu, Yan Yan, Nicu Sebe, Jian Yang

In this paper, we propose a novel Pattern-Affinitive Propagation (PAP) framework to jointly predict depth, surface normal and semantic segmentation.

Monocular Depth Estimation Semantic Segmentation

GazeCorrection:Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks

no code implementations arXiv 2019 Jichao Zhang, Meng Sun, Jingjing Chen, Hao Tang, Yan Yan, Xueying Qin, Nicu Sebe

Gaze correction aims to redirect the person's gaze into the camera by manipulating the eye region, and it can be considered as a specific image resynthesis problem.

Resynthesis

Expression Conditional GAN for Facial Expression-to-Expression Translation

no code implementations14 May 2019 Hao Tang, Wei Wang, Songsong Wu, Xinya Chen, Dan Xu, Nicu Sebe, Yan Yan

In this paper, we focus on the facial expression translation task and propose a novel Expression Conditional GAN (ECGAN) which can learn the mapping from one image domain to another one based on an additional expression attribute.

Attribute Facial expression generation +2

Joint Learning of Self-Representation and Indicator for Multi-View Image Clustering

no code implementations11 May 2019 Songsong Wu, Zhiqiang Lu, Hao Tang, Yan Yan, Songhao Zhu, Xiao-Yuan Jing, Zuoyong Li

Multi-view subspace clustering aims to divide a set of multisource data into several groups according to their underlying subspace structure.

Clustering Multi-view Subspace Clustering

Structured Discriminative Tensor Dictionary Learning for Unsupervised Domain Adaptation

no code implementations11 May 2019 Songsong Wu, Yan Yan, Hao Tang, Jianjun Qian, Jian Zhang, Xiao-Yuan Jing

However, the number of labeled source samples are always limited due to expensive annotation cost in practice, making sub-optimal performance been observed.

Dictionary Learning Pseudo Label +1

Stochastic Primal-Dual Algorithms with Faster Convergence than $O(1/\sqrt{T})$ for Problems without Bilinear Structure

no code implementations23 Apr 2019 Yan Yan, Yi Xu, Qihang Lin, Lijun Zhang, Tianbao Yang

The main contribution of this paper is the design and analysis of new stochastic primal-dual algorithms that use a mixture of stochastic gradient updates and a logarithmic number of deterministic dual updates for solving a family of convex-concave problems with no bilinear structure assumed.

Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation

3 code implementations CVPR 2019 Hao Tang, Dan Xu, Nicu Sebe, Yanzhi Wang, Jason J. Corso, Yan Yan

In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (SelectionGAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map.

Bird View Synthesis Cross-View Image-to-Image Translation +1

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

8 code implementations28 Mar 2019 Hao Tang, Dan Xu, Nicu Sebe, Yan Yan

To handle the limitation, in this paper we propose a novel Attention-Guided Generative Adversarial Network (AGGAN), which can detect the most discriminative semantic object and minimize changes of unwanted part for semantic manipulation problems without using extra data and models.

Generative Adversarial Network Translation +1

Attribute-Guided Sketch Generation

1 code implementation28 Jan 2019 Hao Tang, Xinya Chen, Wei Wang, Dan Xu, Jason J. Corso, Nicu Sebe, Yan Yan

To this end, we propose a novel Attribute-Guided Sketch Generative Adversarial Network (ASGAN) which is an end-to-end framework and contains two pairs of generators and discriminators, one of which is used to generate faces with attributes while the other one is employed for image-to-sketch translation.

Attribute Generative Adversarial Network +1

Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

1 code implementation14 Jan 2019 Hao Tang, Dan Xu, Wei Wang, Yan Yan, Nicu Sebe

State-of-the-art methods for image-to-image translation with Generative Adversarial Networks (GANs) can learn a mapping from one domain to another domain using unpaired image data.

Generative Adversarial Network Image-to-Image Translation +1

Stagewise Training Accelerates Convergence of Testing Error Over SGD

no code implementations NeurIPS 2019 Zhuoning Yuan, Yan Yan, Rong Jin, Tianbao Yang

For convex loss functions and two classes of "nice-behaviored" non-convex objectives that are close to a convex function, we establish faster convergence of stagewise training than the vanilla SGD under the PL condition on both training error and testing error.

DSNet: Deep and Shallow Feature Learning for Efficient Visual Tracking

no code implementations6 Nov 2018 Qiangqiang Wu, Yan Yan, Yanjie Liang, Yi Liu, Hanzi Wang

In recent years, Discriminative Correlation Filter (DCF) based tracking methods have achieved great success in visual tracking.

Image Classification Visual Tracking