Search Results for author: Siwei Lyu

Found 78 papers, 17 papers with code

Open-Eye: An Open Platform to Study Human Performance on Identifying AI-Synthesized Faces

no code implementations13 May 2022 Hui Guo, Shu Hu, Xin Wang, Ming-Ching Chang, Siwei Lyu

In this work, we develop an online platform called Open-eye to study the human performance of AI-synthesized face detection.

Face Detection

Fusing Global and Local Features for Generalized AI-Synthesized Image Detection

no code implementations26 Mar 2022 Yan Ju, Shan Jia, Lipeng Ke, Hongfei Xue, Koki Nagano, Siwei Lyu

Specifically, we design a two-branch model to combine global spatial information from the whole image and local informative features from multiple patches selected by a novel patch selection module.

Model Attribution of Face-swap Deepfake Videos

1 code implementation25 Feb 2022 Shan Jia, Xin Li, Siwei Lyu

Then we take Deepfakes model attribution as a multiclass classification task and propose a spatial and temporal attention based method to explore the differences among Deepfakes in the new dataset.

Face Swapping

GAN-generated Faces Detection: A Survey and New Perspectives (2022)

no code implementations15 Feb 2022 Xin Wang, Hui Guo, Shu Hu, Ming-Ching Chang, Siwei Lyu

Generative Adversarial Networks (GAN) have led to the generation of very realistic face images, which have been used in fake social media accounts and other disinformation matters that can generate profound impacts.

Face Detection

Towards To-a-T Spatio-Temporal Focus for Skeleton-Based Action Recognition

no code implementations4 Feb 2022 Lipeng Ke, Kuan-Chuan Peng, Siwei Lyu

Graph Convolutional Networks (GCNs) have been widely used to model the high-order dynamic dependencies for skeleton-based action recognition.

Action Recognition Skeleton Based Action Recognition

Differentially Private SGDA for Minimax Problems

no code implementations22 Jan 2022 Zhenhuan Yang, Shu Hu, Yunwen Lei, Kush R. Varshney, Siwei Lyu, Yiming Ying

We further provide its utility analysis in the nonconvex-strongly-concave setting which is the first-ever-known result in terms of the primal population risk.

Stochastic Actor-Executor-Critic for Image-to-Image Translation

no code implementations14 Dec 2021 Ziwei Luo, Jing Hu, Xin Wang, Siwei Lyu, Bin Kong, Youbing Yin, Qi Song, Xi Wu

Training a model-free deep reinforcement learning model to solve image-to-image translation is difficult since it involves high-dimensional continuous state and action spaces.

Continuous Control Image-to-Image Translation +2

VocBench: A Neural Vocoder Benchmark for Speech Synthesis

1 code implementation6 Dec 2021 Ehab A. AlBadawy, Andrew Gibiansky, Qing He, JiLong Wu, Ming-Ching Chang, Siwei Lyu

We perform a subjective and objective evaluation to compare the performance of each vocoder along a different axis.

Speech Synthesis

Learnable Discrete Wavelet Pooling (LDW-Pooling) For Convolutional Networks

no code implementations13 Sep 2021 Bor-Shiun Wang, Jun-Wei Hsieh, Ming-Ching Chang, Ping-Yang Chen, Lipeng Ke, Siwei Lyu

We introduce the Learning Discrete Wavelet Pooling (LDW-Pooling) that can be applied universally to replace standard pooling operations to better extract features with improved accuracy and efficiency.

Robust Attentive Deep Neural Network for Exposing GAN-generated Faces

no code implementations5 Sep 2021 Hui Guo, Shu Hu, Xin Wang, Ming-Ching Chang, Siwei Lyu

However, images from existing public datasets do not represent real-world scenarios well enough in terms of view variations and data distributions (where real faces largely outnumber synthetic faces).

Face Detection

Eyes Tell All: Irregular Pupil Shapes Reveal GAN-generated Faces

no code implementations1 Sep 2021 Hui Guo, Shu Hu, Xin Wang, Ming-Ching Chang, Siwei Lyu

Generative adversary network (GAN) generated high-realistic human faces have been used as profile images for fake social media accounts and are visually challenging to discern from real ones.

T$_k$ML-AP: Adversarial Attacks to Top-$k$ Multi-Label Learning

1 code implementation31 Jul 2021 Shu Hu, Lipeng Ke, Xin Wang, Siwei Lyu

Top-$k$ multi-label learning, which returns the top-$k$ predicted labels from an input, has many practical applications such as image annotation, document analysis, and web search engine.

Multi-Label Learning

Sum of Ranked Range Loss for Supervised Learning

1 code implementation7 Jun 2021 Shu Hu, Yiming Ying, Xin Wang, Siwei Lyu

A combination loss of AoRR and TKML is proposed as a new learning objective for improving the robustness of multi-label learning in the face of outliers in sample and labels alike.

Multi-class Classification Multi-Label Learning

Imperceptible Adversarial Examples for Fake Image Detection

no code implementations3 Jun 2021 Quanyu Liao, Yuezun Li, Xin Wang, Bin Kong, Bin Zhu, Siwei Lyu, Youbing Yin, Qi Song, Xi Wu

Fooling people with highly realistic fake images generated with Deepfake or GANs brings a great social disturbance to our society.

Face Swapping Fake Image Detection

Transferable Adversarial Examples for Anchor Free Object Detection

no code implementations3 Jun 2021 Quanyu Liao, Xin Wang, Bin Kong, Siwei Lyu, Bin Zhu, Youbing Yin, Qi Song, Xi Wu

Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbation can completely change prediction result.

Adversarial Attack Object Detection

Detection, Tracking, and Counting Meets Drones in Crowds: A Benchmark

1 code implementation CVPR 2021 Longyin Wen, Dawei Du, Pengfei Zhu, QinGhua Hu, Qilong Wang, Liefeng Bo, Siwei Lyu

To promote the developments of object detection, tracking and counting algorithms in drone-captured videos, we construct a benchmark with a new drone-captured largescale dataset, named as DroneCrowd, formed by 112 video clips with 33, 600 HD frames in various scenarios.

Object Detection

DeepFake-o-meter: An Open Platform for DeepFake Detection

no code implementations2 Mar 2021 Yuezun Li, Cong Zhang, Pu Sun, Honggang Qi, Siwei Lyu

In recent years, the advent of deep learning-based techniques and the significant reduction in the cost of computation resulted in the feasibility of creating realistic videos of human faces, commonly known as DeepFakes.

DeepFake Detection Face Swapping

Landmark Breaker: Obstructing DeepFake By Disturbing Landmark Extraction

no code implementations1 Feb 2021 Pu Sun, Yuezun Li, Honggang Qi, Siwei Lyu

In this paper, we describe Landmark Breaker, the first dedicated method to disrupt facial landmark extraction, and apply it to the obstruction of the generation of DeepFake videos. Our motivation is that disrupting the facial landmark extraction can affect the alignment of input face so as to degrade the DeepFake quality.

Face Swapping

TkML-AP: Adversarial Attacks to Top-k Multi-Label Learning

1 code implementation ICCV 2021 Shu Hu, Lipeng Ke, Xin Wang, Siwei Lyu

Top-k multi-label learning, which returns the top-k predicted labels from an input, has many practical applications such as image annotation, document analysis, and web search engine.

Multi-Label Learning

LandmarkGAN: Synthesizing Faces from Landmarks

no code implementations31 Oct 2020 Pu Sun, Yuezun Li, Honggang Qi, Siwei Lyu

Face synthesis is an important problem in computer vision with many applications.

Face Generation

Learning by Minimizing the Sum of Ranked Range

1 code implementation NeurIPS 2020 Shu Hu, Yiming Ying, Xin Wang, Siwei Lyu

In forming learning objectives, one oftentimes needs to aggregate a set of individual values to a single output.

General Classification Multi-class Classification +1

MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network

no code implementations3 Oct 2020 Yi Wei, Zhe Gan, Wenbo Li, Siwei Lyu, Ming-Ching Chang, Lei Zhang, Jianfeng Gao, Pengchuan Zhang

We present Mask-guided Generative Adversarial Network (MagGAN) for high-resolution face attribute editing, in which semantic facial masks from a pre-trained face parser are used to guide the fine-grained image editing process.

Exposing GAN-generated Faces Using Inconsistent Corneal Specular Highlights

no code implementations24 Sep 2020 Shu Hu, Yuezun Li, Siwei Lyu

We show that such artifacts exist widely in high-quality GAN synthesized faces and further describe an automatic method to extract and compare corneal specular highlights from two eyes.

Cascade Graph Neural Networks for RGB-D Salient Object Detection

1 code implementation ECCV 2020 Ao Luo, Xin Li, Fan Yang, Zhicheng Jiao, Hong Cheng, Siwei Lyu

Current works either simply distill prior knowledge from the corresponding depth map for handling the RGB-image or blindly fuse color and geometric information to generate the coarse depth-aware representations, hindering the performance of RGB-D saliency detectors. In this work, we introduceCascade Graph Neural Networks(Cas-Gnn), a unified framework which is capable of comprehensively distilling and reasoning the mutual benefits between these two data sources through a set of cascade graphs, to learn powerful representations for RGB-D salient object detection.

RGB-D Salient Object Detection RGB Salient Object Detection +1

DeepFake Detection: Current Challenges and Next Steps

no code implementations11 Mar 2020 Siwei Lyu

High quality fake videos and audios generated by AI-algorithms (the deep fakes) have started to challenge the status of videos and audios as definitive evidence of events.

DeepFake Detection Face Swapping

Category-wise Attack: Transferable Adversarial Examples for Anchor Free Object Detection

no code implementations10 Feb 2020 Quanyu Liao, Xin Wang, Bin Kong, Siwei Lyu, Youbing Yin, Qi Song, Xi Wu

Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbations can completely change the classification results.

Object Detection

Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting

no code implementations5 Feb 2020 Xian Zhang, Xin Wang, Bin Kong, Youbing Yin, Qi Song, Siwei Lyu, Jiancheng Lv, Canghong Shi, Xiaojie Li

We firstly represent only face regions using the latent variable as the domain knowledge and combine it with the non-face parts textures to generate high-quality face images with plausible contents.

Facial Inpainting

Learning Semantic Neural Tree for Human Parsing

no code implementations ECCV 2020 Ruyi Ji, Dawei Du, Libo Zhang, Longyin Wen, Yanjun Wu, Chen Zhao, Feiyue Huang, Siwei Lyu

In this paper, we design a novel semantic neural tree for human parsing, which uses a tree architecture to encode physiological structure of human body, and designs a coarse to fine process in a cascade manner to generate accurate results.

Human Parsing Semantic Segmentation

Drone-based Joint Density Map Estimation, Localization and Tracking with Space-Time Multi-Scale Attention Network

1 code implementation4 Dec 2019 Longyin Wen, Dawei Du, Pengfei Zhu, QinGhua Hu, Qilong Wang, Liefeng Bo, Siwei Lyu

This paper proposes a space-time multi-scale attention network (STANet) to solve density map estimation, localization and tracking in dense crowds of video clips captured by drones with arbitrary crowd density, perspective, and flight altitude.

Crowd Counting

Fast Portrait Segmentation with Highly Light-weight Network

no code implementations19 Oct 2019 Yuezun Li, Ao Luo, Siwei Lyu

In this paper, we describe a fast and light-weight portrait segmentation method based on a new highly light-weight backbone (HLB) architecture.

Portrait Segmentation

Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics

6 code implementations CVPR 2020 Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, Siwei Lyu

AI-synthesized face-swapping videos, commonly known as DeepFakes, is an emerging problem threatening the trustworthiness of online information.

DeepFake Detection Face Swapping

Guided Attention Network for Object Detection and Counting on Drones

no code implementations25 Sep 2019 Yuan-Qiang Cai, Dawei Du, Libo Zhang, Longyin Wen, Weiqiang Wang, Yanjun Wu, Siwei Lyu

Object detection and counting are related but challenging problems, especially for drone based scenes with small objects and cluttered background.

Object Detection

Scale Invariant Fully Convolutional Network: Detecting Hands Efficiently

no code implementations11 Jun 2019 Dan Liu, Dawei Du, Libo Zhang, Tiejian Luo, Yanjun Wu, Feiyue Huang, Siwei Lyu

Existing hand detection methods usually follow the pipeline of multiple stages with high computation cost, i. e., feature extraction, region proposal, bounding box regression, and additional layers for rotated region detection.

Hand Detection Region Proposal

Exposing GAN-synthesized Faces Using Landmark Locations

no code implementations30 Mar 2019 Xin Yang, Yuezun Li, Honggang Qi, Siwei Lyu

Generative adversary networks (GANs) have recently led to highly realistic image synthesis results.

General Classification Image Generation

Object-driven Text-to-Image Synthesis via Adversarial Training

1 code implementation CVPR 2019 Wenbo Li, Pengchuan Zhang, Lei Zhang, Qiuyuan Huang, Xiaodong He, Siwei Lyu, Jianfeng Gao

In this paper, we propose Object-driven Attentive Generative Adversarial Newtorks (Obj-GANs) that allow object-centered text-to-image synthesis for complex scenes.

Image Generation

De-identification without losing faces

no code implementations12 Feb 2019 Yuezun Li, Siwei Lyu

In this work, we describe a new face de-identification method that can preserve essential facial attributes in the faces while concealing the identities.


Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding

no code implementations29 Jan 2019 Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Kunlin Cao, Qi Song, Shaoting Zhang, Siwei Lyu, Youbing Yin

In order to address these limitations, we present tree-structured ConvLSTM models for tree-structured image analysis tasks which can be trained end-to-end.

Residual Attention based Network for Hand Bone Age Assessment

no code implementations21 Dec 2018 Eric Wu, Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Shaoting Zhang, Kunlin Cao, Qi Song, Siwei Lyu, Youbing Yin

The hierarchical attention components of the residual attention subnet force our network to focus on the key components of the X-ray images and generate the final predictions as well as the associated visual supports, which is similar to the assessment procedure of clinicians.

Hand Segmentation

Learning Non-Uniform Hypergraph for Multi-Object Tracking

no code implementations10 Dec 2018 Longyin Wen, Dawei Du, Shengkun Li, Xiao Bian, Siwei Lyu

The majority of Multi-Object Tracking (MOT) algorithms based on the tracking-by-detection scheme do not use higher order dependencies among objects or tracklets, which makes them less effective in handling complex scenarios.

Multi-Object Tracking

Evolvement Constrained Adversarial Learning for Video Style Transfer

no code implementations6 Nov 2018 Wenbo Li, Longyin Wen, Xiao Bian, Siwei Lyu

Video style transfer is a useful component for applications such as augmented reality, non-photorealistic rendering, and interactive games.

Optical Flow Estimation Style Transfer +1

Exposing DeepFake Videos By Detecting Face Warping Artifacts

3 code implementations1 Nov 2018 Yuezun Li, Siwei Lyu

Compared to previous methods which use a large amount of real and DeepFake generated images to train CNN classifier, our method does not need DeepFake generated images as negative training examples since we target the artifacts in affine face warping as the distinctive feature to distinguish real and fake images.

Face Swapping

Exposing Deep Fakes Using Inconsistent Head Poses

1 code implementation1 Nov 2018 Xin Yang, Yuezun Li, Siwei Lyu

In this paper, we propose a new method to expose AI-generated fake face images or videos (commonly known as the Deep Fakes).

General Classification

Robust Adversarial Perturbation on Deep Proposal-based Models

no code implementations16 Sep 2018 Yuezun Li, Daniel Tian, Ming-Ching Chang, Xiao Bian, Siwei Lyu

Adversarial noises are useful tools to probe the weakness of deep learning based computer vision algorithms.

Instance Segmentation Region Proposal +1

Exploring the Vulnerability of Single Shot Module in Object Detectors via Imperceptible Background Patches

no code implementations16 Sep 2018 Yuezun Li, Xiao Bian, Ming-Ching Chang, Siwei Lyu

In this paper, we focus on exploring the vulnerability of the Single Shot Module (SSM) commonly used in recent object detectors, by adding small perturbations to patches in the background outside the object.

Region Proposal

Multi-Scale Supervised Network for Human Pose Estimation

no code implementations5 Aug 2018 Lipeng Ke, Ming-Ching Chang, Honggang Qi, Siwei Lyu

Human pose estimation is an important topic in computer vision with many applications including gesture and activity recognition.

Activity Recognition Keypoint Detection

Who did What at Where and When: Simultaneous Multi-Person Tracking and Activity Recognition

no code implementations3 Jul 2018 Wenbo Li, Ming-Ching Chang, Siwei Lyu

We present a bootstrapping framework to simultaneously improve multi-person tracking and activity recognition at individual, interaction and social group activity levels.

Activity Recognition Visual Tracking

Stochastic Proximal Algorithms for AUC Maximization

no code implementations ICML 2018 Michael Natole, Yiming Ying, Siwei Lyu

Stochastic optimization algorithms such as SGDs update the model sequentially with cheap per-iteration costs, making them amenable for large-scale data analysis.

Classification General Classification +2

In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking

3 code implementations7 Jun 2018 Yuezun Li, Ming-Ching Chang, Siwei Lyu

The new developments in deep generative networks have significantly improve the quality and efficiency in generating realistically-looking fake face videos.

Face Swapping

STS Classification with Dual-stream CNN

no code implementations20 May 2018 Shuchen Weng, Wenbo Li, Yi Zhang, Siwei Lyu

Inspired by the dual-stream hypothesis in neural science, we propose a novel dual-stream framework for modeling the interweaved spatiotemporal dependency, and develop a convolutional neural network within this framework that aims to achieve high adaptability and flexibility in STS configurations from various diagonals, i. e., sequential order, dependency range and features.

Activity Recognition Classification +2

A Univariate Bound of Area Under ROC

no code implementations16 Apr 2018 Siwei Lyu, Yiming Ying

In this work, we describe a new surrogate loss based on a reformulation of the AUC risk, which does not require pairwise comparison but rankings of the predictions.

online learning

Tagging like Humans: Diverse and Distinct Image Annotation

no code implementations CVPR 2018 Baoyuan Wu, Weidong Chen, Peng Sun, Wei Liu, Bernard Ghanem, Siwei Lyu

In D2IA, we generate a relevant and distinct tag subset, in which the tags are relevant to the image contents and semantically distinct to each other, using sequential sampling from a determinantal point process (DPP) model.


Multi-label Learning with Missing Labels using Mixed Dependency Graphs

no code implementations31 Mar 2018 Baoyuan Wu, Fan Jia, Wei Liu, Bernard Ghanem, Siwei Lyu

This work focuses on the problem of multi-label learning with missing labels (MLML), which aims to label each test instance with multiple class labels given training instances that have an incomplete/partial set of these labels.

Image Retrieval Multi-Label Learning +1

Contrast Enhancement Estimation for Digital Image Forensics

no code implementations13 Jun 2017 Longyin Wen, Honggang Qi, Siwei Lyu

Our method recovers the original pixel histogram and the contrast enhancement simultaneously from a single image with an iterative algorithm.

Image Forensics

Learning with Average Top-k Loss

no code implementations NeurIPS 2017 Yanbo Fan, Siwei Lyu, Yiming Ying, Bao-Gang Hu

We further give a learning theory analysis of \matk learning on the classification calibration of the \atk loss and the error bounds of \atk-SVM.

General Classification Learning Theory

Robust Localized Multi-view Subspace Clustering

no code implementations22 May 2017 Yanbo Fan, Jian Liang, Ran He, Bao-Gang Hu, Siwei Lyu

In multi-view clustering, different views may have different confidence levels when learning a consensus representation.

Multi-view Subspace Clustering

Stochastic Online AUC Maximization

no code implementations NeurIPS 2016 Yiming Ying, Longyin Wen, Siwei Lyu

From this saddle representation, a stochastic online algorithm (SOLAM) is proposed which has time and space complexity of one datum.

online learning

LSTM with Working Memory

no code implementations6 May 2016 Andrew Pulver, Siwei Lyu

Our architecture is still simple and achieves better performance on the tasks that we tested on.

Geometric Hypergraph Learning for Visual Tracking

no code implementations18 Mar 2016 Dawei Du, Honggang Qi, Longyin Wen, Qi Tian, Qingming Huang, Siwei Lyu

Graph based representation is widely used in visual tracking field by finding correct correspondences between target parts in consecutive frames.

Visual Tracking

Category-Blind Human Action Recognition: A Practical Recognition System

no code implementations ICCV 2015 Wenbo Li, Longyin Wen, Mooi Choo Chuah, Siwei Lyu

In this paper, we propose the category-blind human recognition method (CHARM) which can recognize a human action without making assumptions of the action category.

Action Recognition

ML-MG: Multi-Label Learning With Missing Labels Using a Mixed Graph

no code implementations ICCV 2015 Baoyuan Wu, Siwei Lyu, Bernard Ghanem

This work focuses on the problem of multi-label learning with missing labels (MLML), which aims to label each test instance with multiple class labels given training instances that have an incomplete/partial set of these labels (i. e. some of their labels are missing).

Multi-Label Learning

UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking

no code implementations13 Nov 2015 Longyin Wen, Dawei Du, Zhaowei Cai, Zhen Lei, Ming-Ching Chang, Honggang Qi, Jongwoo Lim, Ming-Hsuan Yang, Siwei Lyu

In this work, we perform a comprehensive quantitative study on the effects of object detection accuracy to the overall MOT performance, using the new large-scale University at Albany DETection and tRACking (UA-DETRAC) benchmark dataset.

Multi-Object Tracking Object Detection

Improving Image Restoration with Soft-Rounding

no code implementations ICCV 2015 Xing Mei, Honggang Qi, Bao-Gang Hu, Siwei Lyu

In this work, we describe an effective and efficient approach to incorporate the knowledge of distinct pixel values of the pristine images into the general regularized least squares restoration framework.

Image Restoration SSIM

On Algorithms for Sparse Multi-factor NMF

no code implementations NeurIPS 2013 Siwei Lyu, Xin Wang

Nonnegative matrix factorization (NMF) is a popular data analysis method, the objective of which is to decompose a matrix with all nonnegative components into the product of two other nonnegative matrices.

Learning with Target Prior

no code implementations NeurIPS 2012 Zuoguan Wang, Siwei Lyu, Gerwin Schalk, Qiang Ji

In this work, we describe a new learning scheme for parametric learning, in which the target variables $\y$ can be modeled with a prior model $p(\y)$ and the relations between data and target variables are estimated through $p(\y)$ and a set of uncorresponded data $\x$ in training.

Pose Estimation

Unifying Non-Maximum Likelihood Learning Objectives with Minimum KL Contraction

no code implementations NeurIPS 2011 Siwei Lyu

When used to learn high dimensional parametric probabilistic models, the clas- sical maximum likelihood (ML) learning often suffers from computational in- tractability, which motivates the active developments of non-ML learning meth- ods.


Divisive Normalization: Justification and Effectiveness as Efficient Coding Transform

no code implementations NeurIPS 2010 Siwei Lyu

Our analysis is based on the use of multivariate {\em t} model to capture some important statistical properties of natural sensory signals.

Reducing statistical dependencies in natural signals using radial Gaussianization

no code implementations NeurIPS 2008 Siwei Lyu, Eero P. Simoncelli

In this case, no linear transform suffices to properly decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial Gaussianization (RG), is able to remove all dependencies.

Cannot find the paper you are looking for? You can Submit a new open access paper.