Search Results for author: Yi Zhou

Found 148 papers, 32 papers with code

On the Transferability of Adversarial Attacks against Neural Text Classifier

no code implementations EMNLP 2021 Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang

Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.

Text Classification

SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities

1 code implementation30 Apr 2022 Pengbo Hu, Xingyu Li, Yi Zhou

Our experiments suggest that for some tasks where different modalities are complementary, the multi-modal models still tend to use the dominant modality alone and ignore the cooperation across modalities.

Data Sampling Affects the Complexity of Online SGD over Dependent Data

no code implementations31 Mar 2022 Shaocong Ma, Ziyi Chen, Yi Zhou, Kaiyi Ji, Yingbin Liang

Moreover, we show that online SGD with mini-batch sampling can further substantially improve the sample complexity over online SGD with periodic data-subsampling over highly dependent data.

Stochastic Optimization

A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimization

no code implementations30 Mar 2022 Ziyi Chen, Bhavya Kailkhura, Yi Zhou

In this work, we study a proximal gradient-type algorithm that adopts the approximate implicit differentiation (AID) scheme for nonconvex bi-level optimization with possibly nonconvex and nonsmooth regularizers.

Delving into the Estimation Shift of Batch Normalization in a Network

1 code implementation21 Mar 2022 Lei Huang, Yi Zhou, Tian Wang, Jie Luo, Xianglong Liu

We define the estimation shift magnitude of BN to quantitatively measure the difference between its estimated population statistics and expected ones.

Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings

1 code implementation14 Mar 2022 Yi Zhou, Masahiro Kaneko, Danushka Bollegala

Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.

Word Embeddings

Extended Load Flexibility of Industrial P2H Plants: A Process Constraint-Aware Scheduling Approach

no code implementations6 Mar 2022 Yiwei Qiu, Buxiang Zhou, Tianlei Zang, Yi Zhou, Ruomei Qi, Jin Lin

The operational flexibility of industrial power-to-hydrogen (P2H) plants enables admittance of volatile renewable power and provides auxiliary regulatory services for the power grid.

Listing Maximal k-Plexes in Large Real-World Graphs

1 code implementation17 Feb 2022 Zhengren Wang, Yi Zhou, Mingyu Xiao, Bakhadyr Khoussainov

Our first contribution is algorithm ListPlex that lists all maximal $k$-plexes in $O^*(\gamma^D)$ time for each constant $k$, where $\gamma$ is a value related to $k$ but strictly smaller than 2, and $D$ is the degeneracy of the graph that is far less than the vertex number $n$ in real-word graphs.

Community Detection

On the Convergence of Gradient Extrapolation Methods for Unbalanced Optimal Transport

no code implementations8 Feb 2022 Quang Minh Nguyen, Hoang H. Nguyen, Yi Zhou, Lam M. Nguyen

We study the Unbalanced Optimal Transport (UOT) between two measures of possibly different masses with at most $n$ components, where marginal constraints of the standard Optimal Transport (OT) are relaxed via Kullback-Leibler divergence with regularization factor $\tau$.

Coordinated Frequency Control through Safe Reinforcement Learning

no code implementations30 Jan 2022 Yi Zhou, Liangcai Zhou, Di Shi, Xiaoying Zhao

With widespread deployment of renewables, the electric power grids are experiencing increasing dynamics and uncertainties, with its secure operation being threatened.

Decision Making reinforcement-learning +1

Accelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learning

no code implementations22 Dec 2021 Ziyi Chen, Shaocong Ma, Yi Zhou

Alternating gradient-descent-ascent (AltGDA) is an optimization algorithm that has been widely used for model training in various machine learning applications, which aims to solve a nonconvex minimax optimization problem.

Slot-VPS: Object-centric Representation Learning for Video Panoptic Segmentation

no code implementations16 Dec 2021 Yi Zhou, HUI ZHANG, Hana Lee, Shuyang Sun, Pingjun Li, Yangguang Zhu, ByungIn Yoo, Xiaojuan Qi, Jae-Joon Han

We encode all panoptic entities in a video, including both foreground instances and background semantics, with a unified representation called panoptic slots.

Panoptic Segmentation Representation Learning

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

no code implementations15 Dec 2021 Yi Zhou, Parikshit Ram, Theodoros Salonidis, Nathalie Baracaldo, Horst Samulowitz, Heiko Ludwig

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO).

Federated Learning

Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks

no code implementations21 Nov 2021 Kaiyuan Liu, Xingyu Li, Yurui Lai, Ge Zhang, Hang Su, Jiachen Wang, Chunxu Guo, Jisong Guan, Yi Zhou

Despite its great success, deep learning severely suffers from robustness; that is, deep neural networks are very vulnerable to adversarial attacks, even the simplest ones.

Event-based Motion Segmentation by Cascaded Two-Level Multi-Model Fitting

no code implementations5 Nov 2021 Xiuyuan Lu, Yi Zhou, Shaojie Shen

In this paper, we present a cascaded two-level multi-model fitting method for identifying independently moving objects (i. e., the motion segmentation problem) with a monocular event camera.

Motion Segmentation

Finding Local Minimax Points via (Stochastic) Cubic-Regularized GDA: Global Convergence and Complexity

no code implementations14 Oct 2021 Ziyi Chen, Qunwei Li, Yi Zhou

Standard gradient descent-ascent (GDA)-type algorithms can only find stationary points in nonconvex minimax optimization, which are far more sub-optimal than local minimax points.

Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy

no code implementations PACLIC 2021 Yi Zhou, Danushka Bollegala

Contextualised word embeddings generated from Neural Language Models (NLMs), such as BERT, represent a word with a vector that considers the semantics of the target word as well its context.

Word Embeddings Word Sense Disambiguation

Sample Efficient Stochastic Policy Extragradient Algorithm for Zero-Sum Markov Game

no code implementations ICLR 2022 Ziyi Chen, Shaocong Ma, Yi Zhou

Two-player zero-sum Markov game is a fundamental problem in reinforcement learning and game theory.

Escaping Saddle Points in Nonconvex Minimax Optimization via Cubic-Regularized Gradient Descent-Ascent

no code implementations29 Sep 2021 Ziyi Chen, Qunwei Li, Yi Zhou

Our result shows that Cubic-GDA achieves an orderwise faster convergence rate than the standard GDA for a wide spectrum of gradient dominant geometry.

How to Improve Sample Complexity of SGD over Highly Dependent Data?

no code implementations29 Sep 2021 Shaocong Ma, Ziyi Chen, Yi Zhou, Kaiyi Ji, Yingbin Liang

Specifically, with a $\phi$-mixing model that captures both exponential and polynomial decay of the data dependence over time, we show that SGD with periodic data-subsampling achieves an improved sample complexity over the standard SGD in the full spectrum of the $\phi$-mixing data dependence.

Stochastic Optimization

Assisted Learning for Organizations with Limited Imbalanced Data

no code implementations29 Sep 2021 Cheng Chen, Jiaying Zhou, Jie Ding, Yi Zhou, Bhavya Kailkhura

We develop an assisted learning framework for assisting organization-level learners to improve their learning performance with limited and imbalanced data.

Decision Making reinforcement-learning

Assisted Learning for Organizations with Limited Data

no code implementations20 Sep 2021 Cheng Chen, Jiaying Zhou, Jie Ding, Yi Zhou

We develop an assisted learning framework for assisting organization-level learners to improve their learning performance with limited and imbalanced data.

Decision Making reinforcement-learning

Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis

no code implementations8 Sep 2021 Ziyi Chen, Yi Zhou, Rongrong Chen, Shaofeng Zou

Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy.

Specificity-preserving RGB-D Saliency Detection

3 code implementations ICCV 2021 Tao Zhou, Deng-Ping Fan, Geng Chen, Yi Zhou, Huazhu Fu

To effectively fuse cross-modal features in the shared learning network, we propose a cross-enhanced integration module (CIM) and then propagate the fused feature to the next layer for integrating cross-level information.

Object Detection Saliency Prediction +1

Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble

no code implementations ACL 2021 Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang

Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.

LEGATO: A LayerwisE Gradient AggregaTiOn Algorithm for Mitigating Byzantine Attacks in Federated Learning

no code implementations26 Jul 2021 Kamala Varma, Yi Zhou, Nathalie Baracaldo, Ali Anwar

This global model can be corrupted when Byzantine workers send malicious gradients, which necessitates robust methods for aggregating gradients that mitigate the adverse effects of Byzantine inputs.

Federated Learning

Exploiting Semantic Embedding and Visual Feature for Facial Action Unit Detection

no code implementations CVPR 2021 Huiyuan Yang, Lijun Yin, Yi Zhou, Jiuxiang Gu

The learned AU semantic embeddings are then used as guidance for the generation of attention maps through a cross-modality attention network.

Action Unit Detection Facial Action Unit Detection

Improving Entity Linking through Semantic Reinforced Entity Embeddings

1 code implementation ACL 2020 Feng Hou, Ruili Wang, Jun He, Yi Zhou

We propose a simple yet effective method, FGS2EE, to inject fine-grained semantic information into entity embeddings to reduce the distinctiveness and facilitate the learning of contextual commonality.

Entity Embeddings Entity Linking +1

Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation

no code implementations NeurIPS 2021 Yue Wang, Shaofeng Zou, Yi Zhou

Temporal-difference learning with gradient correction (TDC) is a two time-scale algorithm for policy evaluation in reinforcement learning.


Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing

no code implementations30 Mar 2021 Cheng Chen, Bhavya Kailkhura, Ryan Goldhahn, Yi Zhou

Federated learning is an emerging data-private distributed learning framework, which, however, is vulnerable to adversarial attacks.

Federated Learning

Multi-Agent Off-Policy TD Learning: Finite-Time Analysis with Near-Optimal Sample Complexity and Communication Complexity

no code implementations24 Mar 2021 Ziyi Chen, Yi Zhou, Rongrong Chen

Under Markovian sampling and linear function approximation, we proved that the finite-time sample complexity of both algorithms for achieving an $\epsilon$-accurate solution is in the order of $\mathcal{O}(\epsilon^{-1}\ln \epsilon^{-1})$, matching the near-optimal sample complexity of centralized TD(0) and TDC.

FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data

no code implementations5 Mar 2021 Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar, James Joshi, Heiko Ludwig

We empirically demonstrate the applicability for multiple types of ML models and show a reduction of 10%-70% of training time and 80% to 90% in data transfer with respect to the state-of-the-art approaches.

Federated Learning

A Deep Emulator for Secondary Motion of 3D Characters

no code implementations CVPR 2021 Mianlun Zheng, Yi Zhou, Duygu Ceylan, Jernej Barbič

Being a local method, our network is independent of the mesh topology and generalizes to arbitrarily shaped 3D character meshes at test time.

Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification

no code implementations26 Feb 2021 Yi Zhou, Lei Huang, Tianfei Zhou, Ling Shao

For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming.

Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry

no code implementations ICLR 2021 Ziyi Chen, Yi Zhou, Tengyu Xu, Yingbin Liang

By leveraging this Lyapunov function and the K{\L} geometry that parameterizes the local geometries of general nonconvex functions, we formally establish the variable convergence of proximal-GDA to a critical point $x^*$, i. e., $x_t\to x^*, y_t\to y^*(x^*)$.

Global existence for semilinear wave equations with scaling invariant damping in 3-D

no code implementations1 Feb 2021 Ning-An Lai, Yi Zhou

Global existence for small data Cauchy problem of semilinear wave equations with scaling invariant damping in 3-D is established in this work, assuming that the data are radial and the constant in front of the damping belongs to $[1. 5, 2)$.

Analysis of PDEs

Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning

no code implementations1 Feb 2021 Syed Zawad, Ahsan Ali, Pin-Yu Chen, Ali Anwar, Yi Zhou, Nathalie Baracaldo, Yuan Tian, Feng Yan

Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks.

Federated Learning

Visual-Textual Attentive Semantic Consistency for Medical Report Generation

no code implementations ICCV 2021 Yi Zhou, Lei Huang, Tao Zhou, Huazhu Fu, Ling Shao

Second, the progressive report decoder consists of a sentence decoder and a word decoder, where we propose image-sentence matching and description accuracy losses to constrain the visual-textual semantic consistency.

Medical Report Generation Word Embeddings

Enhancing Balanced Graph Edge Partition with Effective Local Search

no code implementations17 Dec 2020 Zhenyu Guo, Mingyu Xiao, Yi Zhou, Dongxiang Zhang, Kian-Lee Tan

The graph edge partition problem, which is to split the edge set into multiple balanced parts to minimize the total number of copied vertices, has been widely studied from the view of optimization and algorithms.

Event-based Motion Segmentation with Spatio-Temporal Graph Cuts

1 code implementation16 Dec 2020 Yi Zhou, Guillermo Gallego, Xiuyuan Lu, SiQi Liu, Shaojie Shen

We develop a method to identify independently moving objects acquired with an event-based camera, i. e., to solve the event-based motion segmentation problem.

Motion Segmentation Scene Understanding

Canny-VO: Visual Odometry with RGB-D Cameras based on Geometric 3D-2D Edge Alignment

no code implementations15 Dec 2020 Yi Zhou, Hongdong Li, Laurent Kneip

The present paper reviews the classical problem of free-form curve registration and applies it to an efficient RGBD visual odometry system called Canny-VO, as it efficiently tracks all Canny edge features extracted from the images.

Visual Odometry

Adaptive Histogram-Based Gradient Boosted Trees for Federated Learning

no code implementations11 Dec 2020 Yuya Jeremy Ong, Yi Zhou, Nathalie Baracaldo, Heiko Ludwig

This approach makes the use of gradient boosted trees practical in enterprise federated learning.

Federated Learning

Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation

1 code implementation9 Dec 2020 Xueyi Li, Tianfei Zhou, Jianwu Li, Yi Zhou, Zhaoxiang Zhang

We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths, which can be used for training more accurate segmentation models.

Ranked #20 on Weakly-Supervised Semantic Segmentation on COCO 2014 val (using extra training data)

Structured Prediction Weakly-Supervised Semantic Segmentation

Mitigating Bias in Federated Learning

no code implementations4 Dec 2020 Annie Abay, Yi Zhou, Nathalie Baracaldo, Shashank Rajamoni, Ebube Chuba, Heiko Ludwig

As methods to create discrimination-aware models develop, they focus on centralized ML, leaving federated learning (FL) unexplored.

Fairness Federated Learning

A Statistical Mechanics Framework for Task-Agnostic Sample Design in Machine Learning

no code implementations NeurIPS 2020 Bhavya Kailkhura, Jayaraman J. Thiagarajan, Qunwei Li, Jize Zhang, Yi Zhou, Timo Bremer

Using this framework, we show that space-filling sample designs, such as blue noise and Poisson disk sampling, which optimize spectral properties, outperform random designs in terms of the generalization gap and characterize this gain in a closed-form.

Contrastive Weight Regularization for Large Minibatch SGD

no code implementations17 Nov 2020 Qiwei Yuan, Weizhe Hua, Yi Zhou, Cunxi Yu

The minibatch stochastic gradient descent method (SGD) is widely applied in deep learning due to its efficiency and scalability that enable training deep networks with a large volume of data.

On the Transferability of Adversarial Attacksagainst Neural Text Classifier

no code implementations17 Nov 2020 Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang

Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.

Text Classification

Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study

no code implementations13 Nov 2020 Cheng Chen, Junjie Yang, Yi Zhou

Specifically, we find that the optimization trajectories of successful DNN trainings consistently obey a certain regularity principle that regularizes the model update direction to be aligned with the trajectory direction.

Cross-Lingual Dependency Parsing by POS-Guided Word Reordering

no code implementations Findings of the Association for Computational Linguistics 2020 Lu Liu, Yi Zhou, Jianhan Xu, Xiaoqing Zheng, Kai-Wei Chang, Xuanjing Huang

The words in each sentence of a source language corpus are rearranged to meet the word order in a target language under the guidance of a part-of-speech based language model (LM).

Dependency Parsing Language Modelling +1

Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis

no code implementations NeurIPS 2020 Shaocong Ma, Yi Zhou, Shaofeng Zou

In this work, we develop a variance reduction scheme for the two time-scale TDC algorithm in the off-policy setting and analyze its non-asymptotic convergence rate over both i. i. d.\ and Markovian samples.

Boosting One-Point Derivative-Free Online Optimization via Residual Feedback

no code implementations14 Oct 2020 Yan Zhang, Yi Zhou, Kaiyi Ji, Michael M. Zavlanos

As a result, our regret bounds are much tighter compared to existing regret bounds for ZO with conventional one-point feedback, which suggests that ZO with residual feedback can better track the optimizer of online optimization problems.

UNISON: Unpaired Cross-lingual Image Captioning

no code implementations3 Oct 2020 Jiahui Gao, Yi Zhou, Philip L. H. Yu, Shafiq Joty, Jiuxiang Gu

In this work, we present a novel unpaired cross-lingual method to generate image captions without relying on any caption corpus in the source or the target language.

Image Captioning Machine Translation +1

Group Whitening: Balancing Learning Efficiency and Representational Capacity

1 code implementation CVPR 2021 Lei Huang, Yi Zhou, Li Liu, Fan Zhu, Ling Shao

Results show that GW consistently improves the performance of different architectures, with absolute gains of $1. 02\%$ $\sim$ $1. 49\%$ in top-1 accuracy on ImageNet and $1. 82\%$ $\sim$ $3. 21\%$ in bounding box AP on COCO.

Normalization Techniques in Training DNNs: Methodology, Analysis and Application

no code implementations27 Sep 2020 Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications.

FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling

no code implementations22 Sep 2020 Cheng Chen, Ziyi Chen, Yi Zhou, Bhavya Kailkhura

We develop FedCluster--a novel federated learning framework with improved optimization efficiency, and investigate its theoretical convergence properties.

Federated Learning

Exploring the Hierarchy in Relation Labels for Scene Graph Generation

no code implementations12 Sep 2020 Yi Zhou, Shuyang Sun, Chao Zhang, Yikang Li, Wanli Ouyang

By assigning each relationship a single label, current approaches formulate the relationship detection as a classification problem.

Graph Generation Scene Graph Generation

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

no code implementations22 Aug 2020 Yi Zhou, Boyang Wang, Lei Huang, Shanshan Cui, Ling Shao

This dataset has 1, 842 images with pixel-level DR-related lesion annotations, and 1, 000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency.

Lesion Segmentation Transfer Learning

Learning to Generate Diverse Dance Motions with Transformer

no code implementations18 Aug 2020 Jiaman Li, Yihang Yin, Hang Chu, Yi Zhou, Tingwu Wang, Sanja Fidler, Hao Li

We also introduce new evaluation metrics for the quality of synthesized dance motions, and demonstrate that our system can outperform state-of-the-art methods.

motion synthesis

Spatio-temporal Attention Model for Tactile Texture Recognition

no code implementations10 Aug 2020 Guanqun Cao, Yi Zhou, Danushka Bollegala, Shan Luo

Recently, tactile sensing has attracted great interest in robotics, especially for facilitating exploration of unstructured environments and effective manipulation.

Event-based Stereo Visual Odometry

2 code implementations30 Jul 2020 Yi Zhou, Guillermo Gallego, Shaojie Shen

We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig.

3D Reconstruction Pose Estimation +1

Understanding the Impact of Model Incoherence on Convergence of Incremental SGD with Random Reshuffle

no code implementations ICML 2020 Shaocong Ma, Yi Zhou

Specifically, minimizer incoherence measures the discrepancy between the global minimizers of a sample loss and those of the total loss and affects the convergence error of SGD with random reshuffle.

Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble

1 code implementation20 Jun 2020 Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang

Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples.

A New One-Point Residual-Feedback Oracle For Black-Box Learning and Control

no code implementations18 Jun 2020 Yan Zhang, Yi Zhou, Kaiyi Ji, Michael M. Zavlanos

When optimizing a deterministic Lipschitz function, we show that the query complexity of ZO with the proposed one-point residual feedback matches that of ZO with the existing two-point schemes.

Generative Tweening: Long-term Inbetweening of 3D Human Motions

no code implementations18 May 2020 Yi Zhou, Jingwan Lu, Connelly Barnes, Jimei Yang, Sitao Xiang, Hao Li

We introduce a biomechanically constrained generative adversarial network that performs long-term inbetweening of human motions, conditioned on keyframe constraints.

Momentum with Variance Reduction for Nonconvex Composition Optimization

no code implementations15 May 2020 Ziyi Chen, Yi Zhou

This paper complements the existing literature by developing various momentum schemes with SPIDER-based variance reduction for non-convex composition optimization.

An Investigation into the Stochasticity of Batch Whitening

1 code implementation CVPR 2020 Lei Huang, Lei Zhao, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

Our work originates from the observation that while various whitening transformations equivalently improve the conditioning, they show significantly different behaviors in discriminative scenarios and training Generative Adversarial Networks (GANs).

GFTE: Graph-based Financial Table Extraction

1 code implementation17 Mar 2020 Yiren Li, Zheng Huang, Junchi Yan, Yi Zhou, Fan Ye, Xianhui Liu

Tabular data is a crucial form of information expression, which can organize data in a standard structure for easy information retrieval and comparison.

Information Retrieval Table Extraction

Motion-Attentive Transition for Zero-Shot Video Object Segmentation

1 code implementation9 Mar 2020 Tianfei Zhou, Shunzhou Wang, Yi Zhou, Yazhou Yao, Jianwu Li, Ling Shao

In this paper, we present a novel Motion-Attentive Transition Network (MATNet) for zero-shot video object segmentation, which provides a new way of leveraging motion information to reinforce spatio-temporal object representation.

Ranked #6 on Unsupervised Video Object Segmentation on DAVIS 2016 (using extra training data)

Semantic Segmentation Unsupervised Video Object Segmentation +2

Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization

no code implementations26 Feb 2020 Yi Zhou, Zhe Wang, Kaiyi Ji, Yingbin Liang, Vahid Tarokh

Our APG-restart is designed to 1) allow for adopting flexible parameter restart schemes that cover many existing ones; 2) have a global sub-linear convergence rate in nonconvex and nonsmooth optimization; and 3) have guaranteed convergence to a critical point and have various types of asymptotic convergence rates depending on the parameterization of local geometry in nonconvex and nonsmooth optimization.

TiFL: A Tier-based Federated Learning System

no code implementations25 Jan 2020 Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalie Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, Yue Cheng

To this end, we propose TiFL, a Tier-based Federated Learning System, which divides clients into tiers based on their training performance and selects clients from the same tier in each training round to mitigate the straggler problem caused by heterogeneity in resource and data quantity.

Federated Learning

Reanalysis of Variance Reduced Temporal Difference Learning

no code implementations ICLR 2020 Tengyu Xu, Zhe Wang, Yi Zhou, Yingbin Liang

Furthermore, the variance error (for both i. i. d.\ and Markovian sampling) and the bias error (for Markovian sampling) of VRTD are significantly reduced by the batch size of variance reduction in comparison to those of vanilla TD.

Chinese Named Entity Recognition Augmented with Lexicon Memory

1 code implementation17 Dec 2019 Yi Zhou, Xiaoqing Zheng, Xuanjing Huang

Inspired by a concept of content-addressable retrieval from cognitive science, we propose a novel fragment-based model augmented with a lexicon-based memory for Chinese NER, in which both the character-level and word-level features are combined to generate better feature representations for possible name candidates.

Chinese Named Entity Recognition NER

HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning

no code implementations12 Dec 2019 Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar, Heiko Ludwig

Participants in a federated learning process cooperatively train a model by exchanging model parameters instead of the actual training data, which they might want to keep private.

Federated Learning

DR-GAN: Conditional Generative Adversarial Network for Fine-Grained Lesion Synthesis on Diabetic Retinopathy Images

no code implementations10 Dec 2019 Yi Zhou, Boyang Wang, Xiaodong He, Shanshan Cui, Ling Shao

In this paper, we propose a diabetic retinopathy generative adversarial network (DR-GAN) to synthesize high-resolution fundus images which can be manipulated with arbitrary grading and lesion information.

Data Augmentation Lesion Segmentation

SpiderBoost and Momentum: Faster Variance Reduction Algorithms

no code implementations NeurIPS 2019 Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, Vahid Tarokh

SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonconvex optimization.

A Deep Learning-Based System for PharmaCoNER

no code implementations WS 2019 Ying Xiong, Yedan Shen, Yuanhang Huang, Shuai Chen, Buzhou Tang, Xiaolong Wang, Qingcai Chen, Jun Yan, Yi Zhou

The Biological Text Mining Unit at BSC and CNIO organized the first shared task on chemical {\&} drug mention recognition from Spanish medical texts called PharmaCoNER (Pharmacological Substances, Compounds and proteins and Named Entity Recognition track) in 2019, which includes two tracks: one for NER offset and entity classification (track 1) and the other one for concept indexing (track 2).

General Classification Named Entity Recognition +1

Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization

no code implementations27 Oct 2019 Kaiyi Ji, Zhe Wang, Yi Zhou, Yingbin Liang

Two types of zeroth-order stochastic algorithms have recently been designed for nonconvex optimization respectively based on the first-order techniques SVRG and SARAH/SPIDER.

Faster and Safer Training by Embedding High-Level Knowledge into Deep Reinforcement Learning

no code implementations22 Oct 2019 Haodi Zhang, Zihang Gao, Yi Zhou, Hao Zhang, Kaishun Wu, Fangzhen Lin

Deep reinforcement learning has been successfully used in many dynamic decision making domains, especially those with very large state spaces.

Decision Making reinforcement-learning

History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms

no code implementations ICML 2020 Kaiyi Ji, Zhe Wang, Bowen Weng, Yi Zhou, Wei zhang, Yingbin Liang

In this paper, we propose a novel scheme, which eliminates backtracking line search but still exploits the information along optimization path by adapting the batch size via history stochastic gradients.

Supervised Encoding for Discrete Representation Learning

1 code implementation15 Oct 2019 Cat P. Le, Yi Zhou, Jie Ding, Vahid Tarokh

Classical supervised classification tasks search for a nonlinear mapping that maps each encoded feature directly to a probability mass over the labels.

Representation Learning Style Transfer

Distributed SGD Generalizes Well Under Asynchrony

no code implementations29 Sep 2019 Jayanth Regatti, Gaurav Tendolkar, Yi Zhou, Abhishek Gupta, Yingbin Liang

The performance of fully synchronized distributed systems has faced a bottleneck due to the big data trend, under which asynchronous distributed systems are becoming a major popularity due to their powerful scalability.

An Optimization Principle Of Deep Learning?

no code implementations25 Sep 2019 Cheng Chen, Junjie Yang, Yi Zhou

In particular, we observe that the trainings that apply the training techniques achieve accelerated convergence and obey the principle with a large $\gamma$, which is consistent with the $\mathcal{O}(1/\gamma K)$ convergence rate result under the optimization principle.

Towards Federated Graph Learning for Collaborative Financial Crimes Detection

no code implementations19 Sep 2019 Toyotaro Suzumura, Yi Zhou, Natahalie Baracaldo, Guangnan Ye, Keith Houck, Ryo Kawahara, Ali Anwar, Lucia Larise Stavarache, Yuji Watanabe, Pablo Loyola, Daniel Klyashtorny, Heiko Ludwig, Kumar Bhaskaran

Advances in technology used in this domain, including machine learning based approaches, can improve upon the effectiveness of financial institutions' existing processes, however, a key challenge that most financial institutions continue to face is that they address financial crimes in isolation without any insight from other firms.

Federated Learning Graph Learning

Collaborative Learning of Semi-Supervised Segmentation and Classification for Medical Images

no code implementations CVPR 2019 Yi Zhou, Xiaodong He, Lei Huang, Li Liu, Fan Zhu, Shanshan Cui, Ling Shao

Then, based on initially predicted lesion maps for large quantities of image-level annotated data, a lesion attentive disease grading model is designed to improve the severity classification accuracy.

General Classification Lesion Segmentation +1

A unified variance-reduced accelerated gradient method for convex optimization

no code implementations NeurIPS 2019 Guanghui Lan, Zhize Li, Yi Zhou

Moreover, Varag is the first accelerated randomized incremental gradient method that benefits from the strong convexity of the data-fidelity term to achieve the optimal linear convergence.

Iterative Normalization: Beyond Standardization towards Efficient Whitening

5 code implementations CVPR 2019 Lei Huang, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

With the support of SND, we provide natural explanations to several phenomena from the perspective of optimization, e. g., why group-wise whitening of DBN generally outperforms full-whitening and why the accuracy of BN degenerates with reduced batch sizes.

Momentum Schemes with Stochastic Variance Reduction for Nonconvex Composite Optimization

no code implementations7 Feb 2019 Yi Zhou, Zhe Wang, Kaiyi Ji, Yingbin Liang, Vahid Tarokh

In this paper, we develop novel momentum schemes with flexible coefficient settings to accelerate SPIDER for nonconvex and nonsmooth composite optimization, and show that the resulting algorithms achieve the near-optimal gradient oracle complexity for achieving a generalized first-order stationary condition.

Hybrid coarse-fine classification for head pose estimation

1 code implementation21 Jan 2019 Haofan Wang, Zhenghua Chen, Yi Zhou

In this paper, to do the estimation without facial landmarks, we combine the coarse and fine regression output together for a deep network.

3D Reconstruction Classification +5

SGD Converges to Global Minimum in Deep Learning via Star-convex Path

no code implementations ICLR 2019 Yi Zhou, Junjie Yang, Huishuai Zhang, Yingbin Liang, Vahid Tarokh

Stochastic gradient descent (SGD) has been found to be surprisingly effective in training a variety of deep neural networks.

On the Continuity of Rotation Representations in Neural Networks

5 code implementations CVPR 2019 Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, Hao Li

Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn.

A Hybrid Approach to Privacy-Preserving Federated Learning

no code implementations7 Dec 2018 Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, Yi Zhou

Federated learning facilitates the collaborative training of models without the sharing of raw data.

Federated Learning

MR-GAN: Manifold Regularized Generative Adversarial Networks

no code implementations22 Nov 2018 Qunwei Li, Bhavya Kailkhura, Rushil Anirudh, Yi Zhou, Yingbin Liang, Pramod Varshney

Despite the growing interest in generative adversarial networks (GANs), training GANs remains a challenging problem, both from a theoretical and a practical standpoint.

SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms

no code implementations25 Oct 2018 Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, Vahid Tarokh

SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonconvex optimization.

Cubic Regularization with Momentum for Nonconvex Optimization

no code implementations9 Oct 2018 Zhe Wang, Yi Zhou, Yingbin Liang, Guanghui Lan

However, such a successful acceleration technique has not yet been proposed for second-order algorithms in nonconvex optimization. In this paper, we apply the momentum scheme to cubic regularized (CR) Newton's method and explore the potential for acceleration.

Toward Understanding the Impact of Staleness in Distributed Machine Learning

no code implementations ICLR 2019 Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric P. Xing

Many distributed machine learning (ML) systems adopt the non-synchronous execution in order to alleviate the network communication bottleneck, resulting in stale parameters that do not reflect the latest updates.

Elastic Neural Networks for Classification

3 code implementations1 Oct 2018 Yi Zhou, Yue Bai, Shuvra S. Bhattacharyya, Heikki Huttunen

In this work we propose a framework for improving the performance of any deep neural network that may suffer from vanishing gradients.

Classification General Classification

Asynchronous decentralized accelerated stochastic gradient descent

no code implementations24 Sep 2018 Guanghui Lan, Yi Zhou

In this work, we introduce an asynchronous decentralized accelerated stochastic gradient descent type of method for decentralized stochastic optimization, considering communication and synchronization are the major bottlenecks.

Stochastic Optimization

KDSL: a Knowledge-Driven Supervised Learning Framework for Word Sense Disambiguation

no code implementations28 Aug 2018 Shi Yin, Yi Zhou, Chenguang Li, Shangfei Wang, Jianmin Ji, Xiaoping Chen, Ruili Wang

We propose KDSL, a new word sense disambiguation (WSD) framework that utilizes knowledge to automatically generate sense-labeled data for supervised learning.

Word Sense Disambiguation

A Note on Inexact Condition for Cubic Regularized Newton's Method

no code implementations22 Aug 2018 Zhe Wang, Yi Zhou, Yingbin Liang, Guanghui Lan

This note considers the inexact cubic-regularized Newton's method (CR), which has been shown in \cite{Cartis2011a} to achieve the same order-level convergence rate to a secondary stationary point as the exact CR \citep{Nesterov2006}.

Convergence of Cubic Regularization for Nonconvex Optimization under KL Property

no code implementations NeurIPS 2018 Yi Zhou, Zhe Wang, Yingbin Liang

Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems.

Semi-Dense 3D Reconstruction with a Stereo Event Camera

2 code implementations ECCV 2018 Yi Zhou, Guillermo Gallego, Henri Rebecq, Laurent Kneip, Hongdong Li, Davide Scaramuzza

Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision.

3D Reconstruction Simultaneous Localization and Mapping

When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?

1 code implementation ICLR 2019 Tengyu Xu, Yi Zhou, Kaiyi Ji, Yingbin Liang

We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset.

Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization

no code implementations20 Feb 2018 Zhe Wang, Yi Zhou, Yingbin Liang, Guanghui Lan

Cubic regularization (CR) is an optimization method with emerging popularity due to its capability to escape saddle points and converge to second-order stationary solutions for nonconvex optimization.

Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization

no code implementations19 Feb 2018 Yi Zhou, Yingbin Liang, Huishuai Zhang

With strongly convex regularizers, we further establish the generalization error bounds for nonconvex loss functions under proximal SGD with high-probability guarantee, i. e., exponential concentration in probability.

Critical Points of Linear Neural Networks: Analytical Forms and Landscape Properties

no code implementations ICLR 2018 Yi Zhou, Yingbin Liang

In this paper, we provide a necessary and sufficient characterization of the analytical forms for the critical points (as well as global minimizers) of the square loss functions for linear neural networks.

Random gradient extrapolation for distributed and stochastic optimization

no code implementations15 Nov 2017 Guanghui Lan, Yi Zhou

Furthermore, we demonstrate that for stochastic finite-sum optimization problems, RGEM maintains the optimal ${\cal O}(1/\epsilon)$ complexity (up to a certain logarithmic factor) in terms of the number of stochastic gradient computations, but attains an ${\cal O}(\log(1/\epsilon))$ complexity in terms of communication rounds (each round involves only one agent).

Stochastic Optimization

Critical Points of Neural Networks: Analytical Forms and Landscape Properties

no code implementations30 Oct 2017 Yi Zhou, Yingbin Liang

We show that the analytical forms of the critical points characterize the values of the corresponding loss functions as well as the necessary and sufficient conditions to achieve global minimum.

Characterization of Gradient Dominance and Regularity Conditions for Neural Networks

no code implementations18 Oct 2017 Yi Zhou, Yingbin Liang

The past decade has witnessed a successful application of deep learning to solving many challenging problems in machine learning and artificial intelligence.

Realistic Dynamic Facial Textures From a Single Image Using GANs

no code implementations ICCV 2017 Kyle Olszewski, Zimo Li, Chao Yang, Yi Zhou, Ronald Yu, Zeng Huang, Sitao Xiang, Shunsuke Saito, Pushmeet Kohli, Hao Li

By retargeting the PCA expression geometry from the source, as well as using the newly inferred texture, we can both animate the face and perform video face replacement on the source video using the target appearance.


Learning Latent Space Models with Angular Constraints

no code implementations ICML 2017 Pengtao Xie, Yuntian Deng, Yi Zhou, Abhimanu Kumar, Yao-Liang Yu, James Zou, Eric P. Xing

The large model capacity of latent space models (LSMs) enables them to achieve great performance on various applications, but meanwhile renders LSMs to be prone to overfitting.

Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis

1 code implementation ICLR 2018 Zimo Li, Yi Zhou, Shuangjiu Xiao, Chong He, Zeng Huang, Hao Li

We present a real-time method for synthesizing highly complex human motions using a novel training regime we call the auto-conditioned Recurrent Neural Network (acRNN).

motion synthesis

Combining tabu search and graph reduction to solve the maximum balanced biclique problem

no code implementations20 May 2017 Yi Zhou, Jin-Kao Hao

The Maximum Balanced Biclique Problem is a well-known graph model with relevant applications in diverse domains.


Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization

no code implementations ICML 2017 Qunwei Li, Yi Zhou, Yingbin Liang, Pramod K. Varshney

Then, by exploiting the Kurdyka-{\L}ojasiewicz (\KL) property for a broad class of functions, we establish the linear and sub-linear convergence rates of the function value sequence generated by APGnc.

Structured Production System (extended abstract)

no code implementations26 Apr 2017 Yi Zhou

In this extended abstract, we propose Structured Production Systems (SPS), which extend traditional production systems with well-formed syntactic structures.

Conditional Accelerated Lazy Stochastic Gradient Descent

no code implementations ICML 2017 Guanghui Lan, Sebastian Pokutta, Yi Zhou, Daniel Zink

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate $O\left(\frac{1}{\varepsilon^2}\right)$ improving over the projection-free, Online Frank-Wolfe based stochastic gradient descent of Hazan and Kale [2012] with convergence rate $O\left(\frac{1}{\varepsilon^4}\right)$.

Semi-Dense Visual Odometry for RGB-D Cameras Using Approximate Nearest Neighbour Fields

no code implementations6 Feb 2017 Yi Zhou, Laurent Kneip, Hongdong Li

This paper presents a robust and efficient semi-dense visual odometry solution for RGB-D cameras.

Frame Visual Odometry

Communication-Efficient Algorithms for Decentralized and Stochastic Optimization

no code implementations14 Jan 2017 Guanghui Lan, Soomin Lee, Yi Zhou

Our major contribution is to present a new class of decentralized primal-dual type algorithms, namely the decentralized communication sliding (DCS) methods, which can skip the inter-node communications while agents solve the primal subproblems iteratively through linearizations of their local objective functions.

Stochastic Optimization

From First-Order Logic to Assertional Logic

no code implementations12 Jan 2017 Yi Zhou

Then, we show how to extend it by definitions, which are special kinds of knowledge, i. e., assertions.

DAVE: A Unified Framework for Fast Vehicle Detection and Annotation

no code implementations15 Jul 2016 Yi Zhou, Li Liu, Ling Shao, Matt Mellor

Vehicle detection and annotation for streaming video data with complex scenes is an interesting but challenging task for urban traffic surveillance.

Fast Vehicle Detection

Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations

1 code implementation25 May 2016 Huishuai Zhang, Yi Zhou, Yingbin Liang, Yuejie Chi

We further develop the incremental (stochastic) reshaped Wirtinger flow (IRWF) and show that IRWF converges linearly to the true signal.

A Set Theoretic Approach for Knowledge Representation: the Representation Part

no code implementations11 Mar 2016 Yi Zhou

In this paper, we propose a set theoretic approach for knowledge representation.

DAP3D-Net: Where, What and How Actions Occur in Videos?

no code implementations10 Feb 2016 Li Liu, Yi Zhou, Ling Shao

Action parsing in videos with complex scenes is an interesting but challenging task in computer vision.

Action Localization Action Parsing +2

Analysis of Robust PCA via Local Incoherence

no code implementations NeurIPS 2015 Huishuai Zhang, Yi Zhou, Yingbin Liang

We investigate the robust PCA problem of decomposing an observed matrix into the sum of a low-rank and a sparse error matrices via convex programming Principal Component Pursuit (PCP).

Distributed Machine Learning via Sufficient Factor Broadcasting

no code implementations26 Nov 2015 Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yao-Liang Yu, Eric Xing

Matrix-parametrized models, including multiclass logistic regression and sparse coding, are used in machine learning (ML) applications ranging from computer vision to computational biology.

An optimal randomized incremental gradient method

no code implementations8 Jul 2015 Guanghui Lan, Yi Zhou

We first introduce a deterministic primal-dual gradient (PDG) method that can achieve the optimal black-box iteration complexity for solving these composite optimization problems using a primal-dual termination criterion.

Distributed Machine Learning via Sufficient Factor Broadcasting

no code implementations19 Sep 2014 Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yao-Liang Yu, Eric Xing

Matrix-parametrized models, including multiclass logistic regression and sparse coding, are used in machine learning (ML) applications ranging from computer vision to computational biology.

A Logical Study of Partial Entailment

no code implementations16 Jan 2014 Yi Zhou, Yan Zhang

We introduce a novel logical notion--partial entailment--to propositional logic.

Majority Rule for Belief Evolution in Social Networks

no code implementations3 Sep 2013 Yi Zhou

In this paper, we study how an agent's belief is affected by her neighbors in a social network.

Quantum Orders and Spin Liquids in Cs$_2$CuCl$_4$

1 code implementation30 Oct 2002 Yi Zhou, Xiao-Gang Wen

Motivated by experiments on Cs$_2$CuCl$_4$ samples, we studied and classified the symmetric spin liquids on triangular lattice.

Strongly Correlated Electrons

Cannot find the paper you are looking for? You can Submit a new open access paper.