Search Results for author: Peng Zhao

Found 67 papers, 8 papers with code

Learning with Feature and Distribution Evolvable Streams

no code implementations ICML 2020 Zhen-Yu Zhang, Peng Zhao, Yuan Jiang, Zhi-Hua Zhou

Besides the feature space evolving, it is noteworthy that the data distribution often changes in streaming data.

Exploiting Diffusion Prior for Out-of-Distribution Detection

no code implementations16 Jun 2024 Armando Zhu, Jiabei Liu, Keqin Li, Shuying Dai, Bo Hong, Peng Zhao, Changsong Wei

Out-of-distribution (OOD) detection is crucial for deploying robust machine learning models, especially in areas where security is critical.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

EdgeSync: Faster Edge-model Updating via Adaptive Continuous Learning for Video Data Drift

no code implementations5 Jun 2024 Peng Zhao, Runchu Dong, Guiqin Wang, Cong Zhao

Real-time video analytics systems typically place models with fewer weights on edge devices to reduce latency.

Universal Online Convex Optimization with $1$ Projection per Round

no code implementations30 May 2024 Wenhao Yang, Yibo Wang, Peng Zhao, Lijun Zhang

In this paper, inspired by the black-box reduction of Cutkosky and Orabona (2018), we employ a surrogate loss defined over simpler domains to develop universal OCO algorithms that only require $1$ projection.

Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximation

no code implementations27 May 2024 Long-Fei Li, Yu-Jie Zhang, Peng Zhao, Zhi-Hua Zhou

To the best of our knowledge, this is the first work that achieves almost the same computational and statistical efficiency as linear function approximation while employing non-linear function approximation for reinforcement learning.


Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classification

no code implementations22 Apr 2024 Armando Zhu, Keqin Li, Tong Wu, Peng Zhao, Bo Hong

With wearing masks becoming a new cultural norm, facial expression recognition (FER) while taking masks into account has become a significant challenge.

Facial Expression Recognition Facial Expression Recognition (FER)

Utilizing Deep Learning to Optimize Software Development Processes

no code implementations21 Apr 2024 Keqin Li, Armando Zhu, Peng Zhao, Jintong Song, Jiabei Liu

This study explores the application of deep learning technologies in software development processes, particularly in automating code reviews, error prediction, and test generation to enhance code quality and development efficiency.

Improved Algorithm for Adversarial Linear Mixture MDPs with Bandit Feedback and Unknown Transition

no code implementations7 Mar 2024 Long-Fei Li, Peng Zhao, Zhi-Hua Zhou

We study reinforcement learning with linear function approximation, unknown transition, and adversarial losses in the bandit feedback setting.


Query-decision Regression between Shortest Path and Minimum Steiner Tree

1 code implementation3 Feb 2024 Guangmo Tong, Peng Zhao, Mina Samizadeh

Considering a graph with unknown weights, can we find the shortest path for a pair of nodes if we know the minimal Steiner trees associated with some subset of nodes?

Decision Making regression +1

WiOpen: A Robust Wi-Fi-based Open-set Gesture Recognition Framework

1 code implementation1 Feb 2024 Xiang Zhang, Jingyang Huang, Huan Yan, Peng Zhao, Guohang Zhuang, Zhi Liu, Bin Liu

This uncertainty, resulting from noise and domains, leads to widely scattered and irregular data distributions in collected Wi-Fi sensing data.

Gesture Recognition Uncertainty Quantification

FedLED: Label-Free Equipment Fault Diagnosis with Vertical Federated Transfer Learning

no code implementations29 Dec 2023 Jie Shen, Shusen Yang, Cong Zhao, Xuebin Ren, Peng Zhao, Yuqian Yang, Qing Han, Shuaijun Wu

Intelligent equipment fault diagnosis based on Federated Transfer Learning (FTL) attracts considerable attention from both academia and industry.

Transfer Learning

Efficient LLM inference solution on Intel GPU

no code implementations19 Dec 2023 Hui Wu, Yi Gan, Feng Yuan, Jing Ma, Wei Zhu, Yutao Xu, Hong Zhu, Yuhua Zhu, Xiaoli Liu, Jinghui Gu, Peng Zhao

A customized Scaled-Dot-Product-Attention kernel is designed to match our fusion policy based on the segment KV cache solution.

Decoder Management

Generative Model-based Feature Knowledge Distillation for Action Recognition

1 code implementation14 Dec 2023 Guiqin Wang, Peng Zhao, Yanjiang Shi, Cong Zhao, Shusen Yang

Addressing this gap, our paper introduces an innovative knowledge distillation framework, with the generative model for training a lightweight student model.

Action Detection Action Recognition +3

Efficient Methods for Non-stationary Online Learning

no code implementations16 Sep 2023 Peng Zhao, Yan-Feng Xie, Lijun Zhang, Zhi-Hua Zhou

In this paper, we present efficient methods for optimizing dynamic regret and adaptive regret, which reduce the number of projections per round from $\mathcal{O}(\log T)$ to $1$.

Probabilistic contingent planning based on HTN for high-quality plans

no code implementations14 Aug 2023 Peng Zhao

A more realistic view is that planning ought to take into consideration partial observability beforehand and aim for a more flexible and robust solution.

Universal Online Learning with Gradient Variations: A Multi-layer Online Ensemble Approach

no code implementations NeurIPS 2023 Yu-Hu Yan, Peng Zhao, Zhi-Hua Zhou

Our approach is based on a multi-layer online ensemble framework incorporating novel ingredients, including a carefully designed optimism for unifying diverse function types and cascaded corrections for algorithmic stability.

An Active Learning-based Approach for Hosting Capacity Analysis in Distribution Systems

no code implementations13 May 2023 Kiyeob Lee, Peng Zhao, Anirban Bhattacharya, Bani K. Mallick, Le Xie

Hosting capacity analysis (HCA) examines the amount of DERs that can be safely integrated into the grid and is a challenging task in full generality because there are many possible integration of DERs in foresight.

Active Learning

Improving the Transferability of Adversarial Examples via Direction Tuning

2 code implementations27 Mar 2023 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

Although considerable efforts have been developed on improving the transferability of adversarial examples generated by transfer-based adversarial attacks, our investigation found that, the big deviation between the actual and steepest update directions of the current transfer-based adversarial attacks is caused by the large update step length, resulting in the generated adversarial examples can not converge well.

Network Pruning

Fuzziness-tuned: Improving the Transferability of Adversarial Examples

no code implementations17 Mar 2023 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

In this paper, we first systematically investigated this issue and found that the enormous difference of attack success rates between the surrogate model and victim model is caused by the existence of a special area (known as fuzzy domain in our paper), in which the adversarial examples in the area are classified wrongly by the surrogate model while correctly by the victim model.

Revisiting Weighted Strategy for Non-stationary Parametric Bandits

no code implementations5 Mar 2023 Jing Wang, Peng Zhao, Zhi-Hua Zhou

We propose a refined analysis framework, which simplifies the derivation and importantly produces a simpler weight-based algorithm that is as efficient as window/restart-based algorithms while retaining the same regret as previous studies.

Stochastic Approximation Approaches to Group Distributionally Robust Optimization

no code implementations NeurIPS 2023 Lijun Zhang, Peng Zhao, Zhen-Hua Zhuang, Tianbao Yang, Zhi-Hua Zhou

First, we formulate GDRO as a stochastic convex-concave saddle-point problem, and demonstrate that stochastic mirror descent (SMD), using $m$ samples in each iteration, achieves an $O(m (\log m)/\epsilon^2)$ sample complexity for finding an $\epsilon$-optimal solution, which matches the $\Omega(m/\epsilon^2)$ lower bound up to a logarithmic factor.

Multi-Armed Bandits

Optimistic Online Mirror Descent for Bridging Stochastic and Adversarial Online Convex Optimization

no code implementations9 Feb 2023 Sijia Chen, Yu-Jie Zhang, Wei-Wei Tu, Peng Zhao, Lijun Zhang

Inspired by their work, we investigate the theoretical guarantees of optimistic online mirror descent (OMD) for the SEA model.

Adapting to Continuous Covariate Shift via Online Density Ratio Estimation

no code implementations NeurIPS 2023 Yu-Jie Zhang, Zhen-Yu Zhang, Peng Zhao, Masashi Sugiyama

Our density ratio estimation method is proven to perform well by enjoying a dynamic regret bound, which finally leads to an excess risk guarantee for the predictor.

Density Ratio Estimation

Factorized Fusion Shrinkage for Dynamic Relational Data

1 code implementation30 Sep 2022 Peng Zhao, Anirban Bhattacharya, Debdeep Pati, Bani K. Mallick

Comparing estimated latent factors involves both adjacent and long-term comparisons, with the time range of comparison considered as a variable.

Variational Inference

Structured Optimal Variational Inference for Dynamic Latent Space Models

no code implementations29 Sep 2022 Peng Zhao, Anirban Bhattacharya, Debdeep Pati, Bani K. Mallick

We consider a latent space model for dynamic networks, where our objective is to estimate the pairwise inner products of the latent positions.

Variational Inference

Dynamic Regret of Online Markov Decision Processes

no code implementations26 Aug 2022 Peng Zhao, Long-Fei Li, Zhi-Hua Zhou

For these three models, we propose novel online ensemble algorithms and establish their dynamic regret guarantees respectively, in which the results for episodic (loop-free) SSP are provably minimax optimal in terms of time horizon and certain non-stationarity measure.

Adapting to Online Label Shift with Provable Guarantees

no code implementations5 Jul 2022 Yong Bai, Yu-Jie Zhang, Peng Zhao, Masashi Sugiyama, Zhi-Hua Zhou

In this paper, we formulate and investigate the problem of \emph{online label shift} (OLaS): the learner trains an initial model from the labeled offline data and then deploys it to an unlabeled online environment where the underlying label distribution changes over time but the label-conditional density does not.

FACM: Intermediate Layer Still Retain Effective Features against Adversarial Examples

no code implementations2 Jun 2022 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

To enhance the robustness of the classifier, in our paper, a \textbf{F}eature \textbf{A}nalysis and \textbf{C}onditional \textbf{M}atching prediction distribution (FACM) model is proposed to utilize the features of intermediate layers to correct the classification.

Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reduction

no code implementations2 Jun 2022 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

The empirical and theoretical analysis demonstrates that the MDL loss improves the robustness and generalization of the model simultaneously for natural training.

Gradient Aligned Attacks via a Few Queries

no code implementations19 May 2022 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

Specifically, we propose a gradient aligned mechanism to ensure that the derivatives of the loss function with respect to the logit vector have the same weight coefficients between the surrogate and victim models.

Contrastive Multi-view Hyperbolic Hierarchical Clustering

no code implementations5 May 2022 Fangfei Lin, Bing Bai, Kun Bai, Yazhou Ren, Peng Zhao, Zenglin Xu

Then, we embed the representations into a hyperbolic space and optimize the hyperbolic embeddings via a continuous relaxation of hierarchical clustering loss.


Adaptive Bandit Convex Optimization with Heterogeneous Curvature

no code implementations12 Feb 2022 Haipeng Luo, Mengxiao Zhang, Peng Zhao

We consider the problem of adversarial bandit convex optimization, that is, online learning over a sequence of arbitrary convex loss functions with only one function evaluation for each of them.

Corralling a Larger Band of Bandits: A Case Study on Switching Regret for Linear Bandits

no code implementations12 Feb 2022 Haipeng Luo, Mengxiao Zhang, Peng Zhao, Zhi-Hua Zhou

The CORRAL algorithm of Agarwal et al. (2017) and its variants (Foster et al., 2020a) achieve this goal with a regret overhead of order $\widetilde{O}(\sqrt{MT})$ where $M$ is the number of base algorithms and $T$ is the time horizon.

No-Regret Learning in Time-Varying Zero-Sum Games

no code implementations30 Jan 2022 Mengxiao Zhang, Peng Zhao, Haipeng Luo, Zhi-Hua Zhou

Learning from repeated play in a fixed two-player zero-sum game is a classic problem in game theory and online learning.

Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization

1 code implementation29 Dec 2021 Peng Zhao, Yu-Jie Zhang, Lijun Zhang, Zhi-Hua Zhou

Specifically, we introduce novel online algorithms that can exploit smoothness and replace the dependence on $T$ in dynamic regret with problem-dependent quantities: the variation in gradients of loss functions, the cumulative loss of the comparator sequence, and the minimum of these two terms.

EMDS-7: Environmental Microorganism Image Dataset Seventh Version for Multiple Object Detection Evaluation

no code implementations11 Oct 2021 Hechen Yang, Chen Li, Xin Zhao, Bencheng Cai, Jiawei Zhang, Pingli Ma, Peng Zhao, Ao Chen, Hongzan Sun, Yueyang Teng, Shouliang Qi, Tao Jiang, Marcin Grzegorzek

The Environmental Microorganism Image Dataset Seventh Version (EMDS-7) is a microscopic image data set, including the original Environmental Microorganism images (EMs) and the corresponding object labeling files in ". XML" format file.

Object object-detection +1

A Comparison for Patch-level Classification of Deep Learning Methods on Transparent Environmental Microorganism Images: from Convolutional Neural Networks to Visual Transformers

no code implementations22 Jun 2021 Hechen Yang, Chen Li, Jinghua Zhang, Peng Zhao, Ao Chen, Xin Zhao, Tao Jiang, Marcin Grzegorzek

We conclude that ViT performs the worst in classifying 8 * 8 pixel patches, but it outperforms most convolutional neural networks in classifying 224 * 224 pixel patches.

Optimal Rates of (Locally) Differentially Private Heavy-tailed Multi-Armed Bandits

no code implementations4 Jun 2021 Youming Tao, Yulian Wu, Peng Zhao, Di Wang

Finally, we establish the lower bound to show that the instance-dependent regret of our improved algorithm is optimal.

Multi-Armed Bandits

A Comparison for Anti-noise Robustness of Deep Learning Classification Methods on a Tiny Object Image Dataset: from Convolutional Neural Network to Visual Transformer and Performer

no code implementations3 Jun 2021 Ao Chen, Chen Li, HaoYuan Chen, Hechen Yang, Peng Zhao, Weiming Hu, Wanli Liu, Shuojia Zou, Marcin Grzegorzek

In this paper, we first briefly review the development of Convolutional Neural Network and Visual Transformer in deep learning, and introduce the sources and development of conventional noises and adversarial attacks.

Classification Image Classification

Pinpointing the Memory Behaviors of DNN Training

no code implementations1 Apr 2021 Jiansong Li, Xiao Dong, Guangli Li, Peng Zhao, Xueying Wang, Xiaobing Chen, Xianzhi Yu, Yongxin Yang, Zihan Jiang, Wei Cao, Lei Liu, Xiaobing Feng

The training of deep neural networks (DNNs) is usually memory-hungry due to the limited device memory capacity of DNN accelerators.

Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling

no code implementations22 Mar 2021 Hongying Liu, Peng Zhao, Zhubo Ruan, Fanhua Shang, Yuanyuan Liu

In this paper, we propose a novel deep neural network with Dual Subnet and Multi-stage Communicated Upsampling (DSMC) for super-resolution of videos with large motion.

Motion Compensation Motion Estimation +1

Modeling Multivariate Cyber Risks: Deep Learning Dating Extreme Value Theory

no code implementations15 Mar 2021 Mingyue Zhang Wu, Jinzhu Luo, Xing Fang, Maochao Xu, Peng Zhao

The proposed model not only enjoys the high accurate point predictions via deep learning but also can provide the satisfactory high quantile prediction via extreme value theory.

Non-stationary Linear Bandits Revisited

no code implementations9 Mar 2021 Peng Zhao, Lijun Zhang

Existing studies develop various algorithms and show that they enjoy an $\widetilde{O}(T^{2/3}(1+P_T)^{1/3})$ dynamic regret, where $T$ is the time horizon and $P_T$ is the path-length that measures the fluctuation of the evolving unknown parameter.

Non-stationary Online Learning with Memory and Non-stochastic Control

no code implementations7 Feb 2021 Peng Zhao, Yu-Hu Yan, Yu-Xiang Wang, Zhi-Hua Zhou

We study the problem of Online Convex Optimization (OCO) with memory, which allows loss functions to depend on past decisions and thus captures temporal effects of learning problems.

Latent Dirichlet Allocation Model Training with Differential Privacy

no code implementations9 Oct 2020 Fangyuan Zhao, Xuebin Ren, Shusen Yang, Qing Han, Peng Zhao, Xinyu Yang

To address the privacy issue in LDA, we systematically investigate the privacy protection of the main-stream LDA training algorithm based on Collapsed Gibbs Sampling (CGS) and propose several differentially private LDA algorithms for typical training scenarios.

Privacy Preserving

A Single Frame and Multi-Frame Joint Network for 360-degree Panorama Video Super-Resolution

2 code implementations24 Aug 2020 Hongying Liu, Zhubo Ruan, Chaowei Fang, Peng Zhao, Fanhua Shang, Yuanyuan Liu, Lijun Wang

Spherical videos, also known as \ang{360} (panorama) videos, can be viewed with various virtual reality devices such as computers and head-mounted displays.

Video Super-Resolution

Video Super Resolution Based on Deep Learning: A Comprehensive Survey

no code implementations25 Jul 2020 Hongying Liu, Zhubo Ruan, Peng Zhao, Chao Dong, Fanhua Shang, Yuanyuan Liu, Linlin Yang, Radu Timofte

To the best of our knowledge, this work is the first systematic review on VSR tasks, and it is expected to make a contribution to the development of recent studies in this area and potentially deepen our understanding to the VSR techniques based on deep learning.

speech-recognition Speech Recognition +1

Storage Fit Learning with Feature Evolvable Streams

no code implementations22 Jul 2020 Bo-Jian Hou, Yu-Hu Yan, Peng Zhao, Zhi-Hua Zhou

Our framework is able to fit its behavior to different storage budgets when learning with feature evolvable streams with unlabeled data.

Dynamic Regret of Convex and Smooth Functions

no code implementations NeurIPS 2020 Peng Zhao, Yu-Jie Zhang, Lijun Zhang, Zhi-Hua Zhou

We investigate online convex optimization in non-stationary environments and choose the dynamic regret as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence.

Tail-adaptive Bayesian shrinkage

no code implementations4 Jul 2020 Se Yoon Lee, Peng Zhao, Debdeep Pati, Bani K. Mallick

In this paper, we propose a robust sparse estimation method under diverse sparsity regimes, which has a tail-adaptive shrinkage property.

Improved Analysis for Dynamic Regret of Strongly Convex and Smooth Functions

no code implementations10 Jun 2020 Peng Zhao, Lijun Zhang

In this paper, we present an improved analysis for dynamic regret of strongly convex and smooth functions.

CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning

no code implementations4 May 2020 Yuanrui Dong, Peng Zhao, Hanqiao Yu, Cong Zhao, Shusen Yang

The emerging edge-cloud collaborative Deep Learning (DL) paradigm aims at improving the performance of practical DL implementations in terms of cloud bandwidth consumption, response latency, and data privacy preservation.

Classification General Classification +1

Exploratory Machine Learning with Unknown Unknowns

no code implementations5 Feb 2020 Peng Zhao, Jia-Wei Shan, Yu-Jie Zhang, Zhi-Hua Zhou

In conventional supervised learning, a training dataset is given with ground-truth labels from a known label set, and the learned model will classify unseen instances to known labels.

Attribute BIG-bench Machine Learning

Improving deep forest by confidence screening

no code implementations the 18th IEEE International Conference on Data Mining 2019 Ming Pang, Kai-Ming Ting, Peng Zhao, Zhi-Hua Zhou

Most studies about deep learning are based on neural network models, where many layers of parameterized nonlinear differentiable modules are trained by back propagation.

Representation Learning

An Unbiased Risk Estimator for Learning with Augmented Classes

no code implementations NeurIPS 2020 Yu-Jie Zhang, Peng Zhao, Zhi-Hua Zhou

This paper studies the problem of learning with augmented classes (LAC), where augmented classes unobserved in the training data might emerge in the testing phase.

Bandit Convex Optimization in Non-stationary Environments

no code implementations29 Jul 2019 Peng Zhao, Guanghui Wang, Lijun Zhang, Zhi-Hua Zhou

In this paper, we investigate BCO in non-stationary environments and choose the \emph{dynamic regret} as the performance measure, which is defined as the difference between the cumulative loss incurred by the algorithm and that of any feasible comparator sequence.

Decision Making

High-Dimensional Linear Regression via Implicit Regularization

no code implementations22 Mar 2019 Peng Zhao, Yun Yang, Qiao-Chu He

Many statistical estimators for high-dimensional linear regression are M-estimators, formed through minimizing a data-dependent square loss function plus a regularizer.

regression Vocal Bursts Intensity Prediction

Auto-tuning Neural Network Quantization Framework for Collaborative Inference Between the Cloud and Edge

no code implementations16 Dec 2018 Guangli Li, Lei Liu, Xueying Wang, Xiao Dong, Peng Zhao, Xiaobing Feng

By analyzing the characteristics of layers in DNNs, an auto-tuning neural network quantization framework for collaborative inference is proposed.

Collaborative Inference Quantization

Handling Concept Drift via Model Reuse

no code implementations8 Sep 2018 Peng Zhao, Le-Wen Cai, Zhi-Hua Zhou

In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature.

Distribution-Free One-Pass Learning

no code implementations8 Jun 2017 Peng Zhao, Zhi-Hua Zhou

Moreover, as the whole data volume is unknown when constructing the model, it is desired to scan each data item only once with a storage independent with the data volume.

Cannot find the paper you are looking for? You can Submit a new open access paper.