Search Results for author: Dongsheng Li

Found 71 papers, 17 papers with code

Finding Global Homophily in Graph Neural Networks When Meeting Heterophily

1 code implementation15 May 2022 Xiang Li, Renyu Zhu, Yao Cheng, Caihua Shan, Siqiang Luo, Dongsheng Li, Weining Qian

Further, for other homophilous nodes excluded in the neighborhood, they are ignored for information aggregation.

Enhancing CTR Prediction with Context-Aware Feature Representation Learning

1 code implementation19 Apr 2022 Fangye Wang, Yingxu Wang, Dongsheng Li, Hansu Gu, Tun Lu, Peng Zhang, Ning Gu

However, most methods only learn a fixed representation for each feature without considering the varying importance of each feature under different contexts, resulting in inferior performance.

Click-Through Rate Prediction Representation Learning

Learning Convolutional Neural Networks in the Frequency Domain

1 code implementation14 Apr 2022 Hengyue Pan, Yixin Chen, Xin Niu, Wenbo Zhou, Dongsheng Li

The most important motivation of this research is that we can use the straightforward element-wise multiplication operation to replace the image convolution in the frequency domain based on the Cross-Correlation Theorem, which obviously reduces the computation complexity.

DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation

no code implementations30 Mar 2022 Yu Tang, Chenyu Wang, Yufan Zhang, Yuliang Liu, Xingcheng Zhang, Linbo Qiao, Zhiquan Lai, Dongsheng Li

To the best of our knowledge, we are the first to make a reasonable dynamic runtime scheduler on the combination of tensor swapping and tensor recomputation without user oversight.

CMMD: Cross-Metric Multi-Dimensional Root Cause Analysis

no code implementations30 Mar 2022 Shifu Yan, Caihua Shan, Wenyi Yang, Bixiong Xu, Dongsheng Li, Lili Qiu, Jie Tong, Qi Zhang

To this end, we propose a cross-metric multi-dimensional root cause analysis method, named CMMD, which consists of two key components: 1) relationship modeling, which utilizes graph neural network (GNN) to model the unknown complex calculation among metrics and aggregation function among dimensions from historical data; 2) root cause localization, which adopts the genetic algorithm to efficiently and effectively dive into the raw data and localize the abnormal dimension(s) once the KPI anomalies are detected.

UENAS: A Unified Evolution-based NAS Framework

no code implementations8 Mar 2022 Zimian Wei, Hengyue Pan, Xin Niu, Peijie Dong, Dongsheng Li

To alleviate the huge search cost caused by the expanded search space, three strategies are adopted: First, an adaptive pruning strategy that iteratively trims the average model size in the population without compromising performance.

Neural Architecture Search

VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning

no code implementations17 Feb 2022 Che Wang, Xufang Luo, Keith Ross, Dongsheng Li

We propose a simple but powerful data-driven framework for solving highly challenging visual deep reinforcement learning (DRL) tasks.

Offline RL reinforcement-learning

Neural Piecewise-Constant Delay Differential Equations

no code implementations4 Jan 2022 Qunxi Zhu, Yifei Shen, Dongsheng Li, Wei Lin

Continuous-depth neural networks, such as the Neural Ordinary Differential Equations (ODEs), have aroused a great deal of interest from the communities of machine learning and data science in recent years, which bridge the connection between deep neural networks and dynamical systems.

Reinforcement Learning Enhanced Explainer for Graph Neural Networks

no code implementations NeurIPS 2021 Caihua Shan, Yifei Shen, Yao Zhang, Xiang Li, Dongsheng Li

To address these issues, we propose a RL-enhanced GNN explainer, RG-Explainer, which consists of three main components: starting point selection, iterative graph generation and stopping criteria learning.

Combinatorial Optimization Graph Generation +1

Towards Generating Real-World Time Series Data

no code implementations16 Nov 2021 Hengzhi Pei, Kan Ren, Yuqing Yang, Chang Liu, Tao Qin, Dongsheng Li

In this paper, we propose a novel generative framework for RTS data - RTSGAN to tackle the aforementioned challenges.

Time Series

Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization

no code implementations18 Oct 2021 Tao Sun, Huaming Ling, Zuoqiang Shi, Dongsheng Li, Bao Wang

In this paper, to eliminate the effort for tuning the momentum-related hyperparameter, we propose a new adaptive momentum inspired by the optimal choice of the heavy ball momentum for quadratic optimization.

Image Classification Language Modelling +2

EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks

no code implementations18 Oct 2021 Shengwei Li, Zhiquan Lai, Dongsheng Li, Xiangyu Ye, Yabo Duan

Distributed data-parallel training has been widely used for natural language processing (NLP) neural network models.

S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning

no code implementations5 Oct 2021 Keshi Ge, Yongquan Fu, Zhiquan Lai, Xiaoge Deng, Dongsheng Li

Distributed stochastic gradient descent (SGD) approach has been widely used in large-scale deep learning, and the gradient collective method is vital to ensure the training scalability of the distributed deep learning system.

Deep Ensemble Policy Learning

no code implementations29 Sep 2021 Zhengyu Yang, Kan Ren, Xufang Luo, Weiqing Liu, Jiang Bian, Weinan Zhang, Dongsheng Li

Ensemble learning, which can consistently improve the prediction performance in supervised learning, has drawn increasing attentions in reinforcement learning (RL).

Ensemble Learning

AARL: Automated Auxiliary Loss for Reinforcement Learning

no code implementations29 Sep 2021 Tairan He, Yuge Zhang, Kan Ren, Che Wang, Weinan Zhang, Dongsheng Li, Yuqing Yang

A good state representation is crucial to reinforcement learning (RL) while an ideal representation is hard to learn only with signals from the RL objective.

reinforcement-learning

SANE: Specialization-Aware Neural Network Ensemble

no code implementations29 Sep 2021 Ziyue Li, Kan Ren, Xinyang Jiang, Mingzhe Han, Haipeng Zhang, Dongsheng Li

Real-world data is often generated by some complex distribution, which can be approximated by a composition of multiple simpler distributions.

Ensemble Learning

Adaptive Q-learning for Interaction-Limited Reinforcement Learning

no code implementations29 Sep 2021 Han Zheng, Xufang Luo, Pengfei Wei, Xuan Song, Dongsheng Li, Jing Jiang

Specifically, we explicitly consider the difference between the online and offline data and apply an adaptive update scheme accordingly, i. e., a pessimistic update strategy for the offline dataset and a greedy or no pessimistic update scheme for the online dataset.

Offline RL online learning +2

Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision

no code implementations30 Aug 2021 Bo Li, Xinyang Jiang, Donglin Bai, Yuge Zhang, Ningxin Zheng, Xuanyi Dong, Lu Liu, Yuqing Yang, Dongsheng Li

The energy consumption of deep learning models is increasing at a breathtaking rate, which raises concerns due to potential negative effects on carbon neutrality in the context of global warming and climate change.

Model Compression

How Powerful is Graph Convolution for Recommendation?

1 code implementation17 Aug 2021 Yifei Shen, Yongji Wu, Yao Zhang, Caihua Shan, Jun Zhang, Khaled B. Letaief, Dongsheng Li

In this paper, we endeavor to obtain a better understanding of GCN-based CF methods via the lens of graph signal processing.

Collaborative Filtering

Energy-Based Open-World Uncertainty Modeling for Confidence Calibration

no code implementations ICCV 2021 Yezhen Wang, Bo Li, Tong Che, Kaiyang Zhou, Ziwei Liu, Dongsheng Li

Confidence calibration is of great importance to the reliability of decisions made by machine learning systems.

Invariant Information Bottleneck for Domain Generalization

no code implementations11 Jun 2021 Bo Li, Yifei Shen, Yezhen Wang, Wenzhen Zhu, Colorado J. Reed, Jun Zhang, Dongsheng Li, Kurt Keutzer, Han Zhao

IIB significantly outperforms IRM on synthetic datasets, where the pseudo-invariant features and geometric skews occur, showing the effectiveness of proposed formulation in overcoming failure modes of IRM.

Domain Generalization

Deep Tiny Network for Recognition-Oriented Face Image Quality Assessment

no code implementations9 Jun 2021 Baoyun Peng, Min Liu, Heng Yang, Zhaoning Zhang, Dongsheng Li

Based on the proposed quality measurement, we propose a deep Tiny Face Quality network (tinyFQnet) to learn a quality prediction function from data.

Face Recognition Image Quality Assessment

Scalable and Explainable 1-Bit Matrix Completion via Graph Signal Learning

1 code implementation AAAI 2021 Chao Chen, Dongsheng Li, Junchi Yan, Hanchi Huang, Xiaokang Yang

One-bit matrix completion is an important class of positiveunlabeled (PU) learning problems where the observations consist of only positive examples, eg, in top-N recommender systems.

Collaborative Ranking Matrix Completion +1

Graph Pooling via Coarsened Graph Infomax

no code implementations4 May 2021 Yunsheng Pang, Yunxiang Zhao, Dongsheng Li

Graph pooling that summaries the information in a large graph into a compact form is essential in hierarchical graph representation learning.

Contrastive Learning Graph Representation Learning

Decentralized Federated Averaging

no code implementations23 Apr 2021 Tao Sun, Dongsheng Li, Bao Wang

In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients.

NeuSE: A Neural Snapshot Ensemble Method for Collaborative Filtering

no code implementations15 Apr 2021 Dongsheng Li, Haodong Liu, Chao Chen, Yingying Zhao, Stephen M. Chu, Bo Yang

In collaborative filtering (CF) algorithms, the optimal models are usually learned by globally minimizing the empirical risks averaged over all the observed data.

Collaborative Filtering Ensemble Learning

Hierarchical Adaptive Pooling by Capturing High-order Dependency for Graph Representation Learning

no code implementations13 Apr 2021 Ning Liu, Songlei Jian, Dongsheng Li, Yiming Zhang, Zhiquan Lai, Hongzuo Xu

Graph neural networks (GNN) have been proven to be mature enough for handling graph-structured data on node-level graph representation learning tasks.

Graph Classification Graph Matching +2

A Reinforcement-Learning-Based Energy-Efficient Framework for Multi-Task Video Analytics Pipeline

no code implementations9 Apr 2021 Yingying Zhao, Mingzhi Dong, Yujiang Wang, Da Feng, Qin Lv, Robert P. Dick, Dongsheng Li, Tun Lu, Ning Gu, Li Shang

By monitoring the impact of varying resolution on the quality of high-dimensional video analytics features, hence the accuracy of video analytics results, the proposed end-to-end optimization framework learns the best non-myopic policy for dynamically controlling the resolution of input video streams to globally optimize energy efficiency.

Instance Segmentation Optical Flow Estimation +3

Decentralized Statistical Inference with Unrolled Graph Neural Networks

1 code implementation4 Apr 2021 He Wang, Yifei Shen, Ziyuan Wang, Dongsheng Li, Jun Zhang, Khaled B. Letaief, Jie Lu

In this paper, we investigate the decentralized statistical inference problem, where a network of agents cooperatively recover a (structured) vector from private noisy samples without centralized coordination.

Stability and Generalization of the Decentralized Stochastic Gradient Descent

no code implementations2 Feb 2021 Tao Sun, Dongsheng Li, Bao Wang

The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models.

Inertial Proximal Deep Learning Alternating Minimization for Efficient Neutral Network Training

no code implementations30 Jan 2021 Linbo Qiao, Tao Sun, Hengyue Pan, Dongsheng Li

In recent years, the Deep Learning Alternating Minimization (DLAM), which is actually the alternating minimization applied to the penalty form of the deep neutral networks training, has been developed as an alternative algorithm to overcome several drawbacks of Stochastic Gradient Descent (SGD) algorithms.

Learning content and context with language bias for Visual Question Answering

1 code implementation21 Dec 2020 Chao Yang, Su Feng, Dongsheng Li, HuaWei Shen, Guoqing Wang, Bin Jiang

Many works concentrate on how to reduce language bias which makes models answer questions ignoring visual content and language context.

Question Answering Visual Question Answering +1

Meta-Learning for Neural Relation Classification with Distant Supervision

no code implementations26 Oct 2020 Zhenzhen Li, Jian-Yun Nie, Benyou Wang, Pan Du, Yuhan Zhang, Lixin Zou, Dongsheng Li

Distant supervision provides a means to create a large number of weakly labeled data at low cost for relation classification.

Classification General Classification +2

Dynamic Knowledge Distillation for Black-box Hypothesis Transfer Learning

no code implementations24 Jul 2020 Yiqin Yu, Xu Min, Shiwan Zhao, Jing Mei, Fei Wang, Dongsheng Li, Kenney Ng, Shaochun Li

In real world applications like healthcare, it is usually difficult to build a machine learning prediction model that works universally well across different institutions.

Knowledge Distillation Transfer Learning

Adaptive Temporal Difference Learning with Linear Function Approximation

no code implementations20 Feb 2020 Tao Sun, Han Shen, Tianyi Chen, Dongsheng Li

Typically, the performance of TD(0) and TD($\lambda$) is very sensitive to the choice of stepsizes.

OpenAI Gym reinforcement-learning

Towards Precise End-to-end Weakly Supervised Object Detection Network

1 code implementation ICCV 2019 Ke Yang, Dongsheng Li, Yong Dou

It is challenging for weakly supervised object detection network to precisely predict the positions of the objects, since there are no instance-level category annotations.

Multiple Instance Learning Weakly Supervised Object Detection

XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training

no code implementations24 Oct 2019 Lei Guan, Wotao Yin, Dongsheng Li, Xicheng Lu

It allows the overlapping of the pipelines of multiple micro-batches, including those belonging to different mini-batches.

General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme

no code implementations NeurIPS 2019 Tao Sun, Yuejiao Sun, Dongsheng Li, Qing Liao

In this paper, we propose a general proximal incremental aggregated gradient algorithm, which contains various existing algorithms including the basic incremental aggregated gradient method.

Decentralized Markov Chain Gradient Descent

no code implementations23 Sep 2019 Tao Sun, Dongsheng Li

Decentralized stochastic gradient method emerges as a promising solution for solving large-scale machine learning problems.

A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning

1 code implementation IJCNLP 2019 Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li

Rapid progress has been made in the field of reading comprehension and question answering, where several systems have achieved human parity in some simplified settings.

Question Answering Reading Comprehension

Heavy-ball Algorithms Always Escape Saddle Points

no code implementations23 Jul 2019 Tao Sun, Dongsheng Li, Zhe Quan, Hao Jiang, Shengguo Li, Yong Dou

In this paper, we answer a question: can the nonconvex heavy-ball algorithms with random initialization avoid saddle points?

Exploring Pre-trained Language Models for Event Extraction and Generation

no code implementations ACL 2019 Sen Yang, Dawei Feng, Linbo Qiao, Zhigang Kan, Dongsheng Li

Traditional approaches to the task of ACE event extraction usually depend on manually annotated data, which is often laborious to create and limited in size.

Event Extraction General Classification

IF-TTN: Information Fused Temporal Transformation Network for Video Action Recognition

no code implementations26 Feb 2019 Ke Yang, Peng Qiao, Dongsheng Li, Yong Dou

Focusing on discriminate spatiotemporal feature learning, we propose Information Fused Temporal Transformation Network (IF-TTN) for action recognition on top of popular Temporal Segment Network (TSN) framework.

Action Recognition Optical Flow Estimation

Exploring Frame Segmentation Networks for Temporal Action Localization

no code implementations14 Feb 2019 Ke Yang, Xiaolong Shen, Peng Qiao, Shijie Li, Dongsheng Li, Yong Dou

The proposed FSN can make dense predictions at frame-level for a video clip using both spatial and temporal context information.

Frame Temporal Action Localization

Iteratively reweighted penalty alternating minimization methods with continuation for image deblurring

no code implementations9 Feb 2019 Tao Sun, Dongsheng Li, Hao Jiang, Zhe Quan

In this paper, we consider a class of nonconvex problems with linear constraints appearing frequently in the area of image processing.

Deblurring Image Deblurring

Collaborative Filtering with Stability

no code implementations6 Nov 2018 Dongsheng Li, Chao Chen, Qin Lv, Junchi Yan, Li Shang, Stephen M. Chu

Collaborative filtering (CF) is a popular technique in today's recommender systems, and matrix approximation-based CF methods have achieved great success in both rating prediction and top-N recommendation tasks.

Collaborative Filtering Recommendation Systems

Non-ergodic Convergence Analysis of Heavy-Ball Algorithms

no code implementations5 Nov 2018 Tao Sun, Penghang Yin, Dongsheng Li, Chun Huang, Lei Guan, Hao Jiang

For objective functions satisfying a relaxed strongly convex condition, the linear convergence is established under weaker assumptions on the step size and inertial parameter than made in the existing literature.

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

no code implementations11 Sep 2018 Lei Guan, Linbo Qiao, Dongsheng Li, Tao Sun, Keshi Ge, Xicheng Lu

Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection.

General Classification Variable Selection

Attention-Guided Answer Distillation for Machine Reading Comprehension

no code implementations EMNLP 2018 Minghao Hu, Yuxing Peng, Furu Wei, Zhen Huang, Dongsheng Li, Nan Yang, Ming Zhou

Despite that current reading comprehension systems have achieved significant advancements, their promising performances are often obtained at the cost of making an ensemble of numerous models.

Knowledge Distillation Machine Reading Comprehension

Loss Rank Mining: A General Hard Example Mining Method for Real-time Detectors

no code implementations10 Apr 2018 Hao Yu, Zhaoning Zhang, Zheng Qin, Hao Wu, Dongsheng Li, Jun Zhao, Xicheng Lu

LRM is a general method for real-time detectors, as it utilizes the final feature map which exists in all real-time detectors to mine hard examples.

Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions

3 code implementations27 Mar 2018 Zheng Qin, Zhaoning Zhang, Dongsheng Li, Yiming Zhang, Yuxing Peng

Depthwise convolutions provide significant performance benefits owing to the reduction in both parameters and mult-adds.

Non-ergodic Complexity of Convex Proximal Inertial Gradient Descents

no code implementations23 Jan 2018 Tao Sun, Linbo Qiao, Dongsheng Li

The non-ergodic O(1/k) rate is proved for proximal inertial gradient descent with constant stepzise when the objective function is coercive.

Mixture-Rank Matrix Approximation for Collaborative Filtering

no code implementations NeurIPS 2017 Dongsheng Li, Chao Chen, Wei Liu, Tun Lu, Ning Gu, Stephen Chu

However, our studies show that submatrices with different ranks could coexist in the same user-item rating matrix, so that approximations with fixed ranks cannot perfectly describe the internal structures of the rating matrix, therefore leading to inferior recommendation accuracy.

Collaborative Filtering

Exploring Temporal Preservation Networks for Precise Temporal Action Localization

no code implementations10 Aug 2017 Ke Yang, Peng Qiao, Dongsheng Li, Shaohe Lv, Yong Dou

A newly proposed work exploits Convolutional-Deconvolutional-Convolutional (CDC) filters to upsample the predictions of 3D ConvNets, making it possible to perform per-frame action predictions and achieving promising performance in terms of temporal action localization.

Frame Temporal Action Localization +1

S-OHEM: Stratified Online Hard Example Mining for Object Detection

no code implementations5 May 2017 Minne Li, Zhaoning Zhang, Hao Yu, Xinyuan Chen, Dongsheng Li

S-OHEM exploits OHEM with stratified sampling, a widely-adopted sampling technique, to choose the training examples according to this influence during hard example mining, and thus enhance the performance of object detectors.

Object Detection

Weakly supervised object detection using pseudo-strong labels

no code implementations16 Jul 2016 Ke Yang, Dongsheng Li, Yong Dou, Shaohe Lv, Qiang Wang

Object detection is an import task of computer vision. A variety of methods have been proposed, but methods using the weak labels still do not have a satisfactory result. In this paper, we propose a new framework that using the weakly supervised method's output as the pseudo-strong labels to train a strongly supervised model. One weakly supervised method is treated as black-box to generate class-specific bounding boxes on train dataset. A de-noise method is then applied to the noisy bounding boxes. Then the de-noised pseudo-strong labels are used to train a strongly object detection network. The whole framework is still weakly supervised because the entire process only uses the image-level labels. The experiment results on PASCAL VOC 2007 prove the validity of our framework, and we get result 43. 4% on mean average precision compared to 39. 5% of the previous best result and 34. 5% of the initial method, respectively. And this frame work is simple and distinct, and is promising to be applied to other method easily.

Frame Weakly Supervised Object Detection

Cannot find the paper you are looking for? You can Submit a new open access paper.