Search Results for author: Meng Li

Found 52 papers, 18 papers with code

SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems

no code implementations10 Apr 2022 Xin Dong, Barbara De Salvo, Meng Li, Chiao Liu, Zhongnan Qu, H. T. Kung, Ziyun Li

We design deep neural networks (DNNs) and corresponding networks' splittings to distribute DNNs' workload to camera sensors and a centralized aggregator on head mounted devices to meet system performance targets in inference accuracy and latency under the given hardware resource constraints.

3D Classification Distributed Computing +1

Resilience-Motivated Distribution System Restoration Considering Electricity-Water-Gas Interdependency

no code implementations17 Feb 2022 Jiaxu Li, Yin Xu, Ying Wang, Meng Li, Jinghan He, Chen-Ching Liu, Kevin P. Schneider

In this paper, a distribution system service restoration method considering the electricity-water-gas interdependency is proposed.

Decentralized Unsupervised Learning of Visual Representations

no code implementations21 Nov 2021 Yawen Wu, Zhepeng Wang, Dewen Zeng, Meng Li, Yiyu Shi, Jingtong Hu

To tackle this problem, we propose a collaborative contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching, by which a unified feature space among clients is learned for better data representations.

Contrastive Learning Federated Learning +2

PyTorchVideo: A Deep Learning Library for Video Understanding

1 code implementation18 Nov 2021 Haoqi Fan, Tullie Murrell, Heng Wang, Kalyan Vasudev Alwala, Yanghao Li, Yilei Li, Bo Xiong, Nikhila Ravi, Meng Li, Haichuan Yang, Jitendra Malik, Ross Girshick, Matt Feiszli, Aaron Adcock, Wan-Yen Lo, Christoph Feichtenhofer

We introduce PyTorchVideo, an open-source deep-learning library that provides a rich set of modular, efficient, and reproducible components for a variety of video understanding tasks, including classification, detection, self-supervised learning, and low-level processing.

Self-Supervised Learning Video Understanding

Low-Rank+Sparse Tensor Compression for Neural Networks

no code implementations2 Nov 2021 Cole Hawkins, Haichuan Yang, Meng Li, Liangzhen Lai, Vikas Chandra

Low-rank tensor compression has been proposed as a promising approach to reduce the memory and compute requirements of neural networks for their deployment on edge devices.

Tensor Decomposition

Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation

no code implementations1 Nov 2021 Jiaqi Gu, Hyoukjun Kwon, Dilin Wang, Wei Ye, Meng Li, Yu-Hsin Chen, Liangzhen Lai, Vikas Chandra, David Z. Pan

Therefore, we propose HRViT, which enhances ViTs to learn semantically-rich and spatially-precise multi-scale representations by integrating high-resolution multi-branch architectures with ViTs.

Image Classification Representation Learning +1

LSTM-RPA: A Simple but Effective Long Sequence Prediction Algorithm for Music Popularity Prediction

1 code implementation27 Oct 2021 Kun Li, Meng Li, Yanling Li, Min Lin

The traditional trend prediction models can better predict the short trend than the long trend.

Omni-sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR via Supernet

no code implementations15 Oct 2021 Haichuan Yang, Yuan Shangguan, Dilin Wang, Meng Li, Pierce Chuang, Xiaohui Zhang, Ganesh Venkatesh, Ozlem Kalinli, Vikas Chandra

From wearables to powerful smart devices, modern automatic speech recognition (ASR) models run on a variety of edge devices with different computational budgets.

Automatic Speech Recognition

Contrastive Quant: Quantization Makes Stronger Contrastive Learning

no code implementations29 Sep 2021 Yonggan Fu, Qixuan Yu, Meng Li, Xu Ouyang, Vikas Chandra, Yingyan Lin

Contrastive learning, which learns visual representations by enforcing feature consistency under different augmented views, has emerged as one of the most effective unsupervised learning methods.

Contrastive Learning Quantization

Federated Contrastive Representation Learning with Feature Fusion and Neighborhood Matching

no code implementations29 Sep 2021 Yawen Wu, Zhepeng Wang, Dewen Zeng, Meng Li, Yiyu Shi, Jingtong Hu

Federated learning (FL) enables distributed clients to learn a shared model for prediction while keeping the training data local on each client.

Contrastive Learning Federated Learning +2

NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training

no code implementations ICLR 2022 Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, Qiang Liu, Vikas Chandra

In this work, we observe that the poor performance is due to a gradient conflict issue: the gradients of different sub-networks conflict with that of the supernet more severely in ViTs than CNNs, which leads to early saturation in training and inferior convergence.

Data Augmentation Neural Architecture Search +1

Noisy Training Improves E2E ASR for the Edge

no code implementations9 Jul 2021 Dilin Wang, Yuan Shangguan, Haichuan Yang, Pierce Chuang, Jiatong Zhou, Meng Li, Ganesh Venkatesh, Ozlem Kalinli, Vikas Chandra

We apply noisy training to improve both dense and sparse state-of-the-art Emformer models and observe consistent WER reduction.

Automatic Speech Recognition Data Augmentation

A Rigid Registration Method in TEVAR

no code implementations29 Apr 2021 Meng Li, Changyan Lin, Heng Wu, Jiasong Li, Hongshuai Cao

Since the mapping relationship between definitized intra-interventional X-ray and undefined pre-interventional Computed Tomography(CT) is uncertain, auxiliary positioning devices or body markers, such as medical implants, are commonly used to determine this relationship.

Computed Tomography (CT) Semantic Segmentation

Vision Transformers with Patch Diversification

1 code implementation26 Apr 2021 Chengyue Gong, Dilin Wang, Meng Li, Vikas Chandra, Qiang Liu

To alleviate this problem, in this work, we introduce novel loss functions in vision transformer training to explicitly encourage diversity across patch representations for more discriminative feature extraction.

Image Classification Semantic Segmentation

Long-Running Speech Recognizer:An End-to-End Multi-Task Learning Framework for Online ASR and VAD

no code implementations2 Mar 2021 Meng Li, Shiyu Zhou, Bo Xu

Experimental results on segmented speech data show that the proposed MTL framework outperforms the baseline single-task learning (STL) framework in ASR task.

Action Detection Activity Detection +2

AlphaNet: Improved Training of Supernets with Alpha-Divergence

2 code implementations16 Feb 2021 Dilin Wang, Chengyue Gong, Meng Li, Qiang Liu, Vikas Chandra

Weight-sharing NAS builds a supernet that assembles all the architectures as its sub-networks and jointly trains the supernet with the sub-networks.

Neural Architecture Search

CPT: Efficient Deep Neural Network Training via Cyclic Precision

1 code implementation ICLR 2021 Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, Yingyan Lin

In this paper, we attempt to explore low-precision training from a new perspective as inspired by recent findings in understanding DNN training: we conjecture that DNNs' precision might have a similar effect as the learning rate during DNN training, and advocate dynamic precision along the training trajectory for further boosting the time/energy efficiency of DNN training.

Language Modelling

ScaleNAS: One-Shot Learning of Scale-Aware Representations for Visual Recognition

no code implementations30 Nov 2020 Hsin-Pai Cheng, Feng Liang, Meng Li, Bowen Cheng, Feng Yan, Hai Li, Vikas Chandra, Yiran Chen

We use ScaleNAS to create high-resolution models for two different tasks, ScaleNet-P for human pose estimation and ScaleNet-S for semantic segmentation.

Ranked #2 on Multi-Person Pose Estimation on CrowdPose (using extra training data)

Multi-Person Pose Estimation Neural Architecture Search +2

Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

no code implementations27 Nov 2020 Zejian Liu, Meng Li

For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes.

Gaussian Processes

AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling

2 code implementations CVPR 2021 Dilin Wang, Meng Li, Chengyue Gong, Vikas Chandra

Our discovered model family, AttentiveNAS models, achieves top-1 accuracy from 77. 3% to 80. 7% on ImageNet, and outperforms SOTA models, including BigNAS and Once-for-All networks.

Neural Architecture Search

DNA: Differentiable Network-Accelerator Co-Search

no code implementations28 Oct 2020 Yongan Zhang, Yonggan Fu, Weiwen Jiang, Chaojian Li, Haoran You, Meng Li, Vikas Chandra, Yingyan Lin

Powerful yet complex deep neural networks (DNNs) have fueled a booming demand for efficient DNN solutions to bring DNN-powered intelligence into numerous applications.

Towards High-Performance Solid-State-LiDAR-Inertial Odometry and Mapping

2 code implementations25 Oct 2020 Kailai Li, Meng Li, Uwe D. Hanebeck

LiLi-OM (Livox LiDAR-inertial odometry and mapping) is real-time capable and achieves superior accuracy over state-of-the-art systems for both LiDAR types on public data sets of mechanical LiDARs and in experiments using the Livox Horizon.


NASGEM: Neural Architecture Search via Graph Embedding Method

no code implementations8 Jul 2020 Hsin-Pai Cheng, Tunhou Zhang, Yixing Zhang, Shi-Yu Li, Feng Liang, Feng Yan, Meng Li, Vikas Chandra, Hai Li, Yiran Chen

To preserve graph correlation information in encoding, we propose NASGEM which stands for Neural Architecture Search via Graph Embedding Method.

Graph Embedding Graph Similarity +2

Functional Group Bridge for Simultaneous Regression and Support Estimation

1 code implementation17 Jun 2020 Zhengjia Wang, John Magnotti, Michael S. Beauchamp, Meng Li

In particular, we show that the estimated coefficient functions are rate optimal in the minimax sense under the $L_2$ norm and resemble a phase transition phenomenon.

Methodology Statistics Theory Statistics Theory

On the Estimation of Derivatives Using Plug-in KRR Estimators

no code implementations2 Jun 2020 Zejian Liu, Meng Li

We study the problem of estimating the derivatives of the regression function, which has a wide range of applications as a key nonparametric functional of unknown functions.

Gaussian Processes

SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks

no code implementations5 Mar 2020 Qitao Shi, Ya-Lin Zhang, Longfei Li, Xinxing Yang, Meng Li, Jun Zhou

Machine learning techniques have been widely applied in Internet companies for various tasks, acting as an essential driving force, and feature engineering has been generally recognized as a crucial tache when constructing machine learning systems.

Feature Engineering

Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization

no code implementations13 Feb 2020 Meng Li, Yilei Li, Pierce Chuang, Liangzhen Lai, Vikas Chandra

Neural network accelerator is a key enabler for the on-device AI inference, for which energy efficiency is an important metric.

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

no code implementations13 Feb 2020 Ke Zhang, Meng Li, Zhengchao Zhang, Xi Lin, Fang He

Multi-vehicle routing problem with soft time windows (MVRPSTW) is an indispensable constituent in urban logistics distribution systems.

Multi-agent Reinforcement Learning reinforcement-learning

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

no code implementations10 Feb 2020 Lei Yang, Zheyu Yan, Meng Li, Hyoukjun Kwon, Liangzhen Lai, Tushar Krishna, Vikas Chandra, Weiwen Jiang, Yiyu Shi

Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs).

Neural Architecture Search

Bayesian Median Autoregression for Robust Time Series Forecasting

1 code implementation4 Jan 2020 Zijian Zeng, Meng Li

We develop a Bayesian median autoregressive (BayesMAR) model for time series forecasting.

Applications Econometrics Methodology

CARP: Compression through Adaptive Recursive Partitioning for Multi-dimensional Images

1 code implementation CVPR 2020 Rongjie Liu, Meng Li, Li Ma

Fast and effective image compression for multi-dimensional images has become increasingly important for efficient storage and transfer of massive amounts of high-resolution images and videos.

Image Compression

PGU-net+: Progressive Growing of U-net+ for Automated Cervical Nuclei Segmentation

1 code implementation4 Nov 2019 Jie Zhao, Lei Dai, Mo Zhang, Fei Yu, Meng Li, Hongfeng Li, Wenjia Wang, Li Zhang

The experimental results show that the PGU-net+ has superior accuracy than the previous state-of-the-art methods on cervical nuclei segmentation.

Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent

1 code implementation ICLR 2020 Dilin Wang, Meng Li, Lemeng Wu, Vikas Chandra, Qiang Liu

Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited.

Experimental realization of state transfer by quantum walks with two coins

no code implementations9 Sep 2019 Yun Shang, Meng Li

Quantum state transfer between different sites is a significant problem for quantum networks and quantum computers.

Quantum Physics

Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images

no code implementations22 Aug 2019 Jiexiang Wang, Cheng Bian, Meng Li, Xin Yang, Kai Ma, Wenao Ma, Jin Yuan, Xinghao Ding, Yefeng Zheng

Automatic and accurate segmentation for retinal and choroidal layers of Optical Coherence Tomography (OCT) is crucial for detection of various ocular diseases.

Deep Instance-Level Hard Negative Mining Model for Histopathology Images

1 code implementation24 Jun 2019 Meng Li, Lin Wu, Arnold Wiliem, Kun Zhao, Teng Zhang, Brian C. Lovell

Histopathology image analysis can be considered as a Multiple instance learning (MIL) problem, where the whole slide histopathology image (WSI) is regarded as a bag of instances (i. e, patches) and the task is to predict a single class label to the WSI.

General Classification Multiple Instance Learning

Multistep Speed Prediction on Traffic Networks: A Graph Convolutional Sequence-to-Sequence Learning Approach with Attention Mechanism

no code implementations24 Oct 2018 Zhengchao Zhang, Meng Li, Xi Lin, Yinhai Wang, Fang He

Multistep traffic forecasting on road networks is a crucial task in successful intelligent transportation system applications.

Function-on-Scalar Quantile Regression with Application to Mass Spectrometry Proteomics Data

1 code implementation1 Sep 2018 Yusha Liu, Meng Li, Jeffrey S. Morris

Mass spectrometry proteomics, characterized by spiky, spatially heterogeneous functional data, can be used to identify potential cancer biomarkers.


Computed Tomography Image Enhancement using 3D Convolutional Neural Network

no code implementations18 Jul 2018 Meng Li, Shiwen Shen, Wen Gao, William Hsu, Jason Cong

Computed tomography (CT) is increasingly being used for cancer screening, such as early detection of lung cancer.

Computed Tomography (CT) Image Enhancement +1

Federated Learning with Non-IID Data

1 code implementation2 Jun 2018 Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vikas Chandra

Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.

Federated Learning

Reducing over-clustering via the powered Chinese restaurant process

no code implementations15 Feb 2018 Jun Lu, Meng Li, David Dunson

Dirichlet process mixture (DPM) models tend to produce many small clusters regardless of whether they are needed to accurately characterize the data - this is particularly true for large data sets.

Learning Asymmetric and Local Features in Multi-Dimensional Data through Wavelets with Recursive Partitioning

1 code implementation2 Nov 2017 Meng Li, Li Ma

Effective learning of asymmetric and local features in images and other data observed on multi-dimensional grids is a challenging objective critical for a wide range of image processing applications involving biomedical and natural images.

Bayesian Inference Image Reconstruction

PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training

no code implementations ICLR 2018 Meng Li, Liangzhen Lai, Naveen Suda, Vikas Chandra, David Z. Pan

Massive data exist among user local platforms that usually cannot support deep neural network (DNN) training due to computation and storage resource constraints.

General Classification Image Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.