Search Results for author: Shen Li

Found 32 papers, 13 papers with code

Neural PCA for Flow-Based Representation Learning

1 code implementation23 Aug 2022 Shen Li, Bryan Hooi

Without exploiting any label information, the principal components recovered store the most informative elements in their \emph{leading} dimensions and leave the negligible in the \emph{trailing} ones, allowing for clear performance improvements of $5\%$-$10\%$ in downstream tasks.

Density Estimation Inductive Bias +1

Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations

no code implementations9 Jun 2022 Yanwei Wang, Nadia Figueroa, Shen Li, Ankit Shah, Julie Shah

Learning from demonstration (LfD) methods have shown promise for solving multi-step tasks; however, these approaches do not guarantee successful reproduction of the task given disturbances.

Imitation Learning

Parameter-Efficient Sparsity for Large Language Models Fine-Tuning

2 code implementations23 May 2022 Yuchao Li, Fuli Luo, Chuanqi Tan, Mengdi Wang, Songfang Huang, Shen Li, Junjie Bai

With the dramatically increased number of parameters in language models, sparsity methods have received ever-increasing research focus to compress and accelerate the models.

Neighborhood Attention Transformer

3 code implementations14 Apr 2022 Ali Hassani, Steven Walton, Jiachen Li, Shen Li, Humphrey Shi

We present Neighborhood Attention Transformer (NAT), an efficient, accurate and scalable hierarchical transformer that works well on both image classification and downstream vision tasks.

Image Classification Object Detection +1

A General Framework for Debiasing in CTR Prediction

no code implementations6 Dec 2021 Wenjie Chu, Shen Li, Chao Chen, Longfei Xu, Hengbin Cui, Kaikui Liu

Most of the existing methods for debaising in click-through rate (CTR) prediction depend on an oversimplified assumption, i. e., the click probability is the product of observation probability and relevance probability.

Click-Through Rate Prediction

Enhancing Top-N Item Recommendations by Peer Collaboration

no code implementations31 Oct 2021 Yang Sun, Fajie Yuan, Min Yang, Alexandros Karatzoglou, Shen Li, Xiaoyan Zhao

In this paper, we plan to exploit such redundancy phenomena to improve the performance of RS.

Recommendation Systems

R4: A Framework for Route Representation and Route Recommendation

no code implementations20 Oct 2021 Ran Cheng, Chao Chen, Longfei Xu, Shen Li, Lei Wang, Hengbin Cui, Kaikui Liu, Xiaolong Li

For user representation, we utilize a series of historical navigation to extract user preference.

Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty

no code implementations18 Oct 2021 Shen Li, Theodoros Stouraitis, Michael Gienger, Sethu Vijayakumar, Julie A. Shah

Consistent state estimation is challenging, especially under the epistemic uncertainties arising from learned (nonlinear) dynamic and observation models.

Supervised Bayesian Specification Inference from Demonstrations

no code implementations6 Jul 2021 Ankit Shah, Pritish Kamath, Shen Li, Patrick Craven, Kevin Landers, Kevin Oden, Julie Shah

When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.

Probabilistic Programming

You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient

1 code implementation4 Jun 2021 Shaokun Zhang, Xiawu Zheng, Chenyi Yang, Yuchao Li, Yan Wang, Fei Chao, Mengdi Wang, Shen Li, Jun Yang, Rongrong Ji

Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.

AutoML Model Compression

1xN Pattern for Pruning Convolutional Neural Networks

1 code implementation31 May 2021 Mingbao Lin, Yuxin Zhang, Yuchao Li, Bohong Chen, Fei Chao, Mengdi Wang, Shen Li, Yonghong Tian, Rongrong Ji

We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.

Network Pruning

Tackling Variabilities in Autonomous Driving

no code implementations21 Apr 2021 Yuqiong Qi, Yang Hu, Haibin Wu, Shen Li, Haiyu Mao, Xiaochun Ye, Dongrui Fan, Ninghui Sun

In this work, we aim to extensively explore the above system design challenges and these challenges motivate us to propose a comprehensive framework that synergistically handles the heterogeneous hardware accelerator design principles, system design criteria, and task scheduling mechanism.

Autonomous Driving

Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking

no code implementations18 Apr 2021 Shen Li, Bingpeng Ma, Hong Chang, Shiguang Shan, Xilin Chen

This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN), for visual object tracking.

Visual Object Tracking

PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers

no code implementations5 Feb 2021 Chaoyang He, Shen Li, Mahdi Soltanolkotabi, Salman Avestimehr

PipeTransformer automatically adjusts the pipelining and data parallelism by identifying and freezing some layers during the training, and instead allocates resources for training of the remaining active layers.

Hypersphere Face Uncertainty Learning

no code implementations1 Jan 2021 Shen Li, Jianqing Xu, Xiaqing Xu, Pengcheng Shen, Shaoxin Li, Bryan Hooi

To address these issues, in this paper, we propose a novel framework for face uncertainty learning in hyperspherical space.

Face Verification

PyTorch Distributed: Experiences on Accelerating Data Parallel Training

2 code implementations28 Jun 2020 Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala

This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module.

Quantum Inspired Word Representation and Computation

no code implementations6 Apr 2020 Shen Li, Renfen Hu, Jinshan Wu

Word meaning has different aspects, while the existing word representation "compresses" these aspects into a single vector, and it needs further analysis to recover the information in different dimensions.

Identifying through Flows for Recovering Latent Representations

2 code implementations ICLR 2020 Shen Li, Bryan Hooi, Gim Hee Lee

Yet, most deep generative models do not address the question of identifiability, and thus fail to deliver on the promise of the recovery of the true latent sources that generate the observations.

Representation Learning

Self-Balanced Dropout

1 code implementation6 Aug 2019 Shen Li, Chenhao Su, Renfen Hu, Zhengdong Lu

Dropout is known as an effective way to reduce overfitting via preventing co-adaptations of units.

Bayesian Inference of Temporal Task Specifications from Demonstrations

no code implementations NeurIPS 2018 Ankit Shah, Pritish Kamath, Julie A. Shah, Shen Li

When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.

Probabilistic Programming

Generalize Symbolic Knowledge With Neural Rule Engine

no code implementations30 Aug 2018 Shen Li, Hengru Xu, Zhengdong Lu

As neural networks have dominated the state-of-the-art results in a wide range of NLP tasks, it attracts considerable attention to improve the performance of neural models by integrating symbolic knowledge.

RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations

no code implementations9 Sep 2017 Shuochao Yao, Yiran Zhao, Huajie Shao, Aston Zhang, Chao Zhang, Shen Li, Tarek Abdelzaher

Recent advances in deep learning have led various applications to unprecedented achievements, which could potentially bring higher intelligence to a broad spectrum of mobile and ubiquitous applications.

Cannot find the paper you are looking for? You can Submit a new open access paper.