Search Results for author: Shen Li

Found 44 papers, 16 papers with code

Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning

no code implementations22 Feb 2024 Shen Li, Liuyi Yao, Jinyang Gao, Lan Zhang, Yaliang Li

To support various applications, business owners often seek the customized models that are obtained by fine-tuning a pre-trained LLM through the API provided by LLM owners or cloud servers.

xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein

no code implementations11 Jan 2024 Bo Chen, Xingyi Cheng, Pan Li, Yangli-ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Zeng, Chiming Liu, Aohan Zeng, Yuxiao Dong, Jie Tang, Le Song

We propose a unified protein language model, xTrimoPGLM, to address these two types of tasks simultaneously through an innovative pre-training framework.

Protein Language Model

Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection

1 code implementation28 Sep 2023 Jiaying Wu, Shen Li, Ailin Deng, Miao Xiong, Bryan Hooi

Despite considerable advances in automated fake news detection, due to the timely nature of news, it remains a critical open question how to effectively predict the veracity of news articles based on limited fact-checks.

Fake News Detection

On the Performance of Multidimensional Constellation Shaping for Linear and Nonlinear Optical Fiber Channel

no code implementations17 Aug 2023 Bin Chen, Zhiwei Liang, Shen Li, Yi Lei, Gabriele Liga, Alex Alvarado

Multidimensional constellation shaping of up to 32 dimensions with different spectral efficiencies are compared through AWGN and fiber-optic simulations.

A Robust Integrated Multi-Strategy Bus Control System via Deep Reinforcement Learning

no code implementations16 Aug 2023 Qinghui Nie, Jishun Ou, Haiyang Zhang, Jiawei Lu, Shen Li, Haotian Shi

An efficient urban bus control system has the potential to significantly reduce travel delays and streamline the allocation of transportation resources, thereby offering enhanced and user-friendly transit services to passengers.

How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence

no code implementations2 May 2023 Xuemin Hu, Shen Li, Tingyu Huang, Bo Tang, Rouxing Huai, Long Chen

In general, a large scale of testing in simulation environment is conducted and then the learned driving knowledge is transferred to the real world, so how to adapt driving knowledge learned in simulation to reality becomes a critical issue.

Autonomous Driving

Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness

1 code implementation6 Feb 2023 Ailin Deng, Shen Li, Miao Xiong, Zhirui Chen, Bryan Hooi

Trustworthy machine learning is of primary importance to the practical deployment of deep learning models.

Out-of-Distribution Detection

Probabilistic Knowledge Distillation of Face Ensembles

no code implementations CVPR 2023 Jianqing Xu, Shen Li, Ailin Deng, Miao Xiong, Jiaying Wu, Jiaxiang Wu, Shouhong Ding, Bryan Hooi

Mean ensemble (i. e. averaging predictions from multiple models) is a commonly-used technique in machine learning that improves the performance of each individual model.

Face Image Quality Face Recognition +2

Coordinating CAV Swarms at Intersections with a Deep Learning Model

no code implementations10 Nov 2022 Jiawei Zhang, Shen Li, Li Li

Connected and automated vehicles (CAVs) are viewed as a special kind of robots that have the potential to significantly improve the safety and efficiency of traffic.


lo-fi: distributed fine-tuning without communication

no code implementations19 Oct 2022 Mitchell Wortsman, Suchin Gururangan, Shen Li, Ali Farhadi, Ludwig Schmidt, Michael Rabbat, Ari S. Morcos

When fine-tuning DeiT-base and DeiT-large on ImageNet, this procedure matches accuracy in-distribution and improves accuracy under distribution shift compared to the baseline, which observes the same amount of data but communicates gradients at each step.

Neural PCA for Flow-Based Representation Learning

no code implementations23 Aug 2022 Shen Li, Bryan Hooi

Without exploiting any label information, the principal components recovered store the most informative elements in their \emph{leading} dimensions and leave the negligible in the \emph{trailing} ones, allowing for clear performance improvements of $5\%$-$10\%$ in downstream tasks.

Density Estimation Inductive Bias +1

Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations

no code implementations9 Jun 2022 Yanwei Wang, Nadia Figueroa, Shen Li, Ankit Shah, Julie Shah

In this work, we identify the roots of this challenge as the failure of a learned continuous policy to satisfy the discrete plan implicit in the demonstration.

Imitation Learning

Parameter-Efficient Sparsity for Large Language Models Fine-Tuning

2 code implementations23 May 2022 Yuchao Li, Fuli Luo, Chuanqi Tan, Mengdi Wang, Songfang Huang, Shen Li, Junjie Bai

With the dramatically increased number of parameters in language models, sparsity methods have received ever-increasing research focus to compress and accelerate the models.

Neighborhood Attention Transformer

5 code implementations CVPR 2023 Ali Hassani, Steven Walton, Jiachen Li, Shen Li, Humphrey Shi

We present Neighborhood Attention (NA), the first efficient and scalable sliding-window attention mechanism for vision.

Image Classification Object Detection +1

A General Framework for Debiasing in CTR Prediction

no code implementations6 Dec 2021 Wenjie Chu, Shen Li, Chao Chen, Longfei Xu, Hengbin Cui, Kaikui Liu

Most of the existing methods for debaising in click-through rate (CTR) prediction depend on an oversimplified assumption, i. e., the click probability is the product of observation probability and relevance probability.

Click-Through Rate Prediction

Enhancing Top-N Item Recommendations by Peer Collaboration

no code implementations31 Oct 2021 Yang Sun, Fajie Yuan, Min Yang, Alexandros Karatzoglou, Shen Li, Xiaoyan Zhao

In this paper, we plan to exploit such redundancy phenomena to improve the performance of RS.

Recommendation Systems

R4: A Framework for Route Representation and Route Recommendation

no code implementations20 Oct 2021 Ran Cheng, Chao Chen, Longfei Xu, Shen Li, Lei Wang, Hengbin Cui, Kaikui Liu, Xiaolong Li

For user representation, we utilize a series of historical navigation to extract user preference.


Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty

no code implementations18 Oct 2021 Shen Li, Theodoros Stouraitis, Michael Gienger, Sethu Vijayakumar, Julie A. Shah

Consistent state estimation is challenging, especially under the epistemic uncertainties arising from learned (nonlinear) dynamic and observation models.

Supervised Bayesian Specification Inference from Demonstrations

no code implementations6 Jul 2021 Ankit Shah, Pritish Kamath, Shen Li, Patrick Craven, Kevin Landers, Kevin Oden, Julie Shah

When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.

Probabilistic Programming

You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient

1 code implementation4 Jun 2021 Shaokun Zhang, Xiawu Zheng, Chenyi Yang, Yuchao Li, Yan Wang, Fei Chao, Mengdi Wang, Shen Li, Jun Yang, Rongrong Ji

Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.

AutoML Model Compression

1xN Pattern for Pruning Convolutional Neural Networks

1 code implementation31 May 2021 Mingbao Lin, Yuxin Zhang, Yuchao Li, Bohong Chen, Fei Chao, Mengdi Wang, Shen Li, Yonghong Tian, Rongrong Ji

We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.

Network Pruning

Tackling Variabilities in Autonomous Driving

no code implementations21 Apr 2021 Yuqiong Qi, Yang Hu, Haibin Wu, Shen Li, Haiyu Mao, Xiaochun Ye, Dongrui Fan, Ninghui Sun

In this work, we aim to extensively explore the above system design challenges and these challenges motivate us to propose a comprehensive framework that synergistically handles the heterogeneous hardware accelerator design principles, system design criteria, and task scheduling mechanism.

Autonomous Driving Reinforcement Learning (RL) +1

Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking

no code implementations18 Apr 2021 Shen Li, Bingpeng Ma, Hong Chang, Shiguang Shan, Xilin Chen

This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN), for visual object tracking.

Object Visual Object Tracking

PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers

1 code implementation5 Feb 2021 Chaoyang He, Shen Li, Mahdi Soltanolkotabi, Salman Avestimehr

PipeTransformer automatically adjusts the pipelining and data parallelism by identifying and freezing some layers during the training, and instead allocates resources for training of the remaining active layers.

Hypersphere Face Uncertainty Learning

no code implementations1 Jan 2021 Shen Li, Jianqing Xu, Xiaqing Xu, Pengcheng Shen, Shaoxin Li, Bryan Hooi

To address these issues, in this paper, we propose a novel framework for face uncertainty learning in hyperspherical space.

Face Verification

PyTorch Distributed: Experiences on Accelerating Data Parallel Training

3 code implementations28 Jun 2020 Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala

This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module.

Quantum Inspired Word Representation and Computation

no code implementations6 Apr 2020 Shen Li, Renfen Hu, Jinshan Wu

Word meaning has different aspects, while the existing word representation "compresses" these aspects into a single vector, and it needs further analysis to recover the information in different dimensions.

Identifying through Flows for Recovering Latent Representations

2 code implementations ICLR 2020 Shen Li, Bryan Hooi, Gim Hee Lee

Yet, most deep generative models do not address the question of identifiability, and thus fail to deliver on the promise of the recovery of the true latent sources that generate the observations.

Representation Learning

Self-Balanced Dropout

1 code implementation6 Aug 2019 Shen Li, Chenhao Su, Renfen Hu, Zhengdong Lu

Dropout is known as an effective way to reduce overfitting via preventing co-adaptations of units.

Bayesian Inference of Temporal Task Specifications from Demonstrations

no code implementations NeurIPS 2018 Ankit Shah, Pritish Kamath, Julie A. Shah, Shen Li

When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.

Probabilistic Programming

Generalize Symbolic Knowledge With Neural Rule Engine

no code implementations30 Aug 2018 Shen Li, Hengru Xu, Zhengdong Lu

As neural networks have dominated the state-of-the-art results in a wide range of NLP tasks, it attracts considerable attention to improve the performance of neural models by integrating symbolic knowledge.

RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations

no code implementations9 Sep 2017 Shuochao Yao, Yiran Zhao, Huajie Shao, Aston Zhang, Chao Zhang, Shen Li, Tarek Abdelzaher

Recent advances in deep learning have led various applications to unprecedented achievements, which could potentially bring higher intelligence to a broad spectrum of mobile and ubiquitous applications.

Cannot find the paper you are looking for? You can Submit a new open access paper.