no code implementations • EMNLP 2020 • Zheng Li, Mukul Kumar, William Headden, Bing Yin, Ying WEI, Yu Zhang, Qiang Yang
Recent emergence of multilingual pre-training language model (mPLM) has enabled breakthroughs on various downstream cross-lingual transfer (CLT) tasks.
no code implementations • EMNLP 2021 • Zheng Li, Danqing Zhang, Tianyu Cao, Ying WEI, Yiwei Song, Bing Yin
In this work, we explore multilingual sequence labeling with minimal supervision using a single unified model for multiple languages.
no code implementations • 27 Mar 2023 • Ruijie Wang, Zheng Li, Jingfeng Yang, Tianyu Cao, Chao Zhang, Bing Yin, Tarek Abdelzaher
This paper investigates cross-lingual temporal knowledge graph reasoning problem, which aims to facilitate reasoning on Temporal Knowledge Graphs (TKGs) in low-resource languages by transfering knowledge from TKGs in high-resource ones.
1 code implementation • 3 Mar 2023 • Joseph Kampeas, Yury Nahshan, Hanoch Kremer, Gil Lederman, Shira Zaloshinski, Zheng Li, Emir Haleva
Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources.
no code implementations • 1 Mar 2023 • Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Zhongtian Du
To alleviate the gradient vanishing problem, we propose a Selectively Hard Negative Mining (SelHN) strategy, which chooses whether to mine hard negative samples according to the gradient vanishing condition.
no code implementations • 19 Feb 2023 • Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bin Yin, Tuo Zhao
Since the teacher model has a significantly larger capacity and stronger representation power than the student model, it is very difficult for the student to produce predictions that match the teacher's over a massive amount of open-domain training data.
2 code implementations • 3 Jan 2023 • Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, Yang Zhang
A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset.
1 code implementation • 20 Dec 2022 • Shiqi Wang, Zheng Li, Haifeng Qian, Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, Bing Xiang
Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.
1 code implementation • 29 Nov 2022 • Zheng Li, Xiang Li, Lingfeng Yang, Borui Zhao, RenJie Song, Lei Luo, Jun Li, Jian Yang
In this paper, we propose a simple curriculum-based technique, termed Curriculum Temperature for Knowledge Distillation (CTKD), which controls the task difficulty level during the student's learning career through a dynamic and learnable temperature.
no code implementations • 17 Nov 2022 • Yenho Chen, Carl W. Harris, Xiaoyu Ma, Zheng Li, Francisco Pereira, Charles Y. Zheng
We propose a decoding-based approach to detect context effects on neural codes in longitudinal neural recording data.
no code implementations • 16 Nov 2022 • Juan Zha, Zheng Li, Ying WEI, Yu Zhang
However, most prior works assume that all the tasks are sampled from a single data source, which cannot adapt to real-world scenarios where tasks are heterogeneous and lie in different distributions.
no code implementations • 15 Nov 2022 • Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, Bing Yin
We annotate a large amount of assertions for both plausibility and typicality of an intention that can explain a purchasing or co-purchasing behavior, where the intention can be an open reason or a predicate falling into one of 18 categories aligning with ConceptNet, e. g., IsA, MadeOf, UsedFor, etc.
no code implementations • 20 Oct 2022 • Zheng Li, Caili Guo, Zerun Feng, Jenq-Neng Hwang, Ying Jin, Yufeng Zhang
Such a binary indicator covers only a limited subset of image-text semantic relations, which is insufficient to represent relevance degrees between images and texts described by continuous labels such as image captions.
no code implementations • 16 Oct 2022 • Ruijie Wang, Zheng Li, Dachun Sun, Shengzhong Liu, Jinning Li, Bing Yin, Tarek Abdelzaher
Second, the potentially dynamic distributions from the initially observable facts to the future facts ask for explicitly modeling the evolving characteristics of new entities.
no code implementations • 13 Oct 2022 • Zeyang Sha, Zheng Li, Ning Yu, Yang Zhang
To tackle this problem, we pioneer a systematic study on the detection and attribution of fake images generated by text-to-image generation models.
no code implementations • 8 Oct 2022 • Haoming Jiang, Tianyu Cao, Zheng Li, Chen Luo, Xianfeng Tang, Qingyu Yin, Danqing Zhang, Rahul Goutam, Bing Yin
When applying masking to short search queries, most contextual information is lost and the intent of the search queries may be changed.
no code implementations • 4 Oct 2022 • Xinyue Shen, Xinlei He, Zheng Li, Yun Shen, Michael Backes, Yang Zhang
Different from previous work, we are the first to systematically threat modeling on SSL in every phase of the model supply chain, i. e., pre-training, release, and downstream phases.
no code implementations • 3 Oct 2022 • Yixin Wu, Ning Yu, Zheng Li, Michael Backes, Yang Zhang
The empirical results show that all of the proposed attacks can achieve significant performance, in some cases even close to an accuracy of 1, and thus the corresponding risk is much more severe than that shown by existing membership inference attacks.
no code implementations • 3 Oct 2022 • Zheng Li, Ning Yu, Ahmed Salem, Michael Backes, Mario Fritz, Yang Zhang
Extensive experiments on four popular GAN models trained on two benchmark face datasets show that UnGANable achieves remarkable effectiveness and utility performance, and outperforms multiple baseline methods.
no code implementations • 30 Sep 2022 • Ziqing Yang, Xinlei He, Zheng Li, Michael Backes, Mathias Humbert, Pascal Berrang, Yang Zhang
It is a promising way to solve the above problems as it can use easy-to-collect image-text pairs to construct the training dataset and the raw texts contain almost unlimited categories according to their semantics.
no code implementations • 28 Sep 2022 • Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Jenq-Neng Hwang, Zhongtian Du
More specifically, Triplet loss with Hard Negative mining (Triplet-HN), which is widely used in existing retrieval models to improve the discriminative ability, is easy to fall into local minima in training.
no code implementations • 1 Sep 2022 • Amir Yazdanbakhsh, Ashkan Moradifirouzabadi, Zheng Li, Mingu Kang
The combined in-memory pruning and on-chip recompute of the relevant attention scores enables SPRINT to transform quadratic complexity to a merely linear one.
1 code implementation • 30 Aug 2022 • Prince Grover, Zheng Li, Jianbo Liu, Jakub Zablocki, Hao Zhou, Julia Xu, Anqi Cheng
We hope that FDB helps in the development of customized fraud detection techniques catered to different fraud modus operandi (MOs) as well as in the improvement of AutoML systems that can work well for all datasets in the benchmark.
no code implementations • 23 Aug 2022 • Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, Yang Zhang
Furthermore, we propose a hybrid attack that exploits the exit information to improve the performance of existing attacks.
no code implementations • 22 Aug 2022 • Xinlei He, Zheng Li, Weilin Xu, Cory Cornelius, Yang Zhang
Finally, we find that data augmentation degrades the performance of existing attacks to a larger extent, and we propose an adaptive attack using augmentation to train shadow and attack models that improve attack performance.
3 code implementations • 15 Jun 2022 • Wei Jin, Xianfeng Tang, Haoming Jiang, Zheng Li, Danqing Zhang, Jiliang Tang, Bing Yin
However, existing approaches have their inherent limitations: (1) they are not directly applicable to graphs where the data is discrete; and (2) the condensation process is computationally expensive due to the involved nested optimization.
no code implementations • 31 May 2022 • Jun Shi, Yuanming Zhang, Zheng Li, Xiangmin Han, Saisai Ding, Jun Wang, Shihui Ying
In this work, we propose a pseudo-data based self-supervised federated learning (FL) framework, named SSL-FT-BT, to improve both the diagnostic accuracy and generalization of CAD models.
1 code implementation • Findings (NAACL) 2022 • Yifan Gao, Qingyu Yin, Zheng Li, Rui Meng, Tong Zhao, Bing Yin, Irwin King, Michael R. Lyu
Keyphrase generation is the task of automatically predicting keyphrases given a piece of long text.
no code implementations • 7 Apr 2022 • Zheng Li, Soroush Ghodrati, Amir Yazdanbakhsh, Hadi Esmaeilzadeh, Mingu Kang
To best utilize this mathematical innovation, we devise a bit-serial architecture, dubbed LeOPArd, for transformer language models with bit-level early termination microarchitectural mechanism.
1 code implementation • ACL 2022 • Zijie Huang, Zheng Li, Haoming Jiang, Tianyu Cao, Hanqing Lu, Bing Yin, Karthik Subbian, Yizhou Sun, Wei Wang
In this paper, we explore multilingual KG completion, which leverages limited seed alignment as a bridge, to embrace the collective knowledge from multiple languages.
Ranked #3 on Knowledge Graph Completion on DPB-5L (French)
2 code implementations • ACL 2022 • Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth
Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.
no code implementations • 18 Mar 2022 • Shachi Deshpande, Kaiwen Wang, Dhruv Sreenivas, Zheng Li, Volodymyr Kuleshov
Oftentimes, the confounders are unobserved, but we have access to large amounts of additional unstructured data (images, text) that contain valuable proxy signal about the missing confounders.
no code implementations • 12 Feb 2022 • Ruijie Wang, Zheng Li, Danqing Zhang, Qingyu Yin, Tong Zhao, Bing Yin, Tarek Abdelzaher
And meanwhile, RETE autoregressively accumulates retrieval-enhanced user representations from each time step, to capture evolutionary patterns for joint query and product prediction.
no code implementations • 10 Jan 2022 • Hua Zheng, Wei Xie, Keqi Wang, Zheng Li
Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently guide process control.
1 code implementation • 2 Jan 2022 • Zheng Li, Yue Zhao, Xiyang Hu, Nicola Botta, Cezar Ionescu, George H. Chen
To address these issues, we present a simple yet effective algorithm called ECOD (Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the "rare events" that appear in the tails of a distribution.
1 code implementation • 27 Nov 2021 • Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, Shuchang Zhou
Network quantization significantly reduces model inference complexity and has been widely used in real-world deployments.
Ranked #1 on Quantization on ImageNet
no code implementations • 10 Oct 2021 • Hao Peng, Guofeng Tong, Zheng Li, Yaqi Wang, Yuyuan Shao
The SGNet proposed in this paper has achieved state-of-the-art results for 3D object detection in the KITTI dataset, especially in the detection of small-size objects such as cyclists.
no code implementations • 1 Oct 2021 • Zheng Li, Xiang Li, Lingfeng Yang, Jian Yang, Zhigeng Pan
Knowledge distillation usually transfers the knowledge from a pre-trained cumbersome teacher network to a compact student network, which follows the classical teacher-teaching-student paradigm.
no code implementations • 19 Aug 2021 • Danqing Zhang, Zheng Li, Tianyu Cao, Chen Luo, Tony Wu, Hanqing Lu, Yiwei Song, Bing Yin, Tuo Zhao, Qiang Yang
We study the problem of query attribute value extraction, which aims to identify named entities from user queries as diverse surface form attribute values and afterward transform them into formally canonical forms.
1 code implementation • ICCV 2021 • Zheng Li, Jingwen Ye, Mingli Song, Ying Huang, Zhigeng Pan
However, existing pose distillation works rely on a heavy pre-trained estimator to perform knowledge transfer and require a complex two-stage learning procedure.
no code implementations • 23 Jul 2021 • Binling Wang, Wenxuan Hu, Jing Li, Yiming Zhi, Zheng Li, Qingyang Hong, Lin Li, Dong Wang, Liming Song, Cheng Yang
In addition to the Language Identification (LID) tasks, multilingual Automatic Speech Recognition (ASR) tasks are introduced to OLR 2021 Challenge for the first time.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 5 Jul 2021 • Jing Li, Binling Wang, Yiming Zhi, Zheng Li, Lin Li, Qingyang Hong, Dong Wang
The fifth Oriental Language Recognition (OLR) Challenge focuses on language recognition in a variety of complex environments to promote its development.
no code implementations • 29 Jun 2021 • Zhiyang Lu, Zheng Li, Jun Wang, Jun Shi, Dinggang Shen
To this end, we propose a novel Two-stage Self-supervised Cycle-consistency Network (TSCNet) for MR slice interpolation, in which a two-stage self-supervised learning (SSL) strategy is developed for unsupervised DL network training.
no code implementations • 25 Jun 2021 • Yan Liu, Zheng Li, Lin Li, Qingyang Hong
This paper proposes a multi-task learning network with phoneme-aware and channel-wise attentive learning strategies for text-dependent Speaker Verification (SV).
1 code implementation • Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation 2021 • Jie Zhao, Bojie Li, Wang Nie, Zhen Geng, Renwei Zhang, Xiong Gao, Bin Cheng, Chen Wu, Yun Cheng, Zheng Li, Peng Di, Kun Zhang, Xuefeng Jin
Existing tensor compilers have proven their effectiveness in deploying deep neural networks on general-purpose hardware like CPU and GPU, but optimizing for neural processing units (NPUs) is still challenging due to the heterogeneous compute units and complicated memory hierarchy.
no code implementations • 11 Feb 2021 • Zheng Li, Xiaoyu Nie, Fan Yang, Xiangpei Liu, Dongyu Liu, Xiaolong Dong, Xingchen Zhao, Tao Peng, M. Suhail Zubairy, Marlan O. Scully
We present a novel method to synthesize non-trivial speckles that can enable superresolving second-order correlation imaging.
Optics Image and Video Processing
1 code implementation • 7 Jan 2021 • Xiangyang Li, Yu Xia, Xiang Long, Zheng Li, Sujian Li
In this paper, we describe our system for the AAAI 2021 shared task of COVID-19 Fake News Detection in English, where we achieved the 3rd position with the weighted F1 score of 0. 9859 on the test set.
Ranked #1 on Fake News Detection on Grover-Mega
no code implementations • 22 Dec 2020 • Ming Zhang, Shuqiao Zhang, Haitan Xu, Hankai Zhang, Xiangxu Mu, R. J. Dwayne Miller, Anatoly Ischenko, Oriol Vendrell, Zheng Li
With the ability to directly obtain the Wigner function and density matrix of photon states, quantum tomography (QT) has had a significant impact on quantum optics, quantum computing and quantum information.
no code implementations • 26 Oct 2020 • Shi Pu, Yijiang He, Zheng Li, Mao Zheng
Existing video recommendation systems directly exploit features from different modalities (e. g., user personal data, user behavior data, video titles, video tags, and visual contents) to input deep neural networks, while expecting the networks to online mine user-preferred topics implicitly from these features.
no code implementations • 23 Oct 2020 • Xin Li, Lidong Bing, Wenxuan Zhang, Zheng Li, Wai Lam
Cross-lingual adaptation with multilingual pre-trained language models (mPTLMs) mainly consists of two lines of works: zero-shot approach and translation-based approach, which have been studied extensively on the sequence-level tasks.
no code implementations • 12 Oct 2020 • Zhiyang Lu, Jun Li, Zheng Li, Hongjian He, Jun Shi
In this work, we propose to explore a new value of the high-pass filtered phase data generated in susceptibility weighted imaging (SWI), and develop an end-to-end Cross-connected $\Psi$-Net (C$\Psi$-Net) to reconstruct QSM directly from these phase data in SWI without additional pre-processing.
no code implementations • 2 Oct 2020 • Zheng Li, Ying Huang, Defang Chen, Tianren Luo, Ning Cai, Zhigeng Pan
Extensive experiments proved that our method significantly enhances the diversity among student models and brings better distillation performance.
1 code implementation • 20 Sep 2020 • Zheng Li, Yue Zhao, Jialin Fu
A synthetic dataset is a data object that is generated programmatically, and it may be valuable to creating a single dataset from multiple sources when direct collection is difficult or costly.
2 code implementations • 20 Sep 2020 • Zheng Li, Yue Zhao, Nicola Botta, Cezar Ionescu, Xiyang Hu
In this work, we make three key contributions, 1) propose a novel, parameter-free outlier detection algorithm with both great performance and interpretability, 2) perform extensive experiments on 30 benchmark datasets to show that COPOD outperforms in most cases and is also one of the fastest algorithms, and 3) release an easy-to-use Python implementation for reproducibility.
1 code implementation • 30 Jul 2020 • Zheng Li, Yang Zhang
However, recent research has shown that ML models are vulnerable to attacks against their training data.
no code implementations • 8 Jul 2020 • Zheng Li, Gang Tu, Guang Liu, Zhi-Qiang Zhan, Yi-Jian Liu
The algorithm can not only introduce background knowledge, recognize all kinds of nested phrases in sentences, but also recognize the dependency between phrases.
no code implementations • 7 Jul 2020 • Guang Liu, Gang Tu, Zheng Li, Yi-Jian Liu
At present, most Natural Language Processing technology is based on the results of Word Segmentation for Dependency Parsing, which mainly uses an end-to-end method based on supervised learning.
no code implementations • 16 Jun 2020 • Zerun Feng, Zhimin Zeng, Caili Guo, Zheng Li
Finally, the region features are aggregated to form frame-level features for further encoding to measure video-text similarity.
no code implementations • 4 Jun 2020 • Zheng Li, Miao Zhao, Qingyang Hong, Lin Li, Zhiyuan Tang, Dong Wang, Li-Ming Song, Cheng Yang
Based on Kaldi and Pytorch, recipes for i-vector and x-vector systems are also conducted as baselines for the three tasks.
1 code implementation • 11 Mar 2020 • Yue Zhao, Xiyang Hu, Cheng Cheng, Cong Wang, Changlin Wan, Wen Wang, Jianing Yang, Haoping Bai, Zheng Li, Cao Xiao, Yunlong Wang, Zhi Qiao, Jimeng Sun, Leman Akoglu
Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples with numerous high-stake applications including fraud detection and intrusion detection.
2 code implementations • 4 Nov 2019 • Kai Zhang, Shuhang Gu, Radu Timofte, Zheng Hui, Xiumei Wang, Xinbo Gao, Dongliang Xiong, Shuai Liu, Ruipeng Gang, Nan Nan, Chenghua Li, Xueyi Zou, Ning Kang, Zhan Wang, Hang Xu, Chaofeng Wang, Zheng Li, Lin-Lin Wang, Jun Shi, Wenyu Sun, Zhiqiang Lang, Jiangtao Nie, Wei Wei, Lei Zhang, Yazhe Niu, Peijin Zhuo, Xiangzhen Kong, Long Sun, Wenhao Wang
The challenge had 3 tracks.
no code implementations • WS 2019 • Xiepeng Li, Zhexi Zhang, Wei Zhu, Zheng Li, Yuan Ni, Peng Gao, Junchi Yan, Guotong Xie
We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.
1 code implementation • IJCNLP 2019 • Zheng Li, Xin Li, Ying WEI, Lidong Bing, Yu Zhang, Qiang Yang
Joint extraction of aspects and sentiments can be effectively formulated as a sequence labeling problem.
Aspect-Based Sentiment Analysis (ABSA) Unsupervised Domain Adaptation
1 code implementation • 9 Jul 2019 • Zheng Li, Shi Shu
Inspired by dynamic programming, we propose Stochastic Virtual Gradient Descent (SVGD) algorithm where the Virtual Gradient is defined by computational graph and automatic differentiation.
no code implementations • 28 May 2019 • Xiaocong Du, Gokul Krishnan, Abinash Mohanty, Zheng Li, Gouranga Charan, Yu Cao
Machine learning algorithms have made significant advances in many applications.
no code implementations • 27 May 2019 • Xiaocong Du, Zheng Li, Yu Cao
Today a canonical approach to reduce the computation cost of Deep Neural Networks (DNNs) is to pre-define an over-parameterized model before training to guarantee the learning capacity, and then prune unimportant learning units (filters and neurons) during training to improve model compactness.
no code implementations • 27 May 2019 • Xiaocong Du, Zheng Li, Yufei Ma, Yu Cao
A typical training pipeline to mitigate over-parameterization is to pre-define a DNN structure first with redundant learning units (filters and neurons) under the goal of high accuracy, then to prune redundant learning units after training with the purpose of efficient inference.
no code implementations • ICLR 2019 • Zheng Li, Christopher De Sa
Low-precision training is a promising way of decreasing the time and energy cost of training machine learning models.
2 code implementations • 16 Apr 2019 • Colin Wan, Zheng Li, Alicia Guo, Yue Zhao
Synthetic population generation is the process of combining multiple socioeconomic and demographic datasets from different sources and/or granularity levels, and downscaling them to an individual level.
1 code implementation • 4 Apr 2019 • Chaofeng Wang, Zheng Li, Jun Shi
PyTorch code for our paper "Lightweight Image Super-Resolution with Adaptive Weighted Learning Network"
no code implementations • 13 Mar 2019 • Zheng Li, Ge Han, Yunqing Wei, Shanqing Guo
Steganography refers to the art of concealing secret messages within multiple media carriers so that an eavesdropper is unable to detect the presence and content of the hidden messages.
1 code implementation • 5 Mar 2019 • Zheng Li, Chengyu Hu, Yang Zhang, Shanqing Guo
To fill these gaps, in this paper, we propose a novel intellectual property protection (IPP) framework based on blind-watermark for watermarking deep neural networks that meet the requirements of security and feasibility.
4 code implementations • 6 Jan 2019 • Yue Zhao, Zain Nasrullah, Zheng Li
PyOD is an open-source Python toolbox for performing scalable outlier detection on multivariate data.
1 code implementation • 4 Dec 2018 • Yue Zhao, Zain Nasrullah, Maciej K. Hryniewicki, Zheng Li
The top-performing base detectors in this local region are selected and combined as the model's final output.
1 code implementation • AAAI 2019 2018 • Zheng Li, Ying WEI, Yu Zhang, Xiang Zhang, Xin Li, Qiang Yang
Aspect-level sentiment classification (ASC) aims at identifying sentiment polarities towards aspects in a sentence, where the aspect can behave as a general Aspect Category (AC) or a specific Aspect Term (AT).
1 code implementation • Thirty-Second AAAI Conference on Artificial Intelligence 2018 • Zheng Li, Ying WEI, Yu Zhang, Qiang Yang
Existing cross-domain sentiment classification meth- ods cannot automatically capture non-pivots, i. e., the domain- specific sentiment words, and pivots, i. e., the domain-shared sentiment words, simultaneously.