Search Results for author: Zheng Li

Found 76 papers, 28 papers with code

Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages

no code implementations EMNLP 2020 Zheng Li, Mukul Kumar, William Headden, Bing Yin, Ying WEI, Yu Zhang, Qiang Yang

Recent emergence of multilingual pre-training language model (mPLM) has enabled breakthroughs on various downstream cross-lingual transfer (CLT) tasks.

Cross-Lingual Transfer Graph Learning +1

MetaTS: Meta Teacher-Student Network for Multilingual Sequence Labeling with Minimal Supervision

no code implementations EMNLP 2021 Zheng Li, Danqing Zhang, Tianyu Cao, Ying WEI, Yiwei Song, Bing Yin

In this work, we explore multilingual sequence labeling with minimal supervision using a single unified model for multiple languages.

Meta-Learning

Mutually-paced Knowledge Distillation for Cross-lingual Temporal Knowledge Graph Reasoning

no code implementations27 Mar 2023 Ruijie Wang, Zheng Li, Jingfeng Yang, Tianyu Cao, Chao Zhang, Bing Yin, Tarek Abdelzaher

This paper investigates cross-lingual temporal knowledge graph reasoning problem, which aims to facilitate reasoning on Temporal Knowledge Graphs (TKGs) in low-resource languages by transfering knowledge from TKGs in high-resource ones.

Knowledge Distillation Knowledge Graphs +1

Rotation Invariant Quantization for Model Compression

1 code implementation3 Mar 2023 Joseph Kampeas, Yury Nahshan, Hanoch Kremer, Gil Lederman, Shira Zaloshinski, Zheng Li, Emir Haleva

Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources.

Model Compression Quantization

Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching

no code implementations1 Mar 2023 Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Zhongtian Du

To alleviate the gradient vanishing problem, we propose a Selectively Hard Negative Mining (SelHN) strategy, which chooses whether to mine hard negative samples according to the gradient vanishing condition.

Text Matching

HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers

no code implementations19 Feb 2023 Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bin Yin, Tuo Zhao

Since the teacher model has a significantly larger capacity and stronger representation power than the student model, it is very difficult for the student to produce predictions that match the teacher's over a massive amount of open-domain training data.

Knowledge Distillation Model Compression +1

Backdoor Attacks Against Dataset Distillation

2 code implementations3 Jan 2023 Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, Yang Zhang

A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset.

Backdoor Attack

ReCode: Robustness Evaluation of Code Generation Models

1 code implementation20 Dec 2022 Shiqi Wang, Zheng Li, Haifeng Qian, Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, Bing Xiang

Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.

Code Generation

Curriculum Temperature for Knowledge Distillation

1 code implementation29 Nov 2022 Zheng Li, Xiang Li, Lingfeng Yang, Borui Zhao, RenJie Song, Lei Luo, Jun Li, Jian Yang

In this paper, we propose a simple curriculum-based technique, termed Curriculum Temperature for Knowledge Distillation (CTKD), which controls the task difficulty level during the student's learning career through a dynamic and learnable temperature.

Image Classification Knowledge Distillation

Testing for context-dependent changes in neural encoding in naturalistic experiments

no code implementations17 Nov 2022 Yenho Chen, Carl W. Harris, Xiaoyu Ma, Zheng Li, Francisco Pereira, Charles Y. Zheng

We propose a decoding-based approach to detect context effects on neural codes in longitudinal neural recording data.

Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering

no code implementations16 Nov 2022 Juan Zha, Zheng Li, Ying WEI, Yu Zhang

However, most prior works assume that all the tasks are sampled from a single data source, which cannot adapt to real-world scenarios where tasks are heterogeneous and lie in different distributions.

Few-Shot Text Classification text-classification +1

FolkScope: Intention Knowledge Graph Construction for Discovering E-commerce Commonsense

no code implementations15 Nov 2022 Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, Bing Yin

We annotate a large amount of assertions for both plausibility and typicality of an intention that can explain a purchasing or co-purchasing behavior, where the intention can be an open reason or a predicate falling into one of 18 categories aligning with ConceptNet, e. g., IsA, MadeOf, UsedFor, etc.

graph construction

Image-Text Retrieval with Binary and Continuous Label Supervision

no code implementations20 Oct 2022 Zheng Li, Caili Guo, Zerun Feng, Jenq-Neng Hwang, Ying Jin, Yufeng Zhang

Such a binary indicator covers only a limited subset of image-text semantic relations, which is insufficient to represent relevance degrees between images and texts described by continuous labels such as image captions.

Image Captioning Retrieval +2

Learning to Sample and Aggregate: Few-shot Reasoning over Temporal Knowledge Graphs

no code implementations16 Oct 2022 Ruijie Wang, Zheng Li, Dachun Sun, Shengzhong Liu, Jinning Li, Bing Yin, Tarek Abdelzaher

Second, the potentially dynamic distributions from the initially observable facts to the future facts ask for explicitly modeling the evolving characteristics of new entities.

Knowledge Graphs Meta-Learning

DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models

no code implementations13 Oct 2022 Zeyang Sha, Zheng Li, Ning Yu, Yang Zhang

To tackle this problem, we pioneer a systematic study on the detection and attribution of fake images generated by text-to-image generation models.

Fake Image Detection Text-to-Image Generation

Backdoor Attacks in the Supply Chain of Masked Image Modeling

no code implementations4 Oct 2022 Xinyue Shen, Xinlei He, Zheng Li, Yun Shen, Michael Backes, Yang Zhang

Different from previous work, we are the first to systematically threat modeling on SSL in every phase of the model supply chain, i. e., pre-training, release, and downstream phases.

Contrastive Learning Self-Supervised Learning

Membership Inference Attacks Against Text-to-image Generation Models

no code implementations3 Oct 2022 Yixin Wu, Ning Yu, Zheng Li, Michael Backes, Yang Zhang

The empirical results show that all of the proposed attacks can achieve significant performance, in some cases even close to an accuracy of 1, and thus the corresponding risk is much more severe than that shown by existing membership inference attacks.

Image Classification Text-to-Image Generation

UnGANable: Defending Against GAN-based Face Manipulation

no code implementations3 Oct 2022 Zheng Li, Ning Yu, Ahmed Salem, Michael Backes, Mario Fritz, Yang Zhang

Extensive experiments on four popular GAN models trained on two benchmark face datasets show that UnGANable achieves remarkable effectiveness and utility performance, and outperforms multiple baseline methods.

Face Swapping Misinformation

Data Poisoning Attacks Against Multimodal Encoders

no code implementations30 Sep 2022 Ziqing Yang, Xinlei He, Zheng Li, Michael Backes, Mathias Humbert, Pascal Berrang, Yang Zhang

It is a promising way to solve the above problems as it can use easy-to-collect image-text pairs to construct the training dataset and the raw texts contain almost unlimited categories according to their semantics.

Contrastive Learning Data Poisoning

Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval

no code implementations28 Sep 2022 Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Jenq-Neng Hwang, Zhongtian Du

More specifically, Triplet loss with Hard Negative mining (Triplet-HN), which is widely used in existing retrieval models to improve the discriminative ability, is easy to fall into local minima in training.

Contrastive Learning Retrieval +2

Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation

no code implementations1 Sep 2022 Amir Yazdanbakhsh, Ashkan Moradifirouzabadi, Zheng Li, Mingu Kang

The combined in-memory pruning and on-chip recompute of the relevant attention scores enables SPRINT to transform quadratic complexity to a merely linear one.

FDB: Fraud Dataset Benchmark

1 code implementation30 Aug 2022 Prince Grover, Zheng Li, Jianbo Liu, Jakub Zablocki, Hao Zhou, Julia Xu, Anqi Cheng

We hope that FDB helps in the development of customized fraud detection techniques catered to different fraud modus operandi (MOs) as well as in the improvement of AutoML systems that can work well for all datasets in the benchmark.

AutoML Fraud Detection

Auditing Membership Leakages of Multi-Exit Networks

no code implementations23 Aug 2022 Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, Yang Zhang

Furthermore, we propose a hybrid attack that exploits the exit information to improve the performance of existing attacks.

Membership-Doctor: Comprehensive Assessment of Membership Inference Against Machine Learning Models

no code implementations22 Aug 2022 Xinlei He, Zheng Li, Weilin Xu, Cory Cornelius, Yang Zhang

Finally, we find that data augmentation degrades the performance of existing attacks to a larger extent, and we propose an adaptive attack using augmentation to train shadow and attack models that improve attack performance.

Data Augmentation

Condensing Graphs via One-Step Gradient Matching

3 code implementations15 Jun 2022 Wei Jin, Xianfeng Tang, Haoming Jiang, Zheng Li, Danqing Zhang, Jiliang Tang, Bing Yin

However, existing approaches have their inherent limitations: (1) they are not directly applicable to graphs where the data is discrete; and (2) the condensation process is computationally expensive due to the involved nested optimization.

Dataset Condensation

Pseudo-Data based Self-Supervised Federated Learning for Classification of Histopathological Images

no code implementations31 May 2022 Jun Shi, Yuanming Zhang, Zheng Li, Xiangmin Han, Saisai Ding, Jun Wang, Shihui Ying

In this work, we propose a pseudo-data based self-supervised federated learning (FL) framework, named SSL-FT-BT, to improve both the diagnostic accuracy and generalization of CAD models.

Contrastive Learning Federated Learning +1

Accelerating Attention through Gradient-Based Learned Runtime Pruning

no code implementations7 Apr 2022 Zheng Li, Soroush Ghodrati, Amir Yazdanbakhsh, Hadi Esmaeilzadeh, Mingu Kang

To best utilize this mathematical innovation, we devise a bit-serial architecture, dubbed LeOPArd, for transformer language models with bit-level early termination microarchitectural mechanism.

Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment

1 code implementation ACL 2022 Zijie Huang, Zheng Li, Haoming Jiang, Tianyu Cao, Hanqing Lu, Bing Yin, Karthik Subbian, Yizhou Sun, Wei Wang

In this paper, we explore multilingual KG completion, which leverages limited seed alignment as a bridge, to embrace the collective knowledge from multiple languages.

Knowledge Graph Completion

DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization

2 code implementations ACL 2022 Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth

Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.

Knowledge Distillation Model Compression +2

Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxies

no code implementations18 Mar 2022 Shachi Deshpande, Kaiwen Wang, Dhruv Sreenivas, Zheng Li, Volodymyr Kuleshov

Oftentimes, the confounders are unobserved, but we have access to large amounts of additional unstructured data (images, text) that contain valuable proxy signal about the missing confounders.

Causal Inference Time Series Analysis

RETE: Retrieval-Enhanced Temporal Event Forecasting on Unified Query Product Evolutionary Graph

no code implementations12 Feb 2022 Ruijie Wang, Zheng Li, Danqing Zhang, Qingyu Yin, Tong Zhao, Bing Yin, Tarek Abdelzaher

And meanwhile, RETE autoregressively accumulates retrieval-enhanced user representations from each time step, to capture evolutionary patterns for joint query and product prediction.

Product Recommendation Retrieval

Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control

no code implementations10 Jan 2022 Hua Zheng, Wei Xie, Keqi Wang, Zheng Li

Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently guide process control.

Decision Making Model-based Reinforcement Learning +2

ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions

1 code implementation2 Jan 2022 Zheng Li, Yue Zhao, Xiyang Hu, Nicola Botta, Cezar Ionescu, George H. Chen

To address these issues, we present a simple yet effective algorithm called ECOD (Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the "rare events" that appear in the tails of a distribution.

Anomaly Detection Outlier Detection

FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer

1 code implementation27 Nov 2021 Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, Shuchang Zhou

Network quantization significantly reduces model inference complexity and has been widely used in real-world deployments.

Quantization

3D Object Detection Combining Semantic and Geometric Features from Point Clouds

no code implementations10 Oct 2021 Hao Peng, Guofeng Tong, Zheng Li, Yaqi Wang, Yuyuan Shao

The SGNet proposed in this paper has achieved state-of-the-art results for 3D object detection in the KITTI dataset, especially in the detection of small-size objects such as cyclists.

3D Object Detection object-detection

Student Helping Teacher: Teacher Evolution via Self-Knowledge Distillation

no code implementations1 Oct 2021 Zheng Li, Xiang Li, Lingfeng Yang, Jian Yang, Zhigeng Pan

Knowledge distillation usually transfers the knowledge from a pre-trained cumbersome teacher network to a compact student network, which follows the classical teacher-teaching-student paradigm.

Self-Knowledge Distillation

QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction

no code implementations19 Aug 2021 Danqing Zhang, Zheng Li, Tianyu Cao, Chen Luo, Tony Wu, Hanqing Lu, Yiwei Song, Bing Yin, Tuo Zhao, Qiang Yang

We study the problem of query attribute value extraction, which aims to identify named entities from user queries as diverse surface form attribute values and afterward transform them into formally canonical forms.

Attribute Value Extraction named-entity-recognition +2

Online Knowledge Distillation for Efficient Pose Estimation

1 code implementation ICCV 2021 Zheng Li, Jingwen Ye, Mingli Song, Ying Huang, Zhigeng Pan

However, existing pose distillation works rely on a heavy pre-trained estimator to perform knowledge transfer and require a complex two-stage learning procedure.

Knowledge Distillation Pose Estimation

OLR 2021 Challenge: Datasets, Rules and Baselines

no code implementations23 Jul 2021 Binling Wang, Wenxuan Hu, Jing Li, Yiming Zhi, Zheng Li, Qingyang Hong, Lin Li, Dong Wang, Liming Song, Cheng Yang

In addition to the Language Identification (LID) tasks, multilingual Automatic Speech Recognition (ASR) tasks are introduced to OLR 2021 Challenge for the first time.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Oriental Language Recognition (OLR) 2020: Summary and Analysis

no code implementations5 Jul 2021 Jing Li, Binling Wang, Yiming Zhi, Zheng Li, Lin Li, Qingyang Hong, Dong Wang

The fifth Oriental Language Recognition (OLR) Challenge focuses on language recognition in a variety of complex environments to promote its development.

Dialect Identification

Two-Stage Self-Supervised Cycle-Consistency Network for Reconstruction of Thin-Slice MR Images

no code implementations29 Jun 2021 Zhiyang Lu, Zheng Li, Jun Wang, Jun Shi, Dinggang Shen

To this end, we propose a novel Two-stage Self-supervised Cycle-consistency Network (TSCNet) for MR slice interpolation, in which a two-stage self-supervised learning (SSL) strategy is developed for unsupervised DL network training.

Self-Supervised Learning

Phoneme-aware and Channel-wise Attentive Learning for Text DependentSpeaker Verification

no code implementations25 Jun 2021 Yan Liu, Zheng Li, Lin Li, Qingyang Hong

This paper proposes a multi-task learning network with phoneme-aware and channel-wise attentive learning strategies for text-dependent Speaker Verification (SV).

Multi-Task Learning Text-Dependent Speaker Verification

AKG: Automatic Kernel Generation for Neural Processing Units using Polyhedral Transformations

1 code implementation Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation 2021 Jie Zhao, Bojie Li, Wang Nie, Zhen Geng, Renwei Zhang, Xiong Gao, Bin Cheng, Chen Wu, Yun Cheng, Zheng Li, Peng Di, Kun Zhang, Xuefeng Jin

Existing tensor compilers have proven their effectiveness in deploying deep neural networks on general-purpose hardware like CPU and GPU, but optimizing for neural processing units (NPUs) is still challenging due to the heterogeneous compute units and complicated memory hierarchy.

Code Generation Management +1

Superresolving second-order correlation imaging using synthesized colored noise speckles

no code implementations11 Feb 2021 Zheng Li, Xiaoyu Nie, Fan Yang, Xiangpei Liu, Dongyu Liu, Xiaolong Dong, Xingchen Zhao, Tao Peng, M. Suhail Zubairy, Marlan O. Scully

We present a novel method to synthesize non-trivial speckles that can enable superresolving second-order correlation imaging.

Optics Image and Video Processing

Exploring Text-transformers in AAAI 2021 Shared Task: COVID-19 Fake News Detection in English

1 code implementation7 Jan 2021 Xiangyang Li, Yu Xia, Xiang Long, Zheng Li, Sujian Li

In this paper, we describe our system for the AAAI 2021 shared task of COVID-19 Fake News Detection in English, where we achieved the 3rd position with the weighted F1 score of 0. 9859 on the test set.

Fake News Detection

Tomographic imaging of complete quantum state of matter by ultrafast diffraction

no code implementations22 Dec 2020 Ming Zhang, Shuqiao Zhang, Haitan Xu, Hankai Zhang, Xiangxu Mu, R. J. Dwayne Miller, Anatoly Ischenko, Oriol Vendrell, Zheng Li

With the ability to directly obtain the Wigner function and density matrix of photon states, quantum tomography (QT) has had a significant impact on quantum optics, quantum computing and quantum information.

Quantum Physics

Multimodal Topic Learning for Video Recommendation

no code implementations26 Oct 2020 Shi Pu, Yijiang He, Zheng Li, Mao Zheng

Existing video recommendation systems directly exploit features from different modalities (e. g., user personal data, user behavior data, video titles, video tags, and visual contents) to input deep neural networks, while expecting the networks to online mine user-preferred topics implicitly from these features.

Recommendation Systems

Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond

no code implementations23 Oct 2020 Xin Li, Lidong Bing, Wenxuan Zhang, Zheng Li, Wai Lam

Cross-lingual adaptation with multilingual pre-trained language models (mPTLMs) mainly consists of two lines of works: zero-shot approach and translation-based approach, which have been studied extensively on the sequence-level tasks.

Cross-Lingual Transfer Translation

Reconstruction of Quantitative Susceptibility Maps from Phase of Susceptibility Weighted Imaging with Cross-Connected $Ψ$-Net

no code implementations12 Oct 2020 Zhiyang Lu, Jun Li, Zheng Li, Hongjian He, Jun Shi

In this work, we propose to explore a new value of the high-pass filtered phase data generated in susceptibility weighted imaging (SWI), and develop an end-to-end Cross-connected $\Psi$-Net (C$\Psi$-Net) to reconstruct QSM directly from these phase data in SWI without additional pre-processing.

Online Knowledge Distillation via Multi-branch Diversity Enhancement

no code implementations2 Oct 2020 Zheng Li, Ying Huang, Defang Chen, Tianren Luo, Ning Cai, Zhigeng Pan

Extensive experiments proved that our method significantly enhances the diversity among student models and brings better distillation performance.

Image Classification Knowledge Distillation

SYNC: A Copula based Framework for Generating Synthetic Data from Aggregated Sources

1 code implementation20 Sep 2020 Zheng Li, Yue Zhao, Jialin Fu

A synthetic dataset is a data object that is generated programmatically, and it may be valuable to creating a single dataset from multiple sources when direct collection is difficult or costly.

Feature Engineering Synthetic Data Generation

COPOD: Copula-Based Outlier Detection

2 code implementations20 Sep 2020 Zheng Li, Yue Zhao, Nicola Botta, Cezar Ionescu, Xiyang Hu

In this work, we make three key contributions, 1) propose a novel, parameter-free outlier detection algorithm with both great performance and interpretability, 2) perform extensive experiments on 30 benchmark datasets to show that COPOD outperforms in most cases and is also one of the fastest algorithms, and 3) release an easy-to-use Python implementation for reproducibility.

Outlier Detection

Membership Leakage in Label-Only Exposures

1 code implementation30 Jul 2020 Zheng Li, Yang Zhang

However, recent research has shown that ML models are vulnerable to attacks against their training data.

Face Recognition Inference Attack

Research on multi-dimensional end-to-end phrase recognition algorithm based on background knowledge

no code implementations8 Jul 2020 Zheng Li, Gang Tu, Guang Liu, Zhi-Qiang Zhan, Yi-Jian Liu

The algorithm can not only introduce background knowledge, recognize all kinds of nested phrases in sentences, but also recognize the dependency between phrases.

Research on Annotation Rules and Recognition Algorithm Based on Phrase Window

no code implementations7 Jul 2020 Guang Liu, Gang Tu, Zheng Li, Yi-Jian Liu

At present, most Natural Language Processing technology is based on the results of Word Segmentation for Dependency Parsing, which mainly uses an end-to-end method based on supervised learning.

Dependency Parsing Sentiment Analysis

Exploiting Visual Semantic Reasoning for Video-Text Retrieval

no code implementations16 Jun 2020 Zerun Feng, Zhimin Zeng, Caili Guo, Zheng Li

Finally, the region features are aggregated to form frame-level features for further encoding to measure video-text similarity.

Retrieval Text Retrieval +2

AP20-OLR Challenge: Three Tasks and Their Baselines

no code implementations4 Jun 2020 Zheng Li, Miao Zhao, Qingyang Hong, Lin Li, Zhiyuan Tang, Dong Wang, Li-Ming Song, Cheng Yang

Based on Kaldi and Pytorch, recipes for i-vector and x-vector systems are also conducted as baselines for the three tasks.

Dialect Identification

SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection

1 code implementation11 Mar 2020 Yue Zhao, Xiyang Hu, Cheng Cheng, Cong Wang, Changlin Wan, Wen Wang, Jianing Yang, Haoping Bai, Zheng Li, Cao Xiao, Yunlong Wang, Zhi Qiao, Jimeng Sun, Leman Akoglu

Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples with numerous high-stake applications including fraud detection and intrusion detection.

Dimensionality Reduction Fraud Detection +2

Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks

no code implementations WS 2019 Xiepeng Li, Zhexi Zhang, Wei Zhu, Zheng Li, Yuan Ni, Peng Gao, Junchi Yan, Guotong Xie

We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.

Common Sense Reasoning Machine Reading Comprehension +1

SVGD: A Virtual Gradients Descent Method for Stochastic Optimization

1 code implementation9 Jul 2019 Zheng Li, Shi Shu

Inspired by dynamic programming, we propose Stochastic Virtual Gradient Descent (SVGD) algorithm where the Virtual Gradient is defined by computational graph and automatic differentiation.

Stochastic Optimization

CGaP: Continuous Growth and Pruning for Efficient Deep Learning

no code implementations27 May 2019 Xiaocong Du, Zheng Li, Yu Cao

Today a canonical approach to reduce the computation cost of Deep Neural Networks (DNNs) is to pre-define an over-parameterized model before training to guarantee the learning capacity, and then prune unimportant learning units (filters and neurons) during training to improve model compactness.

Efficient Network Construction through Structural Plasticity

no code implementations27 May 2019 Xiaocong Du, Zheng Li, Yufei Ma, Yu Cao

A typical training pipeline to mitigate over-parameterization is to pre-define a DNN structure first with redundant learning units (filters and neurons) under the goal of high accuracy, then to prune redundant learning units after training with the purpose of efficient inference.

Dimension-Free Bounds for Low-Precision Training

no code implementations ICLR 2019 Zheng Li, Christopher De Sa

Low-precision training is a promising way of decreasing the time and energy cost of training machine learning models.

Quantization

SynC: A Unified Framework for Generating Synthetic Population with Gaussian Copula

2 code implementations16 Apr 2019 Colin Wan, Zheng Li, Alicia Guo, Yue Zhao

Synthetic population generation is the process of combining multiple socioeconomic and demographic datasets from different sources and/or granularity levels, and downscaling them to an individual level.

Feature Engineering

Lightweight Image Super-Resolution with Adaptive Weighted Learning Network

1 code implementation4 Apr 2019 Chaofeng Wang, Zheng Li, Jun Shi

PyTorch code for our paper "Lightweight Image Super-Resolution with Adaptive Weighted Learning Network"

Image Super-Resolution

Learning Symmetric and Asymmetric Steganography via Adversarial Training

no code implementations13 Mar 2019 Zheng Li, Ge Han, Yunqing Wei, Shanqing Guo

Steganography refers to the art of concealing secret messages within multiple media carriers so that an eavesdropper is unable to detect the presence and content of the hidden messages.

How to Prove Your Model Belongs to You: A Blind-Watermark based Framework to Protect Intellectual Property of DNN

1 code implementation5 Mar 2019 Zheng Li, Chengyu Hu, Yang Zhang, Shanqing Guo

To fill these gaps, in this paper, we propose a novel intellectual property protection (IPP) framework based on blind-watermark for watermarking deep neural networks that meet the requirements of security and feasibility.

Association Machine Translation

PyOD: A Python Toolbox for Scalable Outlier Detection

4 code implementations6 Jan 2019 Yue Zhao, Zain Nasrullah, Zheng Li

PyOD is an open-source Python toolbox for performing scalable outlier detection on multivariate data.

Anomaly Detection outlier ensembles

LSCP: Locally Selective Combination in Parallel Outlier Ensembles

1 code implementation4 Dec 2018 Yue Zhao, Zain Nasrullah, Maciej K. Hryniewicki, Zheng Li

The top-performing base detectors in this local region are selected and combined as the model's final output.

Anomaly Detection outlier ensembles

Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification

1 code implementation AAAI 2019 2018 Zheng Li, Ying WEI, Yu Zhang, Xiang Zhang, Xin Li, Qiang Yang

Aspect-level sentiment classification (ASC) aims at identifying sentiment polarities towards aspects in a sentence, where the aspect can behave as a general Aspect Category (AC) or a specific Aspect Term (AT).

General Classification Sentiment Analysis +1

Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification

1 code implementation Thirty-Second AAAI Conference on Artificial Intelligence 2018 Zheng Li, Ying WEI, Yu Zhang, Qiang Yang

Existing cross-domain sentiment classification meth- ods cannot automatically capture non-pivots, i. e., the domain- specific sentiment words, and pivots, i. e., the domain-shared sentiment words, simultaneously.

Classification Cross-Domain Text Classification +4

Cannot find the paper you are looking for? You can Submit a new open access paper.