Search Results for author: Lei LI

Found 294 papers, 144 papers with code

Structural Supervision for Word Alignment and Machine Translation

no code implementations Findings (ACL) 2022 Lei LI, Kai Fan, Hongjia Li, Chun Yuan

Syntactic structure has long been argued to be potentially useful for enforcing accurate word alignment and improving generalization performance of machine translation.

Machine Translation Multi-Task Learning +2

Dispersed EM-VAEs for Interpretable Text Generation

no code implementations ICML 2020 Wenxian Shi, Hao Zhou, Ning Miao, Lei LI

Interpretability is important in text generation for guiding the generation with interpretable attributes.

Text Generation

GLAT: Glancing at Latent Variables for Parallel Text Generation

1 code implementation ACL 2022 Yu Bao, Hao Zhou, ShuJian Huang, Dongqi Wang, Lihua Qian, Xinyu Dai, Jiajun Chen, Lei LI

Recently, parallel text generation has received widespread attention due to its success in generation efficiency.

Text Generation

Extractive Financial Narrative Summarisation based on DPPs

no code implementations FNP (COLING) 2020 Lei LI, Yafei Jiang, Yinan Liu

We participate in the FNS-Summarisation 2020 shared task to be held at FNP 2020 workshop at COLING 2020.

Point Processes

Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization

no code implementations EMNLP 2021 Zhiyuan Zeng, Jiaze Chen, Weiran Xu, Lei LI

Based on the artificial dataset, we train an evaluation model that can not only make accurate and robust factual consistency discrimination but is also capable of making interpretable factual errors tracing by backpropagated gradient distribution on token embeddings.

Abstractive Text Summarization Data Augmentation

Augmenting Legal Judgment Prediction with Contrastive Case Relations

1 code implementation COLING 2022 Dugang Liu, Weihao Du, Lei LI, Weike Pan, Zhong Ming

Existing legal judgment prediction methods usually only consider one single case fact description as input, which may not fully utilize the information in the data such as case relations and frequency.

ImageNetVC: Zero-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories

1 code implementation24 May 2023 Heming Xia, Qingxiu Dong, Lei LI, Jingjing Xu, Ziwei Qin, Zhifang Sui

Recently, Pretrained Language Models (PLMs) have been serving as general-purpose interfaces, posing a significant demand for comprehensive visual knowledge.

Common Sense Reasoning

ALGO: Synthesizing Algorithmic Programs with Generated Oracle Verifiers

1 code implementation24 May 2023 Kexun Zhang, Danqing Wang, Jingtao Xia, William Yang Wang, Lei LI

To address these challenges, we propose ALGO, a framework that synthesizes Algorithmic programs with LLM-Generated Oracles to guide the creation and verify their correctness.

Code Generation

Prompt Optimization of Large Language Model for Interactive Tasks without Gradient and Demonstrations

no code implementations24 May 2023 Siqi Ouyang, Lei LI

Large language models (LLMs) have demonstrated remarkable language proficiency, but they face challenges when solving interactive tasks independently.

Language Modelling

Can Language Models Understand Physical Concepts?

no code implementations23 May 2023 Lei LI, Jingjing Xu, Qingxiu Dong, Ce Zheng, Qi Liu, Lingpeng Kong, Xu sun

Language models~(LMs) gradually become general-purpose interfaces in the interactive and embodied world, where the understanding of physical concepts is an essential prerequisite.

Learn from Mistakes through Cooperative Interaction with Study Assistant

no code implementations23 May 2023 Danqing Wang, Lei LI

Large language models have demonstrated their ability to self-reflect and refine their generation, which can further improve their performance.

Language Modelling

Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

no code implementations23 May 2023 Lean Wang, Lei LI, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie zhou, Xu sun

In-context learning (ICL) emerges as a promising capability of large language models (LLMs) by providing them with demonstration examples to perform diverse tasks.

INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic Feedback

1 code implementation23 May 2023 Wenda Xu, Danqing Wang, Liangming Pan, Zhenqiao Song, Markus Freitag, William Yang Wang, Lei LI

In particular, since the advent of neural metrics, like COMET, BLEURT, and SEScore2, the newest generation of metrics show a high correlation with human judgment.

Text Generation

Extrapolating Multilingual Understanding Models as Multilingual Generators

no code implementations22 May 2023 Bohong Wu, Fei Yuan, Hai Zhao, Lei LI, Jingjing Xu

Considering that encoder-based models have the advantage of efficient generation and self-correction abilities, this paper explores methods to empower multilingual understanding models the generation abilities to get a unified model.

Denoising Machine Translation +5

Can We Edit Factual Knowledge by In-Context Learning?

2 code implementations22 May 2023 Ce Zheng, Lei LI, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang

Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge.

Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter

1 code implementation21 May 2023 Yi Liu, Xiaohan Bi, Lei LI, Sishuo Chen, Wenkai Yang, Xu sun

However, as pre-trained language models (PLMs) continue to increase in size, the communication cost for transmitting parameters during synchronization has become a training speed bottleneck.

Federated Learning Machine Translation +1

Statistical Knowledge Assessment for Generative Language Models

no code implementations17 May 2023 Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Zhifang Sui, Lei LI

Our findings reveal that the knowledge in GLMs with the same backbone architecture adheres to the scaling law, and that tuning on instruction-following data may compromise the model's ability to generate factually correct text consistently.

Instruction Following

Importance Weighted Expectation-Maximization for Protein Sequence Design

no code implementations30 Apr 2023 Zhenqiao Song, Lei LI

How can we efficiently generate diverse and novel protein sequences with high fitness?

Revisiting k-NN for Pre-trained Language Models

1 code implementation18 Apr 2023 Lei LI, Jing Chen, Bozhong Tian, Ningyu Zhang

Pre-trained Language Models (PLMs), as parametric-based eager learners, have become the de-facto choice for current paradigms of Natural Language Processing (NLP).

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

1 code implementation10 Apr 2023 Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, ShuJian Huang, Lingpeng Kong, Jiajun Chen, Lei LI

First, prompt semantics can surprisingly be ignored when given in-context exemplars, where LLMs still show strong performance even with unreasonable prompts.

Machine Translation Translation

Influence of Myocardial Infarction on QRS Properties: A Simulation Study

no code implementations4 Apr 2023 Lei LI, Julia Camps, Zhinuo, Wang, Abhirup Banerjee, Blanca Rodriguez, Vicente Grau

However, the influence of various MI properties on the QRS is not intuitively predictable. In this work, we have systematically investigated the effects of 17 post-MI scenarios, varying the location, size, transmural extent, and conductive level of scarring and border zone area, on the forward-calculated QRS.

Generalizable Local Feature Pre-training for Deformable Shape Analysis

1 code implementation CVPR 2023 Souhaib Attaiki, Lei LI, Maks Ovsjanikov

We observe that with proper training, learned features can be useful in such tasks, but, crucially, only with an appropriate choice of the receptive field size.

Transfer Learning

Aligning Multi-Sequence CMR Towards Fully Automated Myocardial Pathology Segmentation

no code implementations7 Feb 2023 Wangbin Ding, Lei LI, Junyi Qiu, Sihan Wang, Liqin Huang, Yinyin Chen, Shan Yang, Xiahai Zhuang

For instance, balanced steady-state free precession cine sequences present clear anatomical boundaries, while late gadolinium enhancement and T2-weighted CMR sequences visualize myocardial scar and edema of MI, respectively.

Image Registration

Protecting Language Generation Models via Invisible Watermarking

no code implementations6 Feb 2023 Xuandong Zhao, Yu-Xiang Wang, Lei LI

We can then detect the secret message by probing a suspect model to tell if it is distilled from the protected one.

Model extraction Text Generation

Design and Implementation of A Soccer Ball Detection System with Multiple Cameras

no code implementations31 Jan 2023 Lei LI, Tianfang Zhang, Zhongfeng Kang, Wenhan Zhang

This paper designed and implemented football detection system under multiple cameras for the detection and capture of targets in real-time matches.

One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NER

2 code implementations25 Jan 2023 Xiang Chen, Lei LI, Shuofei Qiao, Ningyu Zhang, Chuanqi Tan, Yong Jiang, Fei Huang, Huajun Chen

Previous typical solutions mainly obtain a NER model by pre-trained language models (PLMs) with data from a rich-resource domain and adapt it to the target domain.

NER Text Generation

Geometric ergodicity of SGLD via reflection coupling

no code implementations17 Jan 2023 Lei LI, Jian-Guo Liu, Yuliang Wang

We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings.

BuildSeg: A General Framework for the Segmentation of Buildings

no code implementations15 Jan 2023 Lei LI, Tianfang Zhang, Stefan Oehmcke, Fabian Gieseke, Christian Igel

Building segmentation from aerial images and 3D laser scanning (LiDAR) is a challenging task due to the diversity of backgrounds, building textures, and image quality.

Multi-Target Landmark Detection with Incomplete Images via Reinforcement Learning and Shape Prior

no code implementations13 Jan 2023 Kaiwen Wan, Lei LI, Dengqiang Jia, Shangqi Gao, Wei Qian, Yingzhi Wu, Huandong Lin, Xiongzheng Mu, Xin Gao, Sijia Wang, Fuping Wu, Xiahai Zhuang

This is particularly evident for the learning-based multi-target landmark detection, where algorithms could be misleading to learn primarily the variation of background due to the varying FOV, failing the detection of targets.

Reinforcement Learning (RL)

VQNet 2.0: A New Generation Machine Learning Framework that Unifies Classical and Quantum

no code implementations9 Jan 2023 Huanyu Bian, Zhilong Jia, Menghan Dou, Yuan Fang, Lei LI, Yiming Zhao, Hanchao Wang, Zhaohui Zhou, Wei Wang, Wenyu Zhu, Ye Li, Yang Yang, Weiming Zhang, Nenghai Yu, Zhaoyun Chen, Guoping Guo

Therefore, based on VQNet 1. 0, we further propose VQNet 2. 0, a new generation of unified classical and quantum machine learning framework that supports hybrid optimization.

Quantum Machine Learning Unity

A Survey on In-context Learning

1 code implementation31 Dec 2022 Qingxiu Dong, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu sun, Jingjing Xu, Lei LI, Zhifang Sui

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples.

Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models

no code implementations20 Dec 2022 Jingjing Xu, Qingxiu Dong, Hongyi Liu, Lei LI

With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3.

Language Modelling Masked Language Modeling +2

WACO: Word-Aligned Contrastive Learning for Speech Translation

no code implementations19 Dec 2022 Siqi Ouyang, Rong Ye, Lei LI

In this paper, we propose Word-Aligned COntrastive learning (WACO), a novel method for few-shot speech-to-text translation.

Contrastive Learning Speech-to-Text Translation +1

SEScore2: Retrieval Augmented Pretraining for Text Generation Evaluation

1 code implementation19 Dec 2022 Wenda Xu, Xian Qian, Mingxuan Wang, Lei LI, William Yang Wang

Existing learned metrics have gaps to human judgements, are model-dependent or are limited to the domains or tasks where human ratings are available.

Dialogue Generation Machine Translation +2

Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GAN

no code implementations18 Dec 2022 Lei LI, Tianfang Zhang, Zhongfeng Kang, Xikun Jiang

Fine-grained semantic segmentation of a person's face and head, including facial parts and head components, has progressed a great deal in recent years.

Face Model Face Parsing +1

Pre-trained Language Models can be Fully Zero-Shot Learners

no code implementations14 Dec 2022 Xuandong Zhao, Siqi Ouyang, Zhiguo Yu, Ming Wu, Lei LI

How can we extend a pre-trained model to many language understanding tasks, without labeled or additional unlabeled data?

Retrieval text-classification +3

Accelerating Antimicrobial Peptide Discovery with Latent Sequence-Structure Model

1 code implementation28 Nov 2022 Danqing Wang, Zeyu Wen, Fei Ye, Hao Zhou, Lei LI

Experimental results show that the peptides generated by LSSAMP have a high probability of AMP, and two of the 21 candidates have been verified to have good antimicrobial activity.

MyoPS-Net: Myocardial Pathology Segmentation with Flexible Combination of Multi-Sequence CMR Images

no code implementations6 Nov 2022 Junyi Qiu, Lei LI, Sihan Wang, Ke Zhang, Yinyin Chen, Shan Yang, Xiahai Zhuang

We therefore conducted extensive experiments to investigate the performance of the proposed method in dealing with such complex combinations of different CMR sequences.

Gradient Knowledge Distillation for Pre-trained Language Models

1 code implementation2 Nov 2022 Lean Wang, Lei LI, Xu sun

Knowledge distillation (KD) is an effective framework to transfer knowledge from a large-scale teacher to a compact yet well-performing student.

Knowledge Distillation

Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching

1 code implementation12 Oct 2022 Lei LI, Nicolas Donati, Maks Ovsjanikov

Our approach is not only accurate with near-isometric input, for which a high spectral resolution is typically preferred, but also robust and able to produce reasonable matching even in the presence of significant non-isometric distortion, which poses great challenges to existing methods.

From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models

1 code implementation11 Oct 2022 Lei LI, Yankai Lin, Xuancheng Ren, Guangxiang Zhao, Peng Li, Jie zhou, Xu sun

We then design a Model Uncertainty--aware Knowledge Integration (MUKI) framework to recover the golden supervision for the student.

PARAGEN : A Parallel Generation Toolkit

1 code implementation7 Oct 2022 Jiangtao Feng, Yi Zhou, Jun Zhang, Xian Qian, Liwei Wu, Zhexi Zhang, Yanming Liu, Mingxuan Wang, Lei LI, Hao Zhou

PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation.

Model Selection

Block-Structured Optimization for Subgraph Detection in Interdependent Networks

no code implementations6 Oct 2022 Fei Jie, Chunpai Wang, Feng Chen, Lei LI, Xindong Wu

We propose a generalized framework for block-structured nonconvex optimization, which can be applied to structured subgraph detection in interdependent networks, such as multi-layer networks, temporal networks, networks of networks, and many others.

Safety-based Speed Control of a Wheelchair using Robust Adaptive Model Predictive Control

no code implementations6 Oct 2022 Meng Yuan, Ye Wang, Lei LI, Tianyou Chai, Wei Tech Ang

Electric-powered wheelchair plays an important role in providing accessibility for people with mobility impairment.

Just ClozE! A Fast and Simple Method for Evaluating the Factual Consistency in Abstractive Summarization

no code implementations6 Oct 2022 Yiyang Li, Lei LI, Qing Yang, Marina Litvak, Natalia Vanetik, Dingxin Hu, Yuze Li, Yanquan Zhou, Dongliang Xu, Xuanyu Zhang

We demonstrate that ClozE can reduce the evaluation time by nearly 96$\%$ relative to QA-based metrics while retaining their interpretability and performance through experiments on six human-annotated datasets and a meta-evaluation benchmark GO FIGURE \citep{gabriel2020go}.

Abstractive Text Summarization Language Modelling +1

SRFeat: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence

1 code implementation16 Sep 2022 Lei LI, Souhaib Attaiki, Maks Ovsjanikov

In this work, we present a novel learning-based framework that combines the local accuracy of contrastive learning with the global consistency of geometric approaches, for robust non-rigid matching.

Contrastive Learning

Rethinking the Unpretentious U-net for Medical Ultrasound Image Segmentation

1 code implementation15 Sep 2022 Gongping Chen, Lei LI, Jianxun Zhang, Yu Dai

However, variable tumor morphology, blurred boundary, and similar intensity distributions bring challenges for accurate segmentation of breast tumors.

Image Segmentation Tumor Segmentation

Multi-Modality Cardiac Image Computing: A Survey

no code implementations26 Aug 2022 Lei LI, Wangbin Ding, Liqun Huang, Xiahai Zhuang, Vicente Grau

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases.


A deep learning framework for geodesics under spherical Wasserstein-Fisher-Rao metric and its application for weighted sample generation

no code implementations25 Aug 2022 Yang Jing, Jiaheng Chen, Lei LI, Jianfeng Lu

In this paper, we develop a deep learning framework to compute the geodesics under the spherical WFR metric, and the learned geodesics can be adopted to generate weighted samples.

Bayesian Inference

Deep Computational Model for the Inference of Ventricular Activation Properties

no code implementations8 Aug 2022 Lei LI, Julia Camps, Abhirup Banerjee, Marcel Beetz, Blanca Rodriguez, Vicente Grau

Cardiac digital twins can provide non-invasive characterizations of cardiac functions for individual patients, and therefore are promising for the patient-specific diagnosis and therapy stratification.


Distributional Correlation--Aware Knowledge Distillation for Stock Trading Volume Prediction

1 code implementation4 Aug 2022 Lei LI, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu sun

Traditional knowledge distillation in classification problems transfers the knowledge via class correlations in the soft label produced by teacher models, which are not available in regression problems like stock trading volume prediction.

Knowledge Distillation regression

A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamics

no code implementations19 Jul 2022 Lei LI, Yuliang Wang

We establish a sharp uniform-in-time error estimate for the Stochastic Gradient Langevin Dynamics (SGLD), which is a popular sampling algorithm.

On uniform-in-time diffusion approximation for stochastic gradient descent

no code implementations11 Jul 2022 Lei LI, Yuliang Wang

The main technique is to establish the exponential decay rates of the derivatives of the solution to the backward Kolmogorov equation.

On the Learning of Non-Autoregressive Transformers

no code implementations13 Jun 2022 Fei Huang, Tianhua Tao, Hao Zhou, Lei LI, Minlie Huang

Non-autoregressive Transformer (NAT) is a family of text generation models, which aims to reduce the decoding latency by predicting the whole sentences in parallel.

Text Generation

Decoupling Predictions in Distributed Learning for Multi-Center Left Atrial MRI Segmentation

1 code implementation10 Jun 2022 Zheyao Gao, Lei LI, Fuping Wu, Sihan Wang, Xiahai Zhuang

In this work, we propose a new framework of distributed learning that bridges the gap between two groups, and improves the performance for both generic and local data.

MRI segmentation

Delving into the Openness of CLIP

1 code implementation4 Jun 2022 Shuhuai Ren, Lei LI, Xuancheng Ren, Guangxiang Zhao, Xu sun

However, evaluating the openness of CLIP-like models is challenging, as the models are open to arbitrary vocabulary in theory, but their accuracy varies in practice.

Image Classification Text Matching

Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning

2 code implementations29 May 2022 Xiang Chen, Lei LI, Ningyu Zhang, Xiaozhuan Liang, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, Huajun Chen

Specifically, vanilla prompt learning may struggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.

Few-Shot Text Classification Memorization +5

Enhancing Cross-lingual Transfer by Manifold Mixup

1 code implementation ICLR 2022 Huiyun Yang, Huadong Chen, Hao Zhou, Lei LI

Based on large-scale pre-trained multilingual representations, recent cross-lingual transfer methods have achieved impressive transfer performances.

Cross-Lingual Transfer

Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction

1 code implementation7 May 2022 Xiang Chen, Ningyu Zhang, Lei LI, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, Huajun Chen

To deal with these issues, we propose a novel Hierarchical Visual Prefix fusion NeTwork (HVPNeT) for visual-enhanced entity and relation extraction, aiming to achieve more effective and robust performance.

named-entity-recognition Named Entity Recognition +2

Cross-modal Contrastive Learning for Speech Translation

1 code implementation NAACL 2022 Rong Ye, Mingxuan Wang, Lei LI

Learning similar representations for semantically similar speech and text is important for speech translation.

Contrastive Learning Retrieval +3

Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion

1 code implementation4 May 2022 Xiang Chen, Ningyu Zhang, Lei LI, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, Huajun Chen

Since most MKGs are far from complete, extensive knowledge graph completion studies have been proposed focusing on the multimodal entity, relation extraction and link prediction.

Information Retrieval Link Prediction +4

Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning

1 code implementation4 May 2022 Xiang Chen, Lei LI, Ningyu Zhang, Chuanqi Tan, Fei Huang, Luo Si, Huajun Chen

Note that the previous parametric learning paradigm can be viewed as memorization regarding training data as a book and inference as the close-book test.

Few-Shot Learning Memorization +2

Provably Confidential Language Modelling

1 code implementation NAACL 2022 Xuandong Zhao, Lei LI, Yu-Xiang Wang

Large language models are shown to memorize privacy information such as social security numbers in training data.

Language Modelling Memorization +1

Learning Design and Construction with Varying-Sized Materials via Prioritized Memory Resets

1 code implementation12 Apr 2022 Yunfei Li, Tao Kong, Lei LI, Yi Wu

Can a robot autonomously learn to design and construct a bridge from varying-sized blocks without a blueprint?

Motion Planning

$\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation

1 code implementation5 Apr 2022 Yu Bao, Hao Zhou, ShuJian Huang, Dongqi Wang, Lihua Qian, Xinyu Dai, Jiajun Chen, Lei LI

Recently, parallel text generation has received widespread attention due to its success in generation efficiency.

Text Generation

E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning

no code implementations Findings (ACL) 2022 Jiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi, Zhongqiao Li, Xinbo Zhang, Changzhi Sun, Lei LI, Yanghua Xiao, Hao Zhou

Holding the belief that models capable of reasoning should be right for the right reasons, we propose a first-of-its-kind Explainable Knowledge-intensive Analogical Reasoning benchmark (E-KAR).

Explanation Generation Question Answering

$ \text{T}^3 $OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area

1 code implementation1 Mar 2022 Zheng Yuan, Tianhao Wu, Qinwen Wang, Yiying Yang, Lei LI, Lin Zhang

Although there are some achievements in the field of MVP in the open space environment, the urban area brings complicated road structures and restricted moving spaces as challenges to the resolution of MVP games.

Decision Making

KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Models

no code implementations28 Feb 2022 Daniel Gao, Yantao Jia, Lei LI, Chengzhen Fu, Zhicheng Dou, Hao Jiang, Xinyu Zhang, Lei Chen, Zhao Cao

However, to figure out whether PLMs can be reliable knowledge sources and used as alternative knowledge bases (KBs), we need to further explore some critical features of PLMs.

General Knowledge Memorization +1

Deepfake Network Architecture Attribution

1 code implementation28 Feb 2022 Tianyun Yang, Ziyao Huang, Juan Cao, Lei LI, Xirong Li

With the rapid progress of generation technology, it has become necessary to attribute the origin of fake images.

DeepFake Detection Fake Image Attribution

Personalized Prompt Learning for Explainable Recommendation

1 code implementation15 Feb 2022 Lei LI, Yongfeng Zhang, Li Chen

In the latter case, ID vectors are randomly initialized but the model is trained in advance on large corpora, so they are actually in different learning stages.

Explainable Recommendation Recommendation Systems +1

Cross-Modality Multi-Atlas Segmentation via Deep Registration and Label Fusion

1 code implementation4 Feb 2022 Wangbin Ding, Lei LI, Xiahai Zhuang, Liqin Huang

For the label fusion, we design a similarity estimation network (SimNet), which estimates the fusion weight of each atlas by measuring its similarity to the target image.

Image Registration Image Segmentation +2

AWSnet: An Auto-weighted Supervision Attention Network for Myocardial Scar and Edema Segmentation in Multi-sequence Cardiac Magnetic Resonance Images

1 code implementation14 Jan 2022 Kai-Ni Wang, Xin Yang, Juzheng Miao, Lei LI, Jing Yao, Ping Zhou, Wufeng Xue, Guang-Quan Zhou, Xiahai Zhuang, Dong Ni

Extensive experimental results on a publicly available dataset from Myocardial pathology segmentation combining multi-sequence CMR (MyoPS 2020) demonstrate our method can achieve promising performance compared with other state-of-the-art methods.

Deep Learning Based 3D Point Cloud Regression for Estimating Forest Biomass

no code implementations21 Dec 2021 Stefan Oehmcke, Lei LI, Katerina Trepekli, Jaime Revenga, Thomas Nord-Larsen, Fabian Gieseke, Christian Igel

Quantification of forest biomass stocks and their dynamics is important for implementing effective climate change mitigation measures.

Management regression

Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models

no code implementations14 Dec 2021 Lei LI, Yankai Lin, Xuancheng Ren, Guangxiang Zhao, Peng Li, Jie zhou, Xu sun

As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to reuse these models is vital as it can greatly reduce the retraining computational cost and the potential environmental side-effects.

Unsupervised Editing for Counterfactual Stories

1 code implementation10 Dec 2021 Jiangjie Chen, Chun Gan, Sijie Cheng, Hao Zhou, Yanghua Xiao, Lei LI

We also propose a new metric to alleviate the shortcomings of current automatic metrics and better evaluate the trade-off.

StrokeNet: Stroke Assisted and Hierarchical Graph Reasoning Networks

no code implementations23 Nov 2021 Lei LI, Kai Fan, Chun Yuan

Scene text detection is still a challenging task, as there may be extremely small or low-resolution strokes, and close or arbitrary-shaped texts.

Node Classification Relational Reasoning +1

A Survey on Green Deep Learning

no code implementations8 Nov 2021 Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu, Hao Zhou, Lei LI

In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and computer vision (CV).

Knowledge Distillation Model Compression

Multi-Modality Cardiac Image Analysis with Deep Learning

no code implementations8 Nov 2021 Lei LI, Fuping Wu, Sihang Wang, Xiahai Zhuang

Accurate cardiac computing, analysis and modeling from multi-modality images are important for the diagnosis and treatment of cardiac disease.

Image Segmentation Semantic Segmentation +1

Self-Supervised Speech Denoising Using Only Noisy Audio Signals

1 code implementation30 Oct 2021 Jiasong Wu, Qingchun Li, Guanyu Yang, Lei LI, Lotfi Senhadji, Huazhong Shu

The first module adopts a random audio sub-sampler on each noisy audio to generate training pairs.

Audio Denoising Denoising +1

CNewSum: A Large-scale Chinese News Summarization Dataset with Human-annotated Adequacy and Deducibility Level

no code implementations21 Oct 2021 Danqing Wang, Jiaze Chen, Xianze Wu, Hao Zhou, Lei LI

In this paper, we present a large-scale Chinese news summarization dataset CNewSum, which consists of 304, 307 documents and human-written summaries for the news feed.

News Summarization Text Summarization

Well-classified Examples are Underestimated in Classification with Deep Neural Networks

1 code implementation13 Oct 2021 Guangxiang Zhao, Wenkai Yang, Xuancheng Ren, Lei LI, Yunfang Wu, Xu sun

The conventional wisdom behind learning deep classification models is to focus on bad-classified examples and ignore well-classified examples that are far from the decision boundary.

Graph Classification imbalanced classification +4

LightSeq2: Accelerated Training for Transformer-based Models on GPUs

1 code implementation12 Oct 2021 Xiaohui Wang, Yang Wei, Ying Xiong, Guyue Huang, Xian Qian, Yufei Ding, Mingxuan Wang, Lei LI

In this paper, we present LightSeq2, a system to accelerate training for a general family of Transformer models on GPUs.

Machine Translation Speech Recognition +1

Generating Antimicrobial Peptides from Latent Secondary Structure Space

no code implementations29 Sep 2021 Danqing Wang, Zeyu Wen, Lei LI, Hao Zhou

By sampling in the latent secondary structure space, we can generate peptides with ideal amino acids and secondary structures at the same time.

Drug Discovery

NAIL: A Challenging Benchmark for Na\"ive Logical Reasoning

no code implementations29 Sep 2021 Xinbo Zhang, Changzhi Sun, Yue Zhang, Lei LI, Hao Zhou

Logical reasoning over natural text is an important capability towards human level intelligence.

Logical Reasoning

Dynamic Knowledge Distillation for Pre-trained Language Models

1 code implementation EMNLP 2021 Lei LI, Yankai Lin, Shuhuai Ren, Peng Li, Jie zhou, Xu sun

Knowledge distillation~(KD) has been proved effective for compressing large-scale pre-trained language models.

Knowledge Distillation

Learning When to Translate for Streaming Speech

1 code implementation ACL 2022 Qianqian Dong, Yaoming Zhu, Mingxuan Wang, Lei LI

Given a usually long speech sequence, we develop an efficient monotonic segmentation module inside an encoder-decoder model to accumulate acoustic information incrementally and detect proper speech unit boundaries for the input in speech translation task.

Speech-to-Text Translation Translation

Right Ventricular Segmentation from Short- and Long-Axis MRIs via Information Transition

1 code implementation5 Sep 2021 Lei LI, Wangbin Ding, Liqun Huang, Xiahai Zhuang

In this work, we propose an automatic RV segmentation framework, where the information from long-axis (LA) views is utilized to assist the segmentation of short-axis (SA) views via information transition.

Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification

1 code implementation EMNLP 2021 Shuhuai Ren, Jinchao Zhang, Lei LI, Xu sun, Jie zhou

Data augmentation aims to enrich training samples for alleviating the overfitting issue in low-resource or class-imbalanced situations.

Bayesian Optimization Data Augmentation +2

Secoco: Self-Correcting Encoding for Neural Machine Translation

no code implementations Findings (EMNLP) 2021 Tao Wang, Chengqi Zhao, Mingxuan Wang, Lei LI, Hang Li, Deyi Xiong

This paper presents Self-correcting Encoding (Secoco), a framework that effectively deals with input noise for robust neural machine translation by introducing self-correcting predictors.

Machine Translation NMT +1

WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration

1 code implementation5 Aug 2021 Lei LI, Hongbo Fu, Maks Ovsjanikov

Instead of using a predefined fixed-size local support in voxelization, we propose to learn the optimal support in a data-driven manner.

Metric Learning Point Cloud Registration

Learning to Design and Construct Bridge without Blueprint

no code implementations5 Aug 2021 Yunfei Li, Tao Kong, Lei LI, Yifeng Li, Yi Wu

In this task, the robot needs to first design a feasible bridge architecture for arbitrarily wide cliffs and then manipulate the blocks reliably to construct a stable bridge according to the proposed design.

Motion Planning

Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimation

no code implementations5 Aug 2021 Yiming Li, Tao Kong, Ruihang Chu, Yifeng Li, Peng Wang, Lei LI

In a unified framework, we jointly predict the feasible 6-DoF grasp poses, instance semantic segmentation, and collision information.

Multi-Task Learning Pose Estimation +1

Pre-training Methods for Neural Machine Translation

no code implementations ACL 2021 Mingxuan Wang, Lei LI

This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation.

Machine Translation NMT +1

Follow Your Path: a Progressive Method for Knowledge Distillation

no code implementations20 Jul 2021 Wenxian Shi, Yuxuan Song, Hao Zhou, Bohan Li, Lei LI

However, it has been observed that a converged heavy teacher model is strongly constrained for learning a compact student network and could make the optimization subject to poor local optima.

Knowledge Distillation

Automatic Fatou Property of Law-invariant Risk Measures

no code implementations16 Jul 2021 Shengzhong Chen, Niushan Gao, Denny Leung, Lei LI

In the paper we investigate automatic Fatou property of law-invariant risk measures on a rearrangement-invariant function space $\mathcal{X}$ other than $L^\infty$.

SOLO: A Simple Framework for Instance Segmentation

no code implementations30 Jun 2021 Xinlong Wang, Rufeng Zhang, Chunhua Shen, Tao Kong, Lei LI

Besides instance segmentation, our method yields state-of-the-art results in object detection (from our mask byproduct) and panoptic segmentation.

Image Matting Instance Segmentation +3

Subjective Bias in Abstractive Summarization

1 code implementation18 Jun 2021 Lei LI, Wei Liu, Marina Litvak, Natalia Vanetik, Jiacheng Pei, Yinan Liu, Siya Qi

Due to the subjectivity of the summarization, it is a good practice to have more than one gold summary for each training document.

Abstractive Text Summarization

Medical Image Analysis on Left Atrial LGE MRI for Atrial Fibrillation Studies: A Review

1 code implementation18 Jun 2021 Lei LI, Veronika A. Zimmer, Julia A. Schnabel, Xiahai Zhuang

Late gadolinium enhancement magnetic resonance imaging (LGE MRI) is commonly used to visualize and quantify left atrial (LA) scars.

AtrialGeneral: Domain Generalization for Left Atrial Segmentation of Multi-Center LGE MRIs

no code implementations16 Jun 2021 Lei LI, Veronika A. Zimmer, Julia A. Schnabel, Xiahai Zhuang

Left atrial (LA) segmentation from late gadolinium enhanced magnetic resonance imaging (LGE MRI) is a crucial step needed for planning the treatment of atrial fibrillation.

Domain Generalization Semantic Segmentation +1

Learning to Disentangle GAN Fingerprint for Fake Image Attribution

no code implementations16 Jun 2021 Tianyun Yang, Juan Cao, Qiang Sheng, Lei LI, Jiaqi Ji, Xirong Li, Sheng Tang

Adopting a multi-task framework, we propose a GAN Fingerprint Disentangling Network (GFD-Net) to simultaneously disentangle the fingerprint from GAN-generated images and produce a content-irrelevant representation for fake image attribution.

Fake Image Attribution

Adversarial Option-Aware Hierarchical Imitation Learning

1 code implementation10 Jun 2021 Mingxuan Jing, Wenbing Huang, Fuchun Sun, Xiaojian Ma, Tao Kong, Chuang Gan, Lei LI

In particular, we propose an Expectation-Maximization(EM)-style algorithm: an E-step that samples the options of expert conditioned on the current learned policy, and an M-step that updates the low- and high-level policies of agent simultaneously to minimize the newly proposed option-occupancy measurement between the expert and the agent.

Imitation Learning

UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction

1 code implementation Findings (ACL) 2021 Huanqin Wu, Wei Liu, Lei LI, Dan Nie, Tao Chen, Feng Zhang, Di Wang

Keyphrase Prediction (KP) task aims at predicting several keyphrases that can summarize the main idea of the given document.

Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation

no code implementations NAACL 2021 Hua Zheng, Damai Dai, Lei LI, Tianyu Liu, Zhifang Sui, Baobao Chang, Yang Liu

In this paper, we tackle the task of Definition Generation (DG) in Chinese, which aims at automatically generating a definition for a word.

Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker

2 code implementations ACL 2021 Runxin Xu, Tianyu Liu, Lei LI, Baobao Chang

Existing methods are not effective due to two challenges of this task: a) the target event arguments are scattered across sentences; b) the correlation among events in a document is non-trivial to model.

Document-level Event Extraction Event Extraction

Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge

no code implementations28 May 2021 Yi Zhang, Lei LI, Yunfang Wu, Qi Su, Xu sun

Knowledge facts are typically represented by relational triples, while we observe that some commonsense facts are represented by the triples whose forms are inconsistent with the expression of language.

Cysteine post-translational modifications: ten years from chemical proteomics to bioinformatics

no code implementations28 May 2021 Yanzheng Meng, Lei LI

As the only thiol-bearing amino acid, cysteine (Cys) residues in proteins have the reactive thiol side chain, which is susceptible to a series of post-translational modifications (PTMs).

Personalized Transformer for Explainable Recommendation

1 code implementation ACL 2021 Lei LI, Yongfeng Zhang, Li Chen

Transformer, which is demonstrated with strong language modeling capability, however, is not personalized and fails to make use of the user and item IDs since the ID tokens are not even in the same semantic space as the words.

Explainable Recommendation Language Modelling +1

Contrastive Learning for Many-to-many Multilingual Neural Machine Translation

3 code implementations ACL 2021 Xiao Pan, Mingxuan Wang, Liwei Wu, Lei LI

Existing multilingual machine translation approaches mainly focus on English-centric directions, while the non-English directions still lag behind.

Contrastive Learning Data Augmentation +2

The Volctrans Neural Speech Translation System for IWSLT 2021

1 code implementation ACL (IWSLT) 2021 Chengqi Zhao, Zhicheng Liu, Jian Tong, Tao Wang, Mingxuan Wang, Rong Ye, Qianqian Dong, Jun Cao, Lei LI

For offline speech translation, our best end-to-end model achieves 8. 1 BLEU improvements over the benchmark on the MuST-C test set and is even approaching the results of a strong cascade solution.


Unsupervised Multi-Modality Registration Network based on Spatially Encoded Gradient Information

1 code implementation16 May 2021 Wangbin Ding, Lei LI, Xiahai Zhuang, Liqin Huang

However, it is still challenging to develop a multi-modality registration network due to the lack of robust criteria for network training.

Learning Shared Semantic Space for Speech-to-Text Translation

2 code implementations Findings (ACL) 2021 Chi Han, Mingxuan Wang, Heng Ji, Lei LI

By projecting audio and text features to a common semantic representation, Chimera unifies MT and ST tasks and boosts the performance on ST benchmarks, MuST-C and Augmented Librispeech, to a new state-of-the-art.

Machine Translation Speech-to-Text Translation +1

End-to-end Speech Translation via Cross-modal Progressive Training

1 code implementation21 Apr 2021 Rong Ye, Mingxuan Wang, Lei LI

XSTNet takes both speech and text as input and outputs both transcription and translation text.

Machine Translation Speech-to-Text Translation +1

Few-Shot Meta-Learning on Point Cloud for Semantic Segmentation

no code implementations7 Apr 2021 Xudong Li, Li Feng, Lei LI, Chen Wang

With a good understanding of environmental information, construction robots can work better.

Autonomous Driving Meta-Learning +1

ENPAR:Enhancing Entity and Entity Pair Representations for Joint Entity Relation Extraction

1 code implementation EACL 2021 Yijun Wang, Changzhi Sun, Yuanbin Wu, Hao Zhou, Lei LI, Junchi Yan

Current state-of-the-art systems for joint entity relation extraction (Luan et al., 2019; Wad-den et al., 2019) usually adopt the multi-task learning framework.

coreference-resolution Coreference Resolution +4