Search Results for author: Chen Lin

Found 67 papers, 29 papers with code

EntityBERT: Entity-centric Masking Strategy for Model Pretraining for the Clinical Domain

no code implementations NAACL (BioNLP) 2021 Chen Lin, Timothy Miller, Dmitriy Dligach, Steven Bethard, Guergana Savova

We propose a methodology to produce a model focused on the clinical domain: continued pretraining of a model with a broad representation of biomedical terminology (PubMedBERT) on a clinical corpus along with a novel entity-centric masking strategy to infuse domain knowledge in the learning process.

Negation Negation Detection +2

Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers

1 code implementation9 May 2024 Peng Gao, Le Zhuo, Dongyang Liu, Ruoyi Du, Xu Luo, Longtian Qiu, Yuhang Zhang, Chen Lin, Rongjie Huang, Shijie Geng, Renrui Zhang, Junlin Xi, Wenqi Shao, Zhengkai Jiang, Tianshuo Yang, Weicai Ye, He Tong, Jingwen He, Yu Qiao, Hongsheng Li

Sora unveils the potential of scaling Diffusion Transformer for generating photorealistic images and videos at arbitrary resolutions, aspect ratios, and durations, yet it still lacks sufficient implementation details.

Rho-1: Not All Tokens Are What You Need

3 code implementations11 Apr 2024 Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, Weizhu Chen

After fine-tuning, Rho-1-1B and 7B achieved state-of-the-art results of 40. 6% and 51. 8% on MATH dataset, respectively - matching DeepSeekMath with only 3% of the pretraining tokens.

Continual Pretraining Language Modelling +1

Measuring Gender and Racial Biases in Large Language Models

no code implementations22 Mar 2024 Jiafu An, Difang Huang, Chen Lin, Mingzhu Tai

As AI based decision making tools are increasingly employed across diverse domains, our findings underscore the necessity of understanding and addressing the potential unequal outcomes to ensure equitable outcomes across social groups.

Decision Making Language Modelling +1

Ensuring Safe and High-Quality Outputs: A Guideline Library Approach for Language Models

1 code implementation18 Mar 2024 Yi Luo, Zhenghao Lin, Yuhao Zhang, Jiashuo Sun, Chen Lin, Chengjin Xu, Xiangdong Su, Yelong Shen, Jian Guo, Yeyun Gong

Subsequently, the retrieval model correlates new inputs with relevant guidelines, which guide LLMs in response generation to ensure safe and high-quality outputs, thereby aligning with human values.

Response Generation Retrieval

LOCR: Location-Guided Transformer for Optical Character Recognition

no code implementations4 Mar 2024 Yu Sun, Dongzhan Zhou, Chen Lin, Conghui He, Wanli Ouyang, Han-sen Zhong

Academic documents are packed with texts, equations, tables, and figures, requiring comprehensive understanding for accurate Optical Character Recognition (OCR).

Marketing Optical Character Recognition +1

Self-consistent Validation for Machine Learning Electronic Structure

no code implementations15 Feb 2024 Gengyuan Hu, Gengchen Wei, Zekun Lou, Philip H. S. Torr, Wanli Ouyang, Han-sen Zhong, Chen Lin

Machine learning has emerged as a significant approach to efficiently tackle electronic structure problems.

Active Learning

Revealing Decurve Flows for Generalized Graph Propagation

no code implementations13 Feb 2024 Chen Lin, Liheng Ma, Yiyang Chen, Wanli Ouyang, Michael M. Bronstein, Philip H. S. Torr

\textbf{Secondly}, we propose the {\em Continuous Unified Ricci Curvature} (\textbf{CURC}), an extension of celebrated {\em Ollivier-Ricci Curvature} for directed and weighted graphs.

Graph Learning

Competition-Level Problems are Effective LLM Evaluators

no code implementations4 Dec 2023 Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen

Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about these abilities and the potential data contamination problem recently.

GENET: Unleashing the Power of Side Information for Recommendation via Hypergraph Pre-training

no code implementations22 Nov 2023 Yang Li, Qi'ao Zhao, Chen Lin, Zhenjie Zhang, Xiaomin Zhu

(2) The diverse semantics of side information that describes items and users from multi-level in a context different from recommendation systems.

Sequential Recommendation

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

3 code implementations15 Jul 2023 Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, Heung-Yeung Shum, Jian Guo

Although large language models (LLMs) have achieved significant success in various tasks, they often struggle with hallucination problems, especially in scenarios requiring deep and responsible reasoning.

Hallucination Knowledge Graphs +2

Graph Inductive Biases in Transformers without Message Passing

1 code implementation27 May 2023 Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriano, Puneet K. Dokania, Mark Coates, Philip Torr, Ser-Nam Lim

Graph inductive biases are crucial for Graph Transformers, and previous works incorporate them using message-passing modules and/or positional encodings.

Graph Classification Graph Regression +2

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models

1 code implementation23 Apr 2023 Jiashuo Sun, Yi Luo, Yeyun Gong, Chen Lin, Yelong Shen, Jian Guo, Nan Duan

By utilizing iterative bootstrapping, our approach enables LLMs to autonomously rectify errors, resulting in more precise and comprehensive reasoning chains.

AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators

2 code implementations29 Mar 2023 Xingwei He, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, Weizhu Chen

Many natural language processing (NLP) tasks rely on labeled data to train machine learning models with high performance.

Information Retrieval Retrieval

Algebraically Rigorous Quaternion Framework for the Neural Network Pose Estimation Problem

1 code implementation ICCV 2023 Chen Lin, Andrew J. Hanson, Sonya M. Hanson

The 3D pose estimation problem -- aligning pairs of noisy 3D point clouds -- is a problem with a wide variety of real-world applications.

3D Pose Estimation

OSAN: A One-Stage Alignment Network To Unify Multimodal Alignment and Unsupervised Domain Adaptation

no code implementations CVPR 2023 Ye Liu, Lingfeng Qiao, Changchong Lu, Di Yin, Chen Lin, Haoyuan Peng, Bo Ren

An intuitive way to handle these two problems is to fulfill these tasks in two separate stages: aligning modalities followed by domain adaptation, or vice versa.

Unsupervised Domain Adaptation

Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise

1 code implementation22 Dec 2022 Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, Weizhu Chen

In this paper, we introduce a novel dIffusion language modEl pre-training framework for text generation, which we call GENIE.

Decoder Denoising +2

APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning

2 code implementations14 Dec 2022 Jiashuo Sun, Hang Zhang, Chen Lin, Xiangdong Su, Yeyun Gong, Jian Guo

For the retriever, we adopt a number-aware negative sampling strategy to enable the retriever to be more discriminative on key numerical facts.

Conversational Question Answering

Unsupervised Extractive Summarization with Heterogeneous Graph Embeddings for Chinese Document

no code implementations9 Nov 2022 Chen Lin, Ye Liu, Siyu An, Di Yin

In the scenario of unsupervised extractive summarization, learning high-quality sentence representations is essential to select salient sentences from the input document.

Extractive Summarization Sentence +2

Detecting Elevated Air Pollution Levels by Monitoring Web Search Queries: Deep Learning-Based Time Series Forecasting

no code implementations9 Nov 2022 Chen Lin, Safoora Yousefi, Elvis Kahoro, Payam Karisani, Donghai Liang, Jeremy Sarnat, Eugene Agichtein

Most of the prior work relied on modeling pollutant concentrations collected from ground-based monitors and meteorological data for long-term forecasting of outdoor ozone, oxides of nitrogen, and PM2. 5.

Time Series Time Series Forecasting

Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis

1 code implementation18 Oct 2022 Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, Jian Guo, Nan Duan

Most existing pre-trained language representation models (PLMs) are sub-optimal in sentiment analysis tasks, as they capture the sentiment information from word-level while under-considering sentence-level information.

Contrastive Learning Language Modelling +3

PROD: Progressive Distillation for Dense Retrieval

1 code implementation27 Sep 2022 Zhenghao Lin, Yeyun Gong, Xiao Liu, Hang Zhang, Chen Lin, Anlei Dong, Jian Jiao, Jingwen Lu, Daxin Jiang, Rangan Majumder, Nan Duan

It is common that a better teacher model results in a bad student via distillation due to the nonnegligible gap between teacher and student.

Knowledge Distillation Natural Questions +1

SuperVessel: Segmenting High-resolution Vessel from Low-resolution Retinal Image

1 code implementation28 Jul 2022 Yan Hu, Zhongxi Qiu, Dan Zeng, Li Jiang, Chen Lin, Jiang Liu

Vascular segmentation extracts blood vessels from images and serves as the basis for diagnosing various diseases, like ophthalmic diseases.

Medical Image Segmentation Segmentation +1

Contrastive Graph Multimodal Model for Text Classification in Videos

no code implementations6 Jun 2022 Ye Liu, Changchong Lu, Chen Lin, Di Yin, Bo Ren

However, to our knowledge, there is no existing work focused on the second step of video text classification, which will limit the guidance to downstream tasks such as video indexing and browsing.

Contrastive Learning Optical Character Recognition (OCR) +2

UWC: Unit-wise Calibration Towards Rapid Network Compression

no code implementations17 Jan 2022 Chen Lin, Zheyang Li, Bo Peng, Haoji Hu, Wenming Tan, Ye Ren, ShiLiang Pu

This paper introduces a post-training quantization~(PTQ) method achieving highly efficient Convolutional Neural Network~ (CNN) quantization with high performance.


A Continuous Mapping For Augmentation Design

no code implementations NeurIPS 2021 Keyu Tian, Chen Lin, Ser Nam Lim, Wanli Ouyang, Puneet Dokania, Philip Torr

Automated data augmentation (ADA) techniques have played an important role in boosting the performance of deep models.

Data Augmentation

BN-NAS: Neural Architecture Search with Batch Normalization

1 code implementation ICCV 2021 BoYu Chen, Peixia Li, Baopu Li, Chen Lin, Chuming Li, Ming Sun, Junjie Yan, Wanli Ouyang

We present BN-NAS, neural architecture search with Batch Normalization (BN-NAS), to accelerate neural architecture search (NAS).

Neural Architecture Search

PSViT: Better Vision Transformer via Token Pooling and Attention Sharing

no code implementations7 Aug 2021 BoYu Chen, Peixia Li, Baopu Li, Chuming Li, Lei Bai, Chen Lin, Ming Sun, Junjie Yan, Wanli Ouyang

Then, a compact set of the possible combinations for different token pooling and attention sharing mechanisms are constructed.

Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting

1 code implementation14 Mar 2021 Chen Lin, Zhichao Ouyang, Junqing Zhuang, Jianqiang Chen, Hui Li, Rongxin Wu

In this paper, we present the Block-wise Abstract Syntax Tree Splitting method (BASTS for short), which fully utilizes the rich tree-form syntax structure in ASTs, for improving code summarization.

Code Summarization Source Code Summarization

Sequential Recommendation in Online Games with Multiple Sequences, Tasks and User Levels

no code implementations13 Feb 2021 Si Chen, Yuqiu Qian, Hui Li, Chen Lin

We leverage Graph Neural Network and multi-task learning to design M$^3$Rec in order to model the complex information in the heterogeneous sequential recommendation scenario of Tencent Games.

Multi-Task Learning Sequential Recommendation

Inception Convolution with Efficient Dilation Search

1 code implementation CVPR 2021 Jie Liu, Chuming Li, Feng Liang, Chen Lin, Ming Sun, Junjie Yan, Wanli Ouyang, Dong Xu

To develop a practical method for learning complex inception convolution based on the data, a simple but effective search algorithm, referred to as efficient dilation optimization (EDO), is developed.

Human Detection Instance Segmentation +4

DETR for Crowd Pedestrian Detection

1 code implementation12 Dec 2020 Matthieu Lin, Chuming Li, Xingyuan Bu, Ming Sun, Chen Lin, Junjie Yan, Wanli Ouyang, Zhidong Deng

Furthermore, the bipartite match of ED harms the training efficiency due to the large ground truth number in crowd scenes.

Decoder Pedestrian Detection

PV-NAS: Practical Neural Architecture Search for Video Recognition

no code implementations2 Nov 2020 ZiHao Wang, Chen Lin, Lu Sheng, Junjie Yan, Jing Shao

Recently, deep learning has been utilized to solve video recognition problem due to its prominent representation ability.

Neural Architecture Search Video Recognition

Adaptive Gradient Method with Resilience and Momentum

no code implementations21 Oct 2020 Jie Liu, Chen Lin, Chuming Li, Lu Sheng, Ming Sun, Junjie Yan, Wanli Ouyang

Several variants of stochastic gradient descent (SGD) have been proposed to improve the learning effectiveness and efficiency when training deep neural networks, among which some recent influential attempts would like to adaptively control the parameter-wise learning rate (e. g., Adam and RMSProp).

Improving Auto-Augment via Augmentation-Wise Weight Sharing

1 code implementation NeurIPS 2020 Keyu Tian, Chen Lin, Ming Sun, Luping Zhou, Junjie Yan, Wanli Ouyang

On CIFAR-10, we achieve a top-1 error rate of 1. 24%, which is currently the best performing single model without extra training data.

Powering One-shot Topological NAS with Stabilized Share-parameter Proxy

no code implementations ECCV 2020 Ronghao Guo, Chen Lin, Chuming Li, Keyu Tian, Ming Sun, Lu Sheng, Junjie Yan

Specifically, the difficulties for architecture searching in such a complex space has been eliminated by the proposed stabilized share-parameter proxy, which employs Stochastic Gradient Langevin Dynamics to enable fast shared parameter sampling, so as to achieve stabilized measurement of architecture performance even in search space with complex topological structures.

Neural Architecture Search

Computation Reallocation for Object Detection

no code implementations ICLR 2020 Feng Liang, Chen Lin, Ronghao Guo, Ming Sun, Wei Wu, Junjie Yan, Wanli Ouyang

However, classification allocation pattern is usually adopted directly to object detector, which is proved to be sub-optimal.

Instance Segmentation Neural Architecture Search +4

FLEN: Leveraging Field for Scalable CTR Prediction

7 code implementations12 Nov 2019 Wenqiang Chen, Lizhang Zhan, Yuanlong Ci, Minghua Yang, Chen Lin, Dugang Liu

By suitably exploiting field information, the field-wise bi-interaction pooling captures both inter-field and intra-field feature conjunctions with a small number of model parameters and an acceptable time complexity for industrial applications.

Click-Through Rate Prediction Recommendation Systems

Improving One-shot NAS by Suppressing the Posterior Fading

no code implementations CVPR 2020 Xiang Li, Chen Lin, Chuming Li, Ming Sun, Wei Wu, Junjie Yan, Wanli Ouyang

In this paper, we analyse existing weight sharing one-shot NAS approaches from a Bayesian point of view and identify the posterior fading problem, which compromises the effectiveness of shared weights.

Neural Architecture Search object-detection +2

AM-LFS: AutoML for Loss Function Search

1 code implementation ICCV 2019 Chuming Li, Yuan Xin, Chen Lin, Minghao Guo, Wei Wu, Wanli Ouyang, Junjie Yan

The key contribution of this work is the design of search space which can guarantee the generalization and transferability on different vision tasks by including a bunch of existing prevailing loss functions in a unified formulation.


Synaptic Strength For Convolutional Neural Network

no code implementations NeurIPS 2018 Chen Lin, Zhao Zhong, Wei Wu, Junjie Yan

Inspired by the relevant concept in neural science literature, we propose Synaptic Pruning: a data-driven method to prune connections between input and output feature maps with a newly proposed class of parameters called Synaptic Strength.

Neural Temporal Relation Extraction

no code implementations EACL 2017 Dmitriy Dligach, Timothy Miller, Chen Lin, Steven Bethard, Guergana Savova

We experiment with neural architectures for temporal relation extraction and establish a new state-of-the-art for several scenarios.

Position Relation +3

Cannot find the paper you are looking for? You can Submit a new open access paper.