Search Results for author: Yanlin Wang

Found 40 papers, 20 papers with code

SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks

1 code implementation12 Jun 2025 Lianghong Guo, Yanlin Wang, Caihua Li, Pengyu Yang, Jiachi Chen, Wei Tao, Yingtian Zou, Duyu Tang, Zibin Zheng

Constructing large-scale datasets for the GitHub issue resolution task is crucial for both training and evaluating the software engineering capabilities of Large Language Models (LLMs).

GitHub issue resolution valid

Towards an Understanding of Context Utilization in Code Intelligence

no code implementations11 Apr 2025 Yanlin Wang, Kefeng Duan, Dewu Zheng, Ensheng Shi, Fengji Zhang, Yanli Wang, Jiachi Chen, Xilin Liu, Yuchi Ma, Hongyu Zhang, Qianxiang Wang, Zibin Zheng

(1) A quantitative analysis of the research landscape, including publication trends, venues, and the explored domains; (2) A novel taxonomy of context types used in code intelligence; (3) A task-oriented analysis investigating context integration strategies across diverse code intelligence tasks; (4) A critical evaluation of evaluation methodologies for context-aware methods.

RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation

1 code implementation21 Mar 2025 Linxi Liang, Jing Gong, Mingwei Liu, Chong Wang, Guangsheng Ou, Yanlin Wang, Xin Peng, Zibin Zheng

To address this gap, we present RustEvo, a novel framework for constructing dynamic benchmarks that evaluate the ability of LLMs to adapt to evolving Rust APIs.

Code Generation Navigate +2

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark

2 code implementations24 Dec 2024 Dewu Zheng, Yanlin Wang, Ensheng Shi, Xilin Liu, Yuchi Ma, Hongyu Zhang, Zibin Zheng

With the rapid advancement of large language models (LLMs), extensive research has been conducted to investigate the code generation capabilities of LLMs.

Code Generation Dependency Parsing

RepoTransBench: A Real-World Benchmark for Repository-Level Code Translation

no code implementations23 Dec 2024 Yanli Wang, Yanlin Wang, Suiquan Wang, Daya Guo, Jiachi Chen, John Grundy, Xilin Liu, Yuchi Ma, Mingzhi Mao, Hongyu Zhang, Zibin Zheng

However, even with this improvement, the Success@1 score of the best-performing LLM is only 21%, which may not meet the need for reliable automatic repository-level code translation.

Code Translation Translation

LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation

1 code implementation30 Sep 2024 Ziyao Zhang, Yanlin Wang, Chong Wang, Jiachi Chen, Zibin Zheng

In this paper, we conduct an empirical study to study the phenomena, mechanism, and mitigation of LLM hallucinations within more practical and complex development contexts in repository-level generation scenario.

Code Generation Hallucination +1

RMCBench: Benchmarking Large Language Models' Resistance to Malicious Code

1 code implementation23 Sep 2024 Jiachi Chen, Qingyuan Zhong, Yanlin Wang, Kaiwen Ning, Yongkun Liu, Zenan Xu, Zhe Zhao, Ting Chen, Zibin Zheng

Despite their benefits, LLMs also pose notable risks, including the potential to generate harmful content and being abused by malicious developers to create malicious code.

Benchmarking Code Generation

Agents in Software Engineering: Survey, Landscape, and Vision

1 code implementation13 Sep 2024 Yanlin Wang, Wanjun Zhong, Yanxian Huang, Ensheng Shi, Min Yang, Jiachi Chen, Hui Li, Yuchi Ma, Qianxiang Wang, Zibin Zheng

In recent years, Large Language Models (LLMs) have achieved remarkable success and have been widely used in various downstream tasks, especially in the tasks of the software engineering (SE) field.

Survey

Knowledge Probing for Graph Representation Learning

no code implementations7 Aug 2024 Mingyu Zhao, Xingyu Huang, Ziyu Lyu, Yanlin Wang, Lixin Cui, Lu Bai

Based on the intrinsic properties of graphs, we design three probes to systematically investigate the graph representation learning process from different perspectives, respectively the node-wise level, the path-wise level, and the structural level.

Graph Classification Graph Learning +4

Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models

no code implementations29 Jun 2024 Yanlin Wang, Tianyue Jiang, Mingwei Liu, Jiachi Chen, Zibin Zheng

In this paper, we empirically analyze the differences in coding style between the code generated by mainstream Code LLMs and the code written by human developers, and summarize coding style inconsistency taxonomy.

Code Generation

CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agents

1 code implementation17 Jun 2024 Jing Gong, Yanghui Wu, Linxi Liang, Yanlin Wang, Jiachi Chen, Mingwei Liu, Zibin Zheng

Semantic code search, retrieving code that matches a given natural language query, is an important task to improve productivity in software engineering.

Code Generation Code Search +4

MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution

no code implementations26 Mar 2024 Wei Tao, Yucheng Zhou, Yanlin Wang, Wenqiang Zhang, Hongyu Zhang, Yu Cheng

To overcome this challenge, we empirically study the reason why LLMs fail to resolve GitHub issues and analyze the major factors.

GitHub issue resolution

YODA: Teacher-Student Progressive Learning for Language Models

no code implementations28 Jan 2024 Jianqiao Lu, Wanjun Zhong, YuFei Wang, Zhijiang Guo, Qi Zhu, Wenyong Huang, Yanlin Wang, Fei Mi, Baojun Wang, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu

With the teacher's guidance, the student learns to iteratively refine its answer with feedback, and forms a robust and comprehensive understanding of the posed questions.

GSM8K Math

KADEL: Knowledge-Aware Denoising Learning for Commit Message Generation

1 code implementation16 Jan 2024 Wei Tao, Yucheng Zhou, Yanlin Wang, Hongyu Zhang, Haofen Wang, Wenqiang Zhang

However, previous methods are trained on the entire dataset without considering the fact that a portion of commit messages adhere to good practice (i. e., good-practice commits), while the rest do not.

Denoising

Code Search Debiasing:Improve Search Results beyond Overall Ranking Performance

no code implementations25 Nov 2023 Sheng Zhang, Hui Li, Yanlin Wang, Zhao Wei, Yong Xiu, Juhong Wang, Rongong Ji

To mitigate biases, we develop a general debiasing framework that employs reranking to calibrate search results.

Code Search Reranking

Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoning

1 code implementation1 Oct 2023 Jianpeng Zhou, Wanjun Zhong, Yanlin Wang, Jiahai Wang

These methods employ consistent models, sample sizes, prompting methods and levels of problem decomposition, regardless of the problem complexity.

Computational Efficiency Language Modeling +3

Modeling Orders of User Behaviors via Differentiable Sorting: A Multi-task Framework to Predicting User Post-click Conversion

no code implementations18 Jul 2023 Menghan Wang, Jinming Yang, Yuchen Guo, Yuming Shen, Mengying Zhu, Yanlin Wang

Inspired by recent advances on differentiable sorting, in this paper, we propose a novel multi-task framework that leverages orders of user behaviors to predict user post-click conversion in an end-to-end approach.

Multi-Task Learning Selection bias

MemoryBank: Enhancing Large Language Models with Long-Term Memory

1 code implementation17 May 2023 Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, Yanlin Wang

To mimic anthropomorphic behaviors and selectively preserve memory, MemoryBank incorporates a memory updating mechanism, inspired by the Ebbinghaus Forgetting Curve theory, which permits the AI to forget and reinforce memory based on time elapsed and the relative significance of the memory, thereby offering a human-like memory mechanism.

Chatbot

AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

3 code implementations13 Apr 2023 Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, Nan Duan

Impressively, GPT-4 surpasses average human performance on SAT, LSAT, and math competitions, attaining a 95% accuracy rate on the SAT Math test and a 92. 5% accuracy on the English test of the Chinese national college entrance exam.

Decision Making Math

Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond

1 code implementation11 Apr 2023 Ensheng Shi, Yanlin Wang, Hongyu Zhang, Lun Du, Shi Han, Dongmei Zhang, Hongbin Sun

Our experimental study shows that (1) lexical, syntactic and structural properties of source code are encoded in the lower, intermediate, and higher layers, respectively, while the semantic property spans across the entire model.

software testing

Exploring Representation-Level Augmentation for Code Search

1 code implementation21 Oct 2022 Haochen Li, Chunyan Miao, Cyril Leung, Yanxian Huang, Yuan Huang, Hongyu Zhang, Yanlin Wang

In this paper, we explore augmentation methods that augment data (both code and query) at representation level which does not require additional data processing and training, and based on this we propose a general format of representation-level augmentation that unifies existing methods.

Code Search Contrastive Learning +1

Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing

no code implementations4 Oct 2022 Lunyiu Nie, Jiuding Sun, Yanlin Wang, Lun Du, Lei Hou, Juanzi Li, Shi Han, Dongmei Zhang, Jidong Zhai

The recent prevalence of pretrained language models (PLMs) has dramatically shifted the paradigm of semantic parsing, where the mapping from natural language utterances to structured logical forms is now formulated as a Seq2Seq task.

Decoder Hallucination +2

MM-GNN: Mix-Moment Graph Neural Network towards Modeling Neighborhood Feature Distribution

1 code implementation15 Aug 2022 Wendong Bi, Lun Du, Qiang Fu, Yanlin Wang, Shi Han, Dongmei Zhang

Graph Neural Networks (GNNs) have shown expressive performance on graph representation learning by aggregating information from neighbors.

Graph Neural Network Graph Representation Learning +1

Meta-data Study in Autism Spectrum Disorder Classification Based on Structural MRI

no code implementations9 Jun 2022 Ruimin Ma, Yanlin Wang, Yanjie Wei, Yi Pan

Accurate diagnosis of autism spectrum disorder (ASD) based on neuroimaging data has significant implications, as extracting useful information from neuroimaging data for ASD detection is challenging.

PrivateRec: Differentially Private Training and Serving for Federated News Recommendation

no code implementations18 Apr 2022 Ruixuan Liu, Yanlin Wang, Yang Cao, Lingjuan Lyu, Weike Pan, Yun Chen, Hong Chen

Collecting and training over sensitive personal data raise severe privacy concerns in personalized recommendation systems, and federated learning can potentially alleviate the problem by training models over decentralized user data. However, a theoretically private solution in both the training and serving stages of federated recommendation is essential but still lacking. Furthermore, naively applying differential privacy (DP) to the two stages in federated recommendation would fail to achieve a satisfactory trade-off between privacy and utility due to the high-dimensional characteristics of model gradients and hidden representations. In this work, we propose a federated news recommendation method for achieving a better utility in model training and online serving under a DP guarantee. We first clarify the DP definition over behavior data for each round in the life-circle of federated recommendation systems. Next, we propose a privacy-preserving online serving mechanism under this definition based on the idea of decomposing user embeddings with public basic vectors and perturbing the lower-dimensional combination coefficients.

Federated Learning News Recommendation +2

UniXcoder: Unified Cross-Modal Pre-training for Code Representation

2 code implementations ACL 2022 Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, Jian Yin

Furthermore, we propose to utilize multi-modal contents to learn representation of code fragment with contrastive learning, and then align representations among programming languages using a cross-modal generation task.

Code Completion Code Search +2

No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices

no code implementations16 Feb 2022 Ruixuan Liu, Fangzhao Wu, Chuhan Wu, Yanlin Wang, Lingjuan Lyu, Hong Chen, Xing Xie

In this way, all the clients can participate in the model learning in FL, and the final model can be big and powerful enough.

Federated Learning Knowledge Distillation +1

Game of Privacy: Towards Better Federated Platform Collaboration under Privacy Restriction

no code implementations10 Feb 2022 Chuhan Wu, Fangzhao Wu, Tao Qi, Yanlin Wang, Yuqing Yang, Yongfeng Huang, Xing Xie

To solve the game, we propose a platform negotiation method that simulates the bargaining among platforms and locally optimizes their policies via gradient descent.

Vertical Federated Learning

On the Evaluation of Neural Code Summarization

1 code implementation15 Jul 2021 Ensheng Shi, Yanlin Wang, Lun Du, Junjie Chen, Shi Han, Hongyu Zhang, Dongmei Zhang, Hongbin Sun

To achieve a profound understanding of how far we are from solving this problem and provide suggestions to future research, in this paper, we conduct a systematic and in-depth analysis of 5 state-of-the-art neural code summarization models on 6 widely used BLEU variants, 4 pre-processing operations and their combinations, and 3 widely used datasets.

Code Summarization Source Code Summarization

On the Evaluation of Commit Message Generation Models: An Experimental Study

1 code implementation12 Jul 2021 Wei Tao, Yanlin Wang, Ensheng Shi, Lun Du, Shi Han, Hongyu Zhang, Dongmei Zhang, Wenqiang Zhang

We find that: (1) Different variants of the BLEU metric are used in previous works, which affects the evaluation and understanding of existing methods.

Retrieval

Is a Single Model Enough? MuCoS: A Multi-Model Ensemble Learning for Semantic Code Search

1 code implementation10 Jul 2021 Lun Du, Xiaozhou Shi, Yanlin Wang, Ensheng Shi, Shi Han, Dongmei Zhang

On the other hand, as a specific query may focus on one or several perspectives, it is difficult for a single query representation module to represent different user intents.

Code Search Data Augmentation +2

Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs

no code implementations17 Mar 2021 Yanlin Wang, Hui Li

Code completion has become an essential component of integrated development environments.

Code Completion Graph Attention +2

Cannot find the paper you are looking for? You can Submit a new open access paper.