Search Results for author: Wentao Zhang

Found 129 papers, 69 papers with code

Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning

no code implementations26 Nov 2024 Xinyi Gao, Yayong Li, Tong Chen, Guanhua Ye, Wentao Zhang, Hongzhi Yin

With the increasing computation of training graph neural networks (GNNs) on large-scale graphs, graph condensation (GC) has emerged as a promising solution to synthesize a compact, substitute graph of the large-scale original graph for efficient GNN training.

Graph Generation Self-Supervised Learning

Can LLMs be Good Graph Judger for Knowledge Graph Construction?

1 code implementation26 Nov 2024 Haoyu Huang, Chong Chen, Conghui He, Yang Li, Jiawei Jiang, Wentao Zhang

We seek to utilize the capacity of LLMs to function as a graph judger, a capability superior to their role only as a predictor for KG construction problems.

Denoising graph construction +3

Beyond Sight: Towards Cognitive Alignment in LVLM via Enriched Visual Knowledge

no code implementations25 Nov 2024 Yaqi Zhao, Yuanyang Yin, Lin Li, MingAn Lin, Victor Shea-Jay Huang, Siwei Chen, WeiPeng Chen, Baoqun Yin, Zenan Zhou, Wentao Zhang

Specifically, the VE's representation of visual information may not fully align with LLM's cognitive framework, leading to a mismatch where visual features exceed the language model's interpretive range.

Landmark Recognition Large Language Model

Towards Precise Scaling Laws for Video Diffusion Transformers

no code implementations25 Nov 2024 Yuanyang Yin, Yaqi Zhao, Mingwu Zheng, Ke Lin, Jiarong Ou, Rui Chen, Victor Shea-Jay Huang, Jiahao Wang, Xin Tao, Pengfei Wan, Di Zhang, Baoqun Yin, Wentao Zhang, Kun Gai

Achieving optimal performance of video diffusion transformers within given data and compute budget is crucial due to their high training costs.

Epidemiology-informed Network for Robust Rumor Detection

no code implementations20 Nov 2024 Wei Jiang, Tong Chen, Xinyi Gao, Wentao Zhang, Lizhen Cui, Hongzhi Yin

Given the variations in topics and social impact of the root node, different source information naturally has distinct outreach capabilities, resulting in different heights of propagation trees.

Epidemiology

Toward Personalized Federated Node Classification in One-shot Communication

no code implementations18 Nov 2024 Guochen Yan, Xunkai Li, Luyuan Xie, Wentao Zhang, Qingni Shen, Yuejian Fang, Zhonghai Wu

Specifically, for effective graph learning in a single communication round, our method estimates and aggregates class-wise feature distribution statistics to construct a global pseudo-graph on the server, facilitating the training of a global graph model.

Federated Learning Graph Learning +1

MC-LLaVA: Multi-Concept Personalized Vision-Language Model

1 code implementation18 Nov 2024 Ruichuan An, Sihan Yang, Ming Lu, Kai Zeng, Yulin Luo, Ying Chen, Jiajun Cao, Hao Liang, Qi She, Shanghang Zhang, Wentao Zhang

Specifically, MC-LLaVA uses a joint training strategy incorporating multiple concepts in a single training step, allowing VLMs to perform accurately in multi-concept personalization.

Language Modelling Question Answering +1

VersaTune: An Efficient Data Composition Framework for Training Multi-Capability LLMs

no code implementations18 Nov 2024 Keer Lu, Keshi Zhao, Zheng Liang, Da Pan, Shusen Zhang, Xin Wu, WeiPeng Chen, Zenan Zhou, Guosheng Dong, Bin Cui, Wentao Zhang

Despite their potential, existing work mainly focuses on domain-specific enhancements during fine-tuning, the challenge of which lies in catastrophic forgetting of knowledge across other domains.

EVQAScore: Efficient Video Question Answering Data Evaluation

no code implementations11 Nov 2024 Hao Liang, Zirong Chen, Wentao Zhang

To address this gap, we introduce EVQAScore, a reference-free method that leverages keyword extraction to assess both video caption and video QA data quality.

Keyword Extraction Question Answering +2

RAGraph: A General Retrieval-Augmented Graph Learning Framework

no code implementations31 Oct 2024 Xinke Jiang, Rihong Qiu, Yongxin Xu, Wentao Zhang, Yichen Zhu, Ruizhe Zhang, Yuchen Fang, Xu Chu, Junfeng Zhao, Yasha Wang

Graph Neural Networks (GNNs) have become essential in interpreting relational data across various domains, yet, they often struggle to generalize to unseen graph data that differs markedly from training instances.

Graph Classification Graph Learning +3

VecCity: A Taxonomy-guided Library for Map Entity Representation Learning

1 code implementation31 Oct 2024 Wentao Zhang, Jingyuan Wang, Yifan Yang, Leong Hou U

First, existing research is fragmented, with models classified by the type of map entity, limiting the reusability of techniques across different tasks.

Representation Learning

Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction

no code implementations28 Oct 2024 Qintong Zhang, Victor Shea-Jay Huang, Bin Wang, Junyuan Zhang, Zhengren Wang, Hao Liang, Shawn Wang, Matthieu Lin, Conghui He, Wentao Zhang

Document parsing is essential for converting unstructured and semi-structured documents-such as contracts, academic papers, and invoices-into structured, machine-readable data.

Data Integration Knowledge Base Construction

Nova: A Practical and Advanced Alignment

no code implementations19 Oct 2024 MingAn Lin, Fan Yang, Yanjun Shen, Haoze Sun, Tianpeng Li, Chenzheng Zhu, Tao Zhang, Miao Zheng, Xu Li, Yijie Zhou, Mingyang Chen, Yanzhao Qin, Youquan Li, Hao Liang, Fei Li, Yadong Li, Mang Wang, Guosheng Dong, Kun Fang, Jianhua Xu, Bin Cui, Wentao Zhang, Zenan Zhou, WeiPeng Chen

Importantly, Qwen2-Nova-72B and Llama3-PBM-Nova-70B are instruct versions of the Qwen2-72B and Llama-3-70B base models, optimized through Nova.

Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning

1 code implementation16 Oct 2024 Mingyang Chen, Haoze Sun, Tianpeng Li, Fan Yang, Hao Liang, Keer Lu, Bin Cui, Wentao Zhang, Zenan Zhou, WeiPeng Chen

While current research on function calling by LLMs primarily focuses on single-turn interactions, this paper addresses the overlooked necessity for LLMs to engage in multi-turn function calling--critical for handling compositional, real-world queries that require planning with functions but not only use functions.

8k

Semantic Score Distillation Sampling for Compositional Text-to-3D Generation

1 code implementation11 Oct 2024 Ling Yang, Zixiang Zhang, Junlin Han, Bohan Zeng, Runjia Li, Philip Torr, Wentao Zhang

To overcome these challenges, we introduce a novel SDS approach, Semantic Score Distillation Sampling (SemanticSDS), designed to effectively improve the expressiveness and accuracy of compositional text-to-3D generation.

3D Generation Text to 3D

Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis

1 code implementation9 Oct 2024 Bohan Zeng, Ling Yang, Siyu Li, Jiaming Liu, Zixiang Zhang, Juanxi Tian, Kaixin Zhu, Yongzhen Guo, Fu-Yun Wang, Minkai Xu, Stefano Ermon, Wentao Zhang

Then we propose a geometry-aware 4D transition network to realize a complex scene-level 4D transition based on the plan, which involves expressive geometrical object deformation.

Video Generation

Universal Medical Image Representation Learning with Compositional Decoders

no code implementations30 Sep 2024 Kaini Wang, Ling Yang, Siping Zhou, Guangquan Zhou, Wentao Zhang, Bin Cui, Shuo Li

To address these challenges, we have developed a decomposed-composed universal medical imaging paradigm (UniMed) that supports tasks at all levels.

Decoder Representation Learning

Data Proportion Detection for Optimized Data Management for Large Language Models

1 code implementation26 Sep 2024 Hao Liang, Keshi Zhao, Yajie Yang, Bin Cui, Guosheng Dong, Zenan Zhou, Wentao Zhang

Large language models (LLMs) have demonstrated exceptional performance across a wide range of tasks and domains, with data preparation playing a critical role in achieving these results.

Management

DataSculpt: Crafting Data Landscapes for Long-Context LLMs through Multi-Objective Partitioning

1 code implementation2 Sep 2024 Keer Lu, Xiaonan Nie, Zheng Liang, Da Pan, Shusen Zhang, Keshi Zhao, WeiPeng Chen, Zenan Zhou, Guosheng Dong, Bin Cui, Wentao Zhang

Through extensive experimental analysis, we identified three key challenges in designing effective data management strategies that enable the model to achieve long-context capability without sacrificing performance in other tasks: (1) a shortage of long documents across multiple domains, (2) effective construction of context windows, and (3) efficient organization of large-scale datasets.

Code Completion Combinatorial Optimization +5

OpenFGL: A Comprehensive Benchmarks for Federated Graph Learning

1 code implementation29 Aug 2024 Xunkai Li, Yinlin Zhu, Boyang Pang, Guochen Yan, Yeyu Yan, Zening Li, Zhengyu Wu, Wentao Zhang, Rong-Hua Li, Guoren Wang

Federated graph learning (FGL) has emerged as a promising distributed training paradigm for graph neural networks across multiple local systems without direct data sharing.

Graph Learning

SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs

no code implementations21 Aug 2024 Yuanyang Yin, Yaqi Zhao, YaJie Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Di Zhang, Baoqun Yin, Wentao Zhang

Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable perceptual and reasoning abilities, typically comprising a Vision Encoder, an Adapter, and a Large Language Model (LLM).

Contrastive Learning Language Modelling +2

RAG-Optimized Tibetan Tourism LLMs: Enhancing Accuracy and Personalization

no code implementations21 Aug 2024 Jinhu Qi, Shuai Yan, Yibo Zhang, Wentao Zhang, Rong Jin, Yuwei Hu, Ke Wang

With the development of the modern social economy, tourism has become an important way to meet people's spiritual needs, bringing development opportunities to the tourism industry.

Hallucination RAG +1

SysBench: Can Large Language Models Follow System Messages?

1 code implementation20 Aug 2024 Yanzhao Qin, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, WeiPeng Chen, Zenan Zhou, Wentao Zhang, Bin Cui

Finally, we conduct extensive evaluation across various existing LLMs, measuring their ability to follow specified constraints given in system messages.

TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systems

1 code implementation17 Aug 2024 Xinke Jiang, Yue Fang, Rihong Qiu, Haoyu Zhang, Yongxin Xu, Hao Chen, Wentao Zhang, Ruizhe Zhang, Yuchen Fang, Xu Chu, Junfeng Zhao, Yasha Wang

In the pursuit of enhancing domain-specific Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) emerges as a promising solution to mitigate issues such as hallucinations, outdated knowledge, and limited expertise in highly specialized queries.

RAG Retrieval

CFBench: A Comprehensive Constraints-Following Benchmark for LLMs

1 code implementation2 Aug 2024 Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, MingAn Lin, Yujing Qiao, WeiPeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou

The adeptness of Large Language Models (LLMs) in comprehending and following natural language instructions is critical for their deployment in sophisticated real-world applications.

Are Bigger Encoders Always Better in Vision Large Models?

no code implementations1 Aug 2024 Bozhou Li, Hao Liang, Zimo Meng, Wentao Zhang

Moreover, we analyzed the effects of LLM backbone parameter size and data quality on the pretraining outcomes.

Language Modelling Large Language Model

Synth-Empathy: Towards High-Quality Synthetic Empathy Data

1 code implementation31 Jul 2024 Hao Liang, Linzhuang Sun, Jingxuan Wei, Xijie Huang, Linkun Sun, Bihui Yu, Conghui He, Wentao Zhang

In recent years, with the rapid advancements in large language models (LLMs), achieving excellent empathetic response capabilities has become a crucial prerequisite.

Diversity

SynthVLM: High-Efficiency and High-Quality Synthetic Data for Vision Language Models

1 code implementation30 Jul 2024 Zheng Liu, Hao Liang, Xijie Huang, Wentao Xiong, Qinhan Yu, Linzhuang Sun, Chong Chen, Conghui He, Bin Cui, Wentao Zhang

Crucially, our method's reliance on purely generated data ensures the preservation of privacy, achieving SoTA performance with just 100k data points (only 18% of the official dataset size).

Caption Generation Question Answering

Research on Tibetan Tourism Viewpoints information generation system based on LLM

no code implementations18 Jul 2024 Jinhu Qi, Shuai Yan, Wentao Zhang, Yibo Zhang, Zirui Liu, Ke Wang

Tibet, ensconced within China's territorial expanse, is distinguished by its labyrinthine and heterogeneous topography, a testament to its profound historical heritage, and the cradle of a unique religious ethos.

Language Modelling Large Language Model

Physics-guided Active Sample Reweighting for Urban Flow Prediction

1 code implementation18 Jul 2024 Wei Jiang, Tong Chen, Guanhua Ye, Wentao Zhang, Lizhen Cui, Zi Huang, Hongzhi Yin

Moreover, due to the interval-based predictions and intermittent nature of data filing in many transportation services, the instantaneous dynamics of urban flows can hardly be captured, rendering differential equation-based continuous modeling a loose fit for this setting.

Video In-context Learning

no code implementations10 Jul 2024 Wentao Zhang, Junliang Guo, Tianyu He, Li Zhao, Linli Xu, Jiang Bian

In-context learning for vision data has been underexplored compared with that in natural language.

In-Context Learning

AnyTaskTune: Advanced Domain-Specific Solutions through Task-Fine-Tuning

1 code implementation9 Jul 2024 Jiaxi Cui, Wentao Zhang, Jing Tang, Xudong Tong, Zhenwei Zhang, Amie, Jing Wen, Rongsheng Wang, Pengfei Wu

Our findings demonstrate that models fine-tuned using the \textbf{Task-Fine-Tune} methodology not only achieve superior performance on these specific tasks but also significantly outperform models with higher general capabilities in their respective domains.

Keyword Extraction Sentence

PAS: Data-Efficient Plug-and-Play Prompt Augmentation System

no code implementations8 Jul 2024 Miao Zheng, Hao Liang, Fan Yang, Haoze Sun, Tianpeng Li, Lingchu Xiong, Yan Zhang, Youzhen Wu, Kun Li, Yanjun Shen, MingAn Lin, Tao Zhang, Guosheng Dong, Yujing Qiao, Kun Fang, WeiPeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou

This combination of high performance, efficiency, and flexibility makes PAS a valuable system for enhancing the usability and effectiveness of LLMs through improved prompt engineering.

Prompt Engineering

KeyVideoLLM: Towards Large-scale Video Keyframe Selection

no code implementations3 Jul 2024 Hao Liang, Jiapeng Li, Tianyi Bai, Xijie Huang, Linzhuang Sun, Zhengren Wang, Conghui He, Bin Cui, Chong Chen, Wentao Zhang

Recently, with the rise of web videos, managing and understanding large-scale video datasets has become increasingly important.

Data Compression Management +3

Efficient-Empathy: Towards Efficient and Effective Selection of Empathy Data

no code implementations2 Jul 2024 Linzhuang Sun, Hao Liang, Jingxuan Wei, Linkun Sun, Bihui Yu, Bin Cui, Wentao Zhang

By integrating sensibility and rationality data with a MoE structure, we achieve even higher performance, demonstrating the effectiveness of our Efficient-Empathy algorithm.

Consistency Flow Matching: Defining Straight Flows with Velocity Consistency

1 code implementation2 Jul 2024 Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, Bin Cui

Additionally, we propose a multi-segment training approach for Consistency-FM to enhance expressiveness, achieving a better trade-off between sampling quality and speed.

Image Generation

RobGC: Towards Robust Graph Condensation

no code implementations19 Jun 2024 Xinyi Gao, Hongzhi Yin, Tong Chen, Guanhua Ye, Wentao Zhang, Bin Cui

Graph neural networks (GNNs) have attracted widespread attention for their impressive capability of graph representation learning.

Denoising Graph Representation Learning

Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models

1 code implementation6 Jun 2024 Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, Bin Cui

We introduce Buffer of Thoughts (BoT), a novel and versatile thought-augmented reasoning approach for enhancing accuracy, efficiency and robustness of large language models (LLMs).

Arithmetic Reasoning Code Generation +2

Graph Condensation for Open-World Graph Learning

no code implementations27 May 2024 Xinyi Gao, Tong Chen, Wentao Zhang, Yayong Li, Xiangguo Sun, Hongzhi Yin

Consequently, due to the limited generalization capacity of condensed graphs, applications that employ GC for efficient GNN training end up with sub-optimal GNNs when confronted with evolving graph structures and distributions in dynamic real-world situations.

Graph Learning

A Survey of Multimodal Large Language Model from A Data-centric Perspective

1 code implementation26 May 2024 Tianyi Bai, Hao Liang, Binwang Wan, Yanran Xu, Xi Li, Shiyu Li, Ling Yang, Bozhou Li, Yifan Wang, Bin Cui, Ping Huang, Jiulong Shan, Conghui He, Binhang Yuan, Wentao Zhang

Multimodal large language models (MLLMs) enhance the capabilities of standard large language models by integrating and processing data from multiple modalities, including text, vision, audio, video, and 3D environments.

Language Modelling Large Language Model +2

EditWorld: Simulating World Dynamics for Instruction-Following Image Editing

1 code implementation23 May 2024 Ling Yang, Bohan Zeng, Jiaming Liu, Hong Li, Minghao Xu, Wentao Zhang, Shuicheng Yan

Therefore, this work, EditWorld, introduces a new editing task, namely world-instructed image editing, which defines and categorizes the instructions grounded by various world scenarios.

Instruction Following

Rethinking and Accelerating Graph Condensation: A Training-Free Approach with Class Partition

no code implementations22 May 2024 Xinyi Gao, Tong Chen, Wentao Zhang, Junliang Yu, Guanhua Ye, Quoc Viet Hung Nguyen, Hongzhi Yin

The increasing prevalence of large-scale graphs poses a significant challenge for graph neural network training, attributed to their substantial computational requirements.

Graph Neural Network

Resilient control of networked switched systems subject to deception attack and DoS attack

no code implementations10 May 2024 Rui Zhao, Zhiqiang Zuo, Ying Tan, Yijing Wang, Wentao Zhang

In this paper, the resilient control for switched systems in the presence of deception attack and denial-of-service (DoS) attack is addressed.

Acceleration Algorithms in GNNs: A Survey

1 code implementation7 May 2024 Lu Ma, Zeang Sheng, Xunkai Li, Xinyi Gao, Zhezheng Hao, Ling Yang, Wentao Zhang, Bin Cui

To address the critical challenges, a range of algorithms have been proposed to accelerate training and inference of GNNs, attracting increasing attention from the research community.

Graph Learning Survey

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

4 code implementations7 May 2024 DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, JianZhong Guo, Jiaqi Ni, Jiashi Li, Jin Chen, Jingyang Yuan, Junjie Qiu, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruizhe Pan, Runxin Xu, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Size Zheng, T. Wang, Tian Pei, Tian Yuan, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wei An, Wen Liu, Wenfeng Liang, Wenjun Gao, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Liu, Xin Xie, Xingkai Yu, Xinnan Song, Xinyi Zhou, Xinyu Yang, Xuan Lu, Xuecheng Su, Y. Wu, Y. K. Li, Y. X. Wei, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Zheng, Yichao Zhang, Yiliang Xiong, Yilong Zhao, Ying He, Ying Tang, Yishi Piao, Yixin Dong, Yixuan Tan, Yiyuan Liu, Yongji Wang, Yongqiang Guo, Yuchen Zhu, Yuduan Wang, Yuheng Zou, Yukun Zha, Yunxian Ma, Yuting Yan, Yuxiang You, Yuxuan Liu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhewen Hao, Zhihong Shao, Zhiniu Wen, Zhipeng Xu, Zhongyu Zhang, Zhuoshu Li, Zihan Wang, Zihui Gu, Zilin Li, Ziwei Xie

MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.

Language Modelling Reinforcement Learning (RL)

When Emotional Stimuli meet Prompt Designing: An Auto-Prompt Graphical Paradigm

no code implementations16 Apr 2024 Chenggian Ma, Xiangyu Zhao, Chunhui Zhang, Yanzhao Qin, Wentao Zhang

With the development of Large Language Models (LLM), numerous prompts have been proposed, each with a rich set of features and their own merits.

Distribution-Aware Data Expansion with Diffusion Models

1 code implementation11 Mar 2024 Haowei Zhu, Ling Yang, Jun-Hai Yong, Hongzhi Yin, Jiawei Jiang, Meng Xiao, Wentao Zhang, Bin Wang

In this paper, we propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model.

Image Generation Informativeness

Cradle: Empowering Foundation Agents Towards General Computer Control

1 code implementation5 Mar 2024 Weihao Tan, Wentao Zhang, Xinrun Xu, Haochong Xia, Ziluo Ding, Boyu Li, Bohan Zhou, Junpeng Yue, Jiechuan Jiang, Yewen Li, Ruyi An, Molei Qin, Chuqiao Zong, Longtao Zheng, Yujie Wu, Xiaoqiang Chai, Yifei Bi, Tianbao Xie, Pengjie Gu, Xiyun Li, Ceyao Zhang, Long Tian, Chaojie Wang, Xinrun Wang, Börje F. Karlsson, Bo An, Shuicheng Yan, Zongqing Lu

To handle this issue, we propose the General Computer Control (GCC) setting to restrict foundation agents to interact with software through the most unified and standardized interface, i. e., using screenshots as input and keyboard and mouse actions as output.

Efficient Exploration

Retrieval-Augmented Generation for AI-Generated Content: A Survey

3 code implementations29 Feb 2024 Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, Bin Cui

We first classify RAG foundations according to how the retriever augments the generator, distilling the fundamental abstractions of the augmentation methodologies for various retrievers and generators.

Information Retrieval multimodal generation +3

Kuaiji: the First Chinese Accounting Large Language Model

no code implementations21 Feb 2024 Jiayuan Luo, Songhua Yang, Xiaoling Qiu, Panyu Chen, Yufei Nai, Wenxuan Zeng, Wentao Zhang, Xinke Jiang

Large Language Models (LLMs) like ChatGPT and GPT-4 have demonstrated impressive proficiency in comprehending and generating natural language.

Language Modelling Large Language Model

Rethinking Node-wise Propagation for Large-scale Graph Learning

1 code implementation9 Feb 2024 Xunkai Li, Jingyuan Ma, Zhengyu Wu, Daohan Su, Wentao Zhang, Rong-Hua Li, Guoren Wang

However, (i) Most scalable GNNs tend to treat all nodes in graphs with the same propagation rules, neglecting their topological uniqueness; (ii) Existing node-wise propagation optimization strategies are insufficient on web-scale graphs with intricate topology, where a full portrayal of nodes' local properties is required.

Graph Learning Node Classification

Time Series Supplier Allocation via Deep Black-Litterman Model

no code implementations30 Jan 2024 Jiayuan Luo, Wentao Zhang, Yuchen Fang, Xiaowei Gao, Dingyi Zhuang, Hao Chen, Xinke Jiang

Time Series Supplier Allocation (TSSA) poses a complex NP-hard challenge, aimed at refining future order dispatching strategies to satisfy order demands with maximum supply efficiency fully.

Navigate Time Series

True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning

1 code implementation25 Jan 2024 Weihao Tan, Wentao Zhang, Shanqi Liu, Longtao Zheng, Xinrun Wang, Bo An

Despite the impressive performance across numerous tasks, large language models (LLMs) often fail in solving simple decision-making tasks due to the misalignment of the knowledge in LLMs with environments.

Decision Making Reinforcement Learning (RL)

Graph Condensation: A Survey

1 code implementation22 Jan 2024 Xinyi Gao, Junliang Yu, Tong Chen, Guanhua Ye, Wentao Zhang, Hongzhi Yin

We also empirically compare and analyze representative GC methods with diverse optimization strategies based on the five proposed GC evaluation criteria.

Fairness Graph Generation +1

LightDiC: A Simple yet Effective Approach for Large-scale Digraph Representation Learning

1 code implementation22 Jan 2024 Xunkai Li, Meihao Liao, Zhengyu Wu, Daohan Su, Wentao Zhang, Rong-Hua Li, Guoren Wang

Most existing graph neural networks (GNNs) are limited to undirected graphs, whose restricted scope of the captured relational information hinders their expressive capabilities and deployments in real-world scenarios.

Denoising Representation Learning

FedGTA: Topology-aware Averaging for Federated Graph Learning

1 code implementation22 Jan 2024 Xunkai Li, Zhengyu Wu, Wentao Zhang, Yinlin Zhu, Rong-Hua Li, Guoren Wang

Existing FGL studies fall into two categories: (i) FGL Optimization, which improves multi-client training in existing machine learning models; (ii) FGL Model, which enhances performance with complex local models and multi-client interactions.

Graph Learning

Towards Effective and General Graph Unlearning via Mutual Evolution

1 code implementation22 Jan 2024 Xunkai Li, Yulin Zhao, Zhengyu Wu, Wentao Zhang, Rong-Hua Li, Guoren Wang

With the rapid advancement of AI applications, the growing needs for data privacy and model robustness have highlighted the importance of machine unlearning, especially in thriving graph-based scenarios.

Machine Unlearning

Self-Supervised Deconfounding Against Spatio-Temporal Shifts: Theory and Modeling

1 code implementation21 Nov 2023 Jiahao Ji, Wentao Zhang, Jingyuan Wang, Yue He, Chao Huang

It first encodes traffic data into two disentangled representations for associating invariant and variant ST contexts.

Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools

1 code implementation17 Nov 2023 Wentao Zhang, Yilei Zhao, Shuo Sun, Jie Ying, Yonggang Xie, Zitao Song, Xinrun Wang, Bo An

Specifically, the target stock pool of different investors varies dramatically due to their discrepancy on market states and individual investors may temporally adjust stocks they desire to trade (e. g., adding one popular stocks), which lead to customizable stock pools (CSPs).

Management reinforcement-learning +1

Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning

no code implementations24 Oct 2023 Yuxiang Wang, Xiao Yan, Chuang Hu, Fangcheng Fu, Wentao Zhang, Hao Wang, Shuo Shang, Jiawei Jiang

For graph self-supervised learning (GSSL), masked autoencoder (MAE) follows the generative paradigm and learns to reconstruct masked graph edges or node features.

Contrastive Learning Graph Classification +4

Accelerating Scalable Graph Neural Network Inference with Node-Adaptive Propagation

no code implementations17 Oct 2023 Xinyi Gao, Wentao Zhang, Junliang Yu, Yingxia Shao, Quoc Viet Hung Nguyen, Bin Cui, Hongzhi Yin

To further accelerate Scalable GNNs inference in this inductive setting, we propose an online propagation framework and two novel node-adaptive propagation methods that can customize the optimal propagation depth for each node based on its topological information and thereby avoid redundant feature propagation.

Graph Neural Network

Learning Multiplex Representations on Text-Attributed Graphs with One Language Model Encoder

1 code implementation10 Oct 2023 Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Han Zhao, Jiawei Han

Mainstream text representation learning methods use pretrained language models (PLMs) to generate one embedding for each text unit, expecting that all types of relations between texts can be captured by these single-view embeddings.

Language Modelling Representation Learning

IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts

1 code implementation9 Oct 2023 Bohan Zeng, Shanglin Li, Yutang Feng, Ling Yang, Hong Li, Sicheng Gao, Jiaming Liu, Conghui He, Wentao Zhang, Jianzhuang Liu, Baochang Zhang, Shuicheng Yan

Recent advances in 3D generation have been remarkable, with methods such as DreamFusion leveraging large-scale text-to-image diffusion-based models to guide 3D object generation.

3D Generation Image to 3D +2

To better understand realized ecosystem services: An integrated analysis framework of supply, demand, flow and use

no code implementations27 Sep 2023 Shuyao Wu, Kai-Di Liu, Wentao Zhang, Yuehan Dou, Yuqing Chen, Delong Li

Realized ecosystem services (ES) are the actual use of ES by societies, which is more directly linked to human well-being than potential ES.

EarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading

1 code implementation22 Sep 2023 Molei Qin, Shuo Sun, Wentao Zhang, Haochong Xia, Xinrun Wang, Bo An

In stage II, we construct a pool of diverse RL agents for different market trends, distinguished by return rates, where hundreds of RL agents are trained with different preferences of return rates and only a tiny fraction of them will be selected into the pool based on their profitability.

Algorithmic Trading Hierarchical Reinforcement Learning +1

Agents: An Open-source Framework for Autonomous Language Agents

1 code implementation14 Sep 2023 Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Xiangru Tang, Ningyu Zhang, Huajun Chen, Peng Cui, Mrinmaya Sachan

Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with environments, humans, and other agents using natural language interfaces.

Towards General and Efficient Online Tuning for Spark

no code implementations5 Sep 2023 Yang Li, Huaijun Jiang, Yu Shen, Yide Fang, Xiaofeng Yang, Danqing Huang, Xinyi Zhang, Wentao Zhang, Ce Zhang, Peng Chen, Bin Cui

The distributed data analytic system -- Spark is a common choice for processing massive volumes of heterogeneous data, while it is challenging to tune its parameters to achieve high performance.

Bayesian Optimization Meta-Learning

VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs

1 code implementation4 Aug 2023 Ling Yang, Ye Tian, Minkai Xu, Zhongyi Liu, Shenda Hong, Wei Qu, Wentao Zhang, Bin Cui, Muhan Zhang, Jure Leskovec

To address this issue, we propose to learn a new powerful graph representation space by directly labeling nodes' diverse local structures for GNN-to-MLP distillation.

Knowledge Distillation Quantization +1

Graph Condensation for Inductive Node Representation Learning

no code implementations29 Jul 2023 Xinyi Gao, Tong Chen, Yilong Zang, Wentao Zhang, Quoc Viet Hung Nguyen, Kai Zheng, Hongzhi Yin

To overcome this issue, we propose mapping-aware graph condensation (MCond), explicitly learning the one-to-many node mapping from original nodes to synthetic nodes to seamlessly integrate new nodes into the synthetic graph for inductive representation learning.

Representation Learning

Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization

1 code implementation28 Jun 2023 Ling Yang, Jiayi Zheng, Heyuan Wang, Zhongyi Liu, Zhilin Huang, Shenda Hong, Wentao Zhang, Bin Cui

To remove class spurious feature caused by distribution shifts, we propose Individual Graph Information Bottleneck (I-GIB) which discards irrelevant information by minimizing the mutual information between the input graph and its embeddings.

Graph Learning Out-of-Distribution Generalization

Patton: Language Model Pretraining on Text-Rich Networks

no code implementations20 May 2023 Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, Jiawei Han

A real-world text corpus sometimes comprises not only text documents but also semantic links between them (e. g., academic papers in a bibliographic network are linked by citations and co-authorships).

Language Modelling Masked Language Modeling +1

OpenBox: A Python Toolkit for Generalized Black-box Optimization

1 code implementation26 Apr 2023 Huaijun Jiang, Yu Shen, Yang Li, Beicheng Xu, Sixian Du, Wentao Zhang, Ce Zhang, Bin Cui

Black-box optimization (BBO) has a broad range of applications, including automatic machine learning, experimental design, and database knob tuning.

Experimental Design

Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases

1 code implementation18 Apr 2023 Wentao Zhang, Yujun Huang, Tong Zhang, Qingsong Zou, Wei-Shi Zheng, Ruixuan Wang

In particular, updating an intelligent diagnosis system with training data of new diseases would cause catastrophic forgetting of old disease knowledge.

Continual Learning

GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation

no code implementations8 Apr 2023 Jinming Li, Wentao Zhang, Tian Wang, Guanglei Xiong, Alan Lu, Gerard Medioni

The generated queries naturally serve as interpretable representations of user interests and can be searched to recommend cold-start items.

Diversity Language Modelling +1

Semantic-aware Node Synthesis for Imbalanced Heterogeneous Information Networks

no code implementations27 Feb 2023 Xinyi Gao, Wentao Zhang, Tong Chen, Junliang Yu, Hung Quoc Viet Nguyen, Hongzhi Yin

To tackle the imbalance of minority classes and supplement their inadequate semantics, we present the first method for the semantic imbalance problem in imbalanced HINs named Semantic-aware Node Synthesis (SNS).

Representation Learning

Transfer Learning for Bayesian Optimization: A Survey

no code implementations12 Feb 2023 Tianyi Bai, Yang Li, Yu Shen, Xinyi Zhang, Wentao Zhang, Bin Cui

A wide spectrum of design and decision problems, including parameter tuning, A/B testing and drug design, intrinsically are instances of black-box optimization.

Bayesian Optimization Survey +1

Rover: An online Spark SQL tuning service via generalized transfer learning

no code implementations8 Feb 2023 Yu Shen, Xinyuyang Ren, Yupeng Lu, Huaijun Jiang, Huanyong Xu, Di Peng, Yang Li, Wentao Zhang, Bin Cui

When applying transfer learning to accelerate the tuning process, we notice two domain-specific challenges: 1) most previous work focus on transferring tuning history, while expert knowledge from Spark engineers is of great potential to improve the tuning performance but is not well studied so far; 2) history tasks should be carefully utilized, where using dissimilar ones lead to a deteriorated performance in production.

Bayesian Optimization Transfer Learning

DivBO: Diversity-aware CASH for Ensemble Learning

no code implementations7 Feb 2023 Yu Shen, Yupeng Lu, Yang Li, Yaofeng Tu, Wentao Zhang, Bin Cui

To tackle this issue and further enhance the ensemble performance, we propose DivBO, a diversity-aware framework to inject explicit search of diversity into the CASH problems.

AutoML Bayesian Optimization +2

PAMI: partition input and aggregate outputs for model interpretation

no code implementations7 Feb 2023 Wei Shi, Wentao Zhang, Weishi Zheng, Ruixuan Wang

There is an increasing demand for interpretation of model predictions especially in high-risk applications.

Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training

1 code implementation21 Nov 2022 Ling Yang, Zhilin Huang, Yang song, Shenda Hong, Guohao Li, Wentao Zhang, Bin Cui, Bernard Ghanem, Ming-Hsuan Yang

Generating images from graph-structured inputs, such as scene graphs, is uniquely challenging due to the difficulty of aligning nodes and connections in graphs with objects and their relations in images.

Image Generation

Efficient Graph Neural Network Inference at Large Scale

no code implementations1 Nov 2022 Xinyi Gao, Wentao Zhang, Yingxia Shao, Quoc Viet Hung Nguyen, Bin Cui, Hongzhi Yin

Graph neural networks (GNNs) have demonstrated excellent performance in a wide range of applications.

Graph Neural Network

Distributed Graph Neural Network Training: A Survey

no code implementations1 Nov 2022 Yingxia Shao, Hongzheng Li, Xizhi Gu, Hongbo Yin, Yawen Li, Xupeng Miao, Wentao Zhang, Bin Cui, Lei Chen

In recent years, many efforts have been made on distributed GNN training, and an array of training algorithms and systems have been proposed.

Distributed Computing Graph Neural Network +1

Diffusion Models: A Comprehensive Survey of Methods and Applications

2 code implementations2 Sep 2022 Ling Yang, Zhilong Zhang, Yang song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang

This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration.

Image Super-Resolution Survey +2

Efficient End-to-End AutoML via Scalable Search Space Decomposition

1 code implementation19 Jun 2022 Yang Li, Yu Shen, Wentao Zhang, Ce Zhang, Bin Cui

End-to-end AutoML has attracted intensive interests from both academia and industry which automatically searches for ML pipelines in a space induced by feature engineering, algorithm/model selection, and hyper-parameter tuning.

AutoML Feature Engineering +1

NAFS: A Simple yet Tough-to-beat Baseline for Graph Representation Learning

2 code implementations17 Jun 2022 Wentao Zhang, Zeang Sheng, Mingyu Yang, Yang Li, Yu Shen, Zhi Yang, Bin Cui

First, GNNs can learn higher-order structural information by stacking more layers but can not deal with large depth due to the over-smoothing issue.

Graph Representation Learning Link Prediction +1

Model Degradation Hinders Deep Graph Neural Networks

1 code implementation9 Jun 2022 Wentao Zhang, Zeang Sheng, Ziqi Yin, Yuezihan Jiang, Yikuan Xia, Jun Gao, Zhi Yang, Bin Cui

Graph Neural Networks (GNNs) have achieved great success in various graph mining tasks. However, drastic performance degradation is always observed when a GNN is stacked with many layers.

Attribute Graph Mining

TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning

no code implementations6 Jun 2022 Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Zhi Yang, Ce Zhang, Bin Cui

With the extensive applications of machine learning models, automatic hyperparameter optimization (HPO) has become increasingly important.

Hyperparameter Optimization Neural Architecture Search +2

Transfer Learning based Search Space Design for Hyperparameter Tuning

no code implementations6 Jun 2022 Yang Li, Yu Shen, Huaijun Jiang, Tianyi Bai, Wentao Zhang, Ce Zhang, Bin Cui

The extensive experiments show that our approach considerably boosts BO by designing a promising and compact search space instead of using the entire space, and outperforms the state-of-the-arts on a wide range of benchmarks, including machine learning and deep learning tuning tasks, and neural architecture search.

Bayesian Optimization BIG-bench Machine Learning +2

ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest

1 code implementation20 Mar 2022 Yuezihan Jiang, Yu Cheng, Hanyu Zhao, Wentao Zhang, Xupeng Miao, Yu He, Liang Wang, Zhi Yang, Bin Cui

We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs.

Retrieval

Information Gain Propagation: a new way to Graph Active Learning with Soft Labels

1 code implementation ICLR 2022 Wentao Zhang, Yexin Wang, Zhenbang You, Meng Cao, Ping Huang, Jiulong Shan, Zhi Yang, Bin Cui

Graph Neural Networks (GNNs) have achieved great success in various tasks, but their performance highly relies on a large number of labeled nodes, which typically requires considerable human effort.

Active Learning

PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm

1 code implementation1 Mar 2022 Wentao Zhang, Yu Shen, Zheyu Lin, Yang Li, Xiaosen Li, Wen Ouyang, Yangyu Tao, Zhi Yang, Bin Cui

Through deconstructing the message passing mechanism, PasCa presents a novel Scalable Graph Neural Architecture Paradigm (SGAP), together with a general architecture design space consisting of 150k different designs.

Neural Architecture Search

Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue

no code implementations1 Mar 2022 Wentao Zhang, Shuang Xu, Haoran Huang

We further develop a new method for supervised contrastive learning, referred to as two-level supervised contrastive learning, and employ the method in response selection in multi-turn dialogue.

Contrastive Learning Conversational Response Selection +3

Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale

no code implementations18 Jan 2022 Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Jixiang Li, Ji Liu, Ce Zhang, Bin Cui

The ever-growing demand and complexity of machine learning are putting pressure on hyper-parameter tuning systems: while the evaluation cost of models continues to increase, the scalability of state-of-the-arts starts to become a crucial bottleneck.

Scheduling

K-Core Decomposition on Super Large Graphs with Limited Resources

no code implementations26 Dec 2021 Shicheng Gao, Jie Xu, Xiaosen Li, Fangcheng Fu, Wentao Zhang, Wen Ouyang, Yangyu Tao, Bin Cui

For example, the distributed K-core decomposition algorithm can scale to a large graph with 136 billion edges without losing correctness with our divide-and-conquer technique.

RIM: Reliable Influence-based Active Learning on Graphs

1 code implementation NeurIPS 2021 Wentao Zhang, Yexin Wang, Zhenbang You, Meng Cao, Ping Huang, Jiulong Shan, Zhi Yang, Bin Cui

Message passing is the core of most graph models such as Graph Convolutional Network (GCN) and Label Propagation (LP), which usually require a large number of clean labeled data to smooth out the neighborhood over the graph.

Active Learning