Search Results for author: Weinan Zhang

Found 132 papers, 64 papers with code

Nested Named Entity Recognition with Span-level Graphs

no code implementations ACL 2022 Juncheng Wan, Dongyu Ru, Weinan Zhang, Yong Yu

In this work, we try to improve the span representation by utilizing retrieval-based span-level graphs, connecting spans and entities in the training data based on n-gram features.

named-entity-recognition Named Entity Recognition +3

Play to Your Strengths: Collaborative Intelligence of Conventional Recommender Models and Large Language Models

no code implementations25 Mar 2024 Yunjia Xi, Weiwen Liu, Jianghao Lin, Chuhan Wu, Bo Chen, Ruiming Tang, Weinan Zhang, Yong Yu

The rise of large language models (LLMs) has opened new opportunities in Recommender Systems (RSs) by enhancing user behavior modeling and content understanding.

Language Modelling Large Language Model +1

An Aligning and Training Framework for Multimodal Recommendations

no code implementations19 Mar 2024 Yifan Liu, Kangning Zhang, Xiangyuan Ren, Yanhua Huang, Jiarui Jin, Yingjie Qin, Ruilong Su, Ruiwen Xu, Weinan Zhang

In AlignRec, the recommendation objective is decomposed into three alignments, namely alignment within contents, alignment between content and categorical ID, and alignment between users and items.

Multimodal Recommendation

TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision

1 code implementation10 Mar 2024 Ruiwen Zhou, Yingxuan Yang, Muning Wen, Ying Wen, Wenhao Wang, Chunling Xi, Guoqiang Xu, Yong Yu, Weinan Zhang

Among these works, many of them utilize in-context examples to achieve generalization without the need for fine-tuning, while few of them have considered the problem of how to select and effectively utilize these examples.

Language Modelling Large Language Model +1

Looking Ahead to Avoid Being Late: Solving Hard-Constrained Traveling Salesman Problem

no code implementations8 Mar 2024 Jingxiao Chen, Ziqin Gong, Minghuan Liu, Jun Wang, Yong Yu, Weinan Zhang

To overcome this problem and to have an effective solution against hard constraints, we proposed a novel learning-based method that uses looking-ahead information as the feature to improve the legality of TSP with Time Windows (TSPTW) solutions.

Traveling Salesman Problem

Towards Efficient and Effective Unlearning of Large Language Models for Recommendation

1 code implementation6 Mar 2024 Hangyu Wang, Jianghao Lin, Bo Chen, Yang Yang, Ruiming Tang, Weinan Zhang, Yong Yu

However, in order to protect user privacy and optimize utility, it is also crucial for LLMRec to intentionally forget specific user data, which is generally referred to as recommendation unlearning.

World Knowledge

Offline Fictitious Self-Play for Competitive Games

no code implementations29 Feb 2024 Jingxiao Chen, Weiji Xie, Weinan Zhang, Yong Yu, Ying Wen

Firstly, unaware of the game structure, it is impossible to interact with the opponents and conduct a major learning paradigm, self-play, for competitive games.

Offline RL Reinforcement Learning (RL)

Large-Scale Actionless Video Pre-Training via Discrete Diffusion for Efficient Policy Learning

no code implementations22 Feb 2024 Haoran He, Chenjia Bai, Ling Pan, Weinan Zhang, Bin Zhao, Xuelong Li

In the fine-tuning stage, we harness the imagined future videos to guide low-level action learning trained on a limited set of robot data.

Entropy-Regularized Token-Level Policy Optimization for Large Language Models

1 code implementation9 Feb 2024 Muning Wen, Cheng Deng, Jun Wang, Weinan Zhang, Ying Wen

At the heart of ETPO is our novel per-token soft Bellman update, designed to harmonize the RL process with the principles of language modeling.

Code Generation Decision Making +3

CityFlowER: An Efficient and Realistic Traffic Simulator with Embedded Machine Learning Models

no code implementations9 Feb 2024 Longchao Da, Chen Chu, Weinan Zhang, Hua Wei

Addressing these limitations, we introduce CityFlowER, an advancement over the existing CityFlow simulator, designed for efficient and realistic city-wide traffic simulation.

DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching

no code implementations4 Feb 2024 Guanghe Li, Yixiang Shan, Zhengbang Zhu, Ting Long, Weinan Zhang

In offline reinforcement learning (RL), the performance of the learned policy highly depends on the quality of offline datasets.

D4RL Data Augmentation +4

ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update

1 code implementation1 Feb 2024 Liyuan Mao, Haoran Xu, Weinan Zhang, Xianyuan Zhan

To resolve this issue, we propose a simple yet effective modification that projects the backward gradient onto the normal plane of the forward gradient, resulting in an orthogonal-gradient update, a new learning rule for DICE-based methods.

Imitation Learning Offline RL +1

InfoRank: Unbiased Learning-to-Rank via Conditional Mutual Information Minimization

no code implementations23 Jan 2024 Jiarui Jin, Zexue He, Mengyue Yang, Weinan Zhang, Yong Yu, Jun Wang, Julian McAuley

Subsequently, we minimize the mutual information between the observation estimation and the relevance estimation conditioned on the input features.

Learning-To-Rank Recommendation Systems

D2K: Turning Historical Data into Retrievable Knowledge for Recommender Systems

no code implementations21 Jan 2024 Jiarui Qin, Weiwen Liu, Ruiming Tang, Weinan Zhang, Yong Yu

A personalized knowledge adaptation unit is devised to effectively exploit the information from the knowledge base by adapting the retrieved knowledge to the target samples.

Recommendation Systems

Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges

no code implementations27 Dec 2023 Qingyao Li, Lingyue Fu, Weiming Zhang, Xianyu Chen, Jingwei Yu, Wei Xia, Weinan Zhang, Ruiming Tang, Yong Yu

Online education platforms, leveraging the internet to distribute education resources, seek to provide convenient education but often fall short in real-time communication with students.

Question Answering

GFS: Graph-based Feature Synthesis for Prediction over Relational Databases

no code implementations4 Dec 2023 Han Zhang, Quan Gan, David Wipf, Weinan Zhang

Consequently, the prevalent approach for training machine learning models on data stored in relational databases involves performing feature engineering to merge the data from multiple tables into a single table and subsequently applying single table models.

Feature Engineering Inductive Bias

Vision-Language Foundation Models as Effective Robot Imitators

no code implementations2 Nov 2023 Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, Tao Kong

We believe RoboFlamingo has the potential to be a cost-effective and easy-to-use solution for robotics manipulation, empowering everyone with the ability to fine-tune their own robotics policy.

Imitation Learning

FLIP: Towards Fine-grained Alignment between ID-based Models and Pretrained Language Models for CTR Prediction

no code implementations30 Oct 2023 Hangyu Wang, Jianghao Lin, Xiangyang Li, Bo Chen, Chenxu Zhu, Ruiming Tang, Weinan Zhang, Yong Yu

Specifically, the masked data of one modality (i. e., tokens or features) has to be recovered with the help of the other modality, which establishes the feature-level interaction and alignment via sufficient mutual information extraction between dual modalities.

Click-Through Rate Prediction Contrastive Learning

Specify Robust Causal Representation from Mixed Observations

1 code implementation21 Oct 2023 Mengyue Yang, Xinyu Cai, Furui Liu, Weinan Zhang, Jun Wang

Under the hypothesis that the intrinsic latent factors follow some casual generative models, we argue that by learning a causal representation, which is the minimal sufficient causes of the whole system, we can improve the robustness and generalization performance of machine learning models.

GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models

1 code implementation8 Oct 2023 Hanjing Wang, Man-Kit Sit, Congjie He, Ying Wen, Weinan Zhang, Jun Wang, Yaodong Yang, Luo Mai

This paper introduces a distributed, GPU-centric experience replay system, GEAR, designed to perform scalable reinforcement learning (RL) with large sequence models (such as transformers).

Reinforcement Learning (RL)

Quantifying Zero-shot Coordination Capability with Behavior Preferring Partners

no code implementations8 Oct 2023 Xihuai Wang, Shao Zhang, WenHao Zhang, Wentao Dong, Jingxiao Chen, Ying Wen, Weinan Zhang

Current evaluation methods for ZSC capability still need to improve in constructing diverse evaluation partners and comprehensively measuring the ZSC capability.

Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training

1 code implementation29 Sep 2023 Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, Jun Wang

Empirical results across reasoning, planning, alignment, and decision-making tasks show that TS-LLM outperforms existing approaches and can handle trees with a depth of 64.

Decision Making Language Modelling +1

CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models

1 code implementation5 Sep 2023 Lingyue Fu, Huacan Chai, Shuang Luo, Kounianhua Du, Weiming Zhang, Longteng Fan, Jiayi Lei, Renting Rui, Jianghao Lin, Yuchen Fang, Yifan Liu, Jingkuan Wang, Siyuan Qi, Kangning Zhang, Weinan Zhang, Yong Yu

With the emergence of Large Language Models (LLMs), there has been a significant improvement in the programming capabilities of models, attracting growing attention from researchers.

Code Generation Multiple-choice

ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation

1 code implementation22 Aug 2023 Jianghao Lin, Rong Shan, Chenxu Zhu, Kounianhua Du, Bo Chen, Shigang Quan, Ruiming Tang, Yong Yu, Weinan Zhang

With large language models (LLMs) achieving remarkable breakthroughs in natural language processing (NLP) domains, LLM-enhanced recommender systems have received much attention and have been actively explored currently.

Data Augmentation Language Modelling +3

Through the Lens of Core Competency: Survey on Evaluation of Large Language Models

no code implementations15 Aug 2023 Ziyu Zhuang, Qiguang Chen, Longxuan Ma, Mingda Li, Yi Han, Yushan Qian, Haopeng Bai, Zixian Feng, Weinan Zhang, Ting Liu

From pre-trained language model (PLM) to large language model (LLM), the field of natural language processing (NLP) has witnessed steep performance gains and wide practical uses.

Language Modelling Large Language Model

Replace Scoring with Arrangement: A Contextual Set-to-Arrangement Framework for Learning-to-Rank

no code implementations5 Aug 2023 Jiarui Jin, Xianyu Chen, Weinan Zhang, Mengyue Yang, Yang Wang, Yali Du, Yong Yu, Jun Wang

Notice that these ranking metrics do not consider the effects of the contextual dependence among the items in the list, we design a new family of simulation-based ranking metrics, where existing metrics can be regarded as special cases.

Learning-To-Rank

MAP: A Model-agnostic Pretraining Framework for Click-through Rate Prediction

1 code implementation3 Aug 2023 Jianghao Lin, Yanru Qu, Wei Guo, Xinyi Dai, Ruiming Tang, Yong Yu, Weinan Zhang

The large capacity of neural models helps digest such massive amounts of data under the supervised learning paradigm, yet they fail to utilize the substantial data to its full potential, since the 1-bit click signal is not sufficient to guide the model to learn capable representations of features and instances.

Binary Classification Click-Through Rate Prediction +1

Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance

no code implementations6 Jul 2023 Yuchen Fang, Zhenggang Tang, Kan Ren, Weiqing Liu, Li Zhao, Jiang Bian, Dongsheng Li, Weinan Zhang, Yong Yu, Tie-Yan Liu

Order execution is a fundamental task in quantitative finance, aiming at finishing acquisition or liquidation for a number of trading orders of the specific assets.

Reinforcement Learning (RL)

Is Risk-Sensitive Reinforcement Learning Properly Resolved?

no code implementations2 Jul 2023 Ruiwen Zhou, Minghuan Liu, Kan Ren, Xufang Luo, Weinan Zhang, Dongsheng Li

Due to the nature of risk management in learning applicable policies, risk-sensitive reinforcement learning (RSRL) has been realized as an important direction.

Distributional Reinforcement Learning Management +2

Large Sequence Models for Sequential Decision-Making: A Survey

no code implementations24 Jun 2023 Muning Wen, Runji Lin, Hanjing Wang, Yaodong Yang, Ying Wen, Luo Mai, Jun Wang, Haifeng Zhang, Weinan Zhang

Transformer architectures have facilitated the development of large-scale and general-purpose sequence models for prediction tasks in natural language processing and computer vision, e. g., GPT-3 and Swin Transformer.

Decision Making

Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models

1 code implementation19 Jun 2023 Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, Rui Zhang, Yong Yu

In this work, we propose an Open-World Knowledge Augmented Recommendation Framework with Large Language Models, dubbed KAR, to acquire two types of external knowledge from LLMs -- the reasoning knowledge on user preferences and the factual knowledge on items.

Music Recommendation Recommendation Systems +1

ReLoop2: Building Self-Adaptive Recommendation Models via Responsive Error Compensation Loop

1 code implementation15 Jun 2023 Jieming Zhu, Guohao Cai, JunJie Huang, Zhenhua Dong, Ruiming Tang, Weinan Zhang

The error memory module is designed with fast access capabilities and undergoes continual refreshing with newly observed data samples during the model serving phase to support fast model adaptation.

Recommendation Systems

MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text Classification

1 code implementation15 Jun 2023 Hongyuan Dong, Weinan Zhang, Wanxiang Che

Despite the promising prospects, the performance of prompting model largely depends on the design of prompt template and verbalizer.

Few-Shot Text Classification text-classification

I run as fast as a rabbit, can you? A Multilingual Simile Dialogue Dataset

1 code implementation9 Jun 2023 Longxuan Ma, Weinan Zhang, Shuhan Zhou, Churui Sun, Changxin Ke, Ting Liu

Meanwhile, the MSD data can also be used on dialogue tasks to test the ability of dialogue systems when using similes.

Retrieval Sentence

How Can Recommender Systems Benefit from Large Language Models: A Survey

1 code implementation9 Jun 2023 Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Hao Zhang, Yong liu, Chuhan Wu, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, Weinan Zhang

In this paper, we conduct a comprehensive survey on this research direction from the perspective of the whole pipeline in real-world recommender systems.

Ethics Feature Engineering +5

Set-to-Sequence Ranking-based Concept-aware Learning Path Recommendation

no code implementations7 Jun 2023 Xianyu Chen, Jian Shen, Wei Xia, Jiarui Jin, Yakun Song, Weinan Zhang, Weiwen Liu, Menghui Zhu, Ruiming Tang, Kai Dong, Dingyin Xia, Yong Yu

Noticing that existing approaches fail to consider the correlations of concepts in the path, we propose a novel framework named Set-to-Sequence Ranking-based Concept-aware Learning Path Recommendation (SRC), which formulates the recommendation task under a set-to-sequence paradigm.

Knowledge Tracing Recommendation Systems

Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning

1 code implementation NeurIPS 2023 Haoran He, Chenjia Bai, Kang Xu, Zhuoran Yang, Weinan Zhang, Dong Wang, Bin Zhao, Xuelong Li

Specifically, we propose Multi-Task Diffusion Model (\textsc{MTDiff}), a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis in multi-task offline settings.

Reinforcement Learning (RL)

Privileged Knowledge Distillation for Sim-to-Real Policy Generalization

1 code implementation29 May 2023 Haoran He, Chenjia Bai, Hang Lai, Lingxiao Wang, Weinan Zhang

In this paper, we propose a novel single-stage privileged knowledge distillation method called the Historical Information Bottleneck (HIB) to narrow the sim-to-real gap.

Knowledge Distillation Reinforcement Learning (RL)

MADiff: Offline Multi-agent Learning with Diffusion Models

1 code implementation27 May 2023 Zhengbang Zhu, Minghuan Liu, Liyuan Mao, Bingyi Kang, Minkai Xu, Yong Yu, Stefano Ermon, Weinan Zhang

To the best of our knowledge, MADiff is the first diffusion-based multi-agent offline RL framework, which behaves as both a decentralized policy and a centralized controller.

Offline RL Trajectory Prediction

An Empirical Study on Google Research Football Multi-agent Scenarios

1 code implementation16 May 2023 Yan Song, He Jiang, Zheng Tian, Haifeng Zhang, Yingping Zhang, Jiangcheng Zhu, Zonghong Dai, Weinan Zhang, Jun Wang

Few multi-agent reinforcement learning (MARL) research on Google Research Football (GRF) focus on the 11v11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been released to the public.

Benchmarking Multi-agent Reinforcement Learning +1

Covidia: COVID-19 Interdisciplinary Academic Knowledge Graph

no code implementations14 Apr 2023 Cheng Deng, Jiaxin Ding, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou

In this work, we propose Covidia, COVID-19 interdisciplinary academic knowledge graph to bridge the gap between knowledge of COVID-19 on different domains.

Classification Contrastive Learning +2

FMGNN: Fused Manifold Graph Neural Network

no code implementations3 Apr 2023 Cheng Deng, Fan Xu, Jiaxing Ding, Luoyi Fu, Weinan Zhang, Xinbing Wang

Graph representation learning has been widely studied and demonstrated effectiveness in various graph tasks.

Graph Representation Learning Link Prediction +1

Text Classification in the Wild: a Large-scale Long-tailed Name Normalization Dataset

1 code implementation19 Feb 2023 Jiexing Qi, Shuhao Li, Zhixin Guo, Yusheng Huang, Chenghu Zhou, Weinan Zhang, Xinbing Wang, Zhouhan Lin

In this work, we first collect a large-scale institution name normalization dataset LoT-insts1, which contains over 25k classes that exhibit a naturally long-tailed distribution.

Long-tail Learning open-set classification +4

Order Matters: Agent-by-agent Policy Optimization

no code implementations13 Feb 2023 Xihuai Wang, Zheng Tian, Ziyu Wan, Ying Wen, Jun Wang, Weinan Zhang

In this paper, we propose the \textbf{A}gent-by-\textbf{a}gent \textbf{P}olicy \textbf{O}ptimization (A2PO) algorithm to improve the sample efficiency and retain the guarantees of monotonic improvement for each agent during training.

Visual Imitation Learning with Patch Rewards

1 code implementation2 Feb 2023 Minghuan Liu, Tairan He, Weinan Zhang, Shuicheng Yan, Zhongwen Xu

Specifically, we present Adversarial Imitation Learning with Patch Rewards (PatchAIL), which employs a patch-based discriminator to measure the expertise of different local parts from given images and provide patch rewards.

Imitation Learning

Refined Edge Usage of Graph Neural Networks for Edge Prediction

no code implementations25 Dec 2022 Jiarui Jin, Yangkun Wang, Weinan Zhang, Quan Gan, Xiang Song, Yong Yu, Zheng Zhang, David Wipf

However, existing methods lack elaborate design regarding the distinctions between two tasks that have been frequently overlooked: (i) edges only constitute the topology in the node classification task but can be used as both the topology and the supervisions (i. e., labels) in the edge prediction task; (ii) the node classification makes prediction over each individual node, while the edge prediction is determinated by each pair of nodes.

Link Prediction Node Classification

On Realization of Intelligent Decision-Making in the Real World: A Foundation Decision Model Perspective

1 code implementation24 Dec 2022 Ying Wen, Ziyu Wan, Ming Zhou, Shufang Hou, Zhe Cao, Chenyang Le, Jingxiao Chen, Zheng Tian, Weinan Zhang, Jun Wang

The pervasive uncertainty and dynamic nature of real-world environments present significant challenges for the widespread implementation of machine-driven Intelligent Decision-Making (IDM) systems.

Decision Making Image Captioning +2

Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents

no code implementations18 Dec 2022 Minghuan Liu, Zhengbang Zhu, Menghui Zhu, Yuzheng Zhuang, Weinan Zhang, Jianye Hao

In reinforcement learning applications like robotics, agents usually need to deal with various input/output features when specified with different state/action spaces by their developers or physical restrictions.

Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformer

no code implementations15 Dec 2022 Hang Lai, Weinan Zhang, Xialin He, Chen Yu, Zheng Tian, Yong Yu, Jun Wang

Deep reinforcement learning has recently emerged as an appealing alternative for legged locomotion over multiple terrains by training a policy in physical simulation and then transferring it to the real world (i. e., sim-to-real transfer).

Decision Making

A Bird's-eye View of Reranking: from List Level to Page Level

1 code implementation17 Nov 2022 Yunjia Xi, Jianghao Lin, Weiwen Liu, Xinyi Dai, Weinan Zhang, Rui Zhang, Ruiming Tang, Yong Yu

Moreover, simply applying a shared network for all the lists fails to capture the commonalities and distinctions in user behaviors on different lists.

Recommendation Systems

NeurIPS 2022 Competition: Driving SMARTS

no code implementations14 Nov 2022 Amir Rasouli, Randy Goebel, Matthew E. Taylor, Iuliia Kotseruba, Soheil Alizadeh, Tianpei Yang, Montgomery Alban, Florian Shkurti, Yuzheng Zhuang, Adam Scibior, Kasra Rezaee, Animesh Garg, David Meger, Jun Luo, Liam Paull, Weinan Zhang, Xinyu Wang, Xi Chen

The proposed competition supports methodologically diverse solutions, such as reinforcement learning (RL) and offline learning methods, trained on a combination of naturalistic AD data and open-source simulation platform SMARTS.

Autonomous Driving Reinforcement Learning (RL)

Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems

no code implementations11 Oct 2022 Zhengbang Zhu, Rongjun Qin, JunJie Huang, Xinyi Dai, Yang Yu, Yong Yu, Weinan Zhang

The increase in the measured performance, however, can have two possible attributions: a better understanding of user preferences, and a more proactive ability to utilize human bounded rationality to seduce user over-consumption.

Benchmarking Sequential Recommendation

Forgetting Fast in Recommender Systems

no code implementations14 Aug 2022 Wenyan Liu, Juncheng Wan, Xiaoling Wang, Weinan Zhang, Dell Zhang, Hang Li

In this paper, we investigate fast machine unlearning techniques for recommender systems that can remove the effect of a small amount of training data from the recommendation model without incurring the full cost of retraining.

Machine Unlearning Recommendation Systems

Multi-Scale User Behavior Network for Entire Space Multi-Task Learning

no code implementations3 Aug 2022 Jiarui Jin, Xianyu Chen, Weinan Zhang, Yuanbo Chen, Zaifan Jiang, Zekun Zhu, Zhewen Su, Yong Yu

Modelling the user's multiple behaviors is an essential part of modern e-commerce, whose widely adopted application is to jointly optimize click-through rate (CTR) and conversion rate (CVR) predictions.

Multi-Task Learning Survival Analysis

Bootstrapped Transformer for Offline Reinforcement Learning

no code implementations17 Jun 2022 Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, Dongsheng Li

Offline reinforcement learning (RL) aims at learning policies from previously collected static trajectory data without interacting with the real environment.

Offline RL reinforcement-learning +1

An F-shape Click Model for Information Retrieval on Multi-block Mobile Pages

1 code implementation17 Jun 2022 Lingyue Fu, Jianghao Lin, Weiwen Liu, Ruiming Tang, Weinan Zhang, Rui Zhang, Yong Yu

However, with the development of user interface (UI) design, the layout of displayed items on a result page tends to be multi-block (i. e., multi-list) style instead of a single list, which requires different assumptions to model user behaviors more accurately.

Information Retrieval Retrieval

Multi-Agent Reinforcement Learning is a Sequence Modeling Problem

1 code implementation30 May 2022 Muning Wen, Jakub Grudzien Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, Yaodong Yang

In this paper, we introduce a novel architecture named Multi-Agent Transformer (MAT) that effectively casts cooperative multi-agent reinforcement learning (MARL) into SM problems wherein the task is to map agents' observation sequence to agents' optimal action sequence.

Decision Making Multi-agent Reinforcement Learning +2

Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer

1 code implementation27 May 2022 Bin Lu, Xiaoying Gan, Weinan Zhang, Huaxiu Yao, Luoyi Fu, Xinbing Wang

To address this challenge, cross-city knowledge transfer has shown its promise, where the model learned from data-sufficient cities is leveraged to benefit the learning process of data-scarce cities.

Few-Shot Learning Graph Learning +2

Geometer: Graph Few-Shot Class-Incremental Learning via Prototype Representation

1 code implementation27 May 2022 Bin Lu, Xiaoying Gan, Lina Yang, Weinan Zhang, Luoyi Fu, Xinbing Wang

Instead of replacing and retraining the fully connected neural network classifer, Geometer predicts the label of a node by finding the nearest class prototype.

Few-Shot Class-Incremental Learning Incremental Learning +2

Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemble

no code implementations19 May 2022 Zhengyu Yang, Kan Ren, Xufang Luo, Minghuan Liu, Weiqing Liu, Jiang Bian, Weinan Zhang, Dongsheng Li

Considering the great performance of ensemble methods on both accuracy and generalization in supervised learning (SL), we design a robust and applicable method named Ensemble Proximal Policy Optimization (EPPO), which learns ensemble policies in an end-to-end manner.

reinforcement-learning Reinforcement Learning (RL)

Multi-Level Interaction Reranking with User Behavior History

1 code implementation20 Apr 2022 Yunjia Xi, Weiwen Liu, Jieming Zhu, Xilong Zhao, Xinyi Dai, Ruiming Tang, Weinan Zhang, Rui Zhang, Yong Yu

MIR combines low-level cross-item interaction and high-level set-to-list interaction, where we view the candidate items to be reranked as a set and the users' behavior history in chronological order as a list.

Recommendation Systems

PerfectDou: Dominating DouDizhu with Perfect Information Distillation

1 code implementation30 Mar 2022 Guan Yang, Minghuan Liu, Weijun Hong, Weinan Zhang, Fei Fang, Guangjun Zeng, Yue Lin

To this end, we characterize card and game features for DouDizhu to represent the perfect and imperfect information.

Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects

no code implementations20 Mar 2022 Xihuai Wang, Zhicheng Zhang, Weinan Zhang

Significant advances have recently been achieved in Multi-Agent Reinforcement Learning (MARL) which tackles sequential decision-making problems involving multiple participants.

Decision Making Multi-agent Reinforcement Learning +2

Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization

2 code implementations4 Mar 2022 Minghuan Liu, Zhengbang Zhu, Yuzheng Zhuang, Weinan Zhang, Jianye Hao, Yong Yu, Jun Wang

Recent progress in state-only imitation learning extends the scope of applicability of imitation learning to real-world settings by relieving the need for observing expert actions.

Imitation Learning Transfer Learning

Multi-View Graph Representation for Programming Language Processing: An Investigation into Algorithm Detection

1 code implementation25 Feb 2022 Ting Long, Yutong Xie, Xianyu Chen, Weinan Zhang, Qinxiang Cao, Yong Yu

We thoroughly evaluate our proposed MVG approach in the context of algorithm detection, an important and challenging subfield of PLP.

Neural Re-ranking in Multi-stage Recommender Systems: A Review

1 code implementation14 Feb 2022 Weiwen Liu, Yunjia Xi, Jiarui Qin, Fei Sun, Bo Chen, Weinan Zhang, Rui Zhang, Ruiming Tang

As the final stage of the multi-stage recommender system (MRS), re-ranking directly affects user experience and satisfaction by rearranging the input ranking lists, and thereby plays a critical role in MRS. With the advances in deep learning, neural re-ranking has become a trending topic and been widely applied in industrial applications.

Recommendation Systems Re-Ranking

Who to Watch Next: Two-side Interactive Networks for Live Broadcast Recommendation

no code implementations9 Feb 2022 Jiarui Jin, Xianyu Chen, Yuanbo Chen, Weinan Zhang, Renting Rui, Zaifan Jiang, Zhewen Su, Yong Yu

With the prevalence of live broadcast business nowadays, a new type of recommendation service, called live broadcast recommendation, is widely used in many mobile e-commerce Apps.

Retrieval

Learn over Past, Evolve for Future: Search-based Time-aware Recommendation with Sequential Behavior Data

no code implementations7 Feb 2022 Jiarui Jin, Xianyu Chen, Weinan Zhang, JunJie Huang, Ziming Feng, Yong Yu

More concretely, we first design a search-based module to retrieve a user's relevant historical behaviors, which are then mixed up with her recent records to be fed into a time-aware sequential network for capturing her time-sensitive demands.

Click-Through Rate Prediction

Efficient Policy Space Response Oracles

no code implementations28 Jan 2022 Ming Zhou, Jingxiao Chen, Ying Wen, Weinan Zhang, Yaodong Yang, Yong Yu, Jun Wang

Policy Space Response Oracle methods (PSRO) provide a general solution to learn Nash equilibrium in two-player zero-sum games but suffer from two drawbacks: (1) the computation inefficiency due to the need for consistent meta-game evaluation via simulations, and (2) the exploration inefficiency due to finding the best response against a fixed meta-strategy at every epoch.

Efficient Exploration

Generative Adversarial Exploration for Reinforcement Learning

no code implementations27 Jan 2022 Weijun Hong, Menghui Zhu, Minghuan Liu, Weinan Zhang, Ming Zhou, Yong Yu, Peng Sun

Exploration is crucial for training the optimal reinforcement learning (RL) policy, where the key is to discriminate whether a state visiting is novel.

Generative Adversarial Network Montezuma's Revenge +2

Goal-Conditioned Reinforcement Learning: Problems and Solutions

1 code implementation20 Jan 2022 Minghuan Liu, Menghui Zhu, Weinan Zhang

Goal-conditioned reinforcement learning (GCRL), related to a set of complex RL problems, trains an agent to achieve different goals under particular scenarios.

reinforcement-learning Reinforcement Learning (RL)

PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation

no code implementations COLING 2022 Juncheng Wan, Jian Yang, Shuming Ma, Dongdong Zhang, Weinan Zhang, Yong Yu, Zhoujun Li

While end-to-end neural machine translation (NMT) has achieved impressive progress, noisy input usually leads models to become fragile and unstable.

Machine Translation NMT +1

Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks

1 code implementation6 Dec 2021 Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, Bo Xu

In this paper, we facilitate the research by providing large-scale datasets, and use them to examine the usage of the Decision Transformer in the context of MARL.

Offline RL reinforcement-learning +4

Curriculum Offline Imitating Learning

no code implementations NeurIPS 2021 Minghuan Liu, Hanye Zhao, Zhengyu Yang, Jian Shen, Weinan Zhang, Li Zhao, Tie-Yan Liu

However, IL is usually limited in the capability of the behavioral policy and tends to learn a mediocre behavior from the dataset collected by the mixture of policies.

Continuous Control Imitation Learning +2

Towards Return Parity in Markov Decision Processes

1 code implementation19 Nov 2021 Jianfeng Chi, Jian Shen, Xinyi Dai, Weinan Zhang, Yuan Tian, Han Zhao

We first provide a decomposition theorem for return disparity, which decomposes the return disparity of any two MDPs sharing the same state and action spaces into the distance between group-wise reward functions, the discrepancy of group policies, and the discrepancy between state visitation distributions induced by the group policies.

Fairness Recommendation Systems

QA4PRF: A Question Answering based Framework for Pseudo Relevance Feedback

no code implementations16 Nov 2021 Handong Ma, Jiawei Hou, Chenxu Zhu, Weinan Zhang, Ruiming Tang, Jincai Lai, Jieming Zhu, Xiuqiang He, Yong Yu

Pseudo relevance feedback (PRF) automatically performs query expansion based on top-retrieved documents to better represent the user's information need so as to improve the search results.

Question Answering Semantic Similarity +1

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

1 code implementation5 Nov 2021 Chenxu Zhu, Bo Chen, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, Yong Yu

To address these three issues mentioned above, we propose Automatic Interaction Machine (AIM) with three core components, namely, Feature Interaction Search (FIS), Interaction Function Search (IFS) and Embedding Dimension Search (EDS), to select significant feature interactions, appropriate interaction functions and necessary embedding dimensions automatically in a unified framework.

Click-Through Rate Prediction

Curriculum Offline Imitation Learning

1 code implementation3 Nov 2021 Minghuan Liu, Hanye Zhao, Zhengyu Yang, Jian Shen, Weinan Zhang, Li Zhao, Tie-Yan Liu

However, IL is usually limited in the capability of the behavioral policy and tends to learn a mediocre behavior from the dataset collected by the mixture of policies.

Continuous Control Imitation Learning +2

Context-aware Reranking with Utility Maximization for Recommendation

no code implementations18 Oct 2021 Yunjia Xi, Weiwen Liu, Xinyi Dai, Ruiming Tang, Weinan Zhang, Qing Liu, Xiuqiang He, Yong Yu

As a critical task for large-scale commercial recommender systems, reranking has shown the potential of improving recommendation results by uncovering mutual influence among items.

counterfactual Graph Attention +2

Why Propagate Alone? Parallel Use of Labels and Features on Graphs

no code implementations ICLR 2022 Yangkun Wang, Jiarui Jin, Weinan Zhang, Yongyi Yang, Jiuhai Chen, Quan Gan, Yong Yu, Zheng Zhang, Zengfeng Huang, David Wipf

In this regard, it has recently been proposed to use a randomly-selected portion of the training labels as GNN inputs, concatenated with the original node features for making predictions on the remaining labels.

Node Property Prediction Property Prediction

Graph-Enhanced Exploration for Goal-oriented Reinforcement Learning

no code implementations ICLR 2022 Jiarui Jin, Sijin Zhou, Weinan Zhang, Tong He, Yong Yu, Rasool Fakoor

Goal-oriented Reinforcement Learning (GoRL) is a promising approach for scaling up RL techniques on sparse reward environments requiring long horizon planning.

Continuous Control graph construction +2

Deep Ensemble Policy Learning

no code implementations29 Sep 2021 Zhengyu Yang, Kan Ren, Xufang Luo, Weiqing Liu, Jiang Bian, Weinan Zhang, Dongsheng Li

Ensemble learning, which can consistently improve the prediction performance in supervised learning, has drawn increasing attentions in reinforcement learning (RL).

Ensemble Learning Reinforcement Learning (RL)

Inductive Relation Prediction Using Analogy Subgraph Embeddings

no code implementations ICLR 2022 Jiarui Jin, Yangkun Wang, Kounianhua Du, Weinan Zhang, Zheng Zhang, David Wipf, Yong Yu, Quan Gan

Prevailing methods for relation prediction in heterogeneous graphs aim at learning latent representations (i. e., embeddings) of observed nodes and relations, and thus are limited to the transductive setting where the relation types must be known during training.

Inductive Bias Inductive Relation Prediction +1

AARL: Automated Auxiliary Loss for Reinforcement Learning

no code implementations29 Sep 2021 Tairan He, Yuge Zhang, Kan Ren, Che Wang, Weinan Zhang, Dongsheng Li, Yuqing Yang

A good state representation is crucial to reinforcement learning (RL) while an ideal representation is hard to learn only with signals from the RL objective.

reinforcement-learning Reinforcement Learning (RL)

Task-wise Split Gradient Boosting Trees for Multi-center Diabetes Prediction

1 code implementation16 Aug 2021 Mingcheng Chen, Zhenghui Wang, Zhiyun Zhao, Weinan Zhang, Xiawei Guo, Jian Shen, Yanru Qu, Jieli Lu, Min Xu, Yu Xu, Tiange Wang, Mian Li, Wei-Wei Tu, Yong Yu, Yufang Bi, Weiqing Wang, Guang Ning

To tackle the above challenges, we employ gradient boosting decision trees (GBDT) to handle data heterogeneity and introduce multi-task learning (MTL) to solve data insufficiency.

Diabetes Prediction Multi-Task Learning

Retrieval & Interaction Machine for Tabular Data Prediction

1 code implementation11 Aug 2021 Jiarui Qin, Weinan Zhang, Rong Su, Zhirong Liu, Weiwen Liu, Ruiming Tang, Xiuqiang He, Yong Yu

Prediction over tabular data is an essential task in many data science applications such as recommender systems, online advertising, medical treatment, etc.

Attribute Click-Through Rate Prediction +2

MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning

1 code implementation5 Jun 2021 Ming Zhou, Ziyu Wan, Hanjing Wang, Muning Wen, Runzhe Wu, Ying Wen, Yaodong Yang, Weinan Zhang, Jun Wang

Our framework is comprised of three key components: (1) a centralized task dispatching model, which supports the self-generated tasks and scalable training with heterogeneous policy combinations; (2) a programming architecture named Actor-Evaluator-Learner, which achieves high parallelism for both training and sampling, and meets the evaluation requirement of auto-curriculum learning; (3) a higher-level abstraction of MARL training paradigms, which enables efficient code reuse and flexible deployments on different distributed computing paradigms.

Atari Games Distributed Computing +3

Learning to Select Cuts for Efficient Mixed-Integer Programming

no code implementations28 May 2021 Zeren Huang, Kerong Wang, Furui Liu, Hui-Ling Zhen, Weinan Zhang, Mingxuan Yuan, Jianye Hao, Yong Yu, Jun Wang

In the online A/B testing of the product planning problems with more than $10^7$ variables and constraints daily, Cut Ranking has achieved the average speedup ratio of 12. 42% over the production solver without any accuracy loss of solution.

Multiple Instance Learning

MapGo: Model-Assisted Policy Optimization for Goal-Oriented Tasks

1 code implementation13 May 2021 Menghui Zhu, Minghuan Liu, Jian Shen, Zhicheng Zhang, Sheng Chen, Weinan Zhang, Deheng Ye, Yong Yu, Qiang Fu, Wei Yang

In Goal-oriented Reinforcement learning, relabeling the raw goals in past experience to provide agents with hindsight ability is a major solution to the reward sparsity problem.

Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts

1 code implementation7 May 2021 Weinan Zhang, Xihuai Wang, Jian Shen, Ming Zhou

We specify the dynamics sample complexity and the opponent sample complexity in MARL, and conduct a theoretic analysis of return discrepancy upper bound.

Multi-agent Reinforcement Learning Reinforcement Learning (RL)

Deep Learning for Click-Through Rate Estimation

no code implementations21 Apr 2021 Weinan Zhang, Jiarui Qin, Wei Guo, Ruiming Tang, Xiuqiang He

In this survey, we provide a comprehensive review of deep learning models for CTR estimation tasks.

Recommendation Systems

An Adversarial Imitation Click Model for Information Retrieval

1 code implementation13 Apr 2021 Xinyi Dai, Jianghao Lin, Weinan Zhang, Shuai Li, Weiwen Liu, Ruiming Tang, Xiuqiang He, Jianye Hao, Jun Wang, Yong Yu

Modern information retrieval systems, including web search, ads placement, and recommender systems, typically rely on learning from user feedback.

Imitation Learning Information Retrieval +2

Bag of Tricks for Node Classification with Graph Neural Networks

2 code implementations24 Mar 2021 Yangkun Wang, Jiarui Jin, Weinan Zhang, Yong Yu, Zheng Zhang, David Wipf

Over the past few years, graph neural networks (GNN) and label propagation-based methods have made significant progress in addressing node classification tasks on graphs.

Classification General Classification +2

NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning

3 code implementations1 Feb 2021 Rongjun Qin, Songyi Gao, Xingyuan Zhang, Zhen Xu, Shengkai Huang, Zewen Li, Weinan Zhang, Yang Yu

We evaluate existing offline RL algorithms on NeoRL and argue that the performance of a policy should also be compared with the deterministic version of the behavior policy, instead of the dataset reward.

Offline RL reinforcement-learning +1

Universal Trading for Order Execution with Oracle Policy Distillation

no code implementations28 Jan 2021 Yuchen Fang, Kan Ren, Weiqing Liu, Dong Zhou, Weinan Zhang, Jiang Bian, Yong Yu, Tie-Yan Liu

As a fundamental problem in algorithmic trading, order execution aims at fulfilling a specific trading order, either liquidation or acquirement, for a given instrument.

Algorithmic Trading reinforcement-learning +1

Explore with Dynamic Map: Graph Structured Reinforcement Learning

no code implementations1 Jan 2021 Jiarui Jin, Sijin Zhou, Weinan Zhang, Rasool Fakoor, David Wipf, Tong He, Yong Yu, Zheng Zhang, Alex Smola

In reinforcement learning, a map with states and transitions built based on historical trajectories is often helpful in exploration and exploitation.

reinforcement-learning Reinforcement Learning (RL)

Non-iterative Parallel Text Generation via Glancing Transformer

no code implementations1 Jan 2021 Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, Lei LI

Although non-autoregressive models with one-iteration generation achieves remarkable inference speed-up, they still falls behind their autoregressive counterparts inprediction accuracy.

Language Modelling Text Generation

Regioned Episodic Reinforcement Learning

no code implementations1 Jan 2021 Jiarui Jin, Cong Chen, Ming Zhou, Weinan Zhang, Rasool Fakoor, David Wipf, Yong Yu, Jun Wang, Alex Smola

Goal-oriented reinforcement learning algorithms are often good at exploration, not exploitation, while episodic algorithms excel at exploitation, not exploration.

reinforcement-learning Reinforcement Learning (RL)

Which Heroes to Pick? Learning to Draft in MOBA Games with Neural Networks and Tree Search

no code implementations18 Dec 2020 Sheng Chen, Menghui Zhu, Deheng Ye, Weinan Zhang, Qiang Fu, Wei Yang

Hero drafting is essential in MOBA game playing as it builds the team of each side and directly affects the match outcome.

An Embedding Learning Framework for Numerical Features in CTR Prediction

1 code implementation16 Dec 2020 Huifeng Guo, Bo Chen, Ruiming Tang, Weinan Zhang, Zhenguo Li, Xiuqiang He

In this paper, we propose a novel embedding learning framework for numerical features in CTR prediction (AutoDis) with high model capacity, end-to-end training and unique representation properties preserved.

Click-Through Rate Prediction Feature Engineering +1

Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings

1 code implementation14 Dec 2020 Qipeng Guo, Zhijing Jin, Ziyu Wang, Xipeng Qiu, Weinan Zhang, Jun Zhu, Zheng Zhang, David Wipf

Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.

Knowledge Graphs Text Generation

Towards Generalized Implementation of Wasserstein Distance in GANs

1 code implementation7 Dec 2020 Minkai Xu, Zhiming Zhou, Guansong Lu, Jian Tang, Weinan Zhang, Yong Yu

Wasserstein GANs (WGANs), built upon the Kantorovich-Rubinstein (KR) duality of Wasserstein distance, is one of the most theoretically sound GAN models.

Reciprocal Supervised Learning Improves Neural Machine Translation

1 code implementation5 Dec 2020 Minkai Xu, Mingxuan Wang, Zhouhan Lin, Hao Zhou, Weinan Zhang, Lei LI

Despite the recent success on image classification, self-training has only achieved limited gains on structured prediction tasks such as neural machine translation (NMT).

Image Classification Knowledge Distillation +4

U-rank: Utility-oriented Learning to Rank with Implicit Feedback

no code implementations1 Nov 2020 Xinyi Dai, Jiawei Hou, Qing Liu, Yunjia Xi, Ruiming Tang, Weinan Zhang, Xiuqiang He, Jun Wang, Yong Yu

To this end, we propose a novel ranking framework called U-rank that directly optimizes the expected utility of the ranking list.

Click-Through Rate Prediction Learning-To-Rank +2

Efficient Projection-Free Algorithms for Saddle Point Problems

no code implementations NeurIPS 2020 Cheng Chen, Luo Luo, Weinan Zhang, Yong Yu

The Frank-Wolfe algorithm is a classic method for constrained optimization problems.

Model-based Policy Optimization with Unsupervised Model Adaptation

1 code implementation NeurIPS 2020 Jian Shen, Han Zhao, Weinan Zhang, Yong Yu

However, due to the potential distribution mismatch between simulated data and real data, this could lead to degraded performance.

Continuous Control Model-based Reinforcement Learning +2

Feature-Based Matrix Factorization

no code implementations11 Sep 2011 Tianqi Chen, Zhao Zheng, Qiuxia Lu, Weinan Zhang, Yong Yu

Recommender system has been more and more popular and widely used in many applications recently.

Recommendation Systems

Cannot find the paper you are looking for? You can Submit a new open access paper.