Search Results for author: Rui Zhao

Found 181 papers, 74 papers with code

RBF-Softmax: Learning Deep Representative Prototypes with Radial Basis Function Softmax

1 code implementation ECCV 2020 Xiao Zhang, Rui Zhao, Yu Qiao, Hongsheng Li

To address this problem, this paper introduces a novel Radial Basis Function (RBF) distances to replace the commonly used inner products in the softmax loss function, such that it can adaptively assign losses to regularize the intra-class and inter-class distances by reshaping the relative differences, and thus creating more representative prototypes of classes to improve optimization.

SQL-to-Schema Enhances Schema Linking in Text-to-SQL

no code implementations15 May 2024 Sun Yang, Qiong Su, Zhishuai Li, Ziyue Li, Hangyu Mao, Chenxi Liu, Rui Zhao

Consequently, there is a critical need to filter out unnecessary tables and columns, directing the language models attention to relevant tables and columns with schema-linking, to reduce errors during SQL generation.

Text-To-SQL

Resilient control of networked switched systems subject to deception attack and DoS attack

no code implementations10 May 2024 Rui Zhao, Zhiqiang Zuo, Ying Tan, Yijing Wang, Wentao Zhang

In this paper, the resilient control for switched systems in the presence of deception attack and denial-of-service (DoS) attack is addressed.

Estimating Noisy Class Posterior with Part-level Labels for Noisy Label Learning

1 code implementation8 May 2024 Rui Zhao, Bin Shi, Jianfei Ruan, Tianze Pan, Bo Dong

Utilizing this framework with part-level labels, we can learn the noisy class posteriors more precisely by guiding the model to integrate information from various parts, ultimately improving the classification performance.

Causal Evaluation of Language Models

1 code implementation1 May 2024 Sirui Chen, Bo Peng, Meiqi Chen, Ruiqi Wang, Mengying Xu, Xingyu Zeng, Rui Zhao, Shengjie Zhao, Yu Qiao, Chaochao Lu

Recent advances in language models have expanded the horizons of artificial intelligence across various domains, sparking inquiries into their potential for causal reasoning.

Causal Discovery Causal Inference +1

X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner

no code implementations18 Apr 2024 Haoyuan Jiang, Ziyue Li, Hua Wei, Xuantang Xiong, Jingqing Ruan, Jiaming Lu, Hangyu Mao, Rui Zhao

The effectiveness of traffic light control has been significantly improved by current reinforcement learning-based approaches via better cooperation among multiple traffic lights.

Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model

1 code implementation16 Apr 2024 Hengyuan Zhang, Yanru Wu, Dawei Li, Zacc Yang, Rui Zhao, Yong Jiang, Fei Tan

In an overall evaluation of both speciality and versatility, CoFiTune consistently outperforms baseline methods across diverse tasks and model scales.

Language Modelling Large Language Model

Xiwu: A Basis Flexible and Learnable LLM for High Energy Physics

1 code implementation8 Apr 2024 Zhengde Zhang, Yiyu Zhang, Haodong Yao, Jianwen Luo, Rui Zhao, Bo Huang, Jiameng Zhao, Yipu Liao, Ke Li, Lina Zhao, Jun Cao, Fazhi Qi, Changzheng Yuan

To address this challenge, a sophisticated large language model system named as Xiwu has been developed, allowing you switch between the most advanced foundation models and quickly teach the model domain knowledge.

Code Generation Language Modelling +1

SocialGenPod: Privacy-Friendly Generative AI Social Web Applications with Decentralised Personal Data Stores

1 code implementation15 Mar 2024 Vidminas Vizgirda, Rui Zhao, Naman Goel

Unlike centralised Web and data architectures that keep user data tied to application and service providers, we show how one can use Solid -- a decentralised Web specification -- to decouple user data from generative AI applications.

PET-SQL: A Prompt-enhanced Two-stage Text-to-SQL Framework with Cross-consistency

1 code implementation13 Mar 2024 Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye, Ziyue Li, Rui Zhao, Hangyu Mao

Then, in the first stage, question-SQL pairs are retrieved as few-shot demonstrations, prompting the LLM to generate a preliminary SQL (PreSQL).

In-Context Learning Text-To-SQL

DragAnything: Motion Control for Anything using Entity Representation

2 code implementations12 Mar 2024 Weijia Wu, Zhuang Li, YuChao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, Di Zhang

We introduce DragAnything, which utilizes a entity representation to achieve motion control for any object in controllable video generation.

Object Video Generation

Perennial Semantic Data Terms of Use for Decentralized Web

1 code implementation12 Mar 2024 Rui Zhao, Jun Zhao

We believe this work demonstrates a practicality of a perennial DToU language and the potential of a paradigm shift to how users interact with data and applications in a decentralized Web, offering both improved privacy and usability.

Navigate

Benchmarking the Text-to-SQL Capability of Large Language Models: A Comprehensive Evaluation

no code implementations5 Mar 2024 Bin Zhang, Yuxiao Ye, Guoqing Du, Xiaoru Hu, Zhishuai Li, Sun Yang, Chi Harold Liu, Rui Zhao, Ziyue Li, Hangyu Mao

Then we formulate five evaluation tasks to comprehensively assess the performance of diverse methods across various LLMs throughout the Text-to-SQL process. Our study highlights the performance disparities among LLMs and proposes optimal in-context learning solutions tailored to each task.

Benchmarking In-Context Learning +1

Consistency Matters: Explore LLMs Consistency From a Black-Box Perspective

no code implementations27 Feb 2024 Fufangchen Zhao, Guoqiang Jin, Jiaheng Huang, Rui Zhao, Fei Tan

The solution to this problem is often time-consuming and labor-intensive, and there is also an additional cost of secondary deployment, resulting in economic and time losses.

Non-Neighbors Also Matter to Kriging: A New Contrastive-Prototypical Learning

1 code implementation23 Jan 2024 Zhishuai Li, Yunhao Nie, Ziyue Li, Lei Bai, Yisheng Lv, Rui Zhao

As a pre-trained paradigm, we conduct the Kriging task from a new perspective of representation: we aim to first learn robust and general representations and then recover attributes from representations.

Attribute Self-Supervised Learning

Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment

1 code implementation25 Dec 2023 Rui Zhao, Liang Zhang, Biao Fu, Cong Hu, Jinsong Su, Yidong Chen

The first KL divergence optimizes the conditional variational autoencoder and regularizes the encoder outputs, while the second KL divergence performs a self-distillation from the posterior path to the prior path, ensuring the consistency of decoder outputs.

Decoder Sign Language Translation +1

DuaLight: Enhancing Traffic Signal Control by Leveraging Scenario-Specific and Scenario-Shared Knowledge

1 code implementation22 Dec 2023 Jiaming Lu, Jingqing Ruan, Haoyuan Jiang, Ziyue Li, Hangyu Mao, Rui Zhao

Furthermore, we implement a scenario-shared Co-Train module to facilitate the learning of generalizable dynamics information across different scenarios.

Decision Making

VisionTraj: A Noise-Robust Trajectory Recovery Framework based on Large-scale Camera Network

1 code implementation11 Dec 2023 Zhishuai Li, Ziyue Li, Xiaoru Hu, Guoqing Du, Yunhao Nie, Feng Zhu, Lei Bai, Rui Zhao

Trajectory recovery based on the snapshots from the city-wide multi-camera network facilitates urban mobility sensing and driveway optimization.

Clustering Denoising

VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence

no code implementations4 Dec 2023 YuChao Gu, Yipin Zhou, Bichen Wu, Licheng Yu, Jia-Wei Liu, Rui Zhao, Jay Zhangjie Wu, David Junhao Zhang, Mike Zheng Shou, Kevin Tang

In contrast to previous methods that rely on dense correspondences, we introduce the VideoSwap framework that exploits semantic point correspondences, inspired by our observation that only a small number of semantic points are necessary to align the subject's motion trajectory and modify its shape.

Video Editing

Hulk: A Universal Knowledge Translator for Human-Centric Tasks

2 code implementations4 Dec 2023 Yizhou Wang, Yixuan Wu, Shixiang Tang, Weizhen He, Xun Guo, Feng Zhu, Lei Bai, Rui Zhao, Jian Wu, Tong He, Wanli Ouyang

Human-centric perception tasks, e. g., pedestrian detection, skeleton-based action recognition, and pose estimation, have wide industrial applications, such as metaverse and sports analysis.

3D Human Pose Estimation Action Recognition +8

Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approach

no code implementations23 Nov 2023 Bin Zhang, Hangyu Mao, Jingqing Ruan, Ying Wen, Yang Li, Shao Zhang, Zhiwei Xu, Dapeng Li, Ziyue Li, Rui Zhao, Lijuan Li, Guoliang Fan

The remarkable progress in Large Language Models (LLMs) opens up new avenues for addressing planning and decision-making problems in Multi-Agent Systems (MAS).

Decision Making Hallucination +3

TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems

no code implementations19 Nov 2023 Yilun Kong, Jingqing Ruan, Yihong Chen, Bin Zhang, Tianpeng Bao, Shiwei Shi, Guoqing Du, Xiaoru Hu, Hangyu Mao, Ziyue Li, Xingyu Zeng, Rui Zhao

Large Language Models (LLMs) have demonstrated proficiency in addressing tasks that necessitate a combination of task planning and the usage of external tools that require a blend of task planning and the utilization of external tools, such as APIs.

In-Context Learning Language Modelling +1

What Large Language Models Bring to Text-rich VQA?

no code implementations13 Nov 2023 Xuejing Liu, Wei Tang, Xinzhe Ni, Jinghui Lu, Rui Zhao, Zechao Li, Fei Tan

This pipeline achieved superior performance compared to the majority of existing Multimodal Large Language Models (MLLM) on four text-rich VQA datasets.

Image Comprehension Optical Character Recognition (OCR) +2

KITS: Inductive Spatio-Temporal Kriging with Increment Training Strategy

no code implementations5 Nov 2023 Qianxiong Xu, Cheng Long, Ziyue Li, Sijie Ruan, Rui Zhao, Zhishuai Li

To address this issue, we first present a novel Increment training strategy: instead of masking nodes (and reconstructing them), we add virtual nodes into the training graph so as to mitigate the graph gap issue naturally.

A Critical Perceptual Pre-trained Model for Complex Trajectory Recovery

no code implementations5 Nov 2023 Dedong Li, Ziyue Li, Zhishuai Li, Lei Bai, Qingyuan Gong, Lijun Sun, Wolfgang Ketter, Rui Zhao

Then, we propose a Multi-view Graph and Complexity Aware Transformer (MGCAT) model to encode these semantics in trajectory pre-training from two aspects: 1) adaptively aggregate the multi-view graph features considering trajectory pattern, and 2) higher attention to critical nodes in a complex trajectory.

Decentralised, Scalable and Privacy-Preserving Synthetic Data Generation

no code implementations30 Oct 2023 Vishal Ramesh, Rui Zhao, Naman Goel

Synthetic data is emerging as a promising way to harness the value of data, while reducing privacy risks.

Privacy Preserving Synthetic Data Generation

Reboost Large Language Model-based Text-to-SQL, Text-to-Python, and Text-to-Function -- with Real Applications in Traffic Domain

no code implementations28 Oct 2023 Guanghu Sui, Zhishuai Li, Ziyue Li, Sun Yang, Jingqing Ruan, Hangyu Mao, Rui Zhao

Our experiments with Large Language Models (LLMs) illustrate the significant performance improvement on the business dataset and prove the substantial potential of our method.

Language Modelling Large Language Model +1

DynVideo-E: Harnessing Dynamic NeRF for Large-Scale Motion- and View-Change Human-Centric Video Editing

no code implementations16 Oct 2023 Jia-Wei Liu, Yan-Pei Cao, Jay Zhangjie Wu, Weijia Mao, YuChao Gu, Rui Zhao, Jussi Keppo, Ying Shan, Mike Zheng Shou

To overcome this, we propose to introduce the dynamic Neural Radiance Fields (NeRF) as the innovative video representation, where the editing can be performed in the 3D spaces and propagated to the entire video via the deformation field.

Style Transfer Super-Resolution +1

MotionDirector: Motion Customization of Text-to-Video Diffusion Models

1 code implementation12 Oct 2023 Rui Zhao, YuChao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jiawei Liu, Weijia Wu, Jussi Keppo, Mike Zheng Shou

Given a set of video clips of the same motion concept, the task of Motion Customization is to adapt existing text-to-video diffusion models to generate videos with this motion.

MeanAP-Guided Reinforced Active Learning for Object Detection

no code implementations12 Oct 2023 Zhixuan Liang, Xingyu Zeng, Rui Zhao, Ping Luo

Active learning presents a promising avenue for training high-performance models with minimal labeled data, achieved by judiciously selecting the most informative instances to label and incorporating them into the task learner.

Active Object Detection Object +2

InstructDET: Diversifying Referring Object Detection with Generalized Instructions

1 code implementation8 Oct 2023 Ronghao Dang, Jiangyan Feng, Haodong Zhang, Chongjian Ge, Lin Song, Lijun Gong, Chengju Liu, Qijun Chen, Feng Zhu, Rui Zhao, Yibing Song

In order to encompass common detection expressions, we involve emerging vision-language model (VLM) and large language model (LLM) to generate instructions guided by text prompts and object bbxs, as the generalizations of foundation models are effective to produce human-like expressions (e. g., describing object property, category, and relationship).

Language Modelling Large Language Model +4

Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation

1 code implementation27 Sep 2023 David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, YuChao Gu, Difei Gao, Mike Zheng Shou

In this paper, we are the first to propose a hybrid model, dubbed as Show-1, which marries pixel-based and latent-based VDMs for text-to-video generation.

Text-to-Video Generation Video Alignment +1

t-SOT FNT: Streaming Multi-talker ASR with Text-only Domain Adaptation Capability

no code implementations15 Sep 2023 Jian Wu, Naoyuki Kanda, Takuya Yoshioka, Rui Zhao, Zhuo Chen, Jinyu Li

Token-level serialized output training (t-SOT) was recently proposed to address the challenge of streaming multi-talker automatic speech recognition (ASR).

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Link-Context Learning for Multimodal LLMs

1 code implementation15 Aug 2023 Yan Tai, Weichen Fan, Zhao Zhang, Feng Zhu, Rui Zhao, Ziwei Liu

The ability to learn from context with novel concepts, and deliver appropriate responses are essential in human conversations.

Few-Shot Learning In-Context Learning +1

DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models

1 code implementation NeurIPS 2023 Weijia Wu, Yuzhong Zhao, Hao Chen, YuChao Gu, Rui Zhao, Yefei He, Hong Zhou, Mike Zheng Shou, Chunhua Shen

To showcase the power of the proposed approach, we generate datasets with rich dense pixel-wise labels for a wide range of downstream tasks, including semantic segmentation, instance segmentation, and depth estimation.

Decoder Depth Estimation +6

TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage

no code implementations7 Aug 2023 Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng, Rui Zhao

With recent advancements in natural language processing, Large Language Models (LLMs) have emerged as powerful tools for various real-world applications.

Language Modelling Large Language Model

Relation-Aware Distribution Representation Network for Person Clustering with Multiple Modalities

no code implementations1 Aug 2023 Kaijian Liu, Shixiang Tang, Ziyue Li, Zhishuai Li, Lei Bai, Feng Zhu, Rui Zhao

The distribution representation of a clue is a vector consisting of the relation between this clue and all other clues from all modalities, thus being modality agnostic and good for person clustering.

Clustering Relation

Described Object Detection: Liberating Object Detection with Flexible Expressions

2 code implementations NeurIPS 2023 Chi Xie, Zhao Zhang, Yixuan Wu, Feng Zhu, Rui Zhao, Shuang Liang

In this paper, we advance them to a more practical setting called Described Object Detection (DOD) by expanding category names to flexible language expressions for OVD and overcoming the limitation of REC only grounding the pre-existing object.

Binary Classification Described Object Detection +5

Interaction-Aware Planning With Deep Inverse Reinforcement Learning for Human-Like Autonomous Driving in Merge Scenarios

1 code implementation journal 2023 Jiangfeng Nan, Weiwen Deng, Member, IEEE, Ruzheng Zhang, Ying Wang, Rui Zhao, Juan Ding

To consider the interaction factor, the reward function for planning is utilized to evaluate the joint trajectories of the autonomous driving vehicle (ADV) and traffic vehicles.

Autonomous Driving Decision Making

Patch-Level Contrasting without Patch Correspondence for Accurate and Dense Contrastive Representation Learning

no code implementations23 Jun 2023 Shaofeng Zhang, Feng Zhu, Rui Zhao, Junchi Yan

On classification tasks, for ViT-S, ADCLR achieves 77. 5% top-1 accuracy on ImageNet with linear probing, outperforming our baseline (DINO) without our devised techniques as plug-in, by 0. 5%.

Instance Segmentation object-detection +4

Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

1 code implementation15 Jun 2023 Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, Hongsheng Li

By fine-tuning CLIP on HPD v2, we obtain Human Preference Score v2 (HPS v2), a scoring model that can more accurately predict human preferences on generated images.

Image Generation

Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions

1 code implementation13 Jun 2023 Weizhen He, Yiheng Deng, Shixiang Tang, Qihao Chen, Qingsong Xie, Yizhou Wang, Lei Bai, Feng Zhu, Rui Zhao, Wanli Ouyang, Donglian Qi, Yunfeng Yan

This paper strives to resolve this problem by proposing a new instruct-ReID task that requires the model to retrieve images according to the given image or language instructions.

Person Re-Identification

Correlated Time Series Self-Supervised Representation Learning via Spatiotemporal Bootstrapping

1 code implementation12 Jun 2023 Luxuan Wang, Lei Bai, Ziyue Li, Rui Zhao, Fugee Tsung

We evaluated the effectiveness and flexibility of our representation learning framework on correlated time series forecasting and cold-start transferring the forecasting model to new instances with limited data.

Correlated Time Series Forecasting Representation Learning +1

Dynamic Causal Graph Convolutional Network for Traffic Prediction

1 code implementation12 Jun 2023 Junpeng Lin, Ziyue Li, Zhishuai Li, Lei Bai, Rui Zhao, Chen Zhang

In this work, we propose a novel approach for traffic prediction that embeds time-varying dynamic Bayesian network to capture the fine spatiotemporal topology of traffic data.

Traffic Prediction

MM-DAG: Multi-task DAG Learning for Multi-modal Data -- with Application for Traffic Congestion Analysis

1 code implementation5 Jun 2023 Tian Lan, Ziyue Li, Zhishuai Li, Lei Bai, Man Li, Fugee Tsung, Wolfgang Ketter, Rui Zhao, Chen Zhang

This encourages the multi-task design: with each DAG as a task, the MM-DAG tries to learn the multiple DAGs jointly so that their consensus and consistency are maximized.

Balancing Logit Variation for Long-tailed Semantic Segmentation

1 code implementation CVPR 2023 Yuchao Wang, Jingjing Fei, Haochen Wang, Wei Li, Tianpeng Bao, Liwei Wu, Rui Zhao, Yujun Shen

In this way, we manage to close the gap between the feature areas of different categories, resulting in a more balanced representation.

Semantic Segmentation

ZeroPose: CAD-Model-based Zero-Shot Pose Estimation

no code implementations29 May 2023 Jianqiu Chen, Mingshan Sun, Tianpeng Bao, Rui Zhao, Liwei Wu, Zhenyu He

In this paper, we present a CAD model-based zero-shot pose estimation pipeline called ZeroPose.

Instance Segmentation Object +3

Deeply Coupled Cross-Modal Prompt Learning

1 code implementation29 May 2023 Xuejing Liu, Wei Tang, Jinghui Lu, Rui Zhao, Zhaojun Guo, Fei Tan

Recent advancements in multimodal foundation models (e. g., CLIP) have excelled in zero-shot generalization.

Domain Adaptation Few-Shot Learning +3

Advancing Referring Expression Segmentation Beyond Single Image

1 code implementation ICCV 2023 Yixuan Wu, Zhao Zhang, Xie Chi, Feng Zhu, Rui Zhao

To overcome this limitation, we propose a more realistic and general setting, named Group-wise Referring Expression Segmentation (GRES), which expands RES to a collection of related images, allowing the described objects to be present in a subset of input images.

Co-Salient Object Detection Object +4

Stackelberg Decision Transformer for Asynchronous Action Coordination in Multi-Agent Systems

no code implementations13 May 2023 Bin Zhang, Hangyu Mao, Lijuan Li, Zhiwei Xu, Dapeng Li, Rui Zhao, Guoliang Fan

Our research contributes to the development of an effective and adaptable asynchronous action coordination method that can be widely applied to various task types and environmental configurations in MAS.

Decision Making Multi-agent Reinforcement Learning

Human Preference Score: Better Aligning Text-to-Image Models with Human Preference

1 code implementation ICCV 2023 Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, Hongsheng Li

To address this issue, we collect a dataset of human choices on generated images from the Stable Foundation Discord channel.

CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching

1 code implementation CVPR 2023 Xiaoshi Wu, Feng Zhu, Rui Zhao, Hongsheng Li

To overcome these obstacles, we propose CORA, a DETR-style framework that adapts CLIP for Open-vocabulary detection by Region prompting and Anchor pre-matching.

Ranked #6 on Open Vocabulary Object Detection on MSCOCO (using extra training data)

Described Object Detection object-detection +2

Explore the Power of Synthetic Data on Few-shot Object Detection

no code implementations23 Mar 2023 Shaobo Lin, Kun Wang, Xingyu Zeng, Rui Zhao

To construct a representative synthetic training dataset, we maximize the diversity of the selected images via a sample-based and cluster-based method.

Few-Shot Object Detection Object +3

SpikeCV: Open a Continuous Computer Vision Era

1 code implementation21 Mar 2023 Yajing Zheng, Jiyuan Zhang, Rui Zhao, Jianhao Ding, Shiyan Chen, Ruiqin Xiong, Zhaofei Yu, Tiejun Huang

SpikeCV focuses on encapsulation for spike data, standardization for dataset interfaces, modularization for vision tasks, and real-time applications for challenging scenes.

HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining

1 code implementation CVPR 2023 Shixiang Tang, Cheng Chen, Qingsong Xie, Meilin Chen, Yizhou Wang, Yuanzheng Ci, Lei Bai, Feng Zhu, Haiyang Yang, Li Yi, Rui Zhao, Wanli Ouyang

Specifically, we propose a \textbf{HumanBench} based on existing datasets to comprehensively evaluate on the common ground the generalization abilities of different pretraining methods on 19 datasets from 6 diverse downstream tasks, including person ReID, pose estimation, human parsing, pedestrian attribute recognition, pedestrian detection, and crowd counting.

 Ranked #1 on Pedestrian Attribute Recognition on PA-100K (using extra training data)

Attribute Autonomous Driving +5

UniHCP: A Unified Model for Human-Centric Perceptions

1 code implementation CVPR 2023 Yuanzheng Ci, Yizhou Wang, Meilin Chen, Shixiang Tang, Lei Bai, Feng Zhu, Rui Zhao, Fengwei Yu, Donglian Qi, Wanli Ouyang

When adapted to a specific task, UniHCP achieves new SOTAs on a wide range of human-centric tasks, e. g., 69. 8 mIoU on CIHP for human parsing, 86. 18 mA on PA-100K for attribute prediction, 90. 3 mAP on Market1501 for ReID, and 85. 8 JI on CrowdHuman for pedestrian detection, performing better than specialized models tailored for each task.

2D Pose Estimation Attribute +8

Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation

no code implementations CVPR 2023 Rui Zhao, Wei Li, Zhipeng Hu, Lincheng Li, Zhengxia Zou, Zhenwei Shi, Changjie Fan

In our method, taking the power of large-scale pre-trained multi-modal CLIP and neural rendering, T2P searches both continuous facial parameters and discrete facial parameters in a unified framework.

3D Generation Face Model +3

An Effective Crop-Paste Pipeline for Few-shot Object Detection

no code implementations28 Feb 2023 Shaobo Lin, Kun Wang, Xingyu Zeng, Rui Zhao

Specifically, we first discover the base images which contain the FP of novel categories and select a certain amount of samples from them for the base and novel categories balance.

Data Augmentation Few-Shot Object Detection +1

Efficient Masked Autoencoders with Self-Consistency

no code implementations28 Feb 2023 Zhaowen Li, Yousong Zhu, Zhiyang Chen, Wei Li, Chaoyang Zhao, Liwei Wu, Rui Zhao, Ming Tang, Jinqiao Wang

However, its high random mask ratio would result in two serious problems: 1) the data are not efficiently exploited, which brings inefficient pre-training (\eg, 1600 epochs for MAE $vs.$ 300 epochs for the supervised), and 2) the high uncertainty and inconsistency of the pre-trained model, \ie, the prediction of the same patch may be inconsistent under different mask rounds.

Language Modelling Masked Language Modeling +3

Saliency Guided Contrastive Learning on Scene Images

no code implementations22 Feb 2023 Meilin Chen, Yizhou Wang, Shixiang Tang, Feng Zhu, Haiyang Yang, Lei Bai, Rui Zhao, Donglian Qi, Wanli Ouyang

Despite being feasible, recent works largely overlooked discovering the most discriminative regions for contrastive learning to object representations in scene images.

Contrastive Learning Linear evaluation +2

SparseMAE: Sparse Training Meets Masked Autoencoders

no code implementations ICCV 2023 Aojun Zhou, Yang Li, Zipeng Qin, Jianbo Liu, Junting Pan, Renrui Zhang, Rui Zhao, Peng Gao, Hongsheng Li

In this paper, we aim to reduce model complexity from large vision transformers pretrained by MAE with assistant of sparse training.

Exploring Stochastic Autoregressive Image Modeling for Visual Representation

1 code implementation3 Dec 2022 Yu Qi, Fan Yang, Yousong Zhu, Yufei Liu, Liwei Wu, Rui Zhao, Wei Li

By introducing stochastic prediction and the parallel encoder-decoder, SAIM significantly improve the performance of autoregressive image modeling.

Decoder Self-Supervised Learning

PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasets

1 code implementation27 Nov 2022 Jinghui Lu, Rui Zhao, Brian Mac Namee, Fei Tan

In this work, we present a ``versatile'' model -- the Prompting-based Unified NER system (PUnifiedNER) -- that works with data from different domains and can recognise up to 37 entity types simultaneously, and theoretically it could be as many as possible.

named-entity-recognition Named Entity Recognition +1

MIAD: A Maintenance Inspection Dataset for Unsupervised Anomaly Detection

no code implementations25 Nov 2022 Tianpeng Bao, Jiadong Chen, Wei Li, Xiang Wang, Jingjing Fei, Liwei Wu, Rui Zhao, Ye Zheng

However, existing datasets for unsupervised anomaly detection are biased towards manufacturing inspection, not considering maintenance inspection which is usually conducted under outdoor uncontrolled environment such as varying camera viewpoints, messy background and degradation of object surface after long-term working.

Unsupervised Anomaly Detection

LongFNT: Long-form Speech Recognition with Factorized Neural Transducer

no code implementations17 Nov 2022 Xun Gong, Yu Wu, Jinyu Li, Shujie Liu, Rui Zhao, Xie Chen, Yanmin Qian

This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Geo6D: Geometric Constraints Learning for 6D Pose Estimation

no code implementations20 Oct 2022 Jianqiu Chen, Mingshan Sun, Ye Zheng, Tianpeng Bao, Zhenyu He, Donghai Li, Guoqiang Jin, Rui Zhao, Liwei Wu, Xiaoke Jiang

Numerous 6D pose estimation methods have been proposed that employ end-to-end regression to directly estimate the target pose parameters.

6D Pose Estimation object-detection +3

A Unified Framework with Meta-dropout for Few-shot Learning

no code implementations12 Oct 2022 Shaobo Lin, Xingyu Zeng, Rui Zhao

Conventional training of deep neural networks usually requires a substantial amount of data with expensive human annotations.

Few-Shot Image Classification Few-Shot Learning +2

What Makes Pre-trained Language Models Better Zero-shot Learners?

1 code implementation30 Sep 2022 Jinghui Lu, Dongsheng Zhu, Weidong Han, Rui Zhao, Brian Mac Namee, Fei Tan

Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori.

Language Modelling text-classification +2

Obj2Seq: Formatting Objects as Sequences with Class Prompt for Visual Tasks

2 code implementations28 Sep 2022 Zhiyang Chen, Yousong Zhu, Zhaowen Li, Fan Yang, Wei Li, Haixin Wang, Chaoyang Zhao, Liwei Wu, Rui Zhao, Jinqiao Wang, Ming Tang

Obj2Seq is able to flexibly determine input categories to satisfy customized requirements, and be easily extended to different visual tasks.

Multi-Label Classification Object +2

Jointly Contrastive Representation Learning on Road Network and Trajectory

1 code implementation14 Sep 2022 Zhenyu Mao, Ziyue Li, Dedong Li, Lei Bai, Rui Zhao

Unlike the existing cross-scale contrastive learning methods on graphs that only contrast a graph and its belonging nodes, the contrast between road segment and trajectory is elaborately tailored via novel positive sampling and adaptive weighting strategies.

Contrastive Learning Representation Learning +1

Uni6Dv2: Noise Elimination for 6D Pose Estimation

no code implementations15 Aug 2022 Mingshan Sun, Ye Zheng, Tianpeng Bao, Jianqiu Chen, Guoqiang Jin, Liwei Wu, Rui Zhao, Xiaoke Jiang

Uni6D is the first 6D pose estimation approach to employ a unified backbone network to extract features from both RGB and depth images.

6D Pose Estimation Denoising +2

Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identification

no code implementations1 Aug 2022 Xulin Li, Yan Lu, Bin Liu, Yating Liu, Guojun Yin, Qi Chu, Jinyang Huang, Feng Zhu, Rui Zhao, Nenghai Yu

But we find existing graph-based methods in the visible-infrared person re-identification task (VI-ReID) suffer from bad generalization because of two issues: 1) train-test modality balance gap, which is a property of VI-ReID task.

counterfactual Person Re-Identification

Auto-Encoding Adversarial Imitation Learning

no code implementations22 Jun 2022 Kaifeng Zhang, Rui Zhao, Ziming Zhang, Yang Gao

In this work, we propose Auto-Encoding Adversarial Imitation Learning (AEAIL), a robust and scalable AIL framework.

Imitation Learning Reinforcement Learning (RL)

Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains

no code implementations10 May 2022 Haiyang Yang, Meilin Chen, Yizhou Wang, Shixiang Tang, Feng Zhu, Lei Bai, Rui Zhao, Wanli Ouyang

While recent self-supervised learning methods have achieved good performances with evaluation set on the same domain as the training set, they will have an undesirable performance decrease when tested on a different domain.

Self-Supervised Learning

Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimation

no code implementations CVPR 2022 Xiaoke Jiang, Donghai Li, Hao Chen, Ye Zheng, Rui Zhao, Liwei Wu

They use a 2D CNN for RGB images and a per-pixel point cloud network for depth data, as well as a fusion network for feature fusion.

6D Pose Estimation

UniVIP: A Unified Framework for Self-Supervised Visual Pre-training

no code implementations CVPR 2022 Zhaowen Li, Yousong Zhu, Fan Yang, Wei Li, Chaoyang Zhao, Yingying Chen, Zhiyang Chen, Jiahao Xie, Liwei Wu, Rui Zhao, Ming Tang, Jinqiao Wang

Furthermore, our method can also exploit single-centric-object dataset such as ImageNet and outperforms BYOL by 2. 5% with the same pre-training epochs in linear probing, and surpass current self-supervised object detection methods on COCO dataset, demonstrating its universality and potential.

Image Classification Object +4

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

1 code implementation CVPR 2022 Yuchao Wang, Haochen Wang, Yujun Shen, Jingjing Fei, Wei Li, Guoqiang Jin, Liwei Wu, Rui Zhao, Xinyi Le

A common practice is to select the highly confident predictions as the pseudo ground-truth, but it leads to a problem that most pixels may be left unused due to their unreliability.

Semi-Supervised Semantic Segmentation

Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination

2 code implementations22 Dec 2021 Rui Zhao, Jinming Song, Yufeng Yuan, Hu Haifeng, Yang Gao, Yi Wu, Zhongqian Sun, Yang Wei

We study the problem of training a Reinforcement Learning (RL) agent that is collaborative with humans without using any human data.

Reinforcement Learning (RL)

Feature Erasing and Diffusion Network for Occluded Person Re-Identification

1 code implementation CVPR 2022 Zhikang Wang, Feng Zhu, Shixiang Tang, Rui Zhao, Lihuo He, Jiangning Song

With the guidance of the occlusion scores from OEM, the feature diffusion process is mainly conducted on visible body parts, which guarantees the quality of the synthesized NTP characteristics.

 Ranked #1 on Person Re-Identification on Occluded REID (Rank-1 metric)

Person Re-Identification

FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows

5 code implementations15 Nov 2021 Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, Liwei Wu

However, current methods can not effectively map image features to a tractable base distribution and ignore the relationship between local and global features which are important to identify anomalies.

Unsupervised Anomaly Detection Weakly Supervised Defect Detection

Boundary Distribution Estimation for Precise Object Detection

no code implementations2 Nov 2021 Peng Zhi, Haoran Zhou, Hang Huang, Rui Zhao, Rui Zhou, Qingguo Zhou

In the field of state-of-the-art object detection, the task of object localization is typically accomplished through a dedicated subnet that emphasizes bounding box regression.

Object object-detection +4

Optical Flow Estimation for Spiking Camera

1 code implementation CVPR 2022 Liwen Hu, Rui Zhao, Ziluo Ding, Lei Ma, Boxin Shi, Ruiqin Xiong, Tiejun Huang

Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes.

Event-based vision Motion Estimation +1

Dr.Aid: Supporting Data-governance Rule Compliance for Decentralized Collaboration in an Automated Way

no code implementations3 Oct 2021 Rui Zhao, Malcolm Atkinson, Petros Papapanagiotou, Federica Magnoni, Jacques Fleuriot

It depends on federations sharing data that often have governance rules or external regulations restricting their use.

Auto-Encoding Inverse Reinforcement Learning

no code implementations29 Sep 2021 Kaifeng Zhang, Rui Zhao, Ziming Zhang, Yang Gao

Reinforcement learning (RL) provides a powerful framework for decision-making, but its application in practice often requires a carefully designed reward function.

Imitation Learning reinforcement-learning +1

Zero-CL: Instance and Feature decorrelation for negative-free symmetric contrastive learning

no code implementations ICLR 2022 Shaofeng Zhang, Feng Zhu, Junchi Yan, Rui Zhao, Xiaokang Yang

The proposed two methods (FCL, ICL) can be combined synthetically, called Zero-CL, where ``Zero'' means negative samples are \textbf{zero} relevant, which allows Zero-CL to completely discard negative pairs i. e., with \textbf{zero} negative samples.

Contrastive Learning

MDFL: A UNIFIED FRAMEWORK WITH META-DROPOUT FOR FEW-SHOT LEARNING

no code implementations29 Sep 2021 Shaobo Lin, Xingyu Zeng, Rui Zhao

Conventional training of deep neural networks usually requires a substantial amount of data with expensive human annotations.

Few-Shot Image Classification Few-Shot Learning +2

Multi-Source Video Domain Adaptation with Temporal Attentive Moment Alignment

no code implementations21 Sep 2021 Yuecong Xu, Jianfei Yang, Haozhi Cao, Keyu Wu, Min Wu, Rui Zhao, Zhenghua Chen

Multi-Source Domain Adaptation (MSDA) is a more practical domain adaptation scenario in real-world scenarios.

Unsupervised Domain Adaptation

Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation

1 code implementation10 Sep 2021 Ziluo Ding, Rui Zhao, Jiyuan Zhang, Tianxiao Gao, Ruiqin Xiong, Zhaofei Yu, Tiejun Huang

Recently, many deep learning methods have shown great success in providing promising solutions to many event-based problems, such as optical flow estimation.

Event-based Optical Flow Optical Flow Estimation +1

An Automated Framework for Supporting Data-Governance Rule Compliance in Decentralized MIMO Contexts

no code implementations2 Sep 2021 Rui Zhao

We propose Dr. Aid, a logic-based AI framework for automated compliance checking of data governance rules over data-flow graphs.

MST: Masked Self-Supervised Transformer for Visual Representation

no code implementations NeurIPS 2021 Zhaowen Li, Zhiyang Chen, Fan Yang, Wei Li, Yousong Zhu, Chaoyang Zhao, Rui Deng, Liwei Wu, Rui Zhao, Ming Tang, Jinqiao Wang

More importantly, the masked tokens together with the remaining tokens are further recovered by a global image decoder, which preserves the spatial information of the image and is more friendly to the downstream dense prediction tasks.

Language Modelling Linear evaluation +4

Improving Facial Attribute Recognition by Group and Graph Learning

no code implementations28 May 2021 Zhenghao Chen, Shuhang Gu, Feng Zhu, Jing Xu, Rui Zhao

For the spatial correlation, we aggregate attributes with spatial similarity into a part-based group and then introduce a Group Attention Learning to generate the group attention and the part-based group feature.

Attribute Graph Learning

Neighbourhood-guided Feature Reconstruction for Occluded Person Re-Identification

no code implementations16 May 2021 Shijie Yu, Dapeng Chen, Rui Zhao, Haobin Chen, Yu Qiao

Person images captured by surveillance cameras are often occluded by various obstacles, which lead to defective feature representation and harm person re-identification (Re-ID) performance.

Person Re-Identification

On Addressing Practical Challenges for RNN-Transducer

no code implementations27 Apr 2021 Rui Zhao, Jian Xue, Jinyu Li, Wenning Wei, Lei He, Yifan Gong

The first challenge is solved with a splicing data method which concatenates the speech segments extracted from the source domain data.

speech-recognition Speech Recognition

Memory Enhanced Embedding Learning for Cross-Modal Video-Text Retrieval

no code implementations29 Mar 2021 Rui Zhao, Kecheng Zheng, Zheng-Jun Zha, Hongtao Xie, Jiebo Luo

The cross-modal memory module is employed to record the instance embeddings of all the datasets for global negative mining.

Retrieval Text Retrieval +1

Mutual Information State Intrinsic Control

2 code implementations ICLR 2021 Rui Zhao, Yang Gao, Pieter Abbeel, Volker Tresp, Wei Xu

Reinforcement learning has been shown to be highly successful at many challenging tasks.

Progressive Correspondence Pruning by Consensus Learning

1 code implementation ICCV 2021 Chen Zhao, Yixiao Ge, Feng Zhu, Rui Zhao, Hongsheng Li, Mathieu Salzmann

Correspondence selection aims to correctly select the consistent matches (inliers) from an initial set of putative correspondences.

Denoising Pose Estimation +1

Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition

no code implementations3 Nov 2020 Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong

The external language models (LM) integration remains a challenging task for end-to-end (E2E) automatic speech recognition (ASR) which has no clear division between acoustic and language models.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

The Vulnerability of the Neural Networks Against Adversarial Examples in Deep Learning Algorithms

no code implementations2 Nov 2020 Rui Zhao

Based on current security threats faced by deep learning, this paper introduces the problem of adversarial examples in deep learning, sorts out the existing attack and defense methods of the black box and white box, and classifies them.

Enhancing and Learning Denoiser without Clean Reference

no code implementations9 Sep 2020 Rui Zhao, Daniel P. K. Lun, Kin-Man Lam

Recent studies on learning-based image denoising have achieved promising performance on various noise reduction tasks.

Image Denoising

Transfer Learning Approaches for Streaming End-to-End Speech Recognition System

no code implementations12 Aug 2020 Vikas Joshi, Rui Zhao, Rupesh R. Mehta, Kshitiz Kumar, Jinyu Li

Transfer learning (TL) is widely used in conventional hybrid automatic speech recognition (ASR) system, to transfer the knowledge from source to target language.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Deep Reinforcement Learning Based Mobile Edge Computing for Intelligent Internet of Things

no code implementations1 Aug 2020 Rui Zhao, Xinjie Wang, Junjuan Xia, Liseng Fan

In particular, the system cost of latency and energy consumption can be reduced significantly by the proposed deep reinforcement learning based algorithm.

Edge-computing reinforcement-learning +1

Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability

no code implementations30 Jul 2020 Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong

Because of its streaming nature, recurrent neural network transducer (RNN-T) is a very promising end-to-end (E2E) model that may replace the popular hybrid model for automatic speech recognition.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Learning Individualized Treatment Rules with Estimated Translated Inverse Propensity Score

1 code implementation2 Jul 2020 Zhiliang Wu, Yinchong Yang, Yunpu Ma, Yushan Liu, Rui Zhao, Michael Moor, Volker Tresp

Randomized controlled trials typically analyze the effectiveness of treatments with the goal of making treatment recommendations for patient subgroups.

Enhancement of a CNN-Based Denoiser Based on Spatial and Spectral Analysis

no code implementations28 Jun 2020 Rui Zhao, Kin-Man Lam, Daniel P. K. Lun

Since most of the content or energy of natural images resides in the low-frequency spectrum, their transformed coefficients in the frequency domain are highly imbalanced.

Image Denoising

Continual Representation Learning for Biometric Identification

1 code implementation8 Jun 2020 Bo Zhao, Shixiang Tang, Dapeng Chen, Hakan Bilen, Rui Zhao

With the explosion of digital data in recent years, continuously learning new tasks from a stream of data without forgetting previously acquired knowledge has become increasingly important.

Continual Learning Knowledge Distillation +1

Self-supervising Fine-grained Region Similarities for Large-scale Image Localization

3 code implementations ECCV 2020 Yixiao Ge, Haibo Wang, Feng Zhu, Rui Zhao, Hongsheng Li

The task of large-scale retrieval-based image localization is to estimate the geographical location of a query image by recognizing its nearest reference images from a city-scale dataset.

Image Retrieval Retrieval

Bayesian Adversarial Human Motion Synthesis

1 code implementation CVPR 2020 Rui Zhao, Hui Su, Qiang Ji

By explicitly capturing the distribution of the data and parameters, our model has a more compact parameterization compared to GAN-based generative models.

Bayesian Inference Data Augmentation +1

COCAS: A Large-Scale Clothes Changing Person Dataset for Re-identification

no code implementations CVPR 2020 Shijie Yu, Shihua Li, Dapeng Chen, Rui Zhao, Junjie Yan, Yu Qiao

To address the clothes changing person re-id problem, we construct a novel large-scale re-id benchmark named ClOthes ChAnging Person Set (COCAS), which provides multiple images of the same identity with different clothes.

Person Re-Identification

Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition

no code implementations1 May 2020 Hu Hu, Rui Zhao, Jinyu Li, Liang Lu, Yifan Gong

Recently, the recurrent neural network transducer (RNN-T) architecture has become an emerging trend in end-to-end automatic speech recognition research due to its advantages of being capable for online streaming speech recognition.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Stacked Convolutional Deep Encoding Network for Video-Text Retrieval

no code implementations10 Apr 2020 Rui Zhao, Kecheng Zheng, Zheng-Jun Zha

Existing dominant approaches for cross-modal video-text retrieval task are to learn a joint embedding space to measure the cross-modal similarity.

Language Modelling Retrieval +2

Learning to Cluster Faces via Confidence and Connectivity Estimation

3 code implementations CVPR 2020 Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, Dahua Lin

With the vertex confidence and edge connectivity, we can naturally organize more relevant vertices on the affinity graph and group them into clusters.

Clustering Connectivity Estimation +2

High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model

no code implementations17 Mar 2020 Jinyu Li, Rui Zhao, Eric Sun, Jeremy H. M. Wong, Amit Das, Zhong Meng, Yifan Gong

While the community keeps promoting end-to-end models over conventional hybrid models, which usually are long short-term memory (LSTM) models trained with a cross entropy criterion followed by a sequence discriminative training criterion, we argue that such conventional hybrid models can still be significantly improved.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Structured Domain Adaptation with Online Relation Regularization for Unsupervised Person Re-ID

3 code implementations14 Mar 2020 Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, Xiaogang Wang, Hongsheng Li

To tackle the challenges, we propose an end-to-end structured domain adaptation framework with an online relation-consistency regularization term.

Pseudo Label Relation +3

Towards a computer-interpretable actionable formal model to encode data governance rules

no code implementations19 Nov 2019 Rui Zhao, Malcolm Atkinson

With the needs of science and business, data sharing and re-use has become an intensive activity for various areas.

Bayesian Graph Convolution LSTM for Skeleton Based Action Recognition

no code implementations ICCV 2019 Rui Zhao, Kang Wang, Hui Su, Qiang Ji

Finally, the whole model is extended under the Bayesian framework to a probabilistic model in order to better capture the stochasticity and variation in the data.

Action Recognition Anatomy +2

Self-Supervised State-Control through Intrinsic Mutual Information Rewards

1 code implementation25 Sep 2019 Rui Zhao, Volker Tresp, Wei Xu

Our results show that the mutual information between the context states and the states of interest can be an effective ingredient for overcoming challenges in robotic manipulation tasks with sparse rewards.

OpenAI Gym reinforcement-learning +1

Maximum Entropy-Regularized Multi-Goal Reinforcement Learning

3 code implementations21 May 2019 Rui Zhao, Xudong Sun, Volker Tresp

This objective encourages the agent to maximize the expected return, as well as to achieve more diverse goals.

Multi-Goal Reinforcement Learning OpenAI Gym +2

Neural Networks for Modeling Source Code Edits

no code implementations4 Apr 2019 Rui Zhao, David Bieber, Kevin Swersky, Daniel Tarlow

In this work, we instead treat source code as a dynamic object and tackle the problem of modeling the edits that software developers make to source code files.

Curiosity-Driven Experience Prioritization via Density Estimation

no code implementations20 Feb 2019 Rui Zhao, Volker Tresp

In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning.

Density Estimation OpenAI Gym +3

Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units

no code implementations31 Dec 2018 Amit Das, Jinyu Li, Guoli Ye, Rui Zhao, Yifan Gong

In particular, we introduce Attention CTC, Self-Attention CTC, Hybrid CTC, and Mixed-unit CTC.

Decoder Language Modelling

Energy-Based Hindsight Experience Prioritization

2 code implementations2 Oct 2018 Rui Zhao, Volker Tresp

We evaluate our Energy-Based Prioritization (EBP) approach on four challenging robotic manipulation tasks in simulation.

reinforcement-learning Reinforcement Learning (RL)

Efficient Dialog Policy Learning via Positive Memory Retention

2 code implementations2 Oct 2018 Rui Zhao, Volker Tresp

This paper is concerned with the training of recurrent neural networks as goal-oriented dialog agents using reinforcement learning.

Goal-Oriented Dialog Object Discovery +1