Search Results for author: Siyuan Li

Found 50 papers, 31 papers with code

Lightweight Contrastive Protein Structure-Sequence Transformation

no code implementations19 Mar 2023 Jiangbin Zheng, Ge Wang, Yufei Huang, Bozhen Hu, Siyuan Li, Cheng Tan, Xinwen Fan, Stan Z. Li

In this work, we introduce a novel unsupervised protein structure representation pretraining with a robust protein language model.

Language Modelling Masked Language Modeling

CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment

1 code implementation10 Mar 2023 Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li

In this work, we propose a novel contrastive visual-textual transformation for SLR, CVT-SLR, to fully explore the pretrained knowledge of both the visual and language modalities.

Sign Language Recognition

Explaining Graph Neural Networks via Non-parametric Subgraph Matching

no code implementations7 Jan 2023 Fang Wu, Siyuan Li, Lirong Wu, Dragomir Radev, Yinghui Jiang, Xurui Jin, Zhangming Niu, Stan Z. Li

The great success in graph neural networks (GNNs) provokes the question about explainability: Which fraction of the input graph is the most determinant of the prediction?

Graph Sampling

CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels

1 code implementation25 Nov 2022 Siyuan Li, Li Sun, Qingli Li

The key idea is to fully exploit the cross-modal description ability in CLIP through a set of learnable text tokens for each ID and give them to the text encoder to form ambiguous descriptions.

Image Classification Language Modelling

Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning

1 code implementation12 Nov 2022 Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li, Siyuan Li, Liang Lin, Guanbin Li

We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source-pretrained model is adapted to the target domain without access to the source data.

Contrastive Learning Source-Free Domain Adaptation

Efficient Multi-order Gated Aggregation Network

4 code implementations7 Nov 2022 Siyuan Li, Zedong Wang, Zicheng Liu, Cheng Tan, Haitao Lin, Di wu, ZhiYuan Chen, Jiangbin Zheng, Stan Z. Li

Since the recent success of Vision Transformers (ViTs), explorations toward ViT-style architectures have triggered the resurgence of ConvNets.

 Ranked #1 on Instance Segmentation on COCO test-dev (AP50 metric)

3D Human Pose Estimation Image Classification +5

Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation

no code implementations1 Nov 2022 Jiangbin Zheng, Siyuan Li, Cheng Tan, Chong Wu, Yidong Chen, Stan Z. Li

Therefore, we propose to introduce additional word-level semantic knowledge of sign language linguistics to assist in improving current end-to-end neural SLT models.

Sign Language Translation Translation

Classifying Ambiguous Identities in Hidden-Role Stochastic Games with Multi-Agent Reinforcement Learning

1 code implementation24 Oct 2022 Shijie Han, Siyuan Li, Bo An, Wei Zhao, Peng Liu

In this work, we develop a novel identity detection reinforcement learning (IDRL) framework that allows an agent to dynamically infer the identities of nearby agents and select an appropriate policy to accomplish the task.

Multi-agent Reinforcement Learning reinforcement-learning +1

CUP: Critic-Guided Policy Reuse

1 code implementation15 Oct 2022 Jin Zhang, Siyuan Li, Chongjie Zhang

The ability to reuse previous policies is an important aspect of human intelligence.

OpenMixup: Open Mixup Toolbox and Benchmark for Visual Representation Learning

1 code implementation11 Sep 2022 Siyuan Li, Zedong Wang, Zicheng Liu, Di wu, Stan Z. Li

With the remarkable progress of deep neural networks in computer vision, data mixing augmentation techniques are widely studied to alleviate problems of degraded generalization when the amount of training data is limited.

Representation Learning Self-Supervised Learning +1

MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization

no code implementations1 Sep 2022 Hui Niu, Siyuan Li, Jian Li

We evaluate the proposed approach on three real-world index datasets and compare it to state-of-the-art baselines.

Imitation Learning Management +3

Are Gradients on Graph Structure Reliable in Gray-box Attacks?

1 code implementation7 Aug 2022 Zihan Liu, Yun Luo, Lirong Wu, Siyuan Li, Zicheng Liu, Stan Z. Li

These errors arise from rough gradient usage due to the discreteness of the graph structure and from the unreliability in the meta-gradient on the graph structure.

Tracking Every Thing in the Wild

1 code implementation26 Jul 2022 Siyuan Li, Martin Danelljan, Henghui Ding, Thomas E. Huang, Fisher Yu

Our experiments show that TETA evaluates trackers more comprehensively, and TETer achieves significant improvements on the challenging large-scale datasets BDD100K and TAO compared to the state-of-the-art.

Association Benchmarking +2

DLME: Deep Local-flatness Manifold Embedding

2 code implementations7 Jul 2022 Zelin Zang, Siyuan Li, Di wu, Ge Wang, Lei Shang, Baigui Sun, Hao Li, Stan Z. Li

To overcome the underconstrained embedding problem, we design a loss and theoretically demonstrate that it leads to a more suitable embedding based on the local flatness.

Contrastive Learning Data Augmentation +1

Hyperspherical Consistency Regularization

1 code implementation CVPR 2022 Cheng Tan, Zhangyang Gao, Lirong Wu, Siyuan Li, Stan Z. Li

Though it benefits from taking advantage of both feature-dependent information from self-supervised learning and label-dependent information from supervised learning, this scheme remains suffering from bias of the classifier.

Contrastive Learning Self-Supervised Learning +1

Architecture-Agnostic Masked Image Modeling -- From ViT back to CNN

1 code implementation27 May 2022 Siyuan Li, Di wu, Fang Wu, Zelin Zang, Baigui Sun, Hao Li, Xuansong Xie, Stan. Z. Li

Based on this fact, we propose an Architecture-Agnostic Masked Image Modeling framework (A$^2$MIM), which is compatible with both Transformers and CNNs in a unified way.

Instance Segmentation Object Detection +3

Discovering and Explaining the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

1 code implementation15 May 2022 Fang Wu, Siyuan Li, Lirong Wu, Dragomir Radev, Stan Z. Li

Graph neural networks (GNNs) mainly rely on the message-passing paradigm to propagate node features and build interactions, and different graph learning tasks require different ranges of node interactions.

graph construction Graph Learning +2

A heuristic method for data allocation and task scheduling on heterogeneous multiprocessor systems under memory constraints

no code implementations9 May 2022 Junwen Ding, Liangcai Song, Siyuan Li, Chen Wu, Ronghua He, Zhouxing Su, Zhipeng Lü

Computing workflows in heterogeneous multiprocessor systems are frequently modeled as directed acyclic graphs of tasks and data blocks, which represent computational modules and their dependencies in the form of data produced by a task and used by others.

Job Shop Scheduling Scheduling

neuro2vec: Masked Fourier Spectrum Prediction for Neurophysiological Representation Learning

1 code implementation20 Apr 2022 Di wu, Siyuan Li, Jie Yang, Mohamad Sawan

Extensive data labeling on neurophysiological signals is often prohibitively expensive or impractical, as it may require particular infrastructure or domain expertise.

Electroencephalogram (EEG) Electromyography (EMG) +2

UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection

1 code implementation CVPR 2022 Ye Liu, Siyuan Li, Yang Wu, Chang Wen Chen, Ying Shan, XiaoHu Qie

Finding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.

Highlight Detection Moment Retrieval +2

Decoupled Mixup for Data-efficient Learning

1 code implementation21 Mar 2022 Zicheng Liu, Siyuan Li, Ge Wang, Cheng Tan, Lirong Wu, Stan Z. Li

This also leads to an interesting objective design problem for mixup training that we need to focus on both smoothing the decision boundaries and identifying discriminative features.

Data Augmentation Semi-Supervised Image Classification +1

Style Transformer for Image Inversion and Editing

1 code implementation CVPR 2022 Xueqi Hu, Qiusheng Huang, Zhengyi Shi, Siyuan Li, Changxin Gao, Li Sun, Qingli Li

Existing GAN inversion methods fail to provide latent codes for reliable reconstruction and flexible editing simultaneously.

Image-to-Image Translation

A Shared Representation for Photorealistic Driving Simulators

1 code implementation9 Dec 2021 Saeed Saadatnejad, Siyuan Li, Taylor Mordan, Alexandre Alahi

We build on successful cGAN models to propose a new semantically-aware discriminator that better guides the generator.

Autonomous Vehicles Image Generation +1

Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup

1 code implementation30 Nov 2021 Siyuan Li, Zicheng Liu, Di wu, Zihan Liu, Stan Z. Li

Mixup is a popular data-dependent augmentation technique for deep neural networks, which contains two sub-tasks, mixup generation, and classification.

Data Augmentation Image Classification +2

GenURL: A General Framework for Unsupervised Representation Learning

1 code implementation27 Oct 2021 Siyuan Li, Zicheng Liu, Zelin Zang, Di wu, ZhiYuan Chen, Stan Z. Li

Unsupervised representation learning (URL) that learns compact embeddings of high-dimensional data without supervision has achieved remarkable progress recently.

Contrastive Learning Dimensionality Reduction +3

Offline Reinforcement Learning with Reverse Model-based Imagination

1 code implementation NeurIPS 2021 Jianhao Wang, Wenzhe Li, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, Chongjie Zhang

These reverse imaginations provide informed data augmentation for model-free policy learning and enable conservative generalization beyond the offline dataset.

Data Augmentation Offline RL +2

Improving Discriminative Visual Representation Learning via Automatic Mixup

no code implementations29 Sep 2021 Siyuan Li, Zicheng Liu, Di wu, Stan Z. Li

In this paper, we decompose mixup into two sub-tasks of mixup generation and classification and formulate it for discriminative representations as class- and instance-level mixup.

Data Augmentation Representation Learning

Active Hierarchical Exploration with Stable Subgoal Representation Learning

1 code implementation ICLR 2022 Siyuan Li, Jin Zhang, Jianhao Wang, Yang Yu, Chongjie Zhang

Although GCHRL possesses superior exploration ability by decomposing tasks via subgoals, existing GCHRL methods struggle in temporally extended tasks with sparse external rewards, since the high-level policy learning relies on external rewards.

Continuous Control Hierarchical Reinforcement Learning +1

Unsupervised Deep Manifold Attributed Graph Embedding

1 code implementation27 Apr 2021 Zelin Zang, Siyuan Li, Di wu, Jianzhu Guo, Yongjie Xu, Stan Z. Li

Unsupervised attributed graph representation learning is challenging since both structural and feature information are required to be represented in the latent space.

Graph Embedding Graph Representation Learning +2

AutoMix: Unveiling the Power of Mixup for Stronger Classifiers

2 code implementations24 Mar 2021 Zicheng Liu, Siyuan Li, Di wu, Zihan Liu, ZhiYuan Chen, Lirong Wu, Stan Z. Li

Specifically, AutoMix reformulates the mixup classification into two sub-tasks (i. e., mixed sample generation and mixup classification) with corresponding sub-networks and solves them in a bi-level optimization framework.

Classification Data Augmentation +3

Learning Subgoal Representations with Slow Dynamics

no code implementations ICLR 2021 Siyuan Li, Lulu Zheng, Jianhao Wang, Chongjie Zhang

In goal-conditioned Hierarchical Reinforcement Learning (HRL), a high-level policy periodically sets subgoals for a low-level policy, and the low-level policy is trained to reach those subgoals.

Continuous Control Hierarchical Reinforcement Learning +1

Towards Robust Graph Neural Networks against Label Noise

no code implementations1 Jan 2021 Jun Xia, Haitao Lin, Yongjie Xu, Lirong Wu, Zhangyang Gao, Siyuan Li, Stan Z. Li

A pseudo label is computed from the neighboring labels for each node in the training set using LP; meta learning is utilized to learn a proper aggregation of the original and pseudo label as the final label.

Learning with noisy labels Meta-Learning +2

Invertible Manifold Learning for Dimension Reduction

1 code implementation7 Oct 2020 Siyuan Li, Haitao Lin, Zelin Zang, Lirong Wu, Jun Xia, Stan Z. Li

Dimension reduction (DR) aims to learn low-dimensional representations of high-dimensional data with the preservation of essential information.

Dimensionality Reduction

Deep Clustering and Representation Learning that Preserves Geometric Structures

no code implementations28 Sep 2020 Lirong Wu, Zicheng Liu, Zelin Zang, Jun Xia, Siyuan Li, Stan Z. Li

To overcome the problem that clusteringoriented losses may deteriorate the geometric structure of embeddings in the latent space, an isometric loss is proposed for preserving intra-manifold structure locally and a ranking loss for inter-manifold structure globally.

Deep Clustering Representation Learning

Generalized Clustering and Multi-Manifold Learning with Geometric Structure Preservation

1 code implementation21 Sep 2020 Lirong Wu, Zicheng Liu, Zelin Zang, Jun Xia, Siyuan Li, Stan Z. Li

Though manifold-based clustering has become a popular research topic, we observe that one important factor has been omitted by these works, namely that the defined clustering loss may corrupt the local and global structure of the latent space.

Deep Clustering Representation Learning

Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals

no code implementations CVPR 2020 Siyuan Li, Semih Günel, Mirela Ostrek, Pavan Ramdya, Pascal Fua, Helge Rhodin

We compare our approach with existing domain transfer methods and demonstrate improved pose estimation accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm) and Danio rerio (zebrafish), without requiring any manual annotation on the target domain and despite using simplistic off-the-shelf animal characters for simulation, or simple geometric shapes as models.

Pose Estimation Translation

Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards

1 code implementation NeurIPS 2019 Siyuan Li, Rui Wang, Minxue Tang, Chongjie Zhang

In addition, we also theoretically prove that optimizing low-level skills with this auxiliary reward will increase the task return for the joint policy.

Hierarchical Reinforcement Learning reinforcement-learning +1

Single Image Deraining: A Comprehensive Benchmark Analysis

1 code implementation CVPR 2019 Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K. Tokuda, Roberto Hirata Junior, Roberto Cesar-Junior, Jiawan Zhang, Xiaojie Guo, Xiaochun Cao

We present a comprehensive study and evaluation of existing single image deraining algorithms, using a new large-scale benchmark consisting of both synthetic and real-world rainy images. This dataset highlights diverse data sources and image contents, and is divided into three subsets (rain streak, rain drop, rain and mist), each serving different training or evaluation purposes.

Single Image Deraining

PFLD: A Practical Facial Landmark Detector

18 code implementations28 Feb 2019 Xiaojie Guo, Siyuan Li, Jinke Yu, Jiawan Zhang, Jiayi Ma, Lin Ma, Wei Liu, Haibin Ling

Being accurate, efficient, and compact is essential to a facial landmark detector for practical use.

Face Alignment Facial Landmark Detection

Context-Aware Policy Reuse

no code implementations11 Jun 2018 Siyuan Li, Fangda Gu, Guangxiang Zhu, Chongjie Zhang

Transfer learning can greatly speed up reinforcement learning for a new task by leveraging policies of relevant tasks.

Transfer Learning

Fast Single Image Rain Removal via a Deep Decomposition-Composition Network

no code implementations8 Apr 2018 Siyuan LI, Wenqi Ren, Jiawan Zhang, Jinke Yu, Xiaojie Guo

Rain effect in images typically is annoying for many multimedia and computer vision tasks.

Rain Removal

Deep Eyes: Binocular Depth-from-Focus on Focal Stack Pairs

no code implementations29 Nov 2017 Xinqing Guo, Zhang Chen, Siyuan Li, Yang Yang, Jingyi Yu

We then construct three individual networks: a Focus-Net to extract depth from a single focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from the focal stack, and a Stereo-Net to conduct stereo matching.

Stereo Matching Stereo Matching Hand

An Optimal Online Method of Selecting Source Policies for Reinforcement Learning

no code implementations24 Sep 2017 Siyuan Li, Chongjie Zhang

In this paper, we develop an optimal online method to select source policies for reinforcement learning.

Q-Learning reinforcement-learning +3

Cannot find the paper you are looking for? You can Submit a new open access paper.