no code implementations • 19 Mar 2023 • Jiangbin Zheng, Ge Wang, Yufei Huang, Bozhen Hu, Siyuan Li, Cheng Tan, Xinwen Fan, Stan Z. Li
In this work, we introduce a novel unsupervised protein structure representation pretraining with a robust protein language model.
1 code implementation • 10 Mar 2023 • Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li
In this work, we propose a novel contrastive visual-textual transformation for SLR, CVT-SLR, to fully explore the pretrained knowledge of both the visual and language modalities.
no code implementations • 7 Jan 2023 • Fang Wu, Siyuan Li, Lirong Wu, Dragomir Radev, Yinghui Jiang, Xurui Jin, Zhangming Niu, Stan Z. Li
The great success in graph neural networks (GNNs) provokes the question about explainability: Which fraction of the input graph is the most determinant of the prediction?
no code implementations • 9 Dec 2022 • Haitao Lin, Lirong Wu, Yongjie Xu, Yufei Huang, Siyuan Li, Guojiang Zhao, Stan Z. Li
Solving partial differential equations is difficult.
no code implementations • 2 Dec 2022 • Yiqin Yang, Hao Hu, Wenzhe Li, Siyuan Li, Jun Yang, Qianchuan Zhao, Chongjie Zhang
We show that such lossless primitives can drastically improve the performance of hierarchical policies.
1 code implementation • 25 Nov 2022 • Siyuan Li, Li Sun, Qingli Li
The key idea is to fully exploit the cross-modal description ability in CLIP through a set of learnable text tokens for each ID and give them to the text encoder to form ambiguous descriptions.
1 code implementation • 12 Nov 2022 • Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li, Siyuan Li, Liang Lin, Guanbin Li
We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source-pretrained model is adapted to the target domain without access to the source data.
4 code implementations • 7 Nov 2022 • Siyuan Li, Zedong Wang, Zicheng Liu, Cheng Tan, Haitao Lin, Di wu, ZhiYuan Chen, Jiangbin Zheng, Stan Z. Li
Since the recent success of Vision Transformers (ViTs), explorations toward ViT-style architectures have triggered the resurgence of ConvNets.
Ranked #1 on Instance Segmentation on COCO test-dev (AP50 metric)
no code implementations • 1 Nov 2022 • Jiangbin Zheng, Siyuan Li, Cheng Tan, Chong Wu, Yidong Chen, Stan Z. Li
Therefore, we propose to introduce additional word-level semantic knowledge of sign language linguistics to assist in improving current end-to-end neural SLT models.
1 code implementation • 24 Oct 2022 • Shijie Han, Siyuan Li, Bo An, Wei Zhao, Peng Liu
In this work, we develop a novel identity detection reinforcement learning (IDRL) framework that allows an agent to dynamically infer the identities of nearby agents and select an appropriate policy to accomplish the task.
Multi-agent Reinforcement Learning reinforcement-learning +1
1 code implementation • 15 Oct 2022 • Jin Zhang, Siyuan Li, Chongjie Zhang
The ability to reuse previous policies is an important aspect of human intelligence.
1 code implementation • 11 Sep 2022 • Siyuan Li, Zedong Wang, Zicheng Liu, Di wu, Stan Z. Li
With the remarkable progress of deep neural networks in computer vision, data mixing augmentation techniques are widely studied to alleviate problems of degraded generalization when the amount of training data is limited.
no code implementations • 1 Sep 2022 • Hui Niu, Siyuan Li, Jian Li
We evaluate the proposed approach on three real-world index datasets and compare it to state-of-the-art baselines.
1 code implementation • 7 Aug 2022 • Zihan Liu, Yun Luo, Lirong Wu, Siyuan Li, Zicheng Liu, Stan Z. Li
These errors arise from rough gradient usage due to the discreteness of the graph structure and from the unreliability in the meta-gradient on the graph structure.
1 code implementation • 26 Jul 2022 • Siyuan Li, Martin Danelljan, Henghui Ding, Thomas E. Huang, Fisher Yu
Our experiments show that TETA evaluates trackers more comprehensively, and TETer achieves significant improvements on the challenging large-scale datasets BDD100K and TAO compared to the state-of-the-art.
Ranked #6 on Multiple Object Tracking on BDD100K val
2 code implementations • 7 Jul 2022 • Zelin Zang, Siyuan Li, Di wu, Ge Wang, Lei Shang, Baigui Sun, Hao Li, Stan Z. Li
To overcome the underconstrained embedding problem, we design a loss and theoretically demonstrate that it leads to a more suitable embedding based on the local flatness.
Ranked #1 on Image Classification on ImageNet-100
no code implementations • 24 Jun 2022 • Cheng Tan, Zhangyang Gao, Lirong Wu, Yongjie Xu, Jun Xia, Siyuan Li, Stan Z. Li
Spatiotemporal predictive learning aims to generate future frames by learning from historical frames.
Ranked #11 on Video Prediction on Moving MNIST
1 code implementation • CVPR 2022 • Cheng Tan, Zhangyang Gao, Lirong Wu, Siyuan Li, Stan Z. Li
Though it benefits from taking advantage of both feature-dependent information from self-supervised learning and label-dependent information from supervised learning, this scheme remains suffering from bias of the classifier.
1 code implementation • 27 May 2022 • Siyuan Li, Di wu, Fang Wu, Zelin Zang, Baigui Sun, Hao Li, Xuansong Xie, Stan. Z. Li
Based on this fact, we propose an Architecture-Agnostic Masked Image Modeling framework (A$^2$MIM), which is compatible with both Transformers and CNNs in a unified way.
1 code implementation • 15 May 2022 • Fang Wu, Siyuan Li, Lirong Wu, Dragomir Radev, Stan Z. Li
Graph neural networks (GNNs) mainly rely on the message-passing paradigm to propagate node features and build interactions, and different graph learning tasks require different ranges of node interactions.
no code implementations • 9 May 2022 • Junwen Ding, Liangcai Song, Siyuan Li, Chen Wu, Ronghua He, Zhouxing Su, Zhipeng Lü
Computing workflows in heterogeneous multiprocessor systems are frequently modeled as directed acyclic graphs of tasks and data blocks, which represent computational modules and their dependencies in the form of data produced by a task and used by others.
1 code implementation • 20 Apr 2022 • Di wu, Siyuan Li, Jie Yang, Mohamad Sawan
Extensive data labeling on neurophysiological signals is often prohibitively expensive or impractical, as it may require particular infrastructure or domain expertise.
1 code implementation • CVPR 2022 • Ye Liu, Siyuan Li, Yang Wu, Chang Wen Chen, Ying Shan, XiaoHu Qie
Finding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.
Ranked #1 on Highlight Detection on YouTube Highlights
1 code implementation • 21 Mar 2022 • Zicheng Liu, Siyuan Li, Ge Wang, Cheng Tan, Lirong Wu, Stan Z. Li
This also leads to an interesting objective design problem for mixup training that we need to focus on both smoothing the decision boundaries and identifying discriminative features.
1 code implementation • CVPR 2022 • Xueqi Hu, Qiusheng Huang, Zhengyi Shi, Siyuan Li, Changxin Gao, Li Sun, Qingli Li
Existing GAN inversion methods fail to provide latent codes for reliable reconstruction and flexible editing simultaneously.
1 code implementation • 9 Dec 2021 • Saeed Saadatnejad, Siyuan Li, Taylor Mordan, Alexandre Alahi
We build on successful cGAN models to propose a new semantically-aware discriminator that better guides the generator.
1 code implementation • 30 Nov 2021 • Siyuan Li, Zicheng Liu, Di wu, Zihan Liu, Stan Z. Li
Mixup is a popular data-dependent augmentation technique for deep neural networks, which contains two sub-tasks, mixup generation, and classification.
Ranked #7 on Image Classification on Places205
1 code implementation • 27 Oct 2021 • Siyuan Li, Zicheng Liu, Zelin Zang, Di wu, ZhiYuan Chen, Stan Z. Li
Unsupervised representation learning (URL) that learns compact embeddings of high-dimensional data without supervision has achieved remarkable progress recently.
1 code implementation • NeurIPS 2021 • Jianhao Wang, Wenzhe Li, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, Chongjie Zhang
These reverse imaginations provide informed data augmentation for model-free policy learning and enable conservative generalization beyond the offline dataset.
no code implementations • 29 Sep 2021 • Siyuan Li, Zicheng Liu, Di wu, Stan Z. Li
In this paper, we decompose mixup into two sub-tasks of mixup generation and classification and formulate it for discriminative representations as class- and instance-level mixup.
1 code implementation • 30 Jun 2021 • Di wu, Siyuan Li, Zelin Zang, Stan Z. Li
Self-supervised contrastive learning has demonstrated great potential in learning visual representations.
Ranked #18 on Fine-Grained Image Classification on NABirds
1 code implementation • ICLR 2022 • Siyuan Li, Jin Zhang, Jianhao Wang, Yang Yu, Chongjie Zhang
Although GCHRL possesses superior exploration ability by decomposing tasks via subgoals, existing GCHRL methods struggle in temporally extended tasks with sparse external rewards, since the high-level policy learning relies on external rewards.
1 code implementation • 27 Apr 2021 • Zelin Zang, Siyuan Li, Di wu, Jianzhu Guo, Yongjie Xu, Stan Z. Li
Unsupervised attributed graph representation learning is challenging since both structural and feature information are required to be represented in the latent space.
Ranked #2 on Node Clustering on Wiki
2 code implementations • 24 Mar 2021 • Zicheng Liu, Siyuan Li, Di wu, Zihan Liu, ZhiYuan Chen, Lirong Wu, Stan Z. Li
Specifically, AutoMix reformulates the mixup classification into two sub-tasks (i. e., mixed sample generation and mixup classification) with corresponding sub-networks and solves them in a bi-level optimization framework.
Ranked #8 on Image Classification on Places205
no code implementations • ICLR 2021 • Siyuan Li, Lulu Zheng, Jianhao Wang, Chongjie Zhang
In goal-conditioned Hierarchical Reinforcement Learning (HRL), a high-level policy periodically sets subgoals for a low-level policy, and the low-level policy is trained to reach those subgoals.
no code implementations • 1 Jan 2021 • Jun Xia, Haitao Lin, Yongjie Xu, Lirong Wu, Zhangyang Gao, Siyuan Li, Stan Z. Li
A pseudo label is computed from the neighboring labels for each node in the training set using LP; meta learning is utilized to learn a proper aggregation of the original and pseudo label as the final label.
1 code implementation • 7 Oct 2020 • Siyuan Li, Haitao Lin, Zelin Zang, Lirong Wu, Jun Xia, Stan Z. Li
Dimension reduction (DR) aims to learn low-dimensional representations of high-dimensional data with the preservation of essential information.
no code implementations • 28 Sep 2020 • Lirong Wu, Zicheng Liu, Zelin Zang, Jun Xia, Siyuan Li, Stan Z. Li
To overcome the problem that clusteringoriented losses may deteriorate the geometric structure of embeddings in the latent space, an isometric loss is proposed for preserving intra-manifold structure locally and a ranking loss for inter-manifold structure globally.
1 code implementation • 21 Sep 2020 • Lirong Wu, Zicheng Liu, Zelin Zang, Jun Xia, Siyuan Li, Stan Z. Li
Though manifold-based clustering has become a popular research topic, we observe that one important factor has been omitted by these works, namely that the defined clustering loss may corrupt the local and global structure of the latent space.
no code implementations • International Joint Conference on Artificial Intelligence 2020 • Siyuan Li, Zhi Zhang, Ziyu Liu, Anna Wang, Linglong Qiu, Feng Du
Target localization and proposal generation are two essential subtasks in generic visual tracking, and it is a challenge to address both the two efficiently.
no code implementations • CVPR 2020 • Siyuan Li, Semih Günel, Mirela Ostrek, Pavan Ramdya, Pascal Fua, Helge Rhodin
We compare our approach with existing domain transfer methods and demonstrate improved pose estimation accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm) and Danio rerio (zebrafish), without requiring any manual annotation on the target domain and despite using simplistic off-the-shelf animal characters for simulation, or simple geometric shapes as models.
1 code implementation • NeurIPS 2019 • Siyuan Li, Rui Wang, Minxue Tang, Chongjie Zhang
In addition, we also theoretically prove that optimizing low-level skills with this auxiliary reward will increase the task return for the joint policy.
Hierarchical Reinforcement Learning reinforcement-learning +1
1 code implementation • CVPR 2019 • Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K. Tokuda, Roberto Hirata Junior, Roberto Cesar-Junior, Jiawan Zhang, Xiaojie Guo, Xiaochun Cao
We present a comprehensive study and evaluation of existing single image deraining algorithms, using a new large-scale benchmark consisting of both synthetic and real-world rainy images. This dataset highlights diverse data sources and image contents, and is divided into three subsets (rain streak, rain drop, rain and mist), each serving different training or evaluation purposes.
18 code implementations • 28 Feb 2019 • Xiaojie Guo, Siyuan Li, Jinke Yu, Jiawan Zhang, Jiayi Ma, Lin Ma, Wei Liu, Haibin Ling
Being accurate, efficient, and compact is essential to a facial landmark detector for practical use.
no code implementations • 11 Jun 2018 • Siyuan Li, Fangda Gu, Guangxiang Zhu, Chongjie Zhang
Transfer learning can greatly speed up reinforcement learning for a new task by leveraging policies of relevant tasks.
no code implementations • 8 Apr 2018 • Siyuan LI, Wenqi Ren, Jiawan Zhang, Jinke Yu, Xiaojie Guo
Rain effect in images typically is annoying for many multimedia and computer vision tasks.
no code implementations • 29 Nov 2017 • Xinqing Guo, Zhang Chen, Siyuan Li, Yang Yang, Jingyi Yu
We then construct three individual networks: a Focus-Net to extract depth from a single focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from the focal stack, and a Stereo-Net to conduct stereo matching.
1 code implementation • 17 Oct 2017 • Li Yi, Lin Shao, Manolis Savva, Haibin Huang, Yang Zhou, Qirui Wang, Benjamin Graham, Martin Engelcke, Roman Klokov, Victor Lempitsky, Yuan Gan, Pengyu Wang, Kun Liu, Fenggen Yu, Panpan Shui, Bingyang Hu, Yan Zhang, Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Minki Jeong, Jaehoon Choi, Changick Kim, Angom Geetchandra, Narasimha Murthy, Bhargava Ramu, Bharadwaj Manda, M. Ramanathan, Gautam Kumar, P Preetham, Siddharth Srivastava, Swati Bhugra, Brejesh lall, Christian Haene, Shubham Tulsiani, Jitendra Malik, Jared Lafer, Ramsey Jones, Siyuan Li, Jie Lu, Shi Jin, Jingyi Yu, Qi-Xing Huang, Evangelos Kalogerakis, Silvio Savarese, Pat Hanrahan, Thomas Funkhouser, Hao Su, Leonidas Guibas
We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database.
no code implementations • 24 Sep 2017 • Siyuan Li, Chongjie Zhang
In this paper, we develop an optimal online method to select source policies for reinforcement learning.
no code implementations • 2 Aug 2017 • Zhang Chen, Xinqing Guo, Siyuan Li, Xuan Cao, Jingyi Yu
Depth from defocus (DfD) and stereo matching are two most studied passive depth sensing schemes.