Search Results for author: Lei Zhang

Found 495 papers, 214 papers with code

LST-Net: Learning a Convolutional Neural Network with a Learnable Sparse Transform

no code implementations ECCV 2020 Lida Li, Kun Wang, Shuai Li, Xiangchu Feng, Lei Zhang

The 2D convolutional (Conv2d) layer is the fundamental element to a deep convolutional neural network (CNN).

Momentum Batch Normalization for Deep Learning with Small Batch Size

no code implementations ECCV 2020 Hongwei Yong, Jianqiang Huang, Deyu Meng, Xian-Sheng Hua, Lei Zhang

To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size.

CPPF: A contextual and post-processing-free model for automatic speech recognition

no code implementations14 Sep 2023 Lei Zhang, Zhengkun Tian, Xiang Chen, Jiaming Sun, Hongyu Xiang, Ke Ding, Guanglu Wan

To address this issue, we draw inspiration from the multifaceted capabilities of LLMs and Whisper, and focus on integrating multiple ASR text processing tasks related to speech recognition into the ASR model.

Automatic Speech Recognition speech-recognition +1

Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices

no code implementations5 Sep 2023 Bojia Zi, Xianbiao Qi, Lingzhi Wang, Jianan Wang, Kam-Fai Wong, Lei Zhang

In this paper, we present Delta-LoRA, which is a novel parameter-efficient approach to fine-tune large language models (LLMs).

Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization

1 code implementation28 Aug 2023 Tao Yang, Peiran Ren, Xuansong Xie, Lei Zhang

However, the existing methods along this line either fail to keep faithful pixel-wise image structures or resort to extra skipped connections to reproduce details, which requires additional training in image space and limits their extension to other related tasks in latent space such as image stylization.

Image Stylization Image Super-Resolution

Neural Interactive Keypoint Detection

1 code implementation ICCV 2023 Jie Yang, Ailing Zeng, Feng Li, Shilong Liu, Ruimao Zhang, Lei Zhang

Click-Pose explores how user feedback can cooperate with a neural keypoint detector to correct the predicted keypoints in an interactive way for a faster and more effective annotation process.

Keypoint Detection

Uncovering User Interest from Biased and Noised Watch Time in Video Recommendation

1 code implementation16 Aug 2023 Haiyuan Zhao, Lei Zhang, Jun Xu, Guohao Cai, Zhenhua Dong, Ji-Rong Wen

In the video recommendation, watch time is commonly adopted as an indicator of user interest.

Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance

no code implementations16 Aug 2023 Xinghua Xue, Cheng Liu, Bo Liu, Haitong Huang, Ying Wang, Tao Luo, Lei Zhang, Huawei Li, Xiaowei Li

When it is applied on fault-tolerant neural networks enhanced with fault-aware retraining and constrained activation functions, the resulting model accuracy generally shows significant improvement in presence of various faults.

Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation

1 code implementation ICCV 2023 Yichen Yuan, Yifan Wang, Lijun Wang, Xiaoqi Zhao, Huchuan Lu, Yu Wang, Weibo Su, Lei Zhang

Recent leading zero-shot video object segmentation (ZVOS) works devote to integrating appearance and motion information by elaborately designing feature fusion modules and identically applying them in multiple feature stages.

Semantic Segmentation Video Object Segmentation +2

A Benchmark for Chinese-English Scene Text Image Super-resolution

1 code implementation ICCV 2023 jianqi ma, Zhetong Liang, Wangmeng Xiang, Xi Yang, Lei Zhang

Scene Text Image Super-resolution (STISR) aims to recover high-resolution (HR) scene text images with visually pleasant and readable text content from the given low-resolution (LR) input.

Image Super-Resolution

Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport

1 code implementation ICCV 2023 Wentong Li, Yuqian Yuan, Song Wang, Jianke Zhu, Jianshu Li, Jian Liu, Lei Zhang

Weakly-supervised image segmentation has recently attracted increasing research attentions, aiming to avoid the expensive pixel-wise labeling.

Image Segmentation Panoptic Segmentation

DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting

no code implementations ICCV 2023 Hongyang Li, Hao Zhang, Zhaoyang Zeng, Shilong Liu, Feng Li, Tianhe Ren, Lei Zhang

Existing feature lifting approaches, such as Lift-Splat-based and 2D attention-based, either use estimated depth to get pseudo LiDAR features and then splat them to a 3D space, which is a one-pass operation without feature refinement, or ignore depth and lift features by 2D attention mechanisms, which achieve finer semantics while suffering from a depth ambiguity problem.

3D Object Detection object-detection

CORE: Cooperative Reconstruction for Multi-Agent Perception

1 code implementation ICCV 2023 Binglu Wang, Lei Zhang, Zhaozhong Wang, Yongqiang Zhao, Tianfei Zhou

This paper presents CORE, a conceptually simple, effective and communication-efficient model for multi-agent cooperative perception.

3D Object Detection object-detection +1

Neural Quantile Optimization for Edge-Cloud Computing

no code implementations11 Jul 2023 Bin Du, He Zhang, Xiangle Cheng, Lei Zhang

The network structure reflects the edge-cloud computing topology and is trained to minimize the expectation of the cost function for unconstrained continuous optimization problems.

Cloud Computing

Semantic-SAM: Segment and Recognize Anything at Any Granularity

1 code implementation10 Jul 2023 Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Jianwei Yang, Chunyuan Li, Lei Zhang, Jianfeng Gao

In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity.

Image Segmentation Semantic Segmentation

MomentDiff: Generative Video Moment Retrieval from Random to Real

1 code implementation6 Jul 2023 Pandeng Li, Chen-Wei Xie, Hongtao Xie, Liming Zhao, Lei Zhang, Yun Zheng, Deli Zhao, Yongdong Zhang

Video moment retrieval pursues an efficient and generalized solution to identify the specific temporal segments within an untrimmed video that correspond to a given language description.

Moment Retrieval Retrieval

Impact of UAVs Equipped with ADS-B on the Civil Aviation Monitoring System

no code implementations4 Jul 2023 Yiyang Liao, Lei Zhang, Ziye Jia, Chao Dong, Yifan Zhang, Qihui Wu, Huiling Hu, Bin Wang

However, due to the limited frequency of ADS-B technique, UAVs equipped with ADS-B devices result in the loss of packets to both UAVs and civil aviation.


Steganographic Capacity of Deep Learning Models

no code implementations25 Jun 2023 Lei Zhang, Dong Li, Olha Jurečková, Mark Stamp

We find that the steganographic capacity of the learning models tested is surprisingly high, and that in each case, there is a clear threshold after which model performance rapidly degrades.

Malware Classification

DreamTime: An Improved Optimization Strategy for Text-to-3D Content Creation

no code implementations21 Jun 2023 Yukun Huang, Jianan Wang, Yukai Shi, Xianbiao Qi, Zheng-Jun Zha, Lei Zhang

Text-to-image diffusion models pre-trained on billions of image-text pairs have recently enabled text-to-3D content creation by optimizing a randomly initialized Neural Radiance Fields (NeRF) with score distillation.

Image Generation Text to 3D

Understanding Optimization of Deep Learning

no code implementations15 Jun 2023 Xianbiao Qi, Jianan Wang, Lei Zhang

This article provides a comprehensive understanding of optimization in deep learning, with a primary focus on the challenges of gradient vanishing and gradient exploding, which normally lead to diminished model representational ability and training instability, respectively.

detrex: Benchmarking Detection Transformers

1 code implementation12 Jun 2023 Tianhe Ren, Shilong Liu, Feng Li, Hao Zhang, Ailing Zeng, Jie Yang, Xingyu Liao, Ding Jia, Hongyang Li, He Cao, Jianan Wang, Zhaoyang Zeng, Xianbiao Qi, Yuhui Yuan, Jianwei Yang, Lei Zhang

To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.

Benchmarking object-detection +2

Recognize Anything: A Strong Image Tagging Model

1 code implementation6 Jun 2023 Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, Yandong Guo, Lei Zhang

We are releasing the RAM at \url{https://recognize-anything. github. io/} to foster the advancements of large models in computer vision.

Semantic Parsing

Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning

1 code implementation6 Jun 2023 Peggy Tang, Junbin Gao, Lei Zhang, Zhiyong Wang

Recently, compressive text summarisation offers a balance between the conciseness issue of extractive summarisation and the factual hallucination issue of abstractive summarisation.


Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis

1 code implementation CVPR 2023 Yuxiang Wei, Zhilong Ji, Xiaohe Wu, Jinfeng Bai, Lei Zhang, WangMeng Zuo

Despite the progress in semantic image synthesis, it remains a challenging problem to generate photo-realistic parts from input semantic map.

Image Generation

DreamWaltz: Make a Scene with Complex 3D Animatable Avatars

no code implementations21 May 2023 Yukun Huang, Jianan Wang, Ailing Zeng, He Cao, Xianbiao Qi, Yukai Shi, Zheng-Jun Zha, Lei Zhang

We present DreamWaltz, a novel framework for generating and animating complex 3D avatars given text guidance and parametric human body prior.

Text to 3D

Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Model

1 code implementation20 May 2023 Jie Yang, Bingliang Li, Fengyu Yang, Ailing Zeng, Lei Zhang, Ruimao Zhang

Extensive experiments demonstrate that DiffHOI significantly outperforms the state-of-the-art in regular detection (i. e., 41. 50 mAP) and zero-shot detection.

Ranked #2 on Zero-Shot Human-Object Interaction Detection on HICO-DET (using extra training data)

Human-Object Interaction Detection Zero-Shot Human-Object Interaction Detection

Cognition Guided Human-Object Relationship Detection

1 code implementation journal 2023 Zhitao Zeng, Pengwen Dai, Xuan Zhang, Lei Zhang, Xiaochun Cao

Human-object relationship detection reveals the fine-grained relationship between humans and objects, helping the comprehensive understanding of videos.

Human-Object Relationship Detection

MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks

no code implementations28 Apr 2023 Lei Zhang, Yuge Zhang, Kan Ren, Dongsheng Li, Yuqing Yang

In contrast, though human engineers have the incredible ability to understand tasks and reason about solutions, their experience and knowledge are often sparse and difficult to utilize by quantitative approaches.


A marker-less human motion analysis system for motion-based biomarker discovery in knee disorders

no code implementations26 Apr 2023 Kai Armstrong, Lei Zhang, Yan Wen, Alexander P. Willmott, Paul Lee, Xujioing Ye

In recent years the NHS has been having increased difficulty seeing all low-risk patients, this includes but not limited to suspected osteoarthritis (OA) patients.

A Strong and Reproducible Object Detector with Only Public Datasets

2 code implementations25 Apr 2023 Tianhe Ren, Jianwei Yang, Shilong Liu, Ailing Zeng, Feng Li, Hao Zhang, Hongyang Li, Zhaoyang Zeng, Lei Zhang

This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.

object-detection Object Detection

LipsFormer: Introducing Lipschitz Continuity to Vision Transformers

1 code implementation19 Apr 2023 Xianbiao Qi, Jianan Wang, Yihao Chen, Yukai Shi, Lei Zhang

In contrast to previous practical tricks that address training instability by learning rate warmup, layer normalization, attention formulation, and weight initialization, we show that Lipschitz continuity is a more essential property to ensure training stability.

DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training

1 code implementation CVPR 2023 Yihao Chen, Xianbiao Qi, Jianan Wang, Lei Zhang

In this way, we can reduce the GPU memory consumption of contrastive loss computation from $\bigO(B^2)$ to $\bigO(\frac{B^2}{N})$, where $B$ and $N$ are the batch size and the number of GPUs used for training.

Contrastive Learning

Language Guided Local Infiltration for Interactive Image Retrieval

no code implementations16 Apr 2023 Fuxiang Huang, Lei Zhang

Interactive Image Retrieval (IIR) aims to retrieve images that are generally similar to the reference image but under the requested text modification.

Image Retrieval Retrieval +1

Detection Transformer with Stable Matching

1 code implementation ICCV 2023 Shilong Liu, Tianhe Ren, Jiayu Chen, Zhaoyang Zeng, Hao Zhang, Feng Li, Hongyang Li, Jun Huang, Hang Su, Jun Zhu, Lei Zhang

We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.

HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation

1 code implementation ICCV 2023 Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, Qiang Xu

While such a plug-and-play approach is appealing, the inevitable and uncertain conflicts between the original images produced from the frozen SD branch and the given condition incur significant challenges for the learnable branch, which essentially conducts image feature editing for condition enforcement.

Denoising Image Generation

Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domains

1 code implementation CVPR 2023 Mingjun Xu, Lingyun Qin, WeiJie Chen, ShiLiang Pu, Lei Zhang

In this work, we present an idea to remove non-causal factors from common features by multi-view adversarial training on source domains, because we observe that such insignificant non-causal factors may still be significant in other latent spaces (views) due to the multi-mode structure of data.

Domain Generalization object-detection +1

Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer

no code implementations31 Mar 2023 Yuang Ai, Xiaoqiang Zhou, Huaibo Huang, Lei Zhang, Ran He

Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) by accessing both the source and target data.

Image Super-Resolution Source-Free Domain Adaptation +1

One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer

1 code implementation CVPR 2023 Jing Lin, Ailing Zeng, Haoqian Wang, Lei Zhang, Yu Li

It is challenging to perform this task with a single network due to resolution issues, i. e., the face and hands are usually located in extremely small regions.

3D Human Pose Estimation 3D Human Reconstruction +1

BoxVIS: Video Instance Segmentation with Box Annotations

1 code implementation26 Mar 2023 Minghan Li, Lei Zhang

As a result, the amount of pixel-wise annotations in existing video instance segmentation (VIS) datasets is small, limiting the generalization capability of trained VIS models.

Instance Segmentation Semantic Segmentation +1

Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases

1 code implementation26 Mar 2023 Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, Xiangang Li

However current research rarely studies the impact of different amounts of instruction data on model performance, especially in the real-world use cases.

OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Rendering

1 code implementation CVPR 2023 Zhiyuan Ma, Xiangyu Zhu, GuoJun Qi, Zhen Lei, Lei Zhang

In this paper, we propose One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a generalized controllable tri-plane rendering solution so that each personalized avatar can be constructed from only one portrait as the reference.

MDQE: Mining Discriminative Query Embeddings to Segment Occluded Instances on Challenging Videos

1 code implementation CVPR 2023 Minghan Li, Shuai Li, Wangmeng Xiang, Lei Zhang

The proposed MDQE is the first VIS method with per-clip input that achieves state-of-the-art results on challenging videos and competitive performance on simple videos.

Instance Segmentation Semantic Segmentation +1

Human Guided Ground-truth Generation for Realistic Image Super-resolution

1 code implementation CVPR 2023 Du Chen, Jie Liang, Xindong Zhang, Ming Liu, Hui Zeng, Lei Zhang

A human guided GT image dataset with both positive and negative samples is then constructed, and a loss function is proposed to train the Real-ISR models.

Image Enhancement Image Super-Resolution

One-to-Few Label Assignment for End-to-End Dense Detection

1 code implementation CVPR 2023 Shuai Li, Minghan Li, Ruihuang Li, Chenhang He, Lei Zhang

The positive and negative weights of these soft anchors are dynamically adjusted during training so that they can contribute more to ``representation learning'' in the early training stage, and contribute more to ``duplicated prediction removal'' in the later stage.

Representation Learning

Sharpness-Aware Gradient Matching for Domain Generalization

1 code implementation CVPR 2023 Pengfei Wang, Zhaoxiang Zhang, Zhen Lei, Lei Zhang

In this paper, we present two conditions to ensure that the model could converge to a flat minimum with a small loss, and present an algorithm, named Sharpness-Aware Gradient Matching (SAGM), to meet the two conditions for improving model generalization capability.

Domain Generalization

Towards Diverse Binary Segmentation via A Simple yet General Gated Network

1 code implementation18 Mar 2023 Xiaoqi Zhao, Youwei Pang, Lihe Zhang, Huchuan Lu, Lei Zhang

They ignore two key problems when the encoder exchanges information with the decoder: one is the lack of interference control mechanism between them, the other is without considering the disparity of the contributions from different encoder levels.

Semantic Segmentation

MSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection from Point Cloud Sequences

1 code implementation CVPR 2023 Chenhang He, Ruihuang Li, Yabin Zhang, Shuai Li, Lei Zhang

Current top-performing multi-frame detectors mostly follow a Detect-and-Fuse framework, which extracts features from each frame of the sequence and fuses them to detect the objects in the current frame.

3D Object Detection Autonomous Driving +1

DynaMask: Dynamic Mask Selection for Instance Segmentation

1 code implementation CVPR 2023 Ruihuang Li, Chenhang He, Shuai Li, Yabin Zhang, Lei Zhang

The representative instance segmentation methods mostly segment different object instances with a mask of the fixed resolution, e. g., 28*28 grid.

Instance Segmentation Semantic Segmentation

A Simple Framework for Open-Vocabulary Segmentation and Detection

2 code implementations ICCV 2023 Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianfeng Gao, Jianwei Yang, Lei Zhang

We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets.

Ranked #2 on Instance Segmentation on ADE20K val (using extra training data)

Instance Segmentation Panoptic Segmentation +1

Synthesizing Realistic Image Restoration Training Pairs: A Diffusion Approach

no code implementations13 Mar 2023 Tao Yang, Peiran Ren, Xuansong Xie, Lei Zhang

In supervised image restoration tasks, one key issue is how to obtain the aligned high-quality (HQ) and low-quality (LQ) training image pairs.

Denoising Image Restoration +1

Tag2Text: Guiding Vision-Language Model via Image Tagging

2 code implementations10 Mar 2023 Xinyu Huang, Youcai Zhang, Jinyu Ma, Weiwei Tian, Rui Feng, Yuejie Zhang, Yaqian Li, Yandong Guo, Lei Zhang

This paper presents Tag2Text, a vision language pre-training (VLP) framework, which introduces image tagging into vision-language models to guide the learning of visual-linguistic features.

Language Modelling TAG

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

2 code implementations9 Mar 2023 Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang

To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.

object-detection Referring Expression +3

ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation

1 code implementation ICCV 2023 Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, WangMeng Zuo

In addition to the unprecedented ability in imaginary creation, large text-to-image models are expected to take customized concepts in image generation.

Spatial-Frequency Attention for Image Denoising

no code implementations27 Feb 2023 Shi Guo, Hongwei Yong, Xindong Zhang, jianqi ma, Lei Zhang

In this paper, we propose the spatial-frequency attention network (SFANet) to enhance the network's ability in exploiting long-range dependency.

Image Denoising

Introducing Depth into Transformer-based 3D Object Detection

no code implementations25 Feb 2023 Hao Zhang, Hongyang Li, Ailing Zeng, Feng Li, Shilong Liu, Xingyu Liao, Lei Zhang

To address the second issue, we introduce an auxiliary learning task called Depth-aware Negative Suppression loss.

3D Object Detection Auxiliary Learning +2

Towards a Sustainable Internet-of-Underwater-Things based on AUVs, SWIPT, and Reinforcement Learning

no code implementations21 Feb 2023 Kenechi G. Omeke, Michael Mollel, Syed T. Shah, Lei Zhang, Qammer H. Abbasi, Muhammad Ali Imran

In this paper, we propose a sustainable scheme to improve the throughput and lifetime of underwater networks, enabling them to potentially operate indefinitely.

Decision Making Reinforcement Learning (RL)

Variation Enhanced Attacks Against RRAM-based Neuromorphic Computing System

no code implementations20 Feb 2023 Hao Lv, Bing Li, Lei Zhang, Cheng Liu, Ying Wang

The RRAM-based neuromorphic computing system has amassed explosive interests for its superior data processing capability and energy efficiency than traditional architectures, and thus being widely used in many data-centric applications.

Adversarial Attack

Dual Graph Multitask Framework for Imbalanced Delivery Time Estimation

no code implementations15 Feb 2023 Lei Zhang, Mingliang Wang, Xin Zhou, Xingyu Wu, Yiming Cao, Yonghui Xu, Lizhen Cui, Zhiqi Shen

To address the issue, we propose a novel Dual Graph Multitask framework for imbalanced Delivery Time Estimation (DGM-DTE).

DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional Networks

1 code implementation10 Feb 2023 Lei Zhang, Xiaodong Yan, Jianshan He, Ruopeng Li, Wei Chu

Our experimental results show that our model effectively relieves the problem of over-smoothing in deep GCNs and outperforms the state-of-the-art (SOTA) methods on various benchmark datasets.

Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation

3 code implementations3 Feb 2023 Jie Yang, Ailing Zeng, Shilong Liu, Feng Li, Ruimao Zhang, Lei Zhang

This paper presents a novel end-to-end framework with Explicit box Detection for multi-person Pose estimation, called ED-Pose, where it unifies the contextual learning between human-level (global) and keypoint-level (local) information.

2D Human Pose Estimation Human Detection +3

Adversarial Style Augmentation for Domain Generalization

no code implementations30 Jan 2023 Yabin Zhang, Bin Deng, Ruihuang Li, Kui Jia, Lei Zhang

By updating the model against the adversarial statistics perturbation during training, we allow the model to explore the worst-case domain and hence improve its generalization performance.

Domain Generalization Retrieval

Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning

no code implementations28 Jan 2023 Lei Zhang, Kaixin Bai, Zhaopeng Chen, Yunlei Shi, Jianwei Zhang

In physical robotic experiments, our grasping framework grasped single known objects and novel complex-shaped household objects with a success rate of 90. 91%.

Data Augmentation Robotic Grasping +1

Towards Accurate Acne Detection via Decoupled Sequential Detection Head

no code implementations28 Jan 2023 Xin Wei, Lei Zhang, Jianwei Zhang, Junyou Wang, Wenjie Liu, Jiaqi Li, Xian Jiang

In addition, we build a high-quality acne detection dataset named ACNE-DET to verify the effectiveness of DSDH.

Revisiting Prototypical Network for Cross Domain Few-Shot Learning

1 code implementation CVPR 2023 Fei Zhou, Peng Wang, Lei Zhang, Wei Wei, Yanning Zhang

Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks.

cross-domain few-shot learning Knowledge Distillation

Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle

1 code implementation ICCV 2023 Song Guo, Lei Zhang, Xiawu Zheng, Yan Wang, Yuchao Li, Fei Chao, Chenglin Wu, Shengchuan Zhang, Rongrong Ji

In this paper, we try to solve this problem by introducing a principled and unified framework based on Information Bottleneck (IB) theory, which further guides us to an automatic pruning approach.

Network Pruning

RCA-NOC: Relative Contrastive Alignment for Novel Object Captioning

no code implementations ICCV 2023 Jiashuo Fan, Yaoyuan Liang, Leyao Liu, ShaoLun Huang, Lei Zhang

We evaluate our approach on two datasets and show that our proposed RCA-NOC approach outperforms state-of-the-art methods by a large margin, demonstrating its effectiveness in improving vision-language representation for novel object captioning.

Contrastive Learning TAG

Joint HDR Denoising and Fusion: A Real-World Mobile HDR Image Dataset

1 code implementation CVPR 2023 Shuaizheng Liu, Xindong Zhang, Lingchen Sun, Zhetong Liang, Hui Zeng, Lei Zhang

In this work, we develop, for the first time to our best knowledge, an HDR image dataset by using mobile phone cameras, namely Mobile-HDR dataset.


A General Regret Bound of Preconditioned Gradient Method for DNN Training

1 code implementation CVPR 2023 Hongwei Yong, Ying Sun, Lei Zhang

Though the full-matrix preconditioned gradient methods theoretically have a lower regret bound, they are impractical for use to train DNNs because of the high complexity.

Image Classification object-detection +1

Exploring Vision Transformers as Diffusion Learners

no code implementations28 Dec 2022 He Cao, Jianan Wang, Tianhe Ren, Xianbiao Qi, Yihao Chen, Yuan YAO, Lei Zhang

We further provide a hypothesis on the implication of disentangling the generative backbone as an encoder-decoder structure and show proof-of-concept experiments verifying the effectiveness of a stronger encoder for generative tasks with ASymmetriC ENcoder Decoder (ASCEND).

Accelerating Dataset Distillation via Model Augmentation

2 code implementations CVPR 2023 Lei Zhang, Jie Zhang, Bowen Lei, Subhabrata Mukherjee, Xiang Pan, Bo Zhao, Caiwen Ding, Yao Li, Dongkuan Xu

Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.

Multi-adversarial Faster-RCNN with Paradigm Teacher for Unrestricted Object Detection

no code implementations International Journal of Computer Vision 2022 Zhenwei He, Lei Zhang, Xinbo Gao, David Zhang

Our proposed MAF has two distinct contributions: (1) The Hierarchical Domain Feature Alignment (HDFA) module is introduced to minimize the image-level domain disparity, where Scale Reduction Module (SRM) reduces the feature map size without information loss and increases the training efficiency.

Domain Adaptation Knowledge Distillation +2

Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution

1 code implementation10 Dec 2022 Ruohao Wang, Xiaohui Liu, Zhilu Zhang, Xiaohe Wu, Chun-Mei Feng, Lei Zhang, WangMeng Zuo

On the other hand, alignment algorithms in existing VSR methods perform poorly for real-world videos, leading to unsatisfactory results.

Optical Flow Estimation Video Super-Resolution

Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution

2 code implementations3 Dec 2022 Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Risheng Yu, Xiansheng Hua, Lei Zhang

In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention.

Box-supervised Instance Segmentation

DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding

1 code implementation28 Nov 2022 Shilong Liu, Yaoyuan Liang, Feng Li, Shijia Huang, Hao Zhang, Hang Su, Jun Zhu, Lei Zhang

As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.

object-detection Object Detection +4

Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation

no code implementations17 Nov 2022 Yiyue Hu, Lei Zhang, Nan Mu, Lei Liu

To this end, we propose a parameter-efficient transformer to explore intrinsic inductive bias via position information for medical image segmentation.

feature selection Image Segmentation +3

Point-MA2E: Masked and Affine Transformed AutoEncoder for Self-supervised Point Cloud Learning

1 code implementation13 Nov 2022 Yabin Zhang, Jiehong Lin, Ruihuang Li, Kui Jia, Lei Zhang

Generally, we corrupt the point cloud with affine transformation and masking as input and learn an encoder-decoder model to reconstruct the original point cloud from its corrupted version.

3D Object Detection Denoising +2

Rethinking the transfer learning for FCN based polyp segmentation in colonoscopy

1 code implementation4 Nov 2022 Yan Wen, Lei Zhang, Xiangli Meng, Xujiong Ye

Besides the complex nature of colonoscopy frames with intrinsic frame formation artefacts such as light reflections and the diversity of polyp types/shapes, the publicly available polyp segmentation training datasets are limited, small and imbalanced.

Transfer Learning

Mining Word Boundaries in Speech as Naturally Annotated Word Segmentation Data

no code implementations31 Oct 2022 Lei Zhang, Shilin Zhou, Chen Gong, Zhenghua Li, Zhefeng Wang, Baoxing Huai, Min Zhang

Chinese word segmentation (CWS) models have achieved very high performance when the training data is sufficient and in-domain.

Chinese Word Segmentation

Mitigating spectral bias for the multiscale operator learning with hierarchical attention

no code implementations19 Oct 2022 Xinliang Liu, Bo Xu, Lei Zhang

Neural operators have emerged as a powerful tool for learning the mapping between infinite-dimensional parameter and solution spaces of partial differential equations (PDEs).

Operator learning

Motion correction in MRI using deep learning and a novel hybrid loss function

1 code implementation19 Oct 2022 Lei Zhang, Xiaoke Wang, Michael Rawson, Radu Balan, Edward H. Herskovits, Elias Melhem, Linda Chang, Ze Wang, Thomas Ernst

Evaluation used simulated T1 and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans.


TLDW: Extreme Multimodal Summarisation of News Videos

no code implementations16 Oct 2022 Peggy Tang, Kun Hu, Lei Zhang, Jiebo Luo, Zhiyong Wang

Multimodal summarisation with multimodal output is drawing increasing attention due to the rapid growth of multimedia data.

Learning Dual Memory Dictionaries for Blind Face Restoration

1 code implementation15 Oct 2022 Xiaoming Li, Shiguang Zhang, Shangchen Zhou, Lei Zhang, WangMeng Zuo

Generally, it is a challenging and intractable task to improve the photo-realistic performance of blind restoration and adaptively handle the generic and specific restoration scenarios with a single unified model.

Blind Face Restoration

Attention Diversification for Domain Generalization

1 code implementation9 Oct 2022 Rang Meng, Xianfeng Li, WeiJie Chen, Shicai Yang, Jie Song, Xinchao Wang, Lei Zhang, Mingli Song, Di Xie, ShiLiang Pu

Under this guidance, a novel Attention Diversification framework is proposed, in which Intra-Model and Inter-Model Attention Diversification Regularization are collaborated to reassign appropriate attention to diverse task-related features.

Domain Generalization

From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution

1 code implementation3 Oct 2022 Xiaoming Li, Chaofeng Chen, Xianhui Lin, WangMeng Zuo, Lei Zhang

Notably, LQ face images, which may have the same degradation process as natural images, can be robustly restored with photo-realistic textures by exploiting their strong structural priors.

Image Generation Image Super-Resolution

Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings

no code implementations13 Sep 2022 Zhen Yu, Toan Nguyen, Yaniv Gal, Lie Ju, Shekhar S. Chandra, Lei Zhang, Paul Bonnington, Victoria Mar, Zhiyong Wang, ZongYuan Ge

Accordingly, the learned prototypes preserve the semantic class relations in the embedding space and we can predict the label of an image by assigning its feature to the nearest hyperbolic class prototype.

Statistical Foundation Behind Machine Learning and Its Impact on Computer Vision

no code implementations6 Sep 2022 Lei Zhang, Heung-Yeung Shum

This paper revisits the principle of uniform convergence in statistical learning, discusses how it acts as the foundation behind machine learning, and attempts to gain a better understanding of the essential problem that current deep learning algorithms are solving.

Representation Learning

Recurrent LSTM-based UAV Trajectory Prediction with ADS-B Information

no code implementations1 Sep 2022 Yifan Zhang, Ziye Jia, Chao Dong, Yuntian Liu, Lei Zhang, Qihui Wu

It is noted that the recurrent neural network (RNN) is available for the UAV trajectory prediction, in which the long short-term memory (LSTM) is specialized in dealing with the time-series data.

Time Series Analysis Trajectory Prediction

Generative Action Description Prompts for Skeleton-based Action Recognition

2 code implementations ICCV 2023 Wangmeng Xiang, Chao Li, Yuxuan Zhou, Biao Wang, Lei Zhang

More specifically, we employ a pre-trained large-scale language model as the knowledge engine to automatically generate text descriptions for body parts movements of actions, and propose a multi-modal training scheme by utilizing the text encoder to generate feature vectors for different body parts and supervise the skeleton encoder for action representation learning.

Action Recognition Language Modelling +2

Spatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition

1 code implementation27 Jul 2022 Wangmeng Xiang, Chao Li, Biao Wang, Xihan Wei, Xian-Sheng Hua, Lei Zhang

For 3D video-based tasks such as action recognition, however, directly applying spatiotemporal transformers on video data will bring heavy computation and memory burdens due to the largely increased number of patches and the quadratic complexity of self-attention computation.

Action Classification Action Recognition

Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges

1 code implementation24 Jul 2022 Lei Zhang, Guanyu Gao, Huaizheng Zhang

Then, the learnt knowledge from edge clients will be aggregated by centralized parameter server, where the knowledge will be selectively and attentively distilled from spatial- and temporal-dimension with carefully designed mechanisms.

Continual Learning Federated Learning +2

Auto Machine Learning for Medical Image Analysis by Unifying the Search on Data Augmentation and Neural Architecture

no code implementations21 Jul 2022 Jianwei Zhang, Dong Li, Lituan Wang, Lei Zhang

To address the problem, an improved augmentation search strategy, named Augmented Density Matching, was proposed by randomly sampling policies from a prior distribution for training.

AutoML Data Augmentation

A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restoration

1 code implementation21 Jul 2022 Ming Liu, Yuxiang Wei, Xiaohe Wu, WangMeng Zuo, Lei Zhang

Generative adversarial networks (GANs) have drawn enormous attention due to the simple yet effective training mechanism and superior image generation quality.

Image Generation Image Restoration

Box-supervised Instance Segmentation with Level Set Evolution

2 code implementations19 Jul 2022 Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Xiansheng Hua, Lei Zhang

A simple mask supervised SOLOv2 model is adapted to predict the instance-aware mask map as the level set for each instance.

Box-supervised Instance Segmentation

Mind the Gap: Polishing Pseudo labels for Accurate Semi-supervised Object Detection

1 code implementation17 Jul 2022 Lei Zhang, Yuxuan Sun, Wei Wei

Instead of directly exploiting the pseudo labels produced by the teacher detector, we take the first attempt at reducing their deviation from ground truth using dual polishing learning, where two differently structured polishing networks are elaborately developed and trained using synthesized paired pseudo labels and the corresponding ground truth for categories and bounding boxes on the given annotated objects, respectively.

object-detection Object Detection +2

MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources

1 code implementation15 Jul 2022 Haoran Yin, Meng Ge, Yanjie Fu, Gaoyan Zhang, Longbiao Wang, Lei Zhang, Lin Qiu, Jianwu Dang

These algorithms are usually achieved by mapping the multi-channel audio input to the single output (i. e. overall spatial pseudo-spectrum (SPS) of all sources), that is called MISO.

E2FIF: Push the limit of Binarized Deep Imagery Super-resolution using End-to-end Full-precision Information Flow

1 code implementation14 Jul 2022 Zhiqiang Lang, Chongxing Song, Lei Zhang, Wei Wei

Binary neural network (BNN) provides a promising solution to deploy parameter-intensive deep single image super-resolution (SISR) models onto real devices with limited storage and computational resources.

Image Super-Resolution

Domain Gap Estimation for Source Free Unsupervised Domain Adaptation with Many Classifiers

no code implementations12 Jul 2022 Ziyang Zong, Jun He, Lei Zhang, Hai Huan

However, for source free UDA, the source domain data can not be accessed during adaptation, which poses great challenge of measuring the domain gap.

Unsupervised Domain Adaptation

Learning High-quality Proposals for Acne Detection

1 code implementation8 Jul 2022 Jianwei Zhang, Lei Zhang, Junyou Wang, Xin Wei, Jiaqi Li, Xian Jiang, Dan Du

Acne detection is crucial for interpretative diagnosis and precise treatment of skin disease.

Classification Region Proposal +2

Masked Surfel Prediction for Self-Supervised Point Cloud Learning

1 code implementation7 Jul 2022 Yabin Zhang, Jiehong Lin, Chenhang He, Yongwei Chen, Kui Jia, Lei Zhang

In this work, we make the first attempt, to the best of our knowledge, to consider the local geometry information explicitly into the masked auto-encoding, and propose a novel Masked Surfel Prediction (MaskSurf) method.

Point cloud reconstruction Self-Supervised Learning

Universal Domain Adaptive Object Detector

no code implementations5 Jul 2022 Wenxu Shi, Lei Zhang, WeiJie Chen, ShiLiang Pu

Universal domain adaptive object detection (UniDAOD)is more challenging than domain adaptive object detection (DAOD) since the label space of the source domain may not be the same as that of the target and the scale of objects in the universal scenarios can vary dramatically (i. e, category shift and scale shift).

Multi-Label Learning object-detection +1

Approximating Discontinuous Nash Equilibrial Values of Two-Player General-Sum Differential Games

no code implementations5 Jul 2022 Lei Zhang, Mukesh Ghimire, Wenlong Zhang, Zhe Xu, Yi Ren

This paper investigates two potential solutions to this problem: a hybrid method that leverages both supervised Nash equilibria and the HJI PDE, and a value-hardening method where a sequence of HJIs are solved with a gradually hardening reward.

Autonomous Driving Self-Supervised Learning

Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters

2 code implementations4 Jul 2022 Wenyu Liu, Wentong Li, Jianke Zhu, Miaomiao Cui, Xuansong Xie, Lei Zhang

With DIAL-Filters, we design both unsupervised and supervised frameworks for nighttime driving-scene segmentation, which can be trained in an end-to-end manner.

Autonomous Driving Scene Segmentation

Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation

1 code implementation20 Jun 2022 Tao Chen, Yazhou Yao, Lei Zhang, Qiong Wang, Guo-Sen Xie, Fumin Shen

Specifically, we propose a saliency guided class-agnostic distance module to pull the intra-category features closer by aligning features to their class prototypes.

Pseudo Label Weakly supervised Semantic Segmentation +1

SP-ViT: Learning 2D Spatial Priors for Vision Transformers

1 code implementation15 Jun 2022 Yuxuan Zhou, Wangmeng Xiang, Chao Li, Biao Wang, Xihan Wei, Lei Zhang, Margret Keuper, Xiansheng Hua

Unlike convolutional inductive biases, which are forced to focus exclusively on hard-coded local regions, our proposed SPs are learned by the model itself and take a variety of spatial relations into account.

Image Classification

Learning Domain Adaptive Object Detection with Probabilistic Teacher

2 code implementations13 Jun 2022 Meilin Chen, WeiJie Chen, Shicai Yang, Jie Song, Xinchao Wang, Lei Zhang, Yunfeng Yan, Donglian Qi, Yueting Zhuang, Di Xie, ShiLiang Pu

In addition, we conduct anchor adaptation in parallel with localization adaptation, since anchor can be regarded as a learnable parameter.

object-detection Object Detection

Are Transformers Effective for Time Series Forecasting?

4 code implementations26 May 2022 Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu

Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task.

Anomaly Detection Temporal Relation Extraction +2

Multiple Domain Cyberspace Attack and Defense Game Based on Reward Randomization Reinforcement Learning

no code implementations23 May 2022 Lei Zhang, Yu Pan, Yi Liu, Qibin Zheng, Zhisong Pan

In order to improve the defense ability of defender, a game model based on reward randomization reinforcement learning is proposed.

reinforcement-learning Reinforcement Learning (RL)

SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images

1 code implementation25 Apr 2022 Zhishe Wang, Yanlin Chen, Wenyu Shao, Hui Li, Lei Zhang

The existing deep learning fusion methods mainly concentrate on the convolutional neural networks, and few attempts are made with transformer.

OTExtSum: Extractive Text Summarisation with Optimal Transport

1 code implementation Findings (NAACL) 2022 Peggy Tang, Kun Hu, Rui Yan, Lei Zhang, Junbin Gao, Zhiyong Wang

Optimal sentence extraction is conceptualised as obtaining an optimal summary that minimises the transportation cost to a given document regarding their semantic distributions.

Dense Learning based Semi-Supervised Object Detection

1 code implementation CVPR 2022 Binghui Chen, Pengyu Li, Xiang Chen, Biao Wang, Lei Zhang, Xian-Sheng Hua

Semi-supervised object detection (SSOD) aims to facilitate the training and deployment of object detectors with the help of a large amount of unlabeled data.

object-detection Object Detection +1

Rapid model transfer for medical image segmentation via iterative human-in-the-loop update: from labelled public to unlabelled clinical datasets for multi-organ segmentation in CT

no code implementations13 Apr 2022 Wenao Ma, Shuang Zheng, Lei Zhang, Huimao Zhang, Qi Dou

Despite the remarkable success on medical image analysis with deep learning, it is still under exploration regarding how to rapidly transfer AI models from one dataset to another for clinical applications.

Image Segmentation Medical Image Segmentation +2

Towards Reliable Image Outpainting: Learning Structure-Aware Multimodal Fusion with Depth Guidance

no code implementations12 Apr 2022 Lei Zhang, Kang Liao, Chunyu Lin, Yao Zhao

Concretely, we propose a Depth-Guided Outpainting Network to model different feature representations of two modalities and learn the structure-aware cross-modal fusion.

Image Outpainting

Large-Scale Pre-training for Person Re-identification with Noisy Labels

2 code implementations CVPR 2022 Dengpan Fu, Dongdong Chen, Hao Yang, Jianmin Bao, Lu Yuan, Lei Zhang, Houqiang Li, Fang Wen, Dong Chen

Since theses ID labels automatically derived from tracklets inevitably contain noises, we develop a large-scale Pre-training framework utilizing Noisy Labels (PNL), which consists of three learning modules: supervised Re-ID learning, prototype-based contrastive learning, and label-guided contrastive learning.

Contrastive Learning Multi-Object Tracking +3

Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution

1 code implementation27 Mar 2022 Jie Liang, Hui Zeng, Lei Zhang

Specifically, a tiny regression network is employed to predict the degradation parameters of the input image, while several convolutional experts with the same topology are jointly optimized to specify the network parameters via a non-linear mixture of experts.

Image Super-Resolution

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds

1 code implementation CVPR 2022 Chenhang He, Ruihuang Li, Shuai Li, Lei Zhang

VoxSeT is built upon a voxel-based set attention (VSA) module, which reduces the self-attention in each voxel by two cross-attentions and models features in a hidden space induced by a group of latent codes.

3D Object Detection object-detection

Towards Robust 2D Convolution for Reliable Visual Recognition

no code implementations18 Mar 2022 Lida Li, Shuai Li, Kun Wang, Xiangchu Feng, Lei Zhang

2D convolution (Conv2d), which is responsible for extracting features from the input image, is one of the key modules of a convolutional neural network (CNN).

A Dual Weighting Label Assignment Scheme for Object Detection

1 code implementation CVPR 2022 Shuai Li, Chenhang He, Ruihuang Li, Lei Zhang

Existing LA methods mostly focus on the design of pos weighting function, while the neg weight is directly derived from the pos weight.

object-detection Object Detection +1

Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition

1 code implementation18 Mar 2022 Tao Yang, Peiran Ren, Xuansong Xie, Xiansheng Hua, Lei Zhang

Most of the existing deep learning based VFI methods adopt off-the-shelf optical flow algorithms to estimate the bidirectional flows and interpolate the missing frames accordingly.

Image Generation Image Morphing +3

Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution

2 code implementations CVPR 2022 Jie Liang, Hui Zeng, Lei Zhang

In this paper, we demonstrate that it is possible to train a GAN-based SISR model which can stably generate perceptually realistic details while inhibiting visual artifacts.

Image Super-Resolution

Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization

1 code implementation CVPR 2022 Yabin Zhang, Minghan Li, Ruihuang Li, Kui Jia, Lei Zhang

In this work, we, for the first time to our best knowledge, propose to perform Exact Feature Distribution Matching (EFDM) by exactly matching the empirical Cumulative Distribution Functions (eCDFs) of image features, which could be implemented by applying the Exact Histogram Matching (EHM) in the image feature space.

Domain Generalization Style Transfer

Efficient Long-Range Attention Network for Image Super-resolution

1 code implementation13 Mar 2022 Xindong Zhang, Hui Zeng, Shi Guo, Lei Zhang

A highly efficient long-range attention block (ELAB) is then built by simply cascading two shift-conv with a GMSA module, which is further accelerated by using a shared attention mechanism.

Image Super-Resolution

One-stage Video Instance Segmentation: From Frame-in Frame-out to Clip-in Clip-out

no code implementations12 Mar 2022 Minghan Li, Lei Zhang

Based on the fact that adjacent frames in a short clip are highly coherent in content, we propose to extend the one-stage FiFo framework to a clip-in clip-out (CiCo) one, which performs VIS clip by clip.

Instance Segmentation Semantic Segmentation +1

Unfolded Deep Kernel Estimation for Blind Image Super-resolution

1 code implementation10 Mar 2022 Hongyi Zheng, Hongwei Yong, Lei Zhang

Nonetheless, the existing deep unfolding methods cannot explicitly solve the data term of the unfolding objective function, limiting their capability in blur kernel estimation.

Image Super-Resolution

DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

13 code implementations7 Mar 2022 Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, Heung-Yeung Shum

Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.

 Ranked #1 on Object Detection on COCO 2017 val (box AP metric)

Real-Time Object Detection

Vision-Language Intelligence: Tasks, Representation Learning, and Large Models

no code implementations3 Mar 2022 Feng Li, Hao Zhang, Yi-Fan Zhang, Shilong Liu, Jian Guo, Lionel M. Ni, Pengchuan Zhang, Lei Zhang

This survey is inspired by the remarkable progress in both computer vision and natural language processing, and recent trends shifting from single modality processing to multiple modality comprehension.

Few-Shot Learning Representation Learning

DN-DETR: Accelerate DETR Training by Introducing Query DeNoising

13 code implementations CVPR 2022 Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M. Ni, Lei Zhang

Our method is universal and can be easily plugged into any DETR-like methods by adding dozens of lines of code to achieve a remarkable improvement.

Object Detection

BP-Triplet Net for Unsupervised Domain Adaptation: A Bayesian Perspective

no code implementations19 Feb 2022 Shanshan Wang, Lei Zhang, Pichao Wang

In our work, considering the different importance of pair-wise samples for both feature learning and domain alignment, we deduce our BP-Triplet loss for effective UDA from the perspective of Bayesian learning.

Metric Learning Unsupervised Domain Adaptation

Winograd Convolution: A Perspective from Fault Tolerance

no code implementations17 Feb 2022 Xinghua Xue, Haitong Huang, Cheng Liu, Ying Wang, Tao Luo, Lei Zhang

Winograd convolution is originally proposed to reduce the computing overhead by converting multiplication in neural network (NN) with addition via linear transformation.

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

6 code implementations ICLR 2022 Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang

We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.

Object Detection

Adversarial Examples for Good: Adversarial Examples Guided Imbalanced Learning

1 code implementation28 Jan 2022 Jie Zhang, Lei Zhang, Gang Li, Chao Wu

Adversarial examples are inputs for machine learning models that have been designed by attackers to cause the model to make mistakes.

Online Multi-Object Tracking with Unsupervised Re-Identification Learning and Occlusion Estimation

no code implementations4 Jan 2022 Qiankun Liu, Dongdong Chen, Qi Chu, Lu Yuan, Bin Liu, Lei Zhang, Nenghai Yu

In addition, such practice of re-identification still can not track those highly occluded objects when they are missed by the detector.

Ranked #7 on Multi-Object Tracking on MOT16 (using extra training data)

Multi-Object Tracking Occlusion Estimation +1

Towards Efficient Data Free Black-Box Adversarial Attack

1 code implementation CVPR 2022 Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, Chao Wu

The proposed method can efficiently imitate the target model through a small number of queries and achieve high attack success rate.

Adversarial Attack

Neural Architecture Search With Representation Mutual Information

1 code implementation CVPR 2022 Xiawu Zheng, Xiang Fei, Lei Zhang, Chenglin Wu, Fei Chao, Jianzhuang Liu, Wei Zeng, Yonghong Tian, Rongrong Ji

Building upon RMI, we further propose a new search algorithm termed RMI-NAS, facilitating with a theorem to guarantee the global optimal of the searched architecture.

Neural Architecture Search

DeePN$^2$: A deep learning-based non-Newtonian hydrodynamic model

no code implementations29 Dec 2021 Lidong Fang, Pei Ge, Lei Zhang, Weinan E, Huan Lei

A long standing problem in the modeling of non-Newtonian hydrodynamics of polymeric flows is the availability of reliable and interpretable hydrodynamic models that faithfully encode the underlying micro-scale polymer dynamics.

CausalMTA: Eliminating the User Confounding Bias for Causal Multi-touch Attribution

no code implementations21 Dec 2021 Di Yao, Chang Gong, Lei Zhang, Sheng Chen, Jingping Bi

Existing methods first train a model to predict the conversion probability of the advertisement journeys with historical data and calculate the attribution of each touchpoint using counterfactual predictions.

Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions

1 code implementation15 Dec 2021 Wenyu Liu, Gaofeng Ren, Runsheng Yu, Shi Guo, Jianke Zhu, Lei Zhang

Though deep learning-based object detection methods have achieved promising results on the conventional datasets, it is still challenging to locate objects from the low-quality images captured in adverse weather conditions.

Image Enhancement object-detection +1