Search Results for author:
Found 577 papers, 254 papers with code
A Decoupled Learning Scheme for Real-world Burst Denoising from Raw Images
On one hand, most of the models are trained on video sequences with synthetic noise.
LST-Net: Learning a Convolutional Neural Network with a Learnable Sparse Transform
The 2D convolutional (Conv2d) layer is the fundamental element to a deep convolutional neural network (CNN).
Momentum Batch Normalization for Deep Learning with Small Batch Size
To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size.
Multi-fingered Robotic Hand Grasping in Cluttered Environments through Hand-object Contact Semantic Mapping
The integration of optimization method and generative models has significantly advanced dexterous manipulation techniques for five-fingered hand grasping.
Responsible Visual Editing
To mitigate the negative implications of harmful images on research, we create a transparent and public dataset, AltBear, which expresses harmful information using teddy bears instead of humans.
Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks
Despite providing high-performance solutions for computer vision tasks, the deep neural network (DNN) model has been proved to be extremely vulnerable to adversarial attacks.
Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models
In this paper, we introduce a versatile adaptation approach that can effectively work under all three settings.
An Open-World, Diverse, Cross-Spatial-Temporal Benchmark for Dynamic Wild Person Re-Identification
Person re-identification (ReID) has made great strides thanks to the data-driven deep learning techniques.
T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy
Recognizing the complementary strengths and weaknesses of both text and visual prompts, we introduce T-Rex2 that synergizes both prompts within a single model through contrastive learning.
MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation
Automatic detection of multimodal misinformation has gained a widespread attention recently.
Compress3D: a Compressed Latent Space for 3D Generation from a Single Image
In this paper, we present a triplane autoencoder, which encodes 3D models into a compact triplane latent space to effectively compress both the 3D geometry and texture information.
HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models
Recent advancements indicate that scaling up Multimodal Large Language Models (MLLMs) effectively enhances performance on downstream multimodal tasks.
Ranked #70 on
Visual Question Answering
on MM-Vet
TAPTR: Tracking Any Point with Transformers as Detection
Based on the observation that point tracking bears a great resemblance to object detection and tracking, we borrow designs from DETR-like algorithms to address the task of TAP.
CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility
To this end, this paper proposes a novel text-guided video inpainting model that achieves better consistency, controllability and compatibility.
IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting
Our research addresses the shortfall by introducing a novel approach to VAC, called Irregular Video Action Counting (IVAC).
Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM
Current defense strategies usually train DNNs for a specific adversarial attack method and can achieve good robustness in defense against this type of adversarial attack.
Implicit Discriminative Knowledge Learning for Visible-Infrared Person Re-Identification
To address this issue, we propose a novel Implicit Discriminative Knowledge Learning (IDKL) network to uncover and leverage the implicit discriminative information contained within the modality-specific.
Source Prompt Disentangled Inversion for Boosting Image Editability with Diffusion Models
Text-driven diffusion models have significantly advanced the image editing performance by using text prompts as inputs.
Self-Supervised Video Desmoking for Laparoscopic Surgery
On the other hand, in order to enhance the desmoking performance, we further feed the valuable information from PS frame into models, where a masking strategy and a regularization term are presented to avoid trivial solutions.
A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment
While Multimodal Large Language Models (MLLMs) have experienced significant advancement on visual understanding and reasoning, their potentials to serve as powerful, flexible, interpretable, and text-driven models for Image Quality Assessment (IQA) remains largely unexplored.
Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcription
This consequently limits the performance of downstream tasks, such as image-to-multiview generation and 3D reconstruction.
Vosh: Voxel-Mesh Hybrid Representation for Real-Time View Synthesis
The neural radiance field (NeRF) has emerged as a prominent methodology for synthesizing realistic images of novel views.
Parameterized quantum comb and simpler circuits for reversing unknown qubit-unitary operations
Quantum comb is an essential tool for characterizing complex quantum protocols in quantum information processing.
SGD with Partial Hessian for Deep Neural Networks Optimization
Compared with first-order optimizers, it adopts a certain amount of information from the Hessian matrix to assist optimization, while compared with the existing second-order optimizers, it keeps the good generalization performance of first-order optimizers.
DragTex: Generative Point-Based Texture Editing on 3D Mesh
Creating 3D textured meshes using generative artificial intelligence has garnered significant attention recently.
Learning Causal Features for Incremental Object Detection
\keywords{Object detection, incremental learning, causal feature.
UniVS: Unified and Universal Video Segmentation with Prompts as Queries
Despite the recent advances in unified image segmentation (IS), developing a unified video segmentation (VS) model remains a challenge.
Ranked #2 on
Video Semantic Segmentation
on VSPW
(using extra training data)
Referring Expression Segmentation
Referring Video Object Segmentation
+6
ConSept: Continual Semantic Segmentation via Adapter-based Vision Transformer
Empirical investigations on the adaptation of existing frameworks to vanilla ViT reveal that incorporating visual adapters into ViTs or fine-tuning ViTs with distillation terms is advantageous for enhancing the segmentation capability of novel classes.
Infrared and visible Image Fusion with Language-driven Loss in CLIP Embedding Space
A language-driven fusion model is then constructed in the embedding space, by establishing the relationship among the embedded vectors to represent the fusion objective and input image modalities.
Gradual Residuals Alignment: A Dual-Stream Framework for GAN Inversion and Image Attribute Editing
GAN-based image attribute editing firstly leverages GAN Inversion to project real images into the latent space of GAN and then manipulates corresponding latent codes.
A Collision-Aware Cable Grasping Method in Cluttered Environment
We introduce a Cable Grasping-Convolutional Neural Network designed to facilitate robust cable grasping in cluttered environments.
CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models
The advancement of large language models (LLMs) has enhanced the ability to generalize across a wide range of unseen natural language processing (NLP) tasks through instruction-following.
EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
For instance, on the Wikitext2 dataset with LlamaV1-7B at 70% sparsity, our proposed EBFT achieves a perplexity of 16. 88, surpassing the state-of-the-art DSnoT with a perplexity of 75. 14.
Invariance-powered Trustworthy Defense via Remove Then Restore
Based on this finding, a Pixel Surgery and Semantic Regeneration (PSSR) model following the targeted therapy mechanism is developed, which has three merits: 1) To remove the salient attack, a score-based Pixel Surgery module is proposed, which retains the trivial attack as a kind of invariance information.
LanDA: Language-Guided Multi-Source Domain Adaptation
Multi-Source Domain Adaptation (MSDA) aims to mitigate changes in data distribution when transferring knowledge from multiple labeled source domains to an unlabeled target domain.
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM).
CIM-MLC: A Multi-level Compilation Stack for Computing-In-Memory Accelerators
Based on the proposed abstraction, CIM-MLC can compile tasks onto a wide range of CIM accelerators having different devices, architectures, and programming interfaces.
PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation
In this paper, we introduce a novel prompt-driven adapter into SAM, namely Prompt Adapter Segment Anything Model (PA-SAM), aiming to enhance the segmentation mask quality of the original SAM.
Symbol as Points: Panoptic Symbol Spotting via Point-based Representation
Finally, we propose a KNN interpolation mechanism for the mask attention module of the spotting head to better handle primitive mask downsampling, which is primitive-level in contrast to pixel-level for the image.
PMFSNet: Polarized Multi-scale Feature Self-attention Network For Lightweight Medical Image Segmentation
In this work, we propose PMFSNet, a novel medical imaging segmentation model that effectively balances global and local feature processing while avoiding the computational redundancy typical in larger models.
Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy
While single task image restoration (IR) has achieved significant successes, it remains a challenging issue to train a single model which can tackle multiple IR tasks.
Pontryagin Neural Operator for Solving Parametric General-Sum Differential Games
To address these challenges, we propose in this paper a Pontryagin-mode neural operator that outperforms existing state-of-the-art (SOTA) on safety performance across games with parametric state constraints.
ScatterFormer: Efficient Voxel Transformer with Scattered Linear Attention
Window-based transformers have demonstrated strong ability in large-scale point cloud understanding by capturing context-aware representations with affordable attention computation in a more localized manner.
Improving the Stability of Diffusion Models for Content Consistent Super-Resolution
To improve the stability of diffusion prior-based SR, we propose to employ the diffusion models to refine image structures, while employing the generative adversarial training to enhance image fine details.
A Closed-Loop Multi-perspective Visual Servoing Approach with Reinforcement Learning
Traditional visual servoing methods suffer from serving between scenes from multiple perspectives, which humans can complete with visual signals alone.
Toward Accurate and Temporally Consistent Video Restoration from Raw Data
Extensive experiments demonstrate the leading VJDD performance of our method in term of restoration accuracy, perceptual quality and temporal consistency.
Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspective
High perceptual quality and low distortion degree are two important goals in image restoration tasks such as super-resolution (SR).
One Shot Learning as Instruction Data Prospector for Large Language Models
Nuggets assesses the potential of individual instruction examples to act as effective one shot examples, thereby identifying those that can significantly enhance diverse task performance.
Osprey: Pixel Understanding with Visual Instruction Tuning
In this paper, we propose Osprey, a mask-text instruction tuning approach, to extend MLLMs by incorporating fine-grained mask regions into language instruction, aiming at achieving pixel-wise visual understanding.
Marathon: A Race Through the Realm of Long Context with Large Language Models
Although there are currently many benchmarks available for evaluating the long context understanding and reasoning capability of large language models, with the expansion of the context window in these models, the existing long context benchmarks are no longer sufficient for evaluating the long context understanding and reasoning capability of large language models.
TMP: Temporal Motion Propagation for Online Video Super-Resolution
Online video super-resolution (online-VSR) highly relies on an effective alignment module to aggregate temporal information, while the strict latency requirement makes accurate and efficient alignment very challenging.
Vision-Language Models as a Source of Rewards
Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning.
Guided Image Restoration via Simultaneous Feature and Image Guided Fusion
Currently, joint image filtering-inspired deep learning-based methods represent the state-of-the-art for GIR tasks.
Stable Score Distillation for High-Quality 3D Generation
Although Score Distillation Sampling (SDS) has exhibited remarkable performance in conditional 3D content generation, a comprehensive understanding of its formulation is still lacking, hindering the development of 3D generation.
Dynamic Weighted Combiner for Mixed-Modal Image Retrieval
First, we propose an Editable Modality De-equalizer (EMD) by taking into account the contribution disparity between modalities, containing two modality feature editors and an adaptive weighted combiner.
RCA-NOC: Relative Contrastive Alignment for Novel Object Captioning
We evaluate our approach on two datasets and show that our proposed RCA-NOC approach outperforms state-of-the-art methods by a large margin, demonstrating its effectiveness in improving vision-language representation for novel object captioning.
Audio-Visual LLM for Video Understanding
This paper presents Audio-Visual LLM, a Multimodal Large Language Model that takes both visual and auditory inputs for holistic video understanding.
Compress & Align: Curating Image-Text Data with Human Knowledge
The massive growth of image-text data through web crawling inherently presents the challenge of variability in data quality.
OpenSD: Unified Open-Vocabulary Segmentation and Detection
To address these challenges, we present a universal transformer-based framework, abbreviated as OpenSD, which utilizes the same architecture and network parameters to handle open-vocabulary segmentation and detection tasks.
Ranked #8 on
Zero Shot Segmentation
on Segmentation in the Wild
PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction
To make up for the lack of dynamic HOI scenarios in this area, we introduce the BallPlay dataset that contains eight whole-body basketball skills.
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities.
Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution
To ensure the content consistency among adjacent frames, we exploit the temporal dynamics in LR videos to guide the diffusion process by optimizing the latent sampling path with a motion-guided loss, ensuring that the generated HR video maintains a coherent and continuous visual flow.
Value Approximation for Two-Player General-Sum Differential Games with State Constraints
Solving Hamilton-Jacobi-Isaacs (HJI) PDEs numerically enables equilibrial feedback control in two-player differential games, yet faces the curse of dimensionality (CoD).
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution
First, we train a degradation-aware prompt extractor, which can generate accurate soft and hard semantic prompts even under strong degradation.
RIDE: Real-time Intrusion Detection via Explainable Machine Learning Implemented in a Memristor Hardware Architecture
Deep Learning (DL) based methods have shown great promise in network intrusion detection by identifying malicious network traffic behavior patterns with high accuracy, but their applications to real-time, packet-level detections in high-speed communication networks are challenging due to the high computation time and resource requirements of Deep Neural Networks (DNNs), as well as lack of explainability.
E-CORE: Emotion Correlation Enhanced Empathetic Dialogue Generation
Then we propose an emotion correlation enhanced decoder, with a novel correlation-aware aggregation and soft/hard strategy, respectively improving the emotion perception and response generation.
Parameter Exchange for Robust Dynamic Domain Generalization
The parameters of dynamic networks can be decoupled into a static and a dynamic component, which are designed to learn domain-invariant and domain-specific features, respectively.
Visual In-Context Prompting
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
T-Rex: Counting by Visual Prompting
Guided by the visual feedback from T-Rex, users can also interactively refine the counting results by prompting on missing or falsely-detected objects.
AcademicGPT: Empowering Academic Research
Our training corpus mainly consists of academic papers, thesis, content from some academic domain, high-quality Chinese data and others.
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models.
Ranked #1 on
LMM real-life tasks
on Leaderboard
MatNexus: A Comprehensive Text Mining and Analysis Suite for Materials Discover
MatNexus is a specialized software for the automated collection, processing, and analysis of text from scientific articles.
Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition
However, the high computational cost of optimization-based TTA algorithms makes it intractable to run on resource-constrained edge devices.
A Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication
Generative AI applications are recently catering to a vast user base by creating diverse and high-quality AI-generated content (AIGC).
Adapt Anything: Tailor Any Image Classifiers across Domains And Categories Using Text-to-Image Diffusion Models
Such a one-for-all adaptation paradigm allows us to adapt anything in the world using only one text-to-image generator as well as the corresponding unlabeled target data.
Open-Set Image Tagging with Multi-Grained Text Supervision
Specifically, for predefined commonly used tag categories, RAM++ showcases 10. 2 mAP and 15. 4 mAP enhancements over CLIP on OpenImages and ImageNet.
HumanTOMATO: Text-aligned Whole-body Motion Generation
This work targets a novel text-driven whole-body motion generation task, which takes a given textual description as input and aims at generating high-quality, diverse, and coherent facial expressions, hand gestures, and body motions simultaneously.
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts
Recent text-to-3D generation methods achieve impressive 3D content creation capacity thanks to the advances in image diffusion models and optimizing strategies.
Label-efficient Segmentation via Affinity Propagation
In this work, we formulate the affinity modeling as an affinity propagation process, and propose a local and a global pairwise affinity terms to generate accurate soft pseudo labels.
TOSS:High-quality Text-guided Novel View Synthesis from a Single Image
In this paper, we present TOSS, which introduces text to the task of novel view synthesis (NVS) from just a single RGB image.
Transport-Hub-Aware Spatial-Temporal Adaptive Graph Transformer for Traffic Flow Prediction
Finally, we design an extra spatial-temporal knowledge distillation module for incremental learning of traffic flow prediction tasks.
Consistent123: Improve Consistency for One Image to 3D Object Synthesis
Large image diffusion models enable novel view synthesis with high quality and excellent zero-shot capability.
Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval
Hence, we address this problem by decomposing video information into reconstruction-dependent and semantic-dependent information, which disentangles the semantic extraction from reconstruction constraint.
UniPose: Detecting Any Keypoints
This work proposes a unified framework called UniPose to detect keypoints of any articulated (e. g., human and animal), rigid, and soft objects via visual or textual prompts for fine-grained vision understanding and manipulation.
Ranked #1 on
2D Human Pose Estimation
on Human-Art
(using extra training data)
X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection
Generative adversarial networks (GANs) have remarkably advanced in diverse domains, especially image generation and editing.
Enhancing Accuracy in Deep Learning Using Random Matrix Theory
We explore the applications of random matrix theory (RMT) in the training of deep neural networks (DNNs), focusing on layer pruning that is reducing the number of DNN parameters (weights).
CPPF: A contextual and post-processing-free model for automatic speech recognition
To address this issue, we draw inspiration from the multifaceted capabilities of LLMs and Whisper, and focus on integrating multiple ASR text processing tasks related to speech recognition into the ASR model.
Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices
In this paper, we present Delta-LoRA, which is a novel parameter-efficient approach to fine-tune large language models (LLMs).
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization
Diffusion models have demonstrated impressive performance in various image generation, editing, enhancement and translation tasks.
Neural Interactive Keypoint Detection
Click-Pose explores how user feedback can cooperate with a neural keypoint detector to correct the predicted keypoints in an interactive way for a faster and more effective annotation process.
Uncovering User Interest from Biased and Noised Watch Time in Video Recommendation
In the video recommendation, watch time is commonly adopted as an indicator of user interest.
Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance
When it is applied on fault-tolerant neural networks enhanced with fault-aware retraining and constrained activation functions, the resulting model accuracy generally shows significant improvement in presence of various faults.
Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation
Recent leading zero-shot video object segmentation (ZVOS) works devote to integrating appearance and motion information by elaborately designing feature fusion modules and identically applying them in multiple feature stages.
A Benchmark for Chinese-English Scene Text Image Super-resolution
Scene Text Image Super-resolution (STISR) aims to recover high-resolution (HR) scene text images with visually pleasant and readable text content from the given low-resolution (LR) input.
Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport
Weakly-supervised image segmentation has recently attracted increasing research attentions, aiming to avoid the expensive pixel-wise labeling.
DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting
Existing feature lifting approaches, such as Lift-Splat-based and 2D attention-based, either use estimated depth to get pseudo LiDAR features and then splat them to a 3D space, which is a one-pass operation without feature refinement, or ignore depth and lift features by 2D attention mechanisms, which achieve finer semantics while suffering from a depth ambiguity problem.
CORE: Cooperative Reconstruction for Multi-Agent Perception
This paper presents CORE, a conceptually simple, effective and communication-efficient model for multi-agent cooperative perception.
Neural Quantile Optimization for Edge-Cloud Computing
The network structure reflects the edge-cloud computing topology and is trained to minimize the expectation of the cost function for unconstrained continuous optimization problems.
Semantic-SAM: Segment and Recognize Anything at Any Granularity
In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity.
Lightweight Improved Residual Network for Efficient Inverse Tone Mapping
But the majority of media images on the internet remain in 8-bit standard dynamic range (SDR) format.
MomentDiff: Generative Video Moment Retrieval from Random to Real
Video moment retrieval pursues an efficient and generalized solution to identify the specific temporal segments within an untrimmed video that correspond to a given language description.
Impact of UAVs Equipped with ADS-B on the Civil Aviation Monitoring System
However, due to the limited frequency of ADS-B technique, UAVs equipped with ADS-B devices result in the loss of packets to both UAVs and civil aviation.
Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset
In this paper, we present Motion-X, a large-scale 3D expressive whole-body motion dataset.
Steganographic Capacity of Deep Learning Models
We find that the steganographic capacity of the learning models tested is surprisingly high, and that in each case, there is a clear threshold after which model performance rapidly degrades.
DreamTime: An Improved Optimization Strategy for Text-to-3D Content Creation
Text-to-image diffusion models pre-trained on billions of image-text pairs have recently enabled text-to-3D content creation by optimizing a randomly initialized Neural Radiance Fields (NeRF) with score distillation.
Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant
This article provides a comprehensive understanding of optimization in deep learning, with a primary focus on the challenges of gradient vanishing and gradient exploding, which normally lead to diminished model representational ability and training instability, respectively.
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
Recognize Anything: A Strong Image Tagging Model
We are releasing the RAM at \url{https://recognize-anything. github. io/} to foster the advancements of large models in computer vision.
Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning
Recently, compressive text summarisation offers a balance between the conciseness issue of extractive summarisation and the factual hallucination issue of abstractive summarisation.
Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis
Despite the progress in semantic image synthesis, it remains a challenging problem to generate photo-realistic parts from input semantic map.
Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Model
Extensive experiments demonstrate that DiffHOI significantly outperforms the state-of-the-art in regular detection (i. e., 41. 50 mAP) and zero-shot detection.
Ranked #2 on
Zero-Shot Human-Object Interaction Detection
on HICO-DET
(using extra training data)
Human-Object Interaction Detection
Zero-Shot Human-Object Interaction Detection
Cognition Guided Human-Object Relationship Detection
Human-object relationship detection reveals the fine-grained relationship between humans and objects, helping the comprehensive understanding of videos.
MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks
In contrast, though human engineers have the incredible ability to understand tasks and reason about solutions, their experience and knowledge are often sparse and difficult to utilize by quantitative approaches.
A marker-less human motion analysis system for motion-based biomarker discovery in knee disorders
In recent years the NHS has been having increased difficulty seeing all low-risk patients, this includes but not limited to suspected osteoarthritis (OA) patients.
A Strong and Reproducible Object Detector with Only Public Datasets
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.
Ranked #5 on
Object Detection
on COCO minival
(using extra training data)
Glocal Energy-based Learning for Few-Shot Open-Set Recognition
Few-shot open-set recognition (FSOR) is a challenging task of great practical value.
LipsFormer: Introducing Lipschitz Continuity to Vision Transformers
In contrast to previous practical tricks that address training instability by learning rate warmup, layer normalization, attention formulation, and weight initialization, we show that Lipschitz continuity is a more essential property to ensure training stability.
DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
In this way, we can reduce the GPU memory consumption of contrastive loss computation from $\bigO(B^2)$ to $\bigO(\frac{B^2}{N})$, where $B$ and $N$ are the batch size and the number of GPUs used for training.
Language Guided Local Infiltration for Interactive Image Retrieval
Interactive Image Retrieval (IIR) aims to retrieve images that are generally similar to the reference image but under the requested text modification.
Detection Transformer with Stable Matching
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation
While such a plug-and-play approach is appealing, the inevitable and uncertain conflicts between the original images produced from the frozen SD branch and the given condition incur significant challenges for the learnable branch, which essentially conducts image feature editing for condition enforcement.
Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domains
In this work, we present an idea to remove non-causal factors from common features by multi-view adversarial training on source domains, because we observe that such insignificant non-causal factors may still be significant in other latent spaces (views) due to the multi-mode structure of data.
Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer
Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) by accessing both the source and target data.
One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer
It is challenging to perform this task with a single network due to resolution issues, i. e., the face and hands are usually located in extremely small regions.
Ranked #3 on
3D Human Pose Estimation
on UBody
BoxVIS: Video Instance Segmentation with Box Annotations
As a result, the amount of pixel-wise annotations in existing video instance segmentation (VIS) datasets is small, limiting the generalization capability of trained VIS models.
Ranked #16 on
Video Instance Segmentation
on YouTube-VIS 2021
Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases
However current research rarely studies the impact of different amounts of instruction data on model performance, especially in the real-world use cases.
OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Rendering
In this paper, we propose One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a generalized controllable tri-plane rendering solution so that each personalized avatar can be constructed from only one portrait as the reference.
MDQE: Mining Discriminative Query Embeddings to Segment Occluded Instances on Challenging Videos
The proposed MDQE is the first VIS method with per-clip input that achieves state-of-the-art results on challenging videos and competitive performance on simple videos.
Ranked #13 on
Video Instance Segmentation
on YouTube-VIS 2021
Human Guided Ground-truth Generation for Realistic Image Super-resolution
A human guided GT image dataset with both positive and negative samples is then constructed, and a loss function is proposed to train the Real-ISR models.
One-to-Few Label Assignment for End-to-End Dense Detection
The positive and negative weights of these soft anchors are dynamically adjusted during training so that they can contribute more to ``representation learning'' in the early training stage, and contribute more to ``duplicated prediction removal'' in the later stage.
Towards Diverse Binary Segmentation via A Simple yet General Gated Network
They ignore two key problems when the encoder exchanges information with the decoder: one is the lack of interference control mechanism between them, the other is without considering the disparity of the contributions from different encoder levels.
Sharpness-Aware Gradient Matching for Domain Generalization
In this paper, we present two conditions to ensure that the model could converge to a flat minimum with a small loss, and present an algorithm, named Sharpness-Aware Gradient Matching (SAGM), to meet the two conditions for improving model generalization capability.
MSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection from Point Cloud Sequences
Current top-performing multi-frame detectors mostly follow a Detect-and-Fuse framework, which extracts features from each frame of the sequence and fuses them to detect the objects in the current frame.
A Simple Framework for Open-Vocabulary Segmentation and Detection
We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets.
Ranked #2 on
Instance Segmentation
on ADE20K val
(using extra training data)
SIM: Semantic-aware Instance Mask Generation for Box-Supervised Instance Segmentation
Weakly supervised instance segmentation using only bounding box annotations has recently attracted much research attention.
DynaMask: Dynamic Mask Selection for Instance Segmentation
The representative instance segmentation methods mostly segment different object instances with a mask of the fixed resolution, e. g., 28*28 grid.
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
Synthesizing Realistic Image Restoration Training Pairs: A Diffusion Approach
In supervised image restoration tasks, one key issue is how to obtain the aligned high-quality (HQ) and low-quality (LQ) training image pairs.
MP-Former: Mask-Piloted Transformer for Image Segmentation
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation.
Tag2Text: Guiding Vision-Language Model via Image Tagging
This paper presents Tag2Text, a vision language pre-training (VLP) framework, which introduces image tagging into vision-language models to guide the learning of visual-linguistic features.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.
Ranked #1 on
Zero-Shot Object Detection
on MSCOCO
Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes
Humans have long been recorded in a variety of forms since antiquity.
ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation
In addition to the unprecedented ability in imaginary creation, large text-to-image models are expected to take customized concepts in image generation.
Spatial-Frequency Attention for Image Denoising
In this paper, we propose the spatial-frequency attention network (SFANet) to enhance the network's ability in exploiting long-range dependency.
Introducing Depth into Transformer-based 3D Object Detection
To address the second issue, we introduce an auxiliary learning task called Depth-aware Negative Suppression loss.
Towards a Sustainable Internet-of-Underwater-Things based on AUVs, SWIPT, and Reinforcement Learning
In this paper, we propose a sustainable scheme to improve the throughput and lifetime of underwater networks, enabling them to potentially operate indefinitely.
Variation Enhanced Attacks Against RRAM-based Neuromorphic Computing System
The RRAM-based neuromorphic computing system has amassed explosive interests for its superior data processing capability and energy efficiency than traditional architectures, and thus being widely used in many data-centric applications.
Dual Graph Multitask Framework for Imbalanced Delivery Time Estimation
To address the issue, we propose a novel Dual Graph Multitask framework for imbalanced Delivery Time Estimation (DGM-DTE).
DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional Networks
Our experimental results show that our model effectively relieves the problem of over-smoothing in deep GCNs and outperforms the state-of-the-art (SOTA) methods on various benchmark datasets.
Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation
This paper presents a novel end-to-end framework with Explicit box Detection for multi-person Pose estimation, called ED-Pose, where it unifies the contextual learning between human-level (global) and keypoint-level (local) information.
Ranked #2 on
2D Human Pose Estimation
on Human-Art
Adversarial Style Augmentation for Domain Generalization
By updating the model against the adversarial statistics perturbation during training, we allow the model to explore the worst-case domain and hence improve its generalization performance.
Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning
In physical robotic experiments, our grasping framework grasped single known objects and novel complex-shaped household objects with a success rate of 90. 91%.
Towards Accurate Acne Detection via Decoupled Sequential Detection Head
In addition, we build a high-quality acne detection dataset named ACNE-DET to verify the effectiveness of DSDH.
Human-Timescale Adaptation in an Open-Ended Task Space
Foundation models have shown impressive adaptation and scalability in supervised and self-supervised learning problems, but so far these successes have not fully translated to reinforcement learning (RL).
Joint HDR Denoising and Fusion: A Real-World Mobile HDR Image Dataset
In this work, we develop, for the first time to our best knowledge, an HDR image dataset by using mobile phone cameras, namely Mobile-HDR dataset.
FPR: False Positive Rectification for Weakly Supervised Semantic Segmentation
Without introducing any external supervision and human priors, the proposed FPR effectively suppresses wrong activations from the background objects.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Towards Fairness-aware Adversarial Network Pruning
Network pruning aims to compress models while minimizing loss in accuracy.
A General Regret Bound of Preconditioned Gradient Method for DNN Training
Though the full-matrix preconditioned gradient methods theoretically have a lower regret bound, they are impractical for use to train DNNs because of the high complexity.
Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle
In this paper, we try to solve this problem by introducing a principled and unified framework based on Information Bottleneck (IB) theory, which further guides us to an automatic pruning approach.
Lite DETR: An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
Revisiting Prototypical Network for Cross Domain Few-Shot Learning
Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks.
Exploring Vision Transformers as Diffusion Learners
We further provide a hypothesis on the implication of disentangling the generative backbone as an encoder-decoder structure and show proof-of-concept experiments verifying the effectiveness of a stronger encoder for generative tasks with ASymmetriC ENcoder Decoder (ASCEND).
Accelerating Dataset Distillation via Model Augmentation
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.
Multi-adversarial Faster-RCNN with Paradigm Teacher for Unrestricted Object Detection
Our proposed MAF has two distinct contributions: (1) The Hierarchical Domain Feature Alignment (HDFA) module is introduced to minimize the image-level domain disparity, where Scale Reduction Module (SRM) reduces the feature map size without information loss and increases the training efficiency.
Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution
On the other hand, alignment algorithms in existing VSR methods perform poorly for real-world videos, leading to unsatisfactory results.
Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution
In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention.
Inconsistency Ranking-based Noisy Label Detection for High-quality Data
We apply this technique to the automatic speaker verification (ASV) task as a proof of concept.
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.
Ranked #7 on
Referring Expression Comprehension
on RefCOCO
Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation
To this end, we propose a parameter-efficient transformer to explore intrinsic inductive bias via position information for medical image segmentation.
A Unified Mutual Supervision Framework for Referring Expression Segmentation and Generation
Our mutual supervision contains two directions.
Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning
We also validate the effectiveness of affine transformation corruption with the Transformer backbones, where we decompose the reconstruction of the complete point cloud into the reconstructions of detailed local patches and rough global shape, alleviating the position leakage problem in the reconstruction.
Rethinking the transfer learning for FCN based polyp segmentation in colonoscopy
Besides the complex nature of colonoscopy frames with intrinsic frame formation artefacts such as light reflections and the diversity of polyp types/shapes, the publicly available polyp segmentation training datasets are limited, small and imbalanced.
Mining Word Boundaries in Speech as Naturally Annotated Word Segmentation Data
Inspired by early research on exploring naturally annotated data for Chinese word segmentation (CWS), and also by recent research on integration of speech and text processing, this work for the first time proposes to mine word boundaries from parallel speech/text data.
Mitigating spectral bias for the multiscale operator learning with hierarchical attention
Neural operators have emerged as a powerful tool for learning the mapping between infinite-dimensional parameter and solution spaces of partial differential equations (PDEs).
Motion correction in MRI using deep learning and a novel hybrid loss function
Evaluation used simulated T1 and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans.
TLDW: Extreme Multimodal Summarisation of News Videos
Multimodal summarisation with multimodal output is drawing increasing attention due to the rapid growth of multimedia data.
Learning Dual Memory Dictionaries for Blind Face Restoration
Generally, it is a challenging and intractable task to improve the photo-realistic performance of blind restoration and adaptively handle the generic and specific restoration scenarios with a single unified model.
Attention Diversification for Domain Generalization
Under this guidance, a novel Attention Diversification framework is proposed, in which Intra-Model and Inter-Model Attention Diversification Regularization are collaborated to reassign appropriate attention to diverse task-related features.
From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution
Notably, LQ face images, which may have the same degradation process as natural images, can be robustly restored with photo-realistic textures by exploiting their strong structural priors.
Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings
Accordingly, the learned prototypes preserve the semantic class relations in the embedding space and we can predict the label of an image by assigning its feature to the nearest hyperbolic class prototype.
Statistical Foundation Behind Machine Learning and Its Impact on Computer Vision
This paper revisits the principle of uniform convergence in statistical learning, discusses how it acts as the foundation behind machine learning, and attempts to gain a better understanding of the essential problem that current deep learning algorithms are solving.
Recurrent LSTM-based UAV Trajectory Prediction with ADS-B Information
It is noted that the recurrent neural network (RNN) is available for the UAV trajectory prediction, in which the long short-term memory (LSTM) is specialized in dealing with the time-series data.
Generative Action Description Prompts for Skeleton-based Action Recognition
More specifically, we employ a pre-trained large-scale language model as the knowledge engine to automatically generate text descriptions for body parts movements of actions, and propose a multi-modal training scheme by utilizing the text encoder to generate feature vectors for different body parts and supervise the skeleton encoder for action representation learning.
Ranked #5 on
Skeleton Based Action Recognition
on N-UCLA
Spatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition
For 3D video-based tasks such as action recognition, however, directly applying spatiotemporal transformers on video data will bring heavy computation and memory burdens due to the largely increased number of patches and the quadratic complexity of self-attention computation.
Ranked #9 on
Action Recognition
on Diving-48
Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges
Then, the learnt knowledge from edge clients will be aggregated by centralized parameter server, where the knowledge will be selectively and attentively distilled from spatial- and temporal-dimension with carefully designed mechanisms.
Auto Machine Learning for Medical Image Analysis by Unifying the Search on Data Augmentation and Neural Architecture
To address the problem, an improved augmentation search strategy, named Augmented Density Matching, was proposed by randomly sampling policies from a prior distribution for training.
A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restoration
Generative adversarial networks (GANs) have drawn enormous attention due to the simple yet effective training mechanism and superior image generation quality.
Box-supervised Instance Segmentation with Level Set Evolution
A simple mask supervised SOLOv2 model is adapted to predict the instance-aware mask map as the level set for each instance.
Mind the Gap: Polishing Pseudo labels for Accurate Semi-supervised Object Detection
Instead of directly exploiting the pseudo labels produced by the teacher detector, we take the first attempt at reducing their deviation from ground truth using dual polishing learning, where two differently structured polishing networks are elaborately developed and trained using synthesized paired pseudo labels and the corresponding ground truth for categories and bounding boxes on the given annotated objects, respectively.
Ranked #10 on
Semi-Supervised Object Detection
on COCO 5% labeled data
MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources
These algorithms are usually achieved by mapping the multi-channel audio input to the single output (i. e. overall spatial pseudo-spectrum (SPS) of all sources), that is called MISO.
E2FIF: Push the limit of Binarized Deep Imagery Super-resolution using End-to-end Full-precision Information Flow
Binary neural network (BNN) provides a promising solution to deploy parameter-intensive deep single image super-resolution (SISR) models onto real devices with limited storage and computational resources.
Domain Gap Estimation for Source Free Unsupervised Domain Adaptation with Many Classifiers
However, for source free UDA, the source domain data can not be accessed during adaptation, which poses great challenge of measuring the domain gap.
Learning High-quality Proposals for Acne Detection
Acne detection is crucial for interpretative diagnosis and precise treatment of skin disease.
Masked Surfel Prediction for Self-Supervised Point Cloud Learning
In this work, we make the first attempt, to the best of our knowledge, to consider the local geometry information explicitly into the masked auto-encoding, and propose a novel Masked Surfel Prediction (MaskSurf) method.
Approximating Discontinuous Nash Equilibrial Values of Two-Player General-Sum Differential Games
This paper investigates two potential solutions to this problem: a hybrid method that leverages both supervised Nash equilibria and the HJI PDE, and a value-hardening method where a sequence of HJIs are solved with a gradually hardening reward.
Universal Domain Adaptive Object Detector
Universal domain adaptive object detection (UniDAOD)is more challenging than domain adaptive object detection (DAOD) since the label space of the source domain may not be the same as that of the target and the scale of objects in the universal scenarios can vary dramatically (i. e, category shift and scale shift).
Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters
With DIAL-Filters, we design both unsupervised and supervised frameworks for nighttime driving-scene segmentation, which can be trained in an end-to-end manner.
Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation
Specifically, we propose a saliency guided class-agnostic distance module to pull the intra-category features closer by aligning features to their class prototypes.
SP-ViT: Learning 2D Spatial Priors for Vision Transformers
Unlike convolutional inductive biases, which are forced to focus exclusively on hard-coded local regions, our proposed SPs are learned by the model itself and take a variety of spatial relations into account.
Ranked #153 on
Image Classification
on ImageNet
Learning Domain Adaptive Object Detection with Probabilistic Teacher
In addition, we conduct anchor adaptation in parallel with localization adaptation, since anchor can be regarded as a learnable parameter.
