Search Results for author:
Found 263 papers, 163 papers with code
Deep Continuous Conditional Random Fields with Asymmetric Inter-object Constraints for Online Multi-object Tracking
In addition, inter-object relations are mostly modeled in a symmetric way, which we argue is not an optimal setting.
Learnable Histogram: Statistical Context Features for Deep Neural Networks
Statistical features, such as histogram, Bag-of-Words (BoW) and Fisher Vector, were commonly used with hand-crafted features in conventional classification methods, but attract less attention since the popularity of deep learning methods.
3D Human Pose Estimation in the Wild by Adversarial Learning
Instead of defining hard-coded rules to constrain the pose estimation results, we design a novel multi-source discriminator to distinguish the predicted 3D poses from the ground-truth, which helps to enforce the pose estimator to generate anthropometrically valid poses even with images in the wild.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data
The aim of image captioning is to generate captions by machine to describe image contents.
Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism
The visibility map of the target is learned and used for inferring the spatial attention map.
Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
Vehicle re-identification is an important problem and has many applications in video surveillance and intelligent transportation.
Identity-Aware Textual-Visual Matching with Latent Co-attention
The stage-2 CNN-LSTM network refines the matching results with a latent co-attention mechanism.
Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection
We propose a convolution neural network based algorithm for simultaneously diagnosing diabetic retinopathy and highlighting suspicious regions.
Learning Deep Representations for Scene Labeling with Semantic Context Guided Supervision
The experiments show that our proposed method makes deep models learn more discriminative feature representations without increasing model size or complexity.
CRF-CNN: Modeling Structured Information in Human Pose Estimation
In a classical neural network, there is no message passing between neurons in the same layer.
Structured Feature Learning for Pose Estimation
In this paper, we propose a structured feature learning framework to reason the correlations among body joints at the feature level in human pose estimation.
DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
In this paper, we propose deformable deep convolutional neural networks for generic object detection.
Fast Iteratively Reweighted Least Squares Algorithms for Analysis-Based Sparsity Reconstruction
The convergence rate of the proposed algorithm is almost the same as that of the traditional IRLS algorithms, that is, exponentially fast.
Highly Efficient Forward and Backward Propagation of Convolutional Neural Networks for Pixelwise Classification
The proposed algorithms eliminate all the redundant computation in convolution and pooling on images by introducing novel d-regularly sparse kernels.
DeepID-Net: multi-stage and deformable deep convolutional neural networks for object detection
In the proposed new deep architecture, a new deformation constrained pooling (def-pooling) layer models the deformation of object parts with geometric constraint and penalty.
Person Re-identification with Deep Similarity-Guided Graph Neural Network
However, existing person re-identification models mostly estimate the similarities of different image pairs of probe and gallery images independently while ignores the relationship information between different probe-gallery pairs.
Ranked #2 on
Person Re-Identification
on CUHK03
Generative Adversarial Frontal View to Bird View Synthesis
Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging.
Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association
Person re-identification is an important task that requires learning discriminative visual features for distinguishing different person identities.
Ranked #22 on
Text based Person Retrieval
on CUHK-PEDES
Question-Guided Hybrid Convolution for Visual Question Answering
Most state-of-the-art VQA methods fuse the high-level textual and visual features from the neural network and abandon the visual spatial information when learning multi-modal features. To address these problems, question-guided kernels generated from the input question are designed to convolute with visual features for capturing the textual and visual relationship in the early stage.
Ranked #14 on
Visual Question Answering (VQA)
on CLEVR
HMS-Net: Hierarchical Multi-scale Sparsity-invariant Network for Sparse Depth Completion
Dense depth cues are important and have wide applications in various computer vision tasks.
Ranked #10 on
Depth Completion
on KITTI Depth Completion
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
Video Person Re-Identification With Competitive Snippet-Similarity Aggregation and Co-Attentive Snippet Embedding
The attention weights are obtained based on a query feature, which is learned from the whole probe snippet by an LSTM network, making the resulting embeddings less affected by noisy frames.
Ranked #4 on
Person Re-Identification
on PRID2011
Eliminating Background-Bias for Robust Person Re-Identification
State-of-the-art methods mainly utilize deep learning based approaches for learning visual features for describing person appearances.
Group Consistent Similarity Learning via Deep CRF for Person Re-Identification
Extensive experiments demonstrate the effectiveness of our model that combines DNN and CRF for learning robust multi-scale local similarities.
A^2-Net: Molecular Structure Estimation from Cryo-EM Density Volumes
Constructing of molecular structural models from Cryo-Electron Microscopy (Cryo-EM) density volumes is the critical last step of structure determination by Cryo-EM technologies.
Preconditioning for Accelerated Iteratively Reweighted Least Squares in Structured Sparsity Reconstruction
In this paper, we propose a novel algorithm for structured sparsity reconstruction.
Cross-Scene Crowd Counting via Deep Convolutional Neural Networks
To address this problem, we propose a deep convolutional neural network (CNN) for crowd counting, and it is trained alternatively with two related learning objectives, crowd density and crowd count.
Ranked #15 on
Crowd Counting
on WorldExpo’10
Saliency Detection by Multi-Context Deep Learning
Low-level saliency cues or priors do not produce good enough saliency detection results especially when the salient object presents in a low-contrast background with confusing visual appearance.
Understanding Pedestrian Behaviors From Stationary Crowd Groups
Pedestrian behavior modeling and analysis is important for crowd scene understanding and has various applications in video surveillance.
End-To-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation
In this paper, we propose a novel end-to-end framework for human pose estimation that combines DCNNs with the expressive deformable mixture of parts.
Pedestrian Travel Time Estimation in Crowded Scenes
In this paper, we target on the problem of estimating the statistic of pedestrian travel time within a period from an entrance to a destination in a crowded scene.
Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-Identification
In our vehicle ReID framework, an orientation invariant feature embedding module and a spatial-temporal regularization module are proposed.
Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
Referring expression grounding aims at locating certain objects or persons in an image with a referring expression, where the key challenge is to comprehend and align various types of information from visual and textual domain, such as visual attributes, location and interactions with surrounding regions.
Conditional Adversarial Generative Flow for Controllable Image Synthesis
Flow-based generative models show great potential in image synthesis due to its reversible pipeline and exact log-likelihood target, yet it suffers from weak ability for conditional image synthesis, especially for multi-label or unaware conditions.
P2SGrad: Refined Gradients for Optimizing Deep Face Models
Cosine-based softmax losses significantly improve the performance of deep face recognition networks.
FocusNet: Imbalanced Large and Small Organ Segmentation with an End-to-End Deep Neural Network for Head and Neck CT Images
In this paper, we propose an end-to-end deep neural network for solving the problem of imbalanced large and small organ segmentation in head and neck (HaN) CT images.
Interpolated Convolutional Networks for 3D Point Cloud Understanding
Our InterpConv is shown to be permutation and sparsity invariant, and can directly handle irregular inputs.
Ranked #27 on
3D Part Segmentation
on ShapeNet-Part
Multi-modality Latent Interaction Network for Visual Question Answering
The proposed module learns the cross-modality relationships between latent visual and language summarizations, which summarize visual regions and question into a small number of latent representations to avoid modeling uninformative individual region-word relations.
Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints
Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.
Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimation
Monocular 3D object detection task aims to predict the 3D bounding boxes of objects based on monocular RGB images.
Learning to Predict Context-adaptive Convolution for Semantic Segmentation
Long-range contextual information is essential for achieving high-performance semantic segmentation.
StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching
Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias.
Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers
Given an input video, its associated audio, and a brief caption, the audio-visual scene aware dialog (AVSD) task requires an agent to indulge in a question-answer dialog with a human about the audio-visual content.
Complementary Boundary Generator with Scale-Invariant Relation Modeling for Temporal Action Localization: Submission to ActivityNet Challenge 2020
Specifically, in order to generate high-quality proposals, we consider several factors including the video feature encoder, the proposal generator, the proposal-proposal relations, the scale imbalance, and ensemble strategy.
Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets
In most scenarios, one might obtain annotations of a single or a few organs from one training set, and obtain annotations of the the other organs from another set of training images.
EfficientFCN: Holistically-guided Decoding for Semantic Segmentation
State-of-the-art semantic segmentation algorithms are mostly based on dilated Fully Convolutional Networks (dilatedFCN), which adopt dilated convolutions in the backbone networks to extract high-resolution feature maps for achieving high-performance segmentation performance.
Self-supervised Temporal Learning
Inspired by spatial-based contrastive SSL, we show that significant improvement can be achieved by a proposed temporal-based contrastive learning approach, which includes three novel and efficient modules: temporal augmentations, temporal memory bank and SSTL loss.
SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.
A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection
With the proposed holistically-guided decoder, we implement the EfficientFCN architecture for semantic segmentation and HGD-FPN for object detection and instance segmentation.
Towards Overcoming False Positives in Visual Relationship Detection
We observe that during training, the relationship proposal distribution is highly imbalanced: most of the negative relationship proposals are easy to identify, e. g., the inaccurate object detection, which leads to the under-fitting of low-frequency difficult proposals.
REFINE: Prediction Fusion Network for Panoptic Segmentation
Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results.
Ranked #11 on
Panoptic Segmentation
on COCO test-dev
AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks
However, the automatic design of loss functions for generic tasks with various evaluation metrics remains under-investigated.
Fixing the Teacher-Student Knowledge Discrepancy in Distillation
To solve this problem, in this paper, we propose a novel student-dependent distillation method, knowledge consistent distillation, which makes teacher's knowledge more consistent with the student and provides the best suitable knowledge to different student networks for distillation.
LIFE: Lighting Invariant Flow Estimation
However, local image contents are inevitably ambiguous and error-prone during the cross-image feature matching process, which hinders downstream tasks.
Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
In this way, our BAKE framework achieves online knowledge ensembling across multiple samples with only a single network.
Inverting Generative Adversarial Renderer for Face Reconstruction
Since the GAR learns to model the complicated real-world image, instead of relying on the simplified graphics rules, it is capable of producing realistic images, which essentially inhibits the domain-shift noise in training and optimization.
FNAS: Uncertainty-Aware Fast Neural Architecture Search
Further, an architecture knowledge pool together with a block similarity function is proposed to utilize parameter knowledge and reduces the searching time by 2 times.
Scalable Transformers for Neural Machine Translation
In this paper, we propose a novel Scalable Transformers, which naturally contains sub-Transformers of different scales and have shared parameters.
ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection
These specific designs enable the detector to be trained on meticulously refined pseudo labeled target data with denoised training signals, and thus effectively facilitate adapting an object detector to a target domain without requiring annotations.
LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation
However, two major issues of the fusion between camera and LiDAR hinder its performance, \ie, how to effectively fuse these two modalities and how to precisely align them (suffering from the weak spatiotemporal synchronization problem).
Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
With the effective spatio-temporal relationship modeling, it is possible not only to uncover contextual information in each frame but also to directly capture inter-time dependencies.
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Recently, transformer and multi-layer perceptron (MLP) architectures have achieved impressive results on various vision tasks.
Ranked #239 on
Image Classification
on ImageNet
Pyramid Fusion Transformer for Semantic Segmentation
The recently proposed MaskFormer gives a refreshed perspective on the task of semantic segmentation: it shifts from the popular pixel-level classification paradigm to a mask-level classification method.
Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning
Especially, we develop a variant of ViT for 3D point cloud feature extraction, which also achieves comparable results with existing backbones when combined with our framework, and visualization of the attention maps show that our model does understand the point cloud by combining the global shape information and multiple local structural information, which is consistent with the inspiration of our representation learning method.
Meta Knowledge Distillation
Recent studies pointed out that knowledge distillation (KD) suffers from two degradation problems, the teacher-student gap and the incompatibility with strong data augmentations, making it not applicable to training state-of-the-art models, which are trained with advanced augmentations.
Ranked #135 on
Image Classification
on ImageNet
Learning a Structured Latent Space for Unsupervised Point Cloud Completion
Specifically, we map a series of related partial point clouds into multiple complete shape and occlusion code pairs and fuse the codes to obtain their representations in the unified latent space.
Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network
As for each sub-network, we propose an efficient multi-frequency denoising network to remove noise of different frequencies.
Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object Interaction detection
However, there is still room for improvement in video HOI detection performance.
Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
Towards Robust Face Recognition with Comprehensive Search
To eliminate the bias of single-aspect research and provide an overall understanding of the face recognition model design, we first carefully design the search space for each aspect, then a comprehensive search method is introduced to jointly search optimal data cleaning, architecture, and loss function design.
NeuralMarker: A Framework for Learning General Marker Correspondence
We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker.
Magnetic Resonance Fingerprinting with compressed sensing and distance metric learning
Adaptively learning a distance metric from the undersampled training data can significantly improve the matching accuracy of the query fingerprints.
Collaboration of Pre-trained Models Makes Better Few-shot Learner
In this paper, we propose CoMo, a Collaboration of pre-trained Models that incorporates diverse prior knowledge from various pre-training paradigms for better few-shot learning.
CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
BlinkSim consists of a configurable rendering engine and a flexible engine for event data simulation.
Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehicles
We propose a perception imitation method to simulate results of a certain perception model, and discuss a new heuristic route of autonomous driving simulator without data synthesis.
Segmentation and Vascular Vectorization for Coronary Artery by Geometry-based Cascaded Neural Network
3) A dataset named CCA-200 is collected, consisting of 200 CCTA images with coronary artery disease.
ReasonNet: End-to-End Driving with Temporal and Global Reasoning
The large-scale deployment of autonomous vehicles is yet to come, and one of the major remaining challenges lies in urban dense traffic scenarios.
Ranked #1 on
Autonomous Driving
on CARLA Leaderboard
Voxel2Hemodynamics: An End-to-end Deep Learning Method for Predicting Coronary Artery Hemodynamics
In this work, we proposed an end-to-end deep learning framework, which could predict the coronary artery hemodynamics from CCTA images.
Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
In this paper, we propose to ameliorate the semantic segmentation quality of existing discriminative approaches with a mask prior modeled by a recently-developed denoising diffusion generative model.
Context-PIPs: Persistent Independent Particles Demands Spatial Context Features
We tackle the problem of Persistent Independent Particles (PIPs), also called Tracking Any Point (TAP), in videos, which specifically aims at estimating persistent long-term trajectories of query points in videos.
FlowFormer: A Transformer Architecture and Its Masked Cost Volume Autoencoding for Optical Flow
This paper introduces a novel transformer-based network architecture, FlowFormer, along with the Masked Cost Volume AutoEncoding (MCVA) for pretraining it to tackle the problem of optical flow estimation.
Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models
Video Question Answering (VideoQA) has been significantly advanced from the scaling of recent Large Language Models (LLMs).
Ranked #3 on
Temporal/Casual QA
on NExT-QA
(using extra training data)
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
CCPD transfers the fundamental, point-to-point wayfinding skill that is well trained on the large-scale PointGoal task to ORAN, so as to help ORAN to better master audio-visual navigation with far fewer training samples.
SparseMAE: Sparse Training Meets Masked Autoencoders
In this paper, we aim to reduce model complexity from large vision transformers pretrained by MAE with assistant of sparse training.
Debias the Training of Diffusion Models
Diffusion models have demonstrated compelling generation quality by optimizing the variational lower bound through a simple denoising score matching loss.
Decoupled DETR: Spatially Disentangling Localization and Classification for Improved End-to-End Object Detection
We observe that different regions of interest in the visual feature map are suitable for performing query classification and box localization tasks, even for the same object.
Towards Large-scale Masked Face Recognition
During the COVID-19 coronavirus epidemic, almost everyone is wearing masks, which poses a huge challenge for deep learning-based face recognition algorithms.
Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft
Many reinforcement learning environments (e. g., Minecraft) provide only sparse rewards that indicate task completion or failure with binary values.
Ponymation: Learning 3D Animal Motions from Unlabeled Online Videos
We introduce Ponymation, a new method for learning a generative model of articulated 3D animal motions from raw, unlabeled online videos.
The two-way knowledge interaction interface between humans and neural networks
Through experimental validation, based on this interaction interface, NN can provide humans with easily understandable explanations of the reasoning process.
NODI: Out-Of-Distribution Detection with Noise from Diffusion
Moreover, we studied the robustness of OOD methods by applying different types of image encoders.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
For the first stage, we propose a diffusion-based motion field predictor, which focuses on deducing the trajectories of the reference image's pixels.
Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
Recent advancements in Large Multimodal Models (LMMs) have shown promising results in mathematical reasoning within visual contexts, with models approaching human-level performance on existing benchmarks such as MathVista.
Ranked #1 on
Multimodal Reasoning
on MATH-V
(using extra training data)
ECNet: Effective Controllable Text-to-Image Diffusion Models
Secondly, to overcome the issue of limited conditional supervision, we introduce Diffusion Consistency Loss (DCL), which applies supervision on the denoised latent code at any given time step.
Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior
(2) To handle the unbounded nature of urban scenes, we represent 3D scene with a Scalable Hash Grid structure, incrementally adapting to the growing scale of urban scenes.
GLID: Pre-training a Generalist Encoder-Decoder Vision Model
This paper proposes a GeneraLIst encoder-Decoder (GLID) pre-training method for better handling various downstream computer vision tasks.
Flowmind2Digital: The First Comprehensive Flowmind Recognition and Conversion Approach
Automated conversion methods are essential to overcome manual conversion challenges.
DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generation
In addition to the scene generation, the final part of DiffInDScene can be used as a post-processing module to refine the 3D reconstruction results from multi-view stereo.
Enhancing Visual Grounding and Generalization: A Multi-Task Cycle Training Approach for Vision-Language Models
Furthermore, we contribute a VG dataset, especially with multi-tasks.
MagnifierNet: Towards Semantic Adversary and Fusion for Person Re-identification
Meanwhile, we introduce "Semantic Fusion Branch" to filter out irrelevant noises by selectively fusing semantic region information sequentially.
ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process
In each denoising step, our method first decodes pixels from previous VQ tokens, then generates new VQ tokens from the decoded pixels.
Signet Ring Cell Detection With a Semi-supervised Learning Framework
Our framework achieves accurate signet ring cell detection and can be readily applied in the clinical trails.
InstructSeq: Unifying Vision Tasks with Instruction-conditioned Multi-modal Sequence Generation
In this work, we introduce InstructSeq, an instruction-conditioned multi-modal modeling framework that unifies diverse vision tasks through flexible natural language control and handling of both visual and textual data.
Unsupervised Cross-spectral Stereo Matching by Learning to Synthesize
Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any supervision in the form of ground truth disparity or depth.
Adaptive Zone-Aware Hierarchical Planner for Vision-Language Navigation
In this paper, we propose an Adaptive Zone-aware Hierarchical Planner (AZHP) to explicitly divides the navigation process into two heterogeneous phases, i. e., sub-goal setting via zone partition/selection (high-level action) and sub-goal executing (low-level action), for hierarchical planning.
Semantic Scene Completion via Integrating Instances and Scene in-the-Loop
The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the overall scene.
Ranked #1 on
3D Semantic Scene Completion
on NYUv2
Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification
Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations.
Hybrid Supervision Learning for Pathology Whole Slide Image Classification
To handle this problem, we propose a hybrid supervision learning framework for this kind of high resolution images with sufficient image-level coarse annotations and a few pixel-level fine labels.
Progressive Correspondence Pruning by Consensus Learning
Correspondence selection aims to correctly select the consistent matches (inliers) from an initial set of putative correspondences.
DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks
However, the existing N:M algorithms only address the challenge of how to train N:M sparse neural networks in a uniform fashion (i. e. every layer has the same N:M sparsity) and suffer from a significant accuracy drop for high sparsity (i. e. when sparsity > 80\%).
Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification
With the contribution of the CCD and CRP, our CRCKD algorithm can distill the relational knowledge more comprehensively.
Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretation
We first compose a multi-task training dataset comprising 13. 4 million instruction and ground-truth pairs (with approximately one million radiographs) for the customized tuning, involving both image- and pixel-level tasks.
Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels
Besides, the generated pseudo-labels can be fluctuating and inaccurate at the early stage of training.
Pseudo Label
Weakly-supervised Temporal Action Localization
+1
FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR).
FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
Radiotherapy is a treatment where radiation is used to eliminate cancer cells.
Teach-DETR: Better Training DETR with Teachers
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors.
Structure-Feature based Graph Self-adaptive Pooling
To solve these problems mentioned above, we propose a novel graph self-adaptive pooling method with the following objectives: (1) to construct a reasonable pooled graph topology, structure and feature information of the graph are considered simultaneously, which provide additional veracity and objectivity in node selection; and (2) to make the pooled nodes contain sufficiently effective graph information, node feature information is aggregated before discarding the unimportant nodes; thus, the selected nodes contain information from neighbor nodes, which can enhance the use of features of the unselected nodes.
Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization
In this paper, we present a framework named FAC-Net based on the I3D backbone, on which three branches are appended, named class-wise foreground classification branch, class-agnostic attention branch and multiple instance learning branch.
NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space
Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single image, requiring no 3D inputs.
3D Semantic Scene Completion from a single 2D image
3D Semantic Scene Completion from a single RGB image
Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific Adaptation
We introduce MOTIA Mastering Video Outpainting Through Input-Specific Adaptation, a diffusion-based pipeline that leverages both the intrinsic data-specific patterns of the source video and the image/video generative prior for effective outpainting.
RBF-Softmax: Learning Deep Representative Prototypes with Radial Basis Function Softmax
To address this problem, this paper introduces a novel Radial Basis Function (RBF) distances to replace the commonly used inner products in the softmax loss function, such that it can adaptively assign losses to regularize the intra-class and inter-class distances by reshaping the relative differences, and thus creating more representative prototypes of classes to improve optimization.
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want
In this paper, we introduce the Draw-and-Understand project: a new model, a multi-domain dataset, and a challenging benchmark for visual prompting.
RBGNet: Ray-based Grouping for 3D Object Detection
In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers.
Ranked #13 on
3D Object Detection
on ScanNetV2
GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding
In this paper, we propose Geometry Enhanced Masked Image Modeling (GeoMIM) to transfer the knowledge of the LiDAR model in a pretrain-finetune paradigm for improving the multi-view camera-based 3D detection.
SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Prediction
Context information, such as road maps and surrounding agents' states, provides crucial geometric and semantic information for motion behavior prediction.
Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Existing VQA methods mainly adopt the visual attention mechanism to associate the input question with corresponding image regions for effective question answering.
Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling
The correct ego-motion estimation basically relies on the understanding of correspondences between adjacent LiDAR scans.
Visual CoT: Unleashing Chain-of-Thought Reasoning in Multi-Modal Language Models
This paper presents Visual CoT, a novel pipeline that leverages the reasoning capabilities of multi-modal large language models (MLLMs) by incorporating visual Chain-of-Thought (CoT) reasoning.
Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation
Our method seeks to mine the representative snippets in each video for propagating information between video snippets to generate better pseudo labels.
Pseudo Label
Weakly-supervised Temporal Action Localization
+1
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
Specifically, we propose, for the first time to our best knowledge, post-training approaches for task-agnostic and task-specific expert pruning and skipping of MoE LLMs, tailored to improve deployment efficiency while maintaining model performance across a wide range of tasks.
Container: Context Aggregation Network
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
Ranked #465 on
Image Classification
on ImageNet
Container: Context Aggregation Networks
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
A Simple Long-Tailed Recognition Baseline via Vision-Language Model
Recent advances in large-scale contrastive visual-language pretraining shed light on a new pathway for visual recognition.
Ranked #4 on
Long-tail Learning
on Places-LT
(using extra training data)
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
We propose a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions.
Learning Degradation Representations for Image Deblurring
With the integration, MSDI-Net can handle various and complicated blurry patterns adaptively.
Ranked #14 on
Image Deblurring
on GoPro
Structured Domain Adaptation with Online Relation Regularization for Unsupervised Person Re-ID
To tackle the challenges, we propose an end-to-end structured domain adaptation framework with an online relation-consistency regularization term.
Ranked #4 on
Unsupervised Domain Adaptation
on Market to MSMT
3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior
To this end, we first propose a novel 3D sketch-aware feature embedding to explicitly encode geometric information effectively and efficiently.
3D Semantic Scene Completion from a single RGB image
Hallucination
Decoupled Spatial-Temporal Transformer for Video Inpainting
Seamless combination of these two novel designs forms a better spatial-temporal attention scheme and our proposed model achieves better performance than state-of-the-art video inpainting approaches with significant boosted efficiency.
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning
This has led to a new research direction in parameter-efficient transfer learning.
Ranked #23 on
Action Recognition
on Something-Something V2
(using extra training data)
MoVA: Adapting Mixture of Vision Experts to Multimodal Context
Although some large-scale pretrained vision encoders such as vision encoders in CLIP and DINOv2 have brought promising performance, we found that there is still no single vision encoder that can dominate various image content understanding, e. g., the CLIP vision encoder leads to outstanding results on general image understanding but poor performance on document or chart content.
Object Detection in Videos with Tubelet Proposal Networks
Object detection in videos has drawn increasing attention recently with the introduction of the large-scale ImageNet VID dataset.
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network
Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse.
Balanced Meta-Softmax for Long-Tailed Visual Recognition
In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.
Ranked #7 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
Urban Radiance Field Representation with Deformable Neural Mesh Primitives
To construct urban-level radiance fields efficiently, we design Deformable Neural Mesh Primitive~(DNMP), and propose to parameterize the entire scene with such primitives.
VS-Net: Voting with Segmentation for Visual Localization
To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks.
A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift
In this study, we propose a simple yet effective framework for video restoration.
Ranked #1 on
Deblurring
on GoPro
(using extra training data)
LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector
Compared with the state-of-the-art stereo detector, our method has improved the 3D detection performance of cars, pedestrians, cyclists by 10. 44%, 5. 69%, 5. 97% mAP respectively on the official KITTI benchmark.
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Finally, we integrate configurable operators and DSMs into a unified search space and search with a Reinforcement Learning-based search algorithm to fully explore the optimal combination of the operators.
Ranked #12 on
Neural Architecture Search
on ImageNet
TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers
In this paper, we propose a novel data augmentation technique TokenMix to improve the performance of vision transformers.
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers
In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention-based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency.
Learning Monocular Depth by Distilling Cross-domain Stereo Networks
Monocular depth estimation aims at estimating a pixelwise depth map for a single image, which has wide applications in scene understanding and autonomous driving.
FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis
In this study, we delve into the generation of high-resolution images from pre-trained diffusion models, addressing persistent challenges, such as repetitive patterns and structural distortions, that emerge when models are applied beyond their trained resolutions.
FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
FlowFormer introduces a transformer architecture into optical flow estimation and achieves state-of-the-art performance.
ConQueR: Query Contrast Voxel-DETR for 3D Object Detection
We thus propose a Query Contrast mechanism to explicitly enhance queries towards their best-matched GTs over all unmatched query predictions.
TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses
3D multi-object tracking (MOT) is vital for many applications including autonomous driving vehicles and service robots.
Simulating Fluids in Real-World Still Images
In this work, we tackle the problem of real-world fluid animation from a still image.
End-to-End Deep Kronecker-Product Matching for Person Re-identification
Person re-identification aims to robustly measure similarities between person images.
Deep Group-shuffling Random Walk for Person Re-identification
Person re-identification aims at finding a person of interest in an image gallery by comparing the probe image of this person with all the gallery images.
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
On the contrary, the soft composition operates by stitching different patches into a whole feature map where pixels in overlapping regions are summed up.
Ranked #3 on
Video Inpainting
on DAVIS
Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification
We found that its success can be largely attributed to its powerful skills in generating and executing code, evaluating the output of code execution, and rectifying its solution when receiving unreasonable outputs.
Ranked #1 on
Math Word Problem Solving
on MATH
Rethinking Noise Synthesis and Modeling in Raw Denoising
However, the real raw image noise is contributed by many noise sources and varies greatly among different sensors.
Ranked #2 on
Image Denoising
on SID SonyA7S2 x100
IDR: Self-Supervised Image Denoising via Iterative Data Refinement
To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
MixMAE: Mixed and Masked Autoencoder for Efficient Pretraining of Hierarchical Vision Transformers
In this paper, we propose Mixed and Masked AutoEncoder (MixMAE), a simple but efficient pretraining method that is applicable to various hierarchical Vision Transformers.
Ranked #2 on
Image Classification
on Places205
CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval
Text-image cross-modal retrieval is a challenging task in the field of language and vision.
Ranked #9 on
Image Retrieval
on Flickr30K 1K test
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
Semantic image synthesis aims at generating photorealistic images from semantic layouts.
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching
We further attribute this phenomenon to the diffusion model's insufficient condition utilization, which is caused by its training paradigm.
RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization
The correspondence field estimation and pose refinement are conducted alternatively in each iteration to recover the object poses.
Ranked #1 on
6D Pose Estimation using RGB
on LineMOD
Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
Analysis of the learned SRN model demonstrates that it can effectively capture both semantic and spatial relations of labels for improving classification performance.
Ranked #6 on
Multi-Label Classification
on NUS-WIDE
Person Search with Natural Language Description
Searching persons in large-scale image databases with the query of natural language description has important applications in video surveillance.
Generalizing Monocular 3D Human Pose Estimation in the Wild
We observe that recent innovation in this area mainly focuses on new techniques that explicitly address the generalization issue when using this dataset, because this database is constructed in a highly controlled environment with limited human subjects and background variations.
Ranked #64 on
3D Human Pose Estimation
on Human3.6M
CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching
To overcome these obstacles, we propose CORA, a DETR-style framework that adapts CLIP for Open-vocabulary detection by Region prompting and Anchor pre-matching.
Ranked #6 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
Frozen CLIP Models are Efficient Video Learners
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos.
Ranked #26 on
Action Classification
on Kinetics-400
(using extra training data)
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer
Developing generative models for interleaved image-text data has both research and practical value.
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning
In this paper, we present a method to fine-tune open-source language models, enabling them to use code for modeling and deriving math equations and, consequently, enhancing their mathematical reasoning abilities.
Ranked #4 on
Math Word Problem Solving
on SVAMP
(using extra training data)
End-to-End Object Detection with Adaptive Clustering Transformer
In this paper, a novel variant of transformer named Adaptive Clustering Transformer(ACT) has been proposed to reduce the computation cost for high-resolution input.
Fast Convergence of DETR with Spatially Modulated Co-Attention
The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN.
Fast Convergence of DETR with Spatially Modulated Co-Attention
However, DETR suffers from its slow convergence.
Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction
The HoP approach is straightforward: given the current timestamp t, we generate a pseudo Bird's-Eye View (BEV) feature of timestamp t-k from its adjacent frames and utilize this feature to predict the object set at timestamp t-k. Our approach is motivated by the observation that enforcing the detector to capture both the spatial location and temporal motion of objects occurring at historical timestamps can lead to more accurate BEV feature learning.
Ranked #3 on
3D Object Detection
on nuScenes Camera Only
Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis
Specifically, we compress the light fields for novel view human rendering as conditional implicit neural radiance fields from both geometry and appearance aspects.
KBNet: Kernel Basis Network for Image Restoration
In this paper, we propose a kernel basis attention (KBA) module, which introduces learnable kernel bases to model representative image patterns for spatial information aggregation.
Ranked #1 on
Color Image Denoising
on McMaster sigma50
Crafting GBD-Net for Object Detection
The effectiveness of GBD-Net is shown through experiments on three object detection datasets, ImageNet, Pascal VOC2007 and Microsoft COCO.
Object Detection from Video Tubelets with Convolutional Neural Networks
Deep Convolution Neural Networks (CNNs) have shown impressive performance in various vision tasks such as image classification, object detection and semantic segmentation.
1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020
This technical report introduces our winning solution to the spatio-temporal action localization track, AVA-Kinetics Crossover, in ActivityNet Challenge 2020.
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network, which can maintain the advantages of both unstructured fine-grained sparsity and structured coarse-grained sparsity simultaneously on specifically designed GPUs.
Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training
By fine-tuning on downstream tasks, Point-M2AE achieves 86. 43% accuracy on ScanObjectNN, +3. 36% to the second-best, and largely benefits the few-shot classification, part segmentation and 3D object detection with the hierarchical pre-training scheme.
Ranked #4 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders
Pre-training by numerous image data has become de-facto for robust 2D representations.
Ranked #2 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation
To this end, we propose Multi-level Attention Encoder-Decoder Network (MAED), including a Spatial-Temporal Encoder (STE) and a Kinematic Topology Decoder (KTD) to model multi-level attentions in a unified framework.
Ranked #40 on
3D Human Pose Estimation
on MPI-INF-3DHP
AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations
Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy.
Ranked #6 on
Face Verification
on MegaFace
CameraCtrl: Enabling Camera Control for Text-to-Video Generation
Controllability plays a crucial role in video generation since it allows users to create desired content.
GiT: Towards Generalist Vision Transformer through Universal Language Interface
Due to its simple design, this paradigm holds promise for narrowing the architectural gap between vision and language.
VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation
We first propose a TRi-frame Optical Flow (TROF) module that estimates bi-directional optical flows for the center frame in a three-frame manner.
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
Our evaluations demonstrate that VRWKV surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs.
Learning Feature Pyramids for Human Pose Estimation
We investigate our method on two standard benchmarks for human pose estimation.
Ranked #6 on
Pose Estimation
on Leeds Sports Poses
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
2) Dynamic Shifting for complex point distributions.
Ranked #2 on
Panoptic Segmentation
on SemanticKITTI
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
