Search Results for author:
Found 86 papers, 41 papers with code
Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge
In this report, we introduce our real-time 2D object detection system for the realistic autonomous driving scenario.
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models
We present SPHINX, a versatile multi-modal large language model (MLLM) with a joint mixing of model weights, tuning tasks, and visual embeddings.
Ranked #2 on
Visual Question Answering
on BenchLMM
DER: Dynamically Expandable Representation for Class Incremental Learning
We address the problem of class incremental learning, which is a core step towards achieving adaptive vision intelligence.
Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images
Semi-supervised learning has attracted much attention in medical image segmentation due to challenges in acquiring pixel-wise image annotations, which is a crucial step for building high-performance deep learning methods.
GNeRF: GAN-based Neural Radiance Field without Posed Camera
We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field (NeRF) reconstruction for the complex scenarios with unknown and even randomly initialized camera poses.
Part-aware Prototype Network for Few-shot Semantic Segmentation
In this paper, we propose a novel few-shot semantic segmentation framework based on the prototype representation.
Ranked #3 on
Few-Shot Semantic Segmentation
on Pascal5i
Distribution Alignment: A Unified Framework for Long-tail Visual Recognition
Motivated by our discovery, we propose a unified distribution alignment strategy for long-tail visual recognition.
Ranked #17 on
Long-tail Learning
on Places-LT
Pose-aware Multi-level Feature Network for Human Object Interaction Detection
Reasoning human object interactions is a core problem in human-centric scene understanding and detecting such relations poses a unique challenge to vision systems due to large variations in human-object configurations, multiple co-occurring relation instances and subtle visual difference between relation categories.
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation
Self-supervised vision-and-language pretraining (VLP) aims to learn transferable multi-modal representations from large-scale image-text data and to achieve strong performances on a broad scope of vision-language tasks after finetuning.
LatentGNN: Learning Efficient Non-local Relations for Visual Recognition
A promising strategy is to model the feature context by a fully-connected graph neural network (GNN), which augments traditional convolutional features with an estimated non-local context representation.
Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation
Scene graph generation is an important visual understanding task with a broad range of vision applications.
CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention
Contrastive Language-Image Pre-training (CLIP) has been shown to learn visual representations with great transferability, which achieves promising accuracy for zero-shot classification.
Ranked #4 on
Training-free 3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Training-free 3D Point Cloud Classification
Transfer Learning
+1
SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text
We develop a model that learns to generate visually relevant styled captions from a large corpus of styled text without aligned images.
SGTR: End-to-end Scene Graph Generation with Transformer
Scene Graph Generation (SGG) remains a challenging visual understanding task due to its compositional property.
SGTR+: End-to-end Scene Graph Generation with Transformer
Moreover, we design a graph assembling module to infer the connectivity of the bipartite scene graph based on our entity-aware structure, enabling us to generate the scene graph in an end-to-end manner.
Learning Cross-modal Context Graph for Visual Grounding
To address their limitations, this paper proposes a language-guided graph representation to capture the global context of grounding entities and their relations, and develop a cross-modal graph matching strategy for the multiple-phrase visual grounding task.
HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Models
In detail, we first introduce a novel interaction decoder to extract informative regions in the visual feature map of CLIP via a cross-attention mechanism, which is then fused with the detection backbone by a knowledge integration block for more accurate human-object pair detection.
Ranked #8 on
Human-Object Interaction Detection
on V-COCO
Human-Object Interaction Detection
Knowledge Distillation
+2
Superpixel-guided Iterative Learning from Noisy Labels for Medical Image Segmentation
Learning segmentation from noisy labels is an important task for medical image analysis due to the difficulty in acquiring highquality annotations.
Dynamic Grained Encoder for Vision Transformers
Specifically, we propose a Dynamic Grained Encoder for vision transformers, which can adaptively assign a suitable number of queries to each spatial region.
Human-centric Scene Understanding for 3D Large-scale Scenarios
Human-centric scene understanding is significant for real-world applications, but it is extremely challenging due to the existence of diverse human poses and actions, complex human-environment interactions, severe occlusions in crowds, etc.
Dynamic Context Correspondence Network for Semantic Alignment
We instantiate our strategy by designing an end-to-end learnable deep network, named as Dynamic Context Correspondence Network (DCCNet).
Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts
In this work, we focus on capturing the data-inherent uncertainty (aka aleatoric uncertainty) in segmentation, typically when ambiguities exist in input images.
Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition
The task of skeleton-based action recognition remains a core challenge in human-centred scene understanding due to the multiple granularities and large variation in human motion.
Class-relation Knowledge Distillation for Novel Class Discovery
In addition, to enable a flexible knowledge distillation scheme for each data point in novel classes, we develop a learnable weighting function for the regularization, which adaptively promotes knowledge transfer based on the semantic similarity between the novel and known classes.
Learning Semantic Correspondence with Sparse Annotations
In this paper, we aim to address the challenge of label sparsity in semantic correspondence by enriching supervision signals from sparse keypoint annotations.
Learning Implicit Temporal Alignment for Few-shot Video Classification
Few-shot video classification aims to learn new video categories with only a few labeled examples, alleviating the burden of costly annotation in real-world applications.
ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learning
Real-Time Bidding (RTB) is an important mechanism in modern online advertising systems.
Weakly Supervised Nuclei Segmentation via Instance Learning
Weakly supervised nuclei segmentation is a critical problem for pathological image analysis and greatly benefits the community due to the significant reduction of labeling cost.
Novel Class Discovery for Long-tailed Recognition
In this paper, we consider a more realistic setting for novel class discovery where the distributions of novel and known classes are long-tailed.
P$^2$OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering
Deep clustering, which learns representation and semantic clustering without labels information, poses a great challenge for deep learning-based approaches.
Weakly Supervised Volumetric Segmentation via Self-taught Shape Denoising Model
Moreover, we introduce a weak annotation scheme with a hybrid label design for volumetric images, which improves model learning without increasing the overall annotation cost.
Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training
Firstly, we observe that the CLIP's visual feature of image subregions can achieve closer proximity to the paired caption due to the inherent information loss in text descriptions.
An EM Framework for Online Incremental Learning of Semantic Segmentation
Incremental learning of semantic segmentation has emerged as a promising strategy for visual scene interpretation in the open- world setting.
Single Image 3D Object Estimation with Primitive Graph Networks
To address those challenges, we adopt a primitive-based representation for 3D object, and propose a two-stage graph network for primitive-based 3D object estimation, which consists of a sequential proposal module and a graph reasoning module.
SP$^2$OT: Semantic-Regularized Progressive Partial Optimal Transport for Imbalanced Clustering
To be more precise, we employ the strategy of majorization to reformulate the SP$^2$OT problem into a Progressive Partial Optimal Transport problem, which can be transformed into an unbalanced optimal transport problem with augmented constraints and can be solved efficiently by a fast matrix scaling algorithm.
Predicting Salient Face in Multiple-Face Videos
On the other hand, we find that the attention of different subjects consistently focuses on a single face in each frame of videos involving multiple faces.
MILD: Modeling the Instance Learning Dynamics for Learning with Noisy Labels
Despite deep learning has achieved great success, it often relies on a large amount of training data with accurate labels, which are expensive and time-consuming to collect.
Gradient-Map-Guided Adaptive Domain Generalization for Cross Modality MRI Segmentation
Cross-modal MRI segmentation is of great value for computer-aided medical diagnosis, enabling flexible data acquisition and model generalization.
Simplifying Sentences with Sequence to Sequence Models
We simplify sentences with an attentive neural network sequence to sequence model, dubbed S4.
Geometry-aware Deep Network for Single-Image Novel View Synthesis
By contrast, in this paper, we propose to exploit the 3D geometry of the scene to synthesize a novel view.
Learning deep structured network for weakly supervised change detection
We apply a constrained mean-field algorithm to estimate the pixel-level labels, and use the estimated labels to update the parameters of the CNN in an iterative EM framework.
Boundary-aware Instance Segmentation
In this context, existing methods typically propose candidate objects, usually as bounding boxes, and directly predict a binary mask within each such proposal.
Learning Dynamic Hierarchical Models for Anytime Scene Labeling
With increasing demand for efficient image and video analysis, test-time cost of scene parsing becomes critical for many large-scale or time-sensitive vision applications.
Semantic-Aware Depth Super-Resolution in Outdoor Scenes
Despite much progress, state-of-the-art techniques suffer from two drawbacks: (i) they rely on the assumption that intensity edges coincide with depth discontinuities, which, unfortunately, is only true in controlled environments; and (ii) they typically exploit the availability of high-resolution training depth maps, which can often not be acquired in practice due to the sensors' limitations.
SentiCap: Generating Image Descriptions with Sentiments
We design a system to describe an image with emotions, and present a model that automatically generates captions with positive or negative sentiments.
Structured Depth Prediction in Challenging Monocular Video Sequences
To this end, we first study the problem of depth estimation from a single image.
A unified model of short-range and long-range motion perception
The human vision system is able to effortlessly perceive both short-range and long-range motion patterns in complex dynamic scenes.
Learning Hybrid Models for Image Annotation with Partially Labeled Data
Extensive labeled data for image annotation systems, which learn to assign class labels to image regions, is difficult to obtain.
One-Shot Action Localization by Learning Sequence Matching Network
Learning based temporal action localization methods require vast amounts of training data.
Winding Number for Region-Boundary Consistent Salient Contour Extraction
The main focus is given to how to maintain the consistency (compatibility) between the region cues and the boundary cues.
Learning Structured Hough Voting for Joint Object Detection and Occlusion Reasoning
We propose a structured Hough voting method for detecting objects with heavy occlusion in indoor environments.
An Exemplar-based CRF for Multi-instance Object Segmentation
We address the problem of joint detection and segmentation of multiple object instances in an image, a key step towards scene understanding.
Discrete-Continuous Depth Estimation from a Single Image
The unary potentials in this graphical model are computed by making use of the images with known depth.
Indoor Scene Structure Analysis for Single Image Depth Estimation
We tackle the problem of single image depth estimation, which, without additional knowledge, suffers from many ambiguities.
Multiclass Semantic Video Segmentation With Object-Level Active Inference
To scale up our method, we adopt an active inference strategy to improve the efficiency, which adaptively selects object subgraphs in the object-augmented dense CRF.
Separating Objects and Clutter in Indoor Scenes
Objects' spatial layout estimation and clutter identification are two important tasks to understand indoor scenes.
Learning to Co-Generate Object Proposals With a Deep Structured Network
In particular, we introduce a deep structured network that jointly predicts the objectness scores and the bounding box locations of multiple object candidates.
Indoor Scene Parsing With Instance Segmentation, Semantic Labeling and Support Relationship Inference
In particular, while some of them aim at segmenting the image into regions, such as object or surface instances, others aim at inferring the semantic labels of given regions, or their support relationships.
Structural Kernel Learning for Large Scale Multiclass Object Co-Detection
To exploit the correlations between objects, we build a fully-connected CRF on the candidates, which explicitly incorporates both geometric layout relations across object classes and similarity relations across multiple images.
Deep Free-Form Deformation Network for Object-Mask Registration
In this work, we take a transformation based approach that predicts a 2D non-rigid spatial transform and warps the shape mask onto the target object.
Fixed-price Diffusion Mechanism Design
We consider a fixed-price mechanism design setting where a seller sells one item via a social network, but the seller can only directly communicate with her neighbours initially.
Learning a Layout Transfer Network for Context Aware Object Detection
We present a context aware object detection method based on a retrieve-and-transform scene layout model.
Learning Context-aware Task Reasoning for Efficient Meta-reinforcement Learning
Despite recent success of deep network-based Reinforcement Learning (RL), it remains elusive to achieve human-level efficiency in learning novel tasks.
Towards Purely Unsupervised Disentanglement of Appearance and Shape for Person Images Generation
To be clear, in this paper, we refer unsupervised learning as learning without task-specific human annotations, pairs or any form of weak supervision.)
LGNN: A Context-aware Line Segment Detector
We present a novel real-time line segment detection scheme called Line Graph Neural Network (LGNN).
Confidence-aware Adversarial Learning for Self-supervised Semantic Matching
We are the first that exploit confidence during refinement to improve semantic matching accuracy and develop an end-to-end self-supervised adversarial learning procedure for the entire matching network.
Smoothed Quantile Regression with Large-Scale Inference
Our numerical studies confirm the conquer estimator as a practical and reliable approach to large-scale inference for quantile regression.
Statistics Theory Methodology Statistics Theory
SGTR: Generating Scene Graph by Learning Compositional Triplets with Transformer
We develop a decoding-and-assembling paradigm for the end-to-end scene graph generation.
Budget-aware Few-shot Learning via Graph Convolutional Network
In this work, we introduce a new budget-aware few-shot learning problem that not only aims to learn novel object categories, but also needs to select informative examples to annotate in order to achieve data efficiency.
Cascaded Sparse Feature Propagation Network for Interactive Segmentation
We aim to tackle the problem of point-based interactive segmentation, in which the key challenge is to propagate the user-provided annotations to unlabeled regions efficiently.
Ranked #4 on
Interactive Segmentation
on SBD
General Incremental Learning with Domain-aware Categorical Representations
Continual learning is an important problem for achieving human-level intelligence in real-world applications as an agent must continuously accumulate knowledge in response to streaming data/tasks.
Automatic spinal curvature measurement on ultrasound spine images using Faster R-CNN
The framework consisted of two closely linked modules: 1) the lamina detector for identifying and locating each lamina pairs on ultrasound coronal images, and 2) the spinal curvature estimator for calculating the scoliotic angles based on the chain of detected lamina.
Mutual Information-guided Knowledge Transfer for Novel Class Discovery
Our insight is to utilize mutual information to measure the relation between seen classes and unseen classes in a restricted label space and maximizing mutual information promotes transferring semantic knowledge.
A Novel Unified Conditional Score-based Generative Framework for Multi-modal Medical Image Completion
Multi-modal medical image completion has been extensively applied to alleviate the missing modality issue in a wealth of multi-modal diagnostic tasks.
Part-aware Prototypical Graph Network for One-shot Skeleton-based Action Recognition
In this paper, we study the problem of one-shot skeleton-based action recognition, which poses unique challenges in learning transferable representation from base classes to novel classes, particularly for fine-grained actions.
Generative Negative Text Replay for Continual Vision-Language Pretraining
In this work, we focus on learning a VLP model with sequential chunks of image-text pair data.
Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning
Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building-block for many vision tasks.
Human-Object Interaction Detection
Knowledge Distillation
+3
A Physics-Informed Data-Driven Fault Location Method for Transmission Lines Using Single-Ended Measurements with Field Data Validation
This procedure merges the gap between simulation and practical power systems, and at the same time considers the uncertainty of system and fault parameters in practice.
Grounded Image Text Matching with Mismatched Relation Reasoning
This paper introduces Grounded Image Text Matching with Mismatched Relation (GITM-MR), a novel visual-linguistic joint task that evaluates the relation understanding capabilities of transformer-based pre-trained models.
The Robust Semantic Segmentation UNCV2023 Challenge Results
This paper outlines the winning solutions employed in addressing the MUAD uncertainty quantification challenge held at ICCV 2023.
GenEM: Physics-Informed Generative Cryo-Electron Microscopy
To address this, we introduce physics-informed generative cryo-electron microscopy (GenEM), which for the first time integrates physical-based cryo-EM simulation with a generative unpaired noise translation to generate physically correct synthetic cryo-EM datasets with realistic noises.
RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data.
DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generation
Scene graph generation aims to capture detailed spatial and semantic relationships between objects in an image, which is challenging due to incomplete labelling, long-tailed relationship categories, and relational semantic overlap.
Learning by Correction: Efficient Tuning Task for Zero-Shot Generative Vision-Language Reasoning
Generative vision-language models (VLMs) have shown impressive performance in zero-shot vision-language tasks like image captioning and visual question answering.
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
Scene graph generation (SGG) aims to parse a visual scene into an intermediate graph representation for downstream reasoning tasks.
