Search Results for author:
Found 82 papers, 55 papers with code
LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning
Current high-performance semantic segmentation models are purely data-driven sub-symbolic approaches and blind to the structured nature of the visual world.
ClusterFormer: Clustering As A Universal Visual Learner
This paper presents CLUSTERFORMER, a universal vision model that is based on the CLUSTERing paradigm with TransFORMER.
Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation
Recent advances in semi-supervised semantic segmentation have been heavily reliant on pseudo labeling to compensate for limited labeled data, disregarding the valuable relational knowledge among semantic concepts.
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
CCPD transfers the fundamental, point-to-point wayfinding skill that is well trained on the large-scale PointGoal task to ORAN, so as to help ORAN to better master audio-visual navigation with far fewer training samples.
DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation
VLN-CE is a recently released embodied task, where AI agents need to navigate a freely traversable environment to reach a distant target location, given language instructions.
Bird's-Eye-View Scene Graph for Vision-Language Navigation
Vision-language navigation (VLN), which entails an agent to navigate 3D environments following human instructions, has shown great advances.
Clustering based Point Cloud Representation Learning for 3D Analysis
The mined patterns are, in turn, used to repaint the embedding space, so as to respect the underlying distribution of the entire training dataset and improve the robustness to the variations.
Kefa: A Knowledge Enhanced and Fine-grained Aligned Speaker for Navigation Instruction Generation
We introduce a novel speaker model \textsc{Kefa} for navigation instruction generation.
E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning
Specifically, we introduce a set of learnable key-value prompts and visual prompts into self-attention and input layers, respectively, to improve the effectiveness of model fine-tuning.
Large-Scale Person Detection and Localization using Overhead Fisheye Cameras
Instead of existing efforts devoted to localizing tourist photos captured by perspective cameras, in this article, we focus on devising person positioning solutions using overhead fisheye cameras.
Segment and Track Anything
This report presents a framework called Segment And Track Anything (SAMTrack) that allows users to precisely and effectively segment and track any object in a video.
CLUSTSEG: Clustering for Universal Segmentation
We present CLUSTSEG, a general, transformer-based framework that tackles different image segmentation tasks (i. e., superpixel, semantic, instance, and panoptic) through a unified neural clustering scheme.
Boosting Video Object Segmentation via Space-time Correspondence Learning
Current top-leading solutions for video object segmentation (VOS) typically follow a matching-based regime: for each query frame, the segmentation mask is inferred according to its correspondence to previously processed and the first annotated frames.
ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
To develop a robust VLN-CE agent, we propose a new navigation framework, ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments.
Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentation
The objective of this paper is self-supervised learning of video object segmentation.
Lana: A Language-Capable Navigator for Instruction Following and Generation
Recently, visual-language navigation (VLN) -- entailing robot agents to follow navigation instructions -- has shown great advance.
Towards Versatile Embodied Navigation
With the emergence of varied visual navigation tasks (e. g, image-/object-/audio-goal and vision-language navigation) that specify the target in different ways, the community has made appealing advances in training specialized agents capable of handling individual navigation tasks well.
Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing
Neural-symbolic computing (NeSy), which pursues the integration of the symbolic and statistical paradigms of cognition, has been an active research area of Artificial Intelligence (AI) for many years.
GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models
Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature, class).
Learning Equivariant Segmentation with Instance-Unique Querying
Prevalent state-of-the-art instance segmentation methods fall into a query-based scheme, in which instance masks are derived by querying the image feature using a set of instance-aware embeddings.
Visual Recognition with Deep Nearest Centroids
We devise deep nearest centroids (DNC), a conceptually elegant yet surprisingly effective network for large-scale visual recognition, by revisiting Nearest Centroids, one of the most classic and simple classifiers.
Semi-supervised 3D Object Detection with Proficient Teachers
To reduce the dependence on large supervision, semi-supervised learning (SSL) based approaches have been proposed.
ProposalContrast: Unsupervised Pre-training for LiDAR-based 3D Object Detection
Existing approaches for unsupervised point cloud pre-training are constrained to either scene-level or point/voxel-level instance discrimination.
Reference-based Image Super-Resolution with Deformable Attention Transformer
Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images.
Towards Interpretable Video Super-Resolution via Alternating Optimization
These issues can be alleviated by a cascade of three separate sub-tasks, including video deblurring, frame interpolation, and super-resolution, which, however, would fail to capture the spatial and temporal correlations among video sequences.
Target-Driven Structured Transformer Planner for Vision-Language Navigation
Vision-language navigation is the task of directing an embodied agent to navigate in 3D scenes with natural language instructions.
Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation
Since the rise of vision-language navigation (VLN), great progress has been made in instruction following -- building a follower to navigate environments under the guidance of instructions.
Rethinking Semantic Segmentation: A Prototype View
Prevalent semantic segmentation solutions, despite their different network designs (FCN based or attention based) and mask decoding strategies (parametric softmax based or pixel-query based), can be placed in one category, by considering the softmax weights or query vectors as learnable class prototypes.
Deep Hierarchical Semantic Segmentation
In this paper, we instead address hierarchical semantic segmentation (HSS), which aims at structured, pixel-wise description of visual observation in terms of a class hierarchy.
Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learning
Our target is to learn visual correspondence from unlabeled videos.
Visual Abductive Reasoning
In this paper, we propose a new task and dataset, Visual Abductive Reasoning (VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations.
Scalable Video Object Segmentation with Identification Mechanism
This paper delves into the challenges of achieving scalable and effective multi-object modeling for semi-supervised Video Object Segmentation (VOS).
Semantic Segmentation
Semi-Supervised Video Object Segmentation
+1
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
In light of this, we present Locater (local-global context aware Transformer), which augments the Transformer architecture with a finite memory so as to query the entire video with the language expression in an efficient manner.
Ranked #7 on
Referring Expression Segmentation
on A2D Sentences
Referring Expression Segmentation
Referring Video Object Segmentation
+4
Locality-Aware Inter- and Intra-Video Reconstruction for Self-Supervised Correspondence Learning
Our target is to learn visual correspondence from unlabeled videos.
Collaborative Visual Navigation
As a fundamental problem for Artificial Intelligence, multi-agent system (MAS) is making rapid progress, mainly driven by multi-agent reinforcement learning (MARL) techniques.
A Survey on Deep Learning Technique for Video Segmentation
Video segmentation -- partitioning video frames into multiple segments or objects -- plays a critical role in a broad range of practical applications, from enhancing visual effects in movie, to understanding scenes in autonomous driving, to creating virtual background in video conferencing.
Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference.
One-shot visual object segmentation
Referring Video Object Segmentation
+2
Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentation
Though 3D convolutions are amenable to recognizing which actor is performing the queried actions, it also inevitably introduces misaligned spatial information from adjacent frames, which confuses features of the target frame and yields inaccurate segmentation.
Ranked #8 on
Referring Expression Segmentation
on J-HMDB
Face Forensics in the Wild
On existing public benchmarks, face forgery detection techniques have achieved great success.
Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing
To address the challenging task of instance-aware human part parsing, a new bottom-up regime is proposed to learn category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner.
Structured Scene Memory for Vision-Language Navigation
Recently, numerous algorithms have been developed to tackle the problem of vision-language navigation (VLN), i. e., entailing an agent to navigate 3D environments through following linguistic instructions.
Exploring Cross-Image Pixel Contrast for Semantic Segmentation
Inspired by the recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive framework for semantic segmentation in the fully supervised setting.
Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis
3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world.
LID 2020: The Learning from Imperfect Data Challenge Results
The purpose of the Learning from Imperfect Data (LID) workshop is to inspire and facilitate the research in developing novel approaches that would harness the imperfect data and improve the data-efficiency during training.
Weakly Supervised 3D Object Detection from Lidar Point Cloud
This work proposes a weakly supervised approach for 3D object detection, only requiring a small set of weakly annotated scenes, associated with a few precisely labeled object instances.
Active Visual Information Gathering for Vision-Language Navigation
Vision-language navigation (VLN) is the task of entailing an agent to carry out navigational instructions inside photo-realistic environments.
Video Object Segmentation with Episodic Graph Memory Networks
How to make a segmentation model efficiently adapt to a specific video and to online target appearance variations are fundamentally crucial issues in the field of video object segmentation.
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation
Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge.
Object Localization
Weakly supervised Semantic Segmentation
+1
A Unified Object Motion and Affinity Model for Online Multi-Object Tracking
In this paper, we propose a novel MOT framework that unifies object motion and affinity model into a single network, named UMA, in order to learn a compact feature that is discriminative for both object motion and affinity measure.
Learning Video Object Segmentation from Unlabeled Videos
We propose a new method for video object segmentation (VOS) that addresses object pattern learning from unlabeled videos, unlike most existing methods which rely heavily on extensive annotated data.
Hierarchical Human Parsing with Typed Part-Relation Reasoning
As human bodies are underlying hierarchically structured, how to model human structures is the central theme in this task.
Cascaded Human-Object Interaction Recognition
The interaction recognition network has two crucial parts: a relation ranking module for high-quality HOI proposal selection and a triple-stream classifier for relation prediction.
Human-Aware Motion Deblurring
This paper proposes a human-aware deblurring model that disentangles the motion blur between foreground (FG) humans and background (BG).
Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks
Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation.
Semantic Segmentation
Unsupervised Video Object Segmentation
+3
Learning Compositional Neural Information Fusion for Human Parsing
The bottom-up and top-down inferences explicitly model the compositional and decompositional relations in human bodies, respectively.
See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks
We introduce a novel network, called CO-attention Siamese Network (COSNet), to address the unsupervised video object segmentation task from a holistic view.
Semantic Segmentation
Unsupervised Video Object Segmentation
+2
Improving Neural Machine Translation by Achieving Knowledge Transfer with Sentence Alignment Learning
To alleviate this problem, we propose an NMT approach that heightens the adequacy in machine translation by transferring the semantic knowledge learned from bilingual sentence alignment.
Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning
This paper addresses a new problem of understanding human gaze communication in social videos from both atomic-level and event-level, which is significant for studying human social interactions.
Salient Object Detection With Pyramid Attention and Salient Edges
The first is the exploitation of an essential pyramid attention structure for salient object detection.
An Iterative and Cooperative Top-Down and Bottom-Up Inference Network for Salient Object Detection
The top-down process is used for coarse-to-fine saliency estimation, where high-level saliency is gradually integrated with finer lower-layer features to obtain a fine-grained result.
Learning Pose Grammar for Monocular 3D Pose Estimation
In this paper, we propose a pose grammar to tackle the problem of 3D human pose estimation from a monocular RGB image.
Ranked #13 on
3D Human Pose Estimation
on HumanEva-I
Learning Unsupervised Video Object Segmentation Through Visual Attention
This paper conducts a systematic study on the role of visual attention in Unsupervised Video Object Segmentation (UVOS) tasks.
Semantic Segmentation
Unsupervised Video Object Segmentation
+2
Salient Object Detection in the Deep Learning Era: An In-Depth Survey
As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research attention over the years.
Reasoning Visual Dialogs with Structural and Partial Observations
The answer to a given question is represented by a node with missing value.
Ranked #14 on
Visual Dialog
on VisDial v0.9 val
Pyramid Dilated Deeper ConvLSTM for Video Salient Object Detection
This paper proposes a fast video salient object detection model, based on a novel recurrent network architecture, named Pyramid Dilated Bidirectional ConvLSTM (PDB-ConvLSTM).
Ranked #1 on
Video Salient Object Detection
on UVSD
(using extra training data)
Learning Human-Object Interactions by Graph Parsing Neural Networks
For a given scene, GPNN infers a parse graph that includes i) the HOI graph structure represented by an adjacency matrix, and ii) the node labels.
Ranked #31 on
Human-Object Interaction Detection
on V-COCO
Hyperparameter Optimization for Tracking With Continuous Deep Q-Learning
Hyperparameters are numerical presets whose values are assigned prior to the commencement of the learning process.
Attentive Fashion Grammar Network for Fashion Landmark Detection and Clothing Category Classification
This paper proposes a knowledge-guided fashion network to solve the problem of visual fashion analysis, e. g., fashion landmark localization and clothing category classification.
Salient Object Detection Driven by Fixation Prediction
Salient object detection is then viewed as fine-grained object-level saliency segmentation and is progressively optimized with the guidance of the fixation map in a top-down manner.
Inferring Shared Attention in Social Scene Videos
We collect a new dataset VideoCoAtt from public TV show videos, containing 380 complex video sequences with more than 492, 000 frames that include diverse social scenes for shared attention study.
Optimizing the F-measure for Threshold-free Salient Object Detection
By reformulating the standard F-measure we propose the relaxed F-measure which is differentiable w. r. t the posterior and can be easily appended to the back of CNNs as the loss function.
Learning Descriptor Networks for 3D Shape Synthesis and Analysis
This paper proposes a 3D shape descriptor network, which is a deep convolutional energy-based model, for modeling volumetric shape patterns.
Revisiting Video Saliency: A Large-scale Benchmark and a New Model
Existing video saliency datasets lack variety and generality of common dynamic scenes and fall short in covering challenging situations in unconstrained environments.
Examining CNN Representations with respect to Dataset Bias
We aim to discover representation flaws caused by potential dataset bias.
Deep Cropping via Attention Box Prediction and Aesthetics Assessment
We model the photo cropping problem as a cascade of attention box regression and aesthetic quality classification, based on deep learning.
Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation
In this paper, we propose a pose grammar to tackle the problem of 3D human pose estimation.
Ranked #1 on
3D Absolute Human Pose Estimation
on Human3.6M
(Average MPJPE (mm) metric)
Deep Visual Attention Prediction
Our model is based on a skip-layer network structure, which predicts human attention from multiple convolutional layers with various reception fields.
Selective Video Object Cutout
Conventional video segmentation approaches rely heavily on appearance models.
Super-Trajectory for Video Segmentation
We introduce a novel semi-supervised video segmentation approach based on an efficient video representation, called as "super-trajectory".
Video Salient Object Detection via Fully Convolutional Networks
This paper proposes a deep learning model to efficiently detect salient regions in videos.
Saliency-Aware Geodesic Video Object Segmentation
Building on the observation that foreground areas are surrounded by the regions with high spatiotemporal edge values, geodesic distance provides an initial estimation for foreground and background.
Ranked #5 on
Video Salient Object Detection
on DAVSOD-Difficult20
(using extra training data)
Lazy Random Walks for Superpixel Segmentation
Then, the boundaries of initial superpixels are obtained according to the probabilities and the commute time.
