Search Results for author:
Found 245 papers, 128 papers with code
Boundary Detection Benchmarking: Beyond F-Measures
For an ill-posed problem like boundary detection, human labeled datasets play a critical role.
Bottom-Up Segmentation for Top-Down Detection
When employing the parts, we outperform the original DPM [14] in 19 out of 20 classes, achieving an improvement of 8% AP.
The Role of Context for Object Detection and Semantic Segmentation in the Wild
In this paper we study the role of context in existing state-of-the-art detection and segmentation approaches.
Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
Our model automatically decouples the holistic object or body parts from the model when they are hard to detect.
Parsing Semantic Parts of Cars Using Graphical Models and Segment Appearance Consistency
A novel mixture of graphical models is proposed, which dynamically couples the landmarks to a hierarchy of segments.
Human-Machine CRFs for Identifying Bottlenecks in Holistic Scene Understanding
Recent trends in image understanding have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning, and local appearance based classifiers.
Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations
More precisely, we specify a graphical model for human pose which exploits the fact the local image measurements can be used both to detect parts (or joints) and also to predict the spatial relationships between them (Image Dependent Pairwise Relations).
Ranked #18 on
Pose Estimation
on Leeds Sports Poses
Parsing Occluded People by Flexible Compositions
We model humans using a graphical model which has a tree structure building on recent work [32, 6] and exploit the connectivity prior that, even in presence of occlusion, the visible nodes form a connected subtree of the graphical model.
Semantic Part Segmentation using Compositional Model combining Shape and Appearance
This is more challenging than standard object detection, object segmentation and pose estimation tasks because semantic parts of animals often have similar appearance and highly varying shapes.
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
In this paper, we present a multimodal Recurrent Neural Network (m-RNN) model for generating novel image captions.
Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
In particular, we propose a transposed weight sharing scheme, which not only improves performance on image captioning, but also makes the model more suitable for the novel concept learning task.
Joint Object and Part Segmentation using Deep Learned Potentials
Segmenting semantic objects from images and parsing them into their respective semantic parts are fundamental steps towards detailed object understanding in computer vision.
Pose-Guided Human Parsing with Deep Learned Features
Parsing human body into semantic regions is crucial to human-centric analysis.
Generation and Comprehension of Unambiguous Object Descriptions
We propose a method that can generate an unambiguous description (known as a referring expression) of a specific object or region in an image, and which can also comprehend or interpret such an expression to infer which object is being described.
DOC: Deep OCclusion Estimation From a Single Image
In this paper we propose a deep network architecture, called DOC, which acts on a single image, detects object boundaries and estimates the border ownership (i. e. which side of the boundary is foreground and which is background).
Ground-truth dataset and baseline evaluations for image base-detail separation algorithms
Base-detail separation is a fundamental computer vision problem consisting of modeling a smooth base layer with the coarse structures, and a detail layer containing the texture-like structures.
Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
We address the key question of how object part representations can be found from the internal states of CNNs that are trained for high-level tasks, such as object classification.
Fidelity-Naturalness Evaluation of Single Image Super Resolution
In this paper, we proposed to use both fidelity (the difference with original images) and naturalness (human visual perception of super resolved images) for evaluation.
Multi-Instance Visual-Semantic Embedding
Visual-semantic embedding models have been recently proposed and shown to be effective for image classification and zero-shot learning, by mapping images into a continuous semantic label space.
InterActive: Inter-Layer Activeness Propagation
An increasing number of computer vision tasks can be tackled with deep features, which are the intermediate outputs of a pre-trained Convolutional Neural Network.
Attention Correctness in Neural Image Captioning
Attention mechanisms have recently been introduced in deep learning for various tasks in natural language processing and computer vision.
Geometric Neural Phrase Pooling: Modeling the Spatial Co-occurrence of Neurons
For this, we consider the neurons in the hidden layer as neural words, and construct a set of geometric neural phrases on top of them.
UnrealCV: Connecting Computer Vision to Unreal Engine
Computer graphics can not only generate synthetic images and ground truth but it also offers the possibility of constructing virtual worlds in which: (i) an agent can perceive, navigate, and take actions guided by AI algorithms, (ii) properties of the worlds can be modified (e. g., material and reflectance), (iii) physical simulations can be performed, and (iv) algorithms can be learnt and evaluated.
DeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images
By observing the relationship between the receptive field sizes of the different layers in the network and the skeleton scales they can capture, we introduce two scale-associated side outputs to each stage of the network.
Symmetric Non-Rigid Structure from Motion for Category-Specific Object Structure Estimation
This paper addresses the estimation of 3D structures of symmetric objects from multiple images of the same object category, e. g. different cars, seen from various viewpoints.
Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images
In this paper, we focus on training and evaluating effective word embeddings with both text and visual information.
UnrealStereo: Controlling Hazardous Factors to Analyze Stereo Vision
We generate a large synthetic image dataset with automatically computed hazardous regions and analyze algorithms on these regions.
MAT: A Multimodal Attentive Translator for Image Captioning
In this way, the sequential representation of an image can be naturally translated to a sequence of words, as the target sequence of the RNN model.
Label Distribution Learning Forests
This paper presents label distribution learning forests (LDLFs) - a novel label distribution learning algorithm based on differentiable decision trees, which have several advantages: 1) Decision trees have the potential to model any general form of label distributions by a mixture of leaf node predictions.
Ranked #11 on
Age Estimation
on MORPH album2 (Caucasian)
Deep Collaborative Learning for Visual Recognition
We formulate the function of a convolutional layer as learning a large visual vocabulary, and propose an alternative way, namely Deep Collaborative Learning (DCL), to reduce the computational complexity.
Genetic CNN
The deep Convolutional Neural Network (CNN) is the state-of-the-art solution for large-scale visual recognition.
SORT: Second-Order Response Transform for Visual Recognition
In this paper, we reveal the importance and benefits of introducing second-order operations into deep neural networks.
Recurrent Multimodal Interaction for Referring Image Segmentation
In this paper we are interested in the problem of image segmentation given natural language descriptions, i. e. referring expressions.
Multi-stage Multi-recursive-input Fully Convolutional Networks for Neuronal Boundary Detection
This design is biologically-plausible, as it likes a human visual system to compare different possible segmentation solutions to address the ambiguous boundary issue.
Adversarial Examples for Semantic Segmentation and Object Detection
Our observation is that both segmentation and detection are based on classifying multiple targets on an image (e. g., the basic target is a pixel or a receptive field in segmentation, and an object proposal in detection), which inspires us to optimize a loss function over a set of pixels/proposals for generating adversarial perturbations.
Transfer of View-manifold Learning to Similarity Perception of Novel Objects
We develop a model of perceptual similarity judgment based on re-training a deep convolution neural network (DCNN) that learns to associate different views of each 3D object to capture the notion of object persistence and continuity in our visual experience.
ScaleNet: Guiding Object Proposal Generation in Supermarkets and Beyond
We argue that estimation of object scales in images is helpful for generating object proposals, especially for supermarket images where object scales are usually within a small range.
Few-Shot Image Recognition by Predicting Parameters from Activations
In this paper, we are interested in the few-shot learning problem.
Detecting Semantic Parts on Partially Occluded Objects
Our approach detects semantic parts by accumulating the confidence of local visual cues.
Joint Multi-Person Pose Estimation and Semantic Part Segmentation
To refine part segments, the refined pose and the original part potential are integrated through a Part FCN, where the skeleton feature from pose serves as additional regularization cues for part segments.
Ranked #5 on
Human Part Segmentation
on PASCAL-Part
Mitigating Adversarial Effects Through Randomization
Convolutional neural networks have demonstrated high accuracy on various tasks in recent years.
Visual Concepts and Compositional Voting
We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles.
Few-shot Learning by Exploiting Visual Concepts within CNNs
In this work, we address these limitations of CNNs by developing novel, flexible, and interpretable models for few-shot learning.
Gradually Updated Neural Networks for Large-Scale Image Recognition
Our method is by introducing computation orderings to the channels within convolutional layers or blocks, based on which we gradually compute the outputs in a channel-wise manner.
Progressive Neural Architecture Search
We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms.
Ranked #15 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Deep Regression Forests for Age Estimation
Age estimation from facial images is typically cast as a nonlinear regression problem.
Ranked #6 on
Age Estimation
on FGNET
Unleashing the Potential of CNNs for Interpretable Few-Shot Learning
Our models are based on the idea of encoding objects in terms of visual concepts, which are interpretable visual cues represented by the feature vectors within CNNs.
Deep Co-Training for Semi-Supervised Image Recognition
We present Deep Co-Training, a deep learning based method inspired by the Co-Training framework.
Improving Transferability of Adversarial Examples with Input Diversity
We hope that our proposed attack strategy can serve as a strong benchmark baseline for evaluating the robustness of networks to adversaries and the effectiveness of different defense methods in the future.
Scene Graph Parsing as Dependency Parsing
The scene graphs generated by our learned neural dependency parser achieve an F-score similarity of 49. 67% to ground truth graphs on our evaluation set, surpassing best previous approaches by 5%.
Adversarial Attacks and Defences Competition
To accelerate research on adversarial examples and robustness of machine learning classifiers, Google Brain organized a NIPS 2017 competition that encouraged researchers to develop new methods to generate adversarial examples as well as to develop new ways to defend against them.
SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data
But, this raises an important problem in active vision: given an {\bf infinite} data space, how to effectively sample a {\bf finite} subset to train a visual classifier?
Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
We focus on the problem of training a deep neural network in generations.
Resisting Large Data Variations via Introspective Transformation Network
Experimental results verify that our approach significantly improves the ability of deep networks to resist large variations between training and testing data and achieves classification accuracy improvements on several benchmark datasets, including MNIST, affNIST, SVHN, CIFAR-10 and miniImageNet.
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection
The iterative instance classifier refinement is implemented online using multiple streams in convolutional neural networks, where the first is an MIL network and the others are for instance classifier refinement supervised by the preceding one.
Ranked #1 on
Weakly Supervised Object Detection
on ImageNet
Rethinking Monocular Depth Estimation with Adversarial Training
Most existing approaches treat depth estimation as a regression problem with a local pixel-wise loss function.
Weakly Supervised Region Proposal Network and Object Detection
The Convolutional Neural Network (CNN) based region proposal generation method (i. e. region proposal network), trained using bounding box annotations, is an essential component in modern fully supervised object detectors.
Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
Performance on the five tasks of depth estimation, optical flow estimation, odometry, moving object segmentation and scene flow estimation shows that our approach outperforms other SoTA methods.
OriNet: A Fully Convolutional Network for 3D Human Pose Estimation
We use limb orientations as a new way to represent 3D poses and bind the orientation together with the bounding box of each limb region to better associate images and predictions.
Ranked #76 on
3D Human Pose Estimation
on MPI-INF-3DHP
(AUC metric)
Semantic Part Detection via Matching: Learning to Generalize to Novel Viewpoints from Limited Training Data
In particular, this enables images in the training dataset to be matched to a virtual 3D model of the object (for simplicity, we assume that the object viewpoint can be estimated by standard techniques).
Robust Face Detection via Learning Small Faces on Hard Images
In this paper, we propose that the robustness of a face detector against hard faces can be improved by learning small faces on hard images.
Ranked #8 on
Face Detection
on WIDER Face (Hard)
3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training
Meanwhile, a fully-supervised method based on our approach achieved state-of-the-art performances on both the LiTS liver tumor segmentation and the Medical Segmentation Decathlon (MSD) challenge, demonstrating the robustness and value of our framework, even when fully supervised training is feasible.
Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization
By simply replacing standard optimizers with Neural Rejuvenation, we are able to improve the performances of neural networks by a very large margin while using similar training efforts and maintaining their original resource usages.
Learning Transferable Adversarial Examples via Ghost Networks
The critical principle of ghost networks is to apply feature-level perturbations to an existing model to potentially create a huge set of diverse models.
Feature Denoising for Improving Adversarial Robustness
This study suggests that adversarial perturbations on images lead to noise in the features constructed by these networks.
Ranked #1 on
Adversarial Defense
on CAAD 2018
ELASTIC: Improving CNNs with Dynamic Scaling Policies
We formulate the scaling policy as a non-linear function inside the network's structure that (a) is learned from data, (b) is instance specific, (c) does not add extra computation, and (d) can be applied on any network architecture.
CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
Yet there has been evidence that current benchmark datasets suffer from bias, and current state-of-the-art models cannot be easily evaluated on their intermediate reasoning process.
Ranked #1 on
Referring Expression Segmentation
on CLEVR-Ref+
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
Therefore, we propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space.
Ranked #7 on
Semantic Segmentation
on PASCAL VOC 2012 val
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
Batch Normalization (BN) has become an out-of-box technique to improve deep network training.
Ranked #76 on
Instance Segmentation
on COCO minival
An Alarm System For Segmentation Algorithm Based On Shape Model
Motivated by this, in this paper, we learn a feature space using the shape information which is a strong prior shared among different datasets and robust to the appearance variation of input data. The shape feature is captured using a Variational Auto-Encoder (VAE) network that trained with only the ground truth masks.
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval
Sketch-based image retrieval (SBIR) is widely recognized as an important vision problem which implies a wide range of real-world applications.
Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
Accurate multi-organ abdominal CT segmentation is essential to many clinical applications such as computer-aided intervention.
Structured Prediction using cGANs with Fusion Discriminator
We propose the fusion discriminator, a single unified framework for incorporating conditional information into a generative adversarial network (GAN) for a variety of distinct structured prediction tasks, including image synthesis, semantic segmentation, and depth estimation.
Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models
We test both humans and the above-mentioned computational models in a challenging task of object recognition under extreme occlusion, where target objects are heavily occluded by irrelevant real objects in real backgrounds.
Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion
In this work, we combine DCNNs and compositional object models to retain the best of both approaches: a discriminative model that is robust to partial occlusion and mask attacks.
V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation
Deep learning algorithms, in particular 2D and 3D fully convolutional neural networks (FCNs), have rapidly become the mainstream methodology for volumetric medical image segmentation.
Intriguing properties of adversarial training at scale
This two-domain hypothesis may explain the issue of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics of this mixture distribution is challenging.
Multi-Scale Attentional Network for Multi-Focal Segmentation of Active Bleed after Pelvic Fractures
Trauma is the worldwide leading cause of death and disability in those younger than 45 years, and pelvic fractures are a major source of morbidity and mortality.
Deep Differentiable Random Forests for Age Estimation
Both of them connect split nodes to the top layer of convolutional neural networks (CNNs) and deal with inhomogeneous data by jointly learning input-dependent data partitions at the split nodes and age distributions at the leaf nodes.
FusionNet: Incorporating Shape and Texture for Abnormality Detection in 3D Abdominal CT Scans
Second, a FusionNet is proposed to take both the binary mask and CT image as input and perform a binary classification.
Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation
Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers with an overall five-year survival rate of 8%.
TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion
Despite deep convolutional neural networks' great success in object classification, it suffers from severe generalization performance drop under occlusion due to the inconsistency between training and testing data.
Universal Physical Camouflage Attacks on Object Detectors
In this paper, we study physical adversarial attacks on object detectors in the wild.
Grouped Spatial-Temporal Aggregation for Efficient Action Recognition
This decomposition is more parameter-efficient and enables us to quantitatively analyze the contributions of spatial and temporal features in different layers.
Localizing Occluders with Compositional Convolutional Networks
Our experimental results demonstrate that the proposed extensions increase the model's performance at localizing occluders as well as at classifying partially occluded objects.
Adversarial Examples Improve Image Recognition
We show that AdvProp improves a wide range of models on various image recognition tasks and performs better when the models are bigger.
Ranked #4 on
Domain Generalization
on VizWiz-Classification
Rethinking Normalization and Elimination Singularity in Neural Networks
To address this issue, we propose BatchChannel Normalization (BCN), which uses batch knowledge to avoid the elimination singularities in the training of channel-normalized models.
Identifying Model Weakness with Adversarial Examiner
Different from the existing strategy to always give the same (distribution of) test data, the adversarial examiner will dynamically select the next test data to hand out based on the testing history so far, with the goal being to undermine the model's performance.
Deeply Shape-guided Cascade for Instance Segmentation
The key to a successful cascade architecture for precise instance segmentation is to fully leverage the relationship between bounding box detection and mask segmentation across multiple stages.
RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition
In this paper, we propose the Randomized Simulation as Augmentation (RSA) framework which augments real-world training data with synthetic data to improve the robustness of action recognition networks.
DASZL: Dynamic Action Signatures for Zero-shot Learning
There are many realistic applications of activity recognition where the set of potential activity descriptions is combinatorially large.
Identity Preserve Transform: Understand What Activity Classification Models Have Learnt
This can explain why an activity classification model usually fails to generalize to datasets it is not trained on.
Learning from Synthetic Animals
Despite great success in human parsing, progress for parsing other deformable articulated objects, like animals, is still limited by the lack of labeled data.
AtomNAS: Fine-Grained End-to-End Neural Architecture Search
We propose a fine-grained search space comprised of atomic blocks, a minimal search unit that is much smaller than the ones used in recent NAS algorithms.
Ranked #61 on
Neural Architecture Search
on ImageNet
Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots
Accurate 3D object detection in LiDAR based point clouds suffers from the challenges of data sparsity and irregularities.
Ranked #3 on
3D Object Detection
on KITTI Pedestrians Moderate
When Radiology Report Generation Meets Knowledge Graph
In addition, we proposed a new evaluation metric for radiology image reporting with the assistance of the same composed graph.
Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion
Inspired by the success of compositional models at classifying partially occluded objects, we propose to integrate compositional models and DCNNs into a unified deep model with innate robustness to partial occlusion.
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
In this paper, we attempt to remove this constraint by factorizing 2D self-attention into two 1D self-attentions.
Ranked #4 on
Panoptic Segmentation
on Cityscapes val
(using extra training data)
Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of semantic segmentation, such as autonomous driving and medical image analysis.
Ranked #8 on
Anomaly Detection
on Road Anomaly
(using extra training data)
Are Labels Necessary for Neural Architecture Search?
Existing neural network architectures in computer vision -- whether designed by humans or by machines -- were typically found using both images and their associated labels.
Neural Architecture Search for Lightweight Non-Local Networks
However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks.
Ranked #60 on
Neural Architecture Search
on ImageNet
Context-Aware Group Captioning via Self-Attention and Contrastive Features
In this paper, we introduce a new task, context-aware group captioning, which aims to describe a group of target images in the context of another group of related reference images.
PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
PatchAttack induces misclassifications by superimposing small textured patches on the input image.
Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search
This is the goal of our work, where we introduce stratified organ at risk segmentation (SOARS), an approach that stratifies OARs into anchor, mid-level, and small & hard (S&H) categories.
Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation
In this paper, we present a novel unsupervised domain adaptation (UDA) method, named Domain Adaptive Relational Reasoning (DARR), to generalize 3D multi-organ segmentation models to medical data collected from different scanners and/or protocols (domains).
Robust Object Detection under Occlusion with Context-Aware CompositionalNets
In this work, we propose to overcome two limitations of CompositionalNets which will enable them to detect partially occluded objects: 1) CompositionalNets, as well as other DCNN architectures, do not explicitly separate the representation of the context from the object itself.
JSSR: A Joint Synthesis, Segmentation, and Registration System for 3D Multi-Modal Image Alignment of Large-scale Pathological CT Scans
We extensively evaluate our JSSR system on a large-scale medical image dataset containing 1, 485 patient CT imaging studies of four different phases (i. e., 5, 940 3D CT scans with pathological livers) on the registration, segmentation and synthesis tasks.
Detecting Scatteredly-Distributed, Small, andCritically Important Objects in 3D OncologyImaging via Decision Stratification
We focus on the detection and segmentation of oncology-significant (or suspicious cancer metastasized) lymph nodes (OSLNs), which has not been studied before as a computational task.
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution
In this paper, we explore this mechanism in the backbone design for object detection.
Ranked #2 on
Object Detection
on AI-TOD
Smooth Adversarial Training
SAT also works well with larger networks: it helps EfficientNet-L1 to achieve 82. 2% accuracy and 58. 6% robustness on ImageNet, outperforming the previous state-of-the-art defense by 9. 5% for accuracy and 11. 6% for robustness.
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition under Occlusion
The structure of the compositional model enables CompositionalNets to decompose images into objects and context, as well as to further decompose object representations in terms of individual parts and the objects' pose.
Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
Experiments on the NIH pancreas segmentation dataset and a multi-organ segmentation dataset show state-of-the-art performance of the proposed framework on semi-supervised medical image segmentation.
Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles
As for vehicles, their trajectories are significantly influenced by the lane geometry and how to effectively use the lane information is of active interest.
ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation
In this paper, we further improve spatio-temporal point cloud feature learning with a flexible module called ASAP considering both attention and structure information across frames, which we find as two important factors for successful segmentation in dynamic point clouds.
Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy
Finding, identifying and segmenting suspicious cancer metastasized lymph nodes from 3D multi-modality imaging is a clinical task of paramount importance.
Lymph Node Gross Tumor Volume Detection in Oncology Imaging via Relationship Learning Using Graph Neural Network
Specifically, we first utilize a 3D convolutional neural network with ROI-pooling to extract the GTV$_{LN}$'s instance-wise appearance features.
CoKe: Localized Contrastive Learning for Robust Keypoint Detection
In this paper, we introduce a contrastive learning framework for keypoint detection (CoKe).
CO2: Consistent Contrast for Unsupervised Visual Representation Learning
Regarding the similarity of the query crop to each crop from other images as "unlabeled", the consistency term takes the corresponding similarity of a positive crop as a pseudo label, and encourages consistency between these two similarities.
Shape-Texture Debiased Neural Network Training
To prevent models from exclusively attending on a single cue in representation learning, we augment training data with images with conflicting shape and texture information (eg, an image of chimpanzee shape but with lemon texture) and, most importantly, provide the corresponding supervisions from shape and texture simultaneously.
Ranked #601 on
Image Classification
on ImageNet
Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model
Moreover, by leveraging an outlier process, Bayesian models can further generalize out-of-distribution to segment partially occluded objects and to predict their amodal object boundaries.
Can Temporal Information Help with Contrastive Self-Supervised Learning?
To this end, we present Temporal-aware Contrastive self-supervised learningTaCo, as a general paradigm to enhance video CSL.
Batch Normalization with Enhanced Linear Transformation
Batch normalization (BN) is a fundamental unit in modern deep networks, in which a linear transformation module was designed for improving BN's flexibility of fitting complex data distributions.
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers
As a result, MaX-DeepLab shows a significant 7. 1% PQ gain in the box-free regime on the challenging COCO dataset, closing the gap between box-based and box-free methods for the first time.
Ranked #12 on
Panoptic Segmentation
on COCO test-dev
Robustness Out of the Box: Compositional Representations Naturally Defend Against Black-Box Patch Attacks
Second, we find that compositional deep networks, which have part-based representations that lead to innate robustness to natural occlusion, are robust to patch attacks on PASCAL3D+ and the German Traffic Sign Recognition Benchmark, without adversarial training.
Unsupervised Part Discovery via Feature Alignment
Specifically, given a training image, we find a set of similar images that show instances of the same object category in the same pose, through an affine alignment of their corresponding feature maps.
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion
The improved segmentation masks are, in turn, integrated into the network in a top-down manner to improve the image classification.
ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation
We name this joint task as Depth-aware Video Panoptic Segmentation, and propose a new evaluation metric along with two derived datasets for it, which will be made available to the public.
Ranked #1 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
Depth-aware Video Panoptic Segmentation
Monocular Depth Estimation
+2
Mask Guided Matting via Progressive Refinement Network
We propose Mask Guided (MG) Matting, a robust matting framework that takes a general coarse mask as guidance.
CORL: Compositional Representation Learning for Few-Shot Classification
In particular, during meta-learning, we train a knowledge base that consists of a dictionary of component representations and a dictionary of component activation maps that encode common spatial activation patterns of components.
NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation
Using differentiable rendering we estimate the 3D object pose by minimizing the reconstruction error between NeMo and the feature representation of the target image.
Occluded Video Instance Segmentation: A Benchmark
On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16. 3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario.
Ranked #39 on
Video Instance Segmentation
on OVIS validation
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Semi-supervised learning on class-imbalanced data, although a realistic problem, has been under studied.
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping
Additionally, current approaches that deal with forgetting ignore the problem of catastrophic remembering, i. e. the worsening ability to discriminate between data from different tasks.
Ranked #1 on
Continual Learning
on ImageNet-50 (5 tasks)
Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistency
However, existing approaches which rely only on image-level class labels predominantly suffer from errors due to (a) partial segmentation of objects and (b) missing object predictions.
Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles
We believe our dataset provides a rich testbed to study UDA for part segmentation and will help to significantly push forward research in this area.
CateNorm: Categorical Normalization for Robust Medical Image Segmentation
We propose a new normalization strategy, named categorical normalization (CateNorm), to normalize the activations according to categorical statistics.
A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation
To deal with the large shape variance, we introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space, where we have separate codes for encoding shape and articulation.
Self-Supervised Pillar Motion Learning for Autonomous Driving
Autonomous driving can benefit from motion behavior comprehension when interacting with diverse traffic participants in highly dynamic environments.
Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentation
In this work, we design a new data-driven approach, namely Auto-FedAvg, where aggregation weights are dynamically adjusted, depending on data distributions across data silos and the current training progress of the models.
Visual analogy: Deep learning versus compositional models
We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) directly trained to solve these analogy problems, as well as to that of a compositional model that assesses relational similarity between part-based representations.
Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
In particular, we decouple the training of the representation and the classifier, and systematically investigate the effects of different data re-sampling techniques when training the whole network including a classifier as well as fine-tuning the feature extractor only.
Glance-and-Gaze Vision Transformer
It is motivated by the Glance and Gaze behavior of human beings when recognizing objects in natural scenes, with the ability to efficiently model both long-range dependencies and local context.
Simulated Adversarial Testing of Face Recognition Models
In this work, we propose a framework for learning how to test machine learning algorithms using simulators in an adversarial manner in order to find weaknesses in the model before deploying it in critical scenarios.
Progressive Stage-wise Learning for Unsupervised Feature Representation Enhancement
For a given unsupervised task, we design multilevel tasks and define different learning stages for the deep network.
Locally Enhanced Self-Attention: Combining Self-Attention and Convolution as Local and Context Terms
Self-Attention has become prevalent in computer vision models.
Exploring Simple 3D Multi-Object Tracking for Autonomous Driving
3D multi-object tracking in LiDAR point clouds is a key ingredient for self-driving vehicles.
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
Thus, we propose RobustART, the first comprehensive Robustness investigation benchmark on ImageNet regarding ARchitecture design (49 human-designed off-the-shelf architectures and 1200+ networks from neural architecture search) and Training techniques (10+ techniques, e. g., data augmentation) towards diverse noises (adversarial, natural, and system noises).
SAME: Deformable Image Registration based on Self-supervised Anatomical Embeddings
In this work, we introduce a fast and accurate method for unsupervised 3D medical image registration.
Image BERT Pre-training with Online Tokenizer
The success of language Transformers is primarily attributed to the pretext task of masked language modeling (MLM), where texts are first tokenized into semantically meaningful pieces.
Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images
Our experiments show CCO substantially boosts the performance of neural symbolic methods on real images.
Nuisance-Label Supervision: Robustness Improvement by Free Labels
Nuisance factors are those irrelevant to a task, and an ideal model should be invariant to them.
A Light-weight Interpretable Compositional Model for Nuclei Detection and Weakly-Supervised Segmentation
It only requires annotations on isolated nucleus, rather than on all nuclei in the dataset.
Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose
The model is initialized from a few labelled images and is subsequently used to synthesize feature representations of unseen 3D views.
Are Transformers More Robust Than CNNs?
Transformer emerges as a powerful tool for visual recognition.
Ranked #1 on
Adversarial Robustness
on Stylized ImageNet
Searching for TrioNet: Combining Convolution with Local and Global Self-Attention
In order to effectively search in this huge architecture space, we propose Hierarchical Sampling for better training of the supernet.
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge
To promote the development of occlusion understanding, we collect a large-scale dataset called OVIS for video instance segmentation in the occluded scenario.
iBOT: Image BERT Pre-Training with Online Tokenizer
We present a self-supervised framework iBOT that can perform masked prediction with an online tokenizer.
Ranked #1 on
Unsupervised Image Classification
on ImageNet
TransMix: Attend to Mix for Vision Transformers
The confidence of the label will be larger if the corresponding input image is weighted higher by the attention map.
Learning from Temporal Gradient for Semi-supervised Action Recognition
Our method achieves the state-of-the-art performance on three video action recognition benchmarks (i. e., Kinetics-400, UCF-101, and HMDB-51) under several typical semi-supervised settings (i. e., different ratios of labeled data).
OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors.
PartImageNet: A Large, High-Quality Dataset of Parts
To help address this problem, we propose PartImageNet, a large, high-quality dataset with part segmentation annotations.
MT-TransUNet: Mediating Multi-Task Tokens in Transformers for Skin Lesion Segmentation and Classification
However, these approaches formulated skin cancer diagnosis as a simple classification task, dismissing the potential benefit from lesion segmentation.
Masked Feature Prediction for Self-Supervised Visual Pre-Training
We present Masked Feature Prediction (MaskFeat) for self-supervised pre-training of video models.
Ranked #8 on
Action Recognition
on AVA v2.2
(using extra training data)
Lite Vision Transformer with Enhanced Self-Attention
We propose Lite Vision Transformer (LVT), a novel light-weight transformer network with two enhanced self-attention mechanisms to improve the model performances for mobile deployment.
Point-Level Region Contrast for Object Detection Pre-Training
In this work we present point-level region contrast, a self-supervised pre-training approach for the task of object detection.
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
In this paper, we propose two novel techniques: InverseAug that inverses geometric-related augmentations, e. g., rotation, to enable accurate geometric alignment between lidar points and image pixels, and LearnableAlign that leverages cross-attention to dynamically capture the correlations between image and lidar features during fusion.
CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation
Recent advances in self-supervised contrastive learning yield good image-level representation, which favors classification tasks but usually neglects pixel-level detailed information, leading to unsatisfactory transfer performance to dense prediction tasks such as semantic segmentation.
SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering
By swapping the context object features, the model reliance on context can be suppressed effectively.
Fast AdvProp
Specifically, our modifications in Fast AdvProp are guided by the hypothesis that disentangled learning with adversarial examples is the key for performance improvements, while other training recipes (e. g., paired clean and adversarial training samples, multi-step adversarial attackers) could be largely simplified.
In Defense of Image Pre-Training for Spatiotemporal Recognition
Inspired by this observation, we hypothesize that the key to effectively leveraging image pre-training lies in the decomposition of learning spatial and temporal features, and revisiting image pre-training as the appearance prior to initializing 3D kernels.
VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis
The Gaussian reconstruction kernels have been proposed by Westover (1990) and studied by the computer graphics community back in the 90s, which gives an alternative representation of object 3D geometry from meshes and point clouds.
A Simple Data Mixing Prior for Improving Self-Supervised Learning
More notably, our SDMP is the first method that successfully leverages data mixing to improve (rather than hurt) the performance of Vision Transformers in the self-supervised setting.
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation
We propose Clustering Mask Transformer (CMT-DeepLab), a transformer-based framework for panoptic segmentation designed around clustering.
Ranked #6 on
Panoptic Segmentation
on COCO test-dev
Unsupervised Domain Adaptation through Shape Modeling for Medical Image Segmentation
Previous methods proposed Variational Autoencoder (VAE) based models to learn the distribution of shape for a particular organ and used it to automatically evaluate the quality of a segmentation prediction by fitting it into the learned shape distribution.
kMaX-DeepLab: k-means Mask Transformer
However, we observe that most existing transformer-based vision models simply borrow the idea from NLP, neglecting the crucial difference between languages and images, particularly the extremely large sequence length of spatially flattened pixel features.
Ranked #2 on
Panoptic Segmentation
on COCO test-dev
In Defense of Online Models for Video Instance Segmentation
In recent years, video instance segmentation (VIS) has been largely advanced by offline models, while online models gradually attracted less attention possibly due to their inferior performance.
Ranked #9 on
Video Instance Segmentation
on YouTube-VIS validation
(using extra training data)
Explicit Occlusion Reasoning for Multi-person 3D Human Pose Estimation
Inspired by the remarkable ability of humans to infer occluded joints from visible cues, we develop a method to explicitly model this process that significantly improves bottom-up multi-person human pose estimation with or without occlusions.
Ranked #10 on
3D Multi-Person Pose Estimation (absolute)
on MuPoTS-3D
3D Human Pose Estimation
3D Multi-Person Pose Estimation (absolute)
+2
Masked Autoencoders Enable Efficient Knowledge Distillers
For example, by distilling the knowledge from an MAE pre-trained ViT-L into a ViT-B, our method achieves 84. 0% ImageNet top-1 accuracy, outperforming the baseline of directly distilling a fine-tuned ViT-L by 1. 2%.
Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features
We consider the problem of category-level 6D pose estimation from a single RGB image.
MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models
The tiny-MOAT family is also benchmarked on downstream tasks, serving as a baseline for the community.
Ranked #1 on
Object Detection
on MS COCO
Context-Enhanced Stereo Transformer
We construct our stereo depth estimation model, Context Enhanced Stereo Transformer (CSTR), by plugging CEP into the state-of-the-art stereo depth estimation method Stereo Transformer.
1st Place Solution of The Robust Vision Challenge 2022 Semantic Segmentation Track
The model is trained on a composite dataset consisting of images from 9 datasets (ADE20K, Cityscapes, Mapillary Vistas, ScanNet, VIPER, WildDash 2, IDD, BDD, and COCO) with a simple dataset balancing strategy.
Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification
We hope that this study can direct future research on the application of Transformers to a larger variety of medical imaging tasks.
Synthetic Tumors Make AI Segment Tumors Better
We develop a novel strategy to generate synthetic tumors.
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training
Coupling all these designs allows our method to enjoy both competitive performances on text-to-video retrieval and video question answering tasks, and much less pre-training costs by 1. 9X or more.
LUMix: Improving Mixup by Better Modelling Label Uncertainty
LUMix is simple as it can be implemented in just a few lines of code and can be universally applied to any deep networks \eg CNNs and Vision Transformers, with minimal computational cost.
Localization vs. Semantics: Visual Representations in Unimodal and Multimodal Models
Despite the impressive advancements achieved through vision-and-language pretraining, it remains unclear whether this joint learning paradigm can help understand each individual modality.
Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning
Visual Question Answering (VQA) models often perform poorly on out-of-distribution data and struggle on domain generalization.
AsyInst: Asymmetric Affinity with DepthGrad and Color for Box-Supervised Instance Segmentation
Through systematic analysis, we found that the commonly used pairwise affinity loss has two limitations: (1) it works with color affinity but leads to inferior performance with other modalities such as depth gradient, (2)the original affinity loss does not prevent trivial predictions as intended but actually accelerates this process due to the affinity loss term being symmetric.
Unleashing the Power of Visual Prompting At the Pixel Level
This paper presents a simple and effective visual prompting method for adapting pre-trained models to downstream recognition tasks.
Learning Road Scene-level Representations via Semantic Region Prediction
In this work, we tackle two vital tasks in automated driving systems, i. e., driver intent prediction and risk object identification from egocentric images.
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
The proposed model is developed from an assembly of 14 datasets, using a total of 3, 410 CT scans for training and then evaluated on 6, 162 external CT scans from 3 additional datasets.
Ranked #1 on
Organ Segmentation
on BTCV
Benchmarking Robustness in Neural Radiance Fields
We find that NeRF-based models are significantly degraded in the presence of corruption, and are more sensitive to a different set of corruptions than image recognition models.
CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans
Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases.
PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation
We introduce a learning-based testing method, termed PoseExaminer, that automatically diagnoses HPS algorithms by searching over the parameter space of human pose images to find the failure modes.
InstMove: Instance Motion for Object-centric Video Segmentation
A common solution is to use optical flow to provide motion information, but essentially it only considers pixel-level motion, which still relies on appearance similarity and hence is often inaccurate under occlusion and fast movement.
Label-Free Liver Tumor Segmentation
We demonstrate that AI models can accurately segment liver tumors without the need for manual annotation by using synthetic tumors in CT scans.
Ranked #1 on
Tumor Segmentation
on LiTS17
Diffusion Models as Masked Autoencoders
There has been a longstanding belief that generation can facilitate a true understanding of visual data.
OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
Enhancing the robustness of vision algorithms in real-world scenarios is challenging.
