Search Results for author:
Found 79 papers, 33 papers with code
Detecting and Aligning Faces by Image Retrieval
In order to overcome these challenges, we present a novel and robust exemplarbased face detector that integrates image retrieval and discriminative learning.
Towards Unified Human Parsing and Pose Estimation
We study the problem of human body configuration analysis, more specifically, human parsing and human pose estimation.
Efficient Boosted Exemplar-based Face Detection
Despite the fact that face detection has been studied intensively over the past several decades, the problem is still not completely solved.
Deep Human Parsing with Active Template Regression
The first CNN network is with max-pooling, and designed to predict the template coefficients for each label mask, while the second CNN network is without max-pooling to preserve sensitivity to label mask position and accurately predict the active shape parameters.
Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
Under the classic K Nearest Neighbor (KNN)-based nonparametric framework, the parametric Matching Convolutional Neural Network (M-CNN) is proposed to predict the matching confidence and displacements of the best matched region in the testing image for a particular semantic region in one KNN image.
Joint Object and Part Segmentation using Deep Learned Potentials
Segmenting semantic objects from images and parsing them into their respective semantic parts are fundamental steps towards detailed object understanding in computer vision.
Inner and Inter Label Propagation: Salient Object Detection in the Wild
For most natural images, some boundary superpixels serve as the background labels and the saliency of other superpixels are determined by ranking their similarities to the boundary labels based on an inner propagation scheme.
A Convolutional Neural Network Cascade for Face Detection
To improve localization effectiveness, and reduce the number of candidates at later stages, we introduce a CNN-based calibration stage after each of the detection stages in the cascade.
Towards Unified Depth and Semantic Prediction From a Single Image
By allowing for interactions between the depth and semantic information, the joint network provides more accurate depth prediction than a state-of-the-art CNN trained solely for depth prediction [5].
LCNN: Low-level Feature Embedded CNN for Salient Object Detection
In this paper, we propose a novel deep neural network framework embedded with low-level features (LCNN) for salient object detection in complex images.
Proposal-free Network for Instance-level Object Segmentation
Instance-level object segmentation is an important yet under-explored task.
STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
Then, a better network called Enhanced-DCNN is learned with supervision from the predicted segmentation masks of simple images based on the Initial-DCNN as well as the image-level annotations.
Automatic Content-Aware Color and Tone Stylization
We introduce a new technique that automatically generates diverse, visually compelling stylizations for a photograph in an unsupervised manner.
Semantic Object Parsing with Local-Global Long Short-Term Memory
The long chains of sequential computation by stacked LG-LSTM layers also enable each pixel to sense a much larger region for inference benefiting from the memorization of previous dependencies in all positions along all dimensions.
Human Parsing With Contextualized Convolutional Neural Network
In this work, we address the human parsing task with a novel Contextualized Convolutional Neural Network (Co-CNN) architecture, which well integrates the cross-layer context, global image-level context, within-super-pixel context and cross-super-pixel neighborhood context into a unified network.
Minimum Barrier Salient Object Detection at 80 FPS
Powered by this fast MBD transform algorithm, the proposed salient object detection method runs at 80 FPS, and significantly outperforms previous methods with similar speed on four large benchmark datasets, and achieves comparable or better performance than state-of-the-art methods.
Ranked #6 on
Video Salient Object Detection
on VOS-T
(using extra training data)
Deep Multi-Patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation
We propose a deep multi-patch aggregation network training approach, which allows us to train models using multiple patches generated from one image.
Ranked #8 on
Aesthetics Quality Assessment
on AVA
Semantic Object Parsing with Graph LSTM
By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data.
A Multi-Level Contextual Model For Person Recognition in Photo Albums
Our new framework enables efficient use of these complementary multi-level contextual cues to improve overall recognition rates on the photo album person recognition task, as demonstrated through state-of-the-art results on a challenging public dataset.
Shortlist Selection With Residual-Aware Distance Estimator for K-Nearest Neighbor Search
We have tested the proposed method with the inverted index and multi-index on a diverse set of benchmarks including up to one billion data points with varying dimensions, and found that our method robustly improves the accuracy of shortlists (up to 127% relatively higher) over the state-of-the-art techniques with a comparable or even faster computational cost.
Event-Specific Image Importance
In this paper, we show that the selection of important images is consistent among different viewers, and that this selection process is related to the event type of the album.
Unconstrained Salient Object Detection via Proposal Subset Optimization
Our system leverages a Convolutional-Neural-Network model to generate location proposals of salient objects.
Photo Aesthetics Ranking Network with Attributes and Content Adaptation
In this work, we propose to learn a deep convolutional neural network to rank photo aesthetics in which the relative ranking of photo aesthetics are directly modeled in the loss function.
Ranked #7 on
Aesthetics Quality Assessment
on AVA
Progressive Attention Networks for Visual Attribute Prediction
We propose a novel attention model that can accurately attends to target objects of various scales and shapes in images.
Salient Object Subitizing
We study the problem of Salient Object Subitizing, i. e. predicting the existence and the number of salient objects in an image using holistic cues.
Top-down Neural Attention by Excitation Backprop
We aim to model the top-down attention of a Convolutional Neural Network (CNN) classifier for generating task-specific attention maps.
SURGE: Surface Regularized Geometry Estimation from a Single Image
This paper introduces an approach to regularize 2. 5D surface normal and depth predictions at each pixel given a single input image.
Video Scene Parsing with Predictive Feature Learning
In this way, the network can effectively learn to capture video dynamics and temporal context, which are critical clues for video scene parsing, without requiring extra manual annotations.
Learning to Detect Multiple Photographic Defects
Our new dataset enables us to formulate the problem as a multi-task learning problem and train a multi-column deep convolutional neural network (CNN) to simultaneously predict the severity of all the defects.
Interpretable Structure-Evolving LSTM
Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization.
Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
Human parsing has recently attracted a lot of research interests due to its huge application potentials.
Ranked #13 on
Semantic Segmentation
on LIP val
Recurrent Multimodal Interaction for Referring Image Segmentation
In this paper we are interested in the problem of image segmentation given natural language descriptions, i. e. referring expressions.
Learning to Predict Indoor Illumination from a Single Image
We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene.
Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition
Furthermore, our algorithm can generate descriptions with varied length, benefiting from the separate control of the skeleton and attributes.
Recognizing and Curating Photo Albums via Event-Specific Image Importance
Automatic organization of personal photos is a problem with many real world ap- plications, and can be divided into two main tasks: recognizing the event type of the photo collection, and selecting interesting images from the collection.
FoveaNet: Perspective-aware Urban Scene Parsing
Thus, they suffer from heterogeneous object scales caused by perspective projection of cameras on actual scenes and inevitably encounter parsing failures on distant objects as well as other boundary and recognition errors.
Personalized Image Aesthetics
To accommodate our study, we first collect two distinct datasets, a large image dataset from Flickr and annotated by Amazon Mechanical Turk, and a small dataset of real personal albums rated by owners.
Learning to Segment Human by Watching YouTube
An intuition on human segmentation is that when a human is moving in a video, the video-context (e. g., appearance and motion clues) may potentially infer reasonable mask information for the whole human body.
Scene Parsing with Global Context Embedding
We present a scene parsing method that utilizes global context information based on both the parametric and non- parametric models.
Predicting Scene Parsing and Motion Dynamics in the Future
The ability of predicting the future is important for intelligent systems, e. g. autonomous vehicles and robots to plan early and make decisions accordingly.
MAttNet: Modular Attention Network for Referring Expression Comprehension
In this paper, we address referring expression comprehension: localizing an image region described by a natural language expression.
Generalized Referring Expression Segmentation
Referring Expression
+1
Generative Image Inpainting with Contextual Attention
Motivated by these observations, we propose a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark
To further explore and take advantage of the semantic correlation of these two tasks, we propose a novel joint human parsing and pose estimation network to explore efficient context modeling, which can simultaneously predict parsing and pose with extremely high quality.
Ranked #10 on
Semantic Segmentation
on LIP val
Towards Interpretable Face Recognition
Deep CNNs have been pushing the frontier of visual recognition over past years.
Learning to Understand Image Blur
In this paper, we propose a unified framework to estimate a spatially-varying blur map and understand its desirability in terms of image quality at the same time.
Good View Hunting: Learning Photo Composition From Dense View Pairs
Finding views with good photo composition is a challenging task for machine learning methods.
Free-Form Image Inpainting with Gated Convolution
We present a generative image inpainting system to complete images with free-form mask and guidance.
Ranked #3 on
Image Inpainting
on Places2 val
Concept Mask: Large-Scale Segmentation from Semantic Concepts
Then, we refine and extend the embedding network to predict an attention map, using a curated dataset with bounding box annotations on 750 concepts.
Compositing-aware Image Search
We present a new image search technique that, given a background image, returns compatible foreground objects for image compositing tasks.
Learning to Blend Photos
Specifically, given a foreground image and a background image, our proposed method automatically generates a set of blending photos with scores that indicate the aesthetics quality with the proposed quality network and policy network.
DeepLens: Shallow Depth Of Field From A Single Image
To achieve this, we propose a novel neural network model comprised of a depth prediction module, a lens blur module, and a guided upsampling module.
Sequence-to-Segment Networks for Segment Detection
Detecting segments of interest from an input sequence is a challenging problem which often requires not only good knowledge of individual target segments, but also contextual understanding of the entire input sequence and the relationships between the target segments.
Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
We observe the property of regional homogeneity in adversarial perturbations and suggest that the defenses are less robust to regionally homogeneous perturbations.
Graphonomy: Universal Human Parsing via Graph Transfer Learning
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity.
Fashion Editing with Adversarial Parsing Learning
Interactive fashion image manipulation, which enables users to edit images with sketches and color strokes, is an interesting research problem with great application value.
EnlightenGAN: Deep Light Enhancement without Paired Supervision
Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data?
Anatomy-aware 3D Human Pose Estimation with Bone-based Pose Decomposition
In this work, we propose a new solution to 3D human pose estimation in videos.
Ranked #12 on
Monocular 3D Human Pose Estimation
on Human3.6M
Human Motion Transfer from Poses in the Wild
In this paper, we tackle the problem of human motion transfer, where we synthesize novel motion video for a target person that imitates the movement from a reference video.
Unsupervised Low-light Image Enhancement with Decoupled Networks
In this paper, we tackle the problem of enhancing real-world low-light images with significant noise in an unsupervised fashion.
A Unified 3D Human Motion Synthesis Model via Conditional Variational Auto-Encoder
Notably, by considering this problem as a conditional generation process, we estimate a parametric distribution of the missing regions based on the input conditions, from which to sample and synthesize the full motion series.
DynOcc: Learning Single-View Depth from Dynamic Occlusion Cues
Recently, significant progress has been made in single-view depth estimation thanks to increasingly large and diverse depth datasets.
Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis
In this paper, we explore open-domain sketch-to-photo translation, which aims to synthesize a realistic photo from a freehand sketch with its class label, even if the sketches of that class are missing in the training data.
Ranked #1 on
Sketch-to-Image Translation
on Scribble
Semantic StyleGAN
SemanticStyleGAN presents a method where a generator is trained to model local semantic parts separately and synthesizes images in a compositional way.
Video Salient Object Detection via Contrastive Features and Attention Modules
Video salient object detection aims to find the most visually distinctive objects in a video.
SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing
When combined with editing methods designed for StyleGANs, it can achieve a more fine-grained control to edit synthesized or real images.
Contrastive Masked Autoencoders are Stronger Vision Learners
The momentum encoder, fed with the full images, enhances the feature discriminability via contrastive learning with its online counterpart.
R2Former: Unified Retrieval and Reranking Transformer for Place Recognition
Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database.
MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval
The first is a Temporal Adaptation Module that is incorporated in the video branch to introduce global and local temporal contexts.
$R^{2}$Former: Unified $R$etrieval and $R$eranking Transformer for Place Recognition
Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database.
Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP
The proposed FC-CLIP, benefits from the following observations: the frozen CLIP backbone maintains the ability of open-vocabulary classification and can also serve as a strong mask generator, and the convolutional CLIP generalizes well to a larger input resolution than the one used during contrastive image-text pretraining.
Ranked #1 on
Open Vocabulary Semantic Segmentation
on Cityscapes
Open Vocabulary Panoptic Segmentation
Open Vocabulary Semantic Segmentation
+1
Towards Open-Ended Visual Recognition with Large Language Model
Localizing and recognizing objects in the open-ended physical world poses a long-standing challenge within the domain of machine perception.
MaXTron: Mask Transformer with Trajectory Attention for Video Panoptic Segmentation
To alleviate the issue, we propose to adapt the trajectory attention for both the dense pixel features and object queries, aiming to improve the short-term and long-term tracking results, respectively.
Ranked #1 on
Video Panoptic Segmentation
on VIPSeg
ViTamin: Designing Scalable Vision Models in the Vision-Language Era
To this end, we introduce ViTamin, a new vision models tailored for VLMs.
COCONut: Modernizing COCO Segmentation
By enhancing the annotation quality and expanding the dataset to encompass 383K images with more than 5. 18M panoptic masks, we introduce COCONut, the COCO Next Universal segmenTation dataset.
Video Object Detection via Object-level Temporal Aggregation
While single-image object detectors can be naively applied to videos in a frame-by-frame fashion, the prediction is often temporally inconsistent.
Learning Progressive Joint Propagation for Human Motion Prediction
Last, in order to incorporate a general motion space for high-quality prediction, we build a memory-based dictionary, which aims to preserve the global motion patterns in training data to guide the predictions.
