Search Results for author:
Found 59 papers, 21 papers with code
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click
We propose to employ phrase expressions as another interaction input to infer the attributes of target object.
MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis
MetaShadow combines the strengths of two cooperative components: Shadow Analyzer, for object-centered shadow detection and removal, and Shadow Synthesizer, for reference-based controllable shadow synthesis.
FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication
Recent dataset deduplication techniques have demonstrated that content-aware dataset pruning can dramatically reduce the cost of training Vision-Language Pretrained (VLP) models without significant performance losses compared to training on the original dataset.
FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction
To address this, we propose FineMatch, a new aspect-based fine-grained text and image matching benchmark, focusing on text and image mismatch detection and correction.
IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation
Generative object compositing emerges as a promising new avenue for compositional image editing.
Latent Feature-Guided Diffusion Models for Shadow Removal
Recovering textures under shadows has remained a challenging problem due to the difficulty of inferring shadow-free scenes from shadow images.
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data
To solve this problem, we propose an auto-regressive model that given a subject, it predicts its relations, objects, and object locations by casting this output as a sequence of tokens.
GamutMLP: A Lightweight MLP for Color Loss Recovery
Inspired by neural implicit representations for 2D images, we propose a method that optimizes a lightweight multi-layer-perceptron (MLP) model during the gamut reduction step to predict the clipped values.
TopNet: Transformer-based Object Placement Network for Image Compositing
Given a background image and a segmented object, the goal is to train a model to predict plausible placements (location and scale) of the object for compositing.
ObjectStitch: Object Compositing With Diffusion Model
Object compositing based on 2D images is a challenging problem since it typically involves multiple processing stages such as color harmonization, geometry correction and shadow generation to generate realistic results.
Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators
Moreover, the object-level discriminators take aligned instances as inputs to enforce the realism of individual objects.
ObjectStitch: Generative Object Compositing
Object compositing based on 2D images is a challenging problem since it typically involves multiple processing stages such as color harmonization, geometry correction and shadow generation to generate realistic results.
GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing
To move a step further, this paper proposes GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open-world image compositing.
CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training
We propose cascaded modulation GAN (CM-GAN), a new network design consisting of an encoder with Fourier convolution blocks that extract multi-scale feature representations from the input image with holes and a dual-stream decoder with a novel cascaded global-spatial modulation block at each scale level.
Ranked #3 on
Image Inpainting
on Places2
Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning
Generalized few-shot semantic segmentation was introduced to move beyond only evaluating few-shot segmentation models on novel classes to include testing their ability to remember base classes.
Learning to Predict Visual Attributes in the Wild
In this paper, we introduce a large-scale in-the-wild visual attribute prediction dataset consisting of over 927K attribute annotations for over 260K object instances.
AESOP: Abstract Encoding of Stories, Objects, and Pictures
Visual storytelling and story comprehension are uniquely human skills that play a central role in how we learn about and experience the world.
Semantic Layout Manipulation with High-Resolution Sparse Attention
A core problem of this task is how to transfer visual details from the input images to the new semantic layout while making the resulting image visually realistic.
PhraseCut: Language-based Image Segmentation in the Wild
We consider the problem of segmenting image regions given a natural language phrase, and study it on a novel dataset of 77, 262 images and 345, 486 phrase-region pairs.
Ranked #4 on
Referring Expression Segmentation
on PhraseCut
Objectness-Aware Few-Shot Semantic Segmentation
We demonstrate how to increase overall model capacity to achieve improved performance, by introducing objectness, which is class-agnostic and so not prone to overfitting, for complementary use with class-specific features.
DeepStrip: High Resolution Boundary Refinement
In this paper, we target refining the boundaries in high resolution images given low resolution masks.
Getting to 99% Accuracy in Interactive Segmentation
We propose a novel interactive architecture and a novel training scheme that are both tailored to better exploit the user workflow.
Deep Visual Template-Free Form Parsing
Automatic, template-free extraction of information from form images is challenging due to the variety of form layouts.
Unconstrained Foreground Object Search
Many people search for foreground objects to use when editing images.
Answering Questions about Data Visualizations using Efficient Bimodal Fusion
Chart question answering (CQA) is a newly proposed visual question answering (VQA) task where an algorithm must answer questions about data visualizations, e. g. bar charts, pie charts, and line graphs.
Figure Captioning with Reasoning and Sequence-Level Training
In this work, we investigate the problem of figure captioning where the goal is to automatically generate a natural language description of the figure.
Image Recoloring Based on Object Color Distributions
We present a method to perform automatic image recoloring based on the distribution of colors associated with objects present in an image.
YouTube-VOS: Sequence-to-Sequence Video Object Segmentation
End-to-end sequential learning to explore spatial-temporal features for video segmentation is largely limited by the scale of available video segmentation datasets, i. e., even the largest video segmentation dataset only contains 90 short video clips.
Ranked #12 on
Video Object Segmentation
on YouTube-VOS 2018
(F-Measure (Unseen) metric)
Start, Follow, Read: End-to-End Full-Page Handwriting Recognition
Despite decades of research, offline handwriting recognition (HWR) of degraded historical documents remains a challenging problem, which if solved could greatly improve the searchability of online cultural heritage archives.
Ranked #13 on
Handwritten Text Recognition
on IAM
Interactive Boundary Prediction for Object Selection
Instead of relying on pre-defined low-level image features, our method adaptively predicts object boundaries according to image content and user interactions.
Concept Mask: Large-Scale Segmentation from Semantic Concepts
Then, we refine and extend the embedding network to predict an attention map, using a curated dataset with bounding box annotations on 750 concepts.
Guided Image Inpainting: Replacing an Image Region by Pulling Content from Another Image
Deep generative models have shown success in automatically synthesizing missing image regions using surrounding context.
Discriminability objective for training descriptive captions
One property that remains lacking in image captions generated by contemporary methods is discriminability: being able to tell two images apart given the caption for one of them.
DVQA: Understanding Data Visualizations via Question Answering
Bar charts are an effective way to convey numeric information, but today's algorithms cannot parse them.
Group-Theme Recoloring for Multi-Image Color Consistency
Methods such as color transfer are effective in making an image share similar colors with a target image; however, color transfer is not suitable for modifying multiple images.
Deep GrabCut for Object Selection
In this paper, we propose a novel segmentation approach that uses a rectangle as a soft constraint by transforming it into an Euclidean distance map.
Depth From Defocus in the Wild
We consider the problem of two-frame depth from defocus in conditions unsuitable for existing methods yet typical of everyday photography: a handheld cellphone camera, a small aperture, a non-stationary scene and sparse surface texture.
Relationship Proposal Networks
We demonstrate the ability of our Rel-PN to localize relationships with only a few thousand proposals.
Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition
Furthermore, our algorithm can generate descriptions with varied length, benefiting from the separate control of the skeleton and attributes.
Forecasting Human Dynamics from Static Images
This paper presents the first study on forecasting human dynamics from static images.
Deep Image Matting
We evaluate our algorithm on the image matting benchmark, our testing set, and a wide variety of real images.
SURGE: Surface Regularized Geometry Estimation from a Single Image
This paper introduces an approach to regularize 2. 5D surface normal and depth predictions at each pixel given a single input image.
Progressive Attention Networks for Visual Attribute Prediction
We propose a novel attention model that can accurately attends to target objects of various scales and shapes in images.
Two Illuminant Estimation and User Correction Preference
Existing methods attempt to estimate a spatially varying illumination map, however, results are error prone and the resulting illumination maps are too low-resolution to be used for proper spatially varying white-balance correction.
Interactive Segmentation on RGBD Images via Cue Selection
While these methods achieve better results than color-based methods, they are still limited in either using depth as an additional color channel or simply combining depth with color in a linear way.
Automatic Annotation of Structured Facts in Images
Motivated by the application of fact-level image understanding, we present an automatic method for data collection of structured visual facts from images with captions.
Object Contour Detection with a Fully Convolutional Encoder-Decoder Network
We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network.
Deep Interactive Object Selection
Interactive object selection is a very important research problem and has many applications.
Ranked #11 on
Interactive Segmentation
on SBD
Beyond White: Ground Truth Colors for Color Constancy Correction
A limitation in color constancy research is the inability to establish ground truth colors for evaluating corrected images.
Sherlock: Scalable Fact Learning in Images
We show that learning visual facts in a structured way enables not only a uniform but also generalizable visual understanding.
Towards Unified Depth and Semantic Prediction From a Single Image
By allowing for interactions between the depth and semantic information, the joint network provides more accurate depth prediction than a state-of-the-art CNN trained solely for depth prediction [5].
Effective Learning-Based Illuminant Estimation Using Simple Features
More recent state-of-the-art methods employ learning-based techniques that produce better results, but often rely on complex features and have long evaluation and training times.
PatchCut: Data-Driven Object Segmentation via Local Shape Transfer
The transferred local shape masks constitute a patch-level segmentation solution space and we thus develop a novel cascade algorithm, PatchCut, for coarse-to-fine object segmentation.
Joint Object and Part Segmentation using Deep Learned Potentials
Segmenting semantic objects from images and parsing them into their respective semantic parts are fundamental steps towards detailed object understanding in computer vision.
Context Driven Scene Parsing with Attention to Rare Classes
This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching.
Semantic Object Selection
With the tag provided by the user we do a text query of an image database to gather exemplars of the object.
Large Displacement Optical Flow from Nearest Neighbor Fields
We use approximate nearest neighbor fields to compute an initial motion field and use a robust algorithm to compute a set of similarity transformations as the motion candidates for segmentation.
Improving Image Matting Using Comprehensive Sampling Sets
The first is that the range in which the foreground and background are sampled is often limited to such an extent that the true foreground and background colors are not present.
Fast Image Super-Resolution Based on In-Place Example Regression
Extensive experiments on benchmark and realworld images demonstrate that our algorithm can produce natural-looking results with sharp edges and preserved fine details, while the current state-of-the-art algorithms are prone to visual artifacts.
