Search Results for author:
Found 48 papers, 22 papers with code
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
We introduce Florence-2, a novel vision foundation model with a unified, prompt-based representation for a variety of computer vision and vision-language tasks.
MM-VID: Advancing Video Understanding with GPT-4V(ision)
We present MM-VID, an integrated system that harnesses the capabilities of GPT-4V, combined with specialized tools in vision, audio, and speech, to facilitate advanced video understanding.
Self-Enforced Job Matching
The classic two-sided many-to-one job matching model assumes that firms treat workers as substitutes and workers ignore colleagues when choosing where to work.
Indiscernible Object Counting in Underwater Scenes
We further advance the frontier of this field by systematically studying a new challenge named indiscernible object counting (IOC), the goal of which is to count objects that are blended with respect to their surroundings.
Single Image Depth Prediction Made Better: A Multivariate Gaussian Take
Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution.
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action
We propose MM-REACT, a system paradigm that integrates ChatGPT with a pool of vision experts to achieve multimodal reasoning and action.
Ranked #6 on
Visual Question Answering
on MM-Vet
VA-DepthNet: A Variational Approach to Single Image Depth Prediction
While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene.
Ranked #10 on
Monocular Depth Estimation
on NYU-Depth V2
Learning Customized Visual Models with Retrieval-Augmented Knowledge
Image-text contrastive learning models such as CLIP have demonstrated strong task transfer ability.
Ranked #1 on
Semi-Supervised Image Classification
on ImageNet - 10% labeled data
(using extra training data)
X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion
We demonstrate for the first time that using a text2image model to generate images or zero-shot recognition model to filter noisily crawled images for different object categories is a feasible way to make Copy-Paste truly scalable.
Ranked #6 on
Instance Segmentation
on LVIS v1.0 val
ReCo: Region-Controlled Text-to-Image Generation
Human evaluation on PaintSkill shows that ReCo is +19. 28% and +17. 21% more accurate in generating images with correct object count and spatial relationship than the T2I model.
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
This paper presents OmniVL, a new foundation model to support both image-language and video-language tasks using one universal architecture.
Ranked #4 on
Cross-Modal Retrieval
on Flickr30k
(using extra training data)
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
Vision-language (VL) pre-training has recently received considerable attention.
Ranked #1 on
Phrase Grounding
on Flickr30k Entities Dev
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling
In this work, we explore a unified VidL framework LAVENDER, where Masked Language Modeling (MLM) is used as the common interface for all pre-training and downstream tasks.
Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning
Thanks to the strong zero-shot capability of foundation models, we start by constructing a rich semantic representation of the image (e. g., image tags, object attributes / locations, captions) as a structured textual prompt, called visual clues, using a vision foundation model.
GIT: A Generative Image-to-text Transformer for Vision and Language
In this paper, we design and train a Generative Image-to-text Transformer, GIT, to unify vision-language tasks such as image/video captioning and question answering.
Ranked #1 on
Image Captioning
on nocaps near-domain
K-LITE: Learning Transferable Visual Models with External Knowledge
We propose K-LITE, a simple strategy to leverage external knowledge for building transferable visual systems: In training, it enriches entities in text with WordNet and Wiktionary knowledge, leading to an efficient and scalable approach to learning image representations that uses knowledge about the visual concepts.
Unified Contrastive Learning in Image-Text-Label Space
Particularly, it attains gains up to 9. 2% and 14. 5% in average on zero-shot recognition benchmarks over the language-image contrastive learning and supervised learning methods, respectively.
MaskGIT: Masked Generative Image Transformer
At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation.
Ranked #1 on
Image Outpainting
on LHQC
(Block-FID (Right Extend) metric)
ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction
The surface embeddings are implemented as coordinate-based MLPs that are fit to each video via consistency and contrastive reconstruction losses. Experimental results show that ViSER compares favorably against prior work on challenging videos of humans with loose clothing and unusual poses as well as animals videos from DAVIS and YTVOS.
Pyramid Adversarial Training Improves ViT Performance
In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance.
Ranked #6 on
Domain Generalization
on ImageNet-C
(using extra training data)
Florence: A New Foundation Model for Computer Vision
Computer vision foundation models, which are trained on diverse, large-scale dataset and can be adapted to a wide range of downstream tasks, are critical for this mission to solve real-world computer vision applications.
Ranked #1 on
Action Recognition In Videos
on Kinetics-600
Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition
Decomposing a scene into its shape, reflectance and illumination is a fundamental problem in computer vision and graphics.
SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting
We present SLIDE, a modular and unified system for single image 3D photography that uses a simple yet effective soft layering strategy to better preserve appearance details in novel views.
ViTGAN: Training GANs with Vision Transformers
Recently, Vision Transformers (ViTs) have shown competitive performance on image recognition while requiring less vision-specific inductive biases.
Ranked #61 on
Image Generation
on CIFAR-10
OCONet: Image Extrapolation by Object Completion
Existing methods struggle to extrapolate images with salient objects in the foreground or are limited to very specific objects such as humans, but tend to work well on indoor/outdoor scenes.
LASR: Learning Articulated Shape Reconstruction from a Monocular Video
Remarkable progress has been made in 3D reconstruction of rigid structures from a video or a collection of images.
COMISR: Compression-Informed Video Super-Resolution
Most video super-resolution methods focus on restoring high-resolution video frames from low-resolution videos without taking into account compression.
AutoFlow: Learning a Better Training Set for Optical Flow
Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications.
Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and Extracting Them from 2D Renderings
Digital watermarking is widely used for copyright protection.
Regularizing Generative Adversarial Networks under Limited Data
Recent years have witnessed the rapid progress of generative adversarial networks (GANs).
Ranked #1 on
Image Generation
on 25% ImageNet 128x128
NeRD: Neural Reflectance Decomposition from Image Collections
This problem is inherently more challenging when the illumination is not a single light source under laboratory conditions but is instead an unconstrained environmental illumination.
Robust image stitching with multiple registrations
Here, we observe that the use of a single registration often leads to errors, especially in scenes with significant depth variation or object motion.
Stability in Repeated Matching Markets
This paper develops a framework for repeated matching markets.
Supervised Contrastive Learning
Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models.
Ranked #2 on
Class Incremental Learning
on cifar100
Depth Extraction from Video Using Non-parametric Sampling
We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling.
DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling.
Boundless: Generative Adversarial Networks for Image Extension
Image extension models have broad applications in image editing, computational photography and computer graphics.
Ranked #2 on
Uncropping
on Places2 val
Learning the Depths of Moving People by Watching Frozen People
We present a method for predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving.
Sparse, Smart Contours to Represent and Edit Images
We study the problem of reconstructing an image from information stored at contour locations.
Smart, Sparse Contours to Represent and Edit Images
We study the problem of reconstructing an image from information stored at contour locations.
On the Effectiveness of Visible Watermarks
Since such an attack relies on the consistency of watermarks across image collection, we explore and evaluate how it is affected by various types of inconsistencies in the watermark embedding that could potentially be used to make watermarking more secured.
Deep Convolutional Neural Network for Image Deconvolution
Many fundamental image-related problems involve deconvolution operators.
Ranked #1 on
Image Compression
on FER2013
Local Layering for Joint Motion Estimation and Occlusion Detection
To handle such situations, we propose a local layering model where motion and occlusion relationships are inferred jointly.
A Compositional Model for Low-Dimensional Image Set Representation
Learning a low-dimensional representation of images is useful for various applications in graphics and computer vision.
Unsupervised Joint Object Discovery and Segmentation in Internet Images
In contrast to previous co-segmentation methods, our algorithm performs well even in the presence of significant amounts of noise images (images not containing a common object), as typical for datasets collected from Internet search.
Probabilistic Label Trees for Efficient Large Scale Image Classification
Large-scale recognition problems with thousands of classes pose a particular challenge because applying the classifier requires more computation as the number of classes grows.
Deformable Spatial Pyramid Matching for Fast Dense Correspondences
We introduce a fast deformable spatial pyramid (DSP) matching algorithm for computing dense pixel correspondences.
