Search Results for author:
Found 46 papers, 27 papers with code
NICE: Neural Image Commenting with Empathy
Based on these annotations, we define two different tasks for the NICE dataset.
Learning 3D Photography Videos via Self-supervised Diffusion on Single Images
3D photography renders a static image into a video with appealing 3D visual effects.
NP-Match: Towards a New Probabilistic Model for Semi-Supervised Learning
In this work, we adjust neural processes (NPs) to the semi-supervised image classification task, resulting in a new method named NP-Match.
Generalized Decoding for Pixel, Image, and Language
We present X-Decoder, a generalized decoding model that can predict pixel-level segmentation and language tokens seamlessly.
Ranked #3 on
Instance Segmentation
on ADE20K val
(using extra training data)
GRiT: A Generative Region-to-text Transformer for Object Understanding
Specifically, GRiT consists of a visual encoder to extract image features, a foreground object extractor to localize objects, and a text decoder to generate open-set object descriptions.
Ranked #1 on
Dense Captioning
on Visual Genome
ReCo: Region-Controlled Text-to-Image Generation
Human evaluation on PaintSkill shows that ReCo is +19. 28% and +17. 21% more accurate in generating images with correct object count and spatial relationship than the T2I model.
Exploring Discrete Diffusion Models for Image Captioning
The image captioning task is typically realized by an auto-regressive method that decodes the text tokens one by one.
Prompting GPT-3 To Be Reliable
While reliability is a broad and vaguely defined term, we decompose reliability into four main facets that correspond to the existing framework of ML safety and are well-recognized to be important: generalizability, social biases, calibration, and factuality.
NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis
In this paper, we present NUWA-Infinity, a generative model for infinite visual synthesis, which is defined as the task of generating arbitrarily-sized high-resolution images or long-duration videos.
Ranked #1 on
Image Outpainting
on LHQC
NP-Match: When Neural Processes meet Semi-Supervised Learning
Semi-supervised learning (SSL) has been widely explored in recent years, and it is an effective way of leveraging unlabeled data to reduce the reliance on labeled data.
Rethinking Bayesian Deep Learning Methods for Semi-Supervised Volumetric Medical Image Segmentation
Secondly, in fact, they are only partially based on Bayesian deep learning, as their overall architectures are not designed under the Bayesian framework.
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
Vision-language (VL) pre-training has recently received considerable attention.
Ranked #1 on
Phrase Grounding
on Flickr30k Entities Dev
Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding
We design a detection hub to dynamically adapt queries on category embedding based on the different distributions of datasets.
GIT: A Generative Image-to-text Transformer for Vision and Language
In this paper, we design and train a Generative Image-to-text Transformer, GIT, to unify vision-language tasks such as image/video captioning and question answering.
Ranked #1 on
Image Captioning
on nocaps-XD out-of-domain
The Overlooked Classifier in Human-Object Interaction Recognition
Human-Object Interaction (HOI) recognition is challenging due to two factors: (1) significant imbalance across classes and (2) requiring multiple labels per image.
The Overlooked Classifier in Human-Object Interaction Recognition
Human-Object Interaction (HOI) recognition is challenging due to two factors: (1) significant imbalance across classes and (2) requiring multiple labels per image.
Ranked #1 on
Human-Object Interaction Detection
on HICO
Injecting Semantic Concepts into End-to-End Image Captioning
In this paper, we are concerned with a better-performing detector-free image captioning model, and propose a pure vision transformer-based image captioning model, dubbed as ViTCAP, in which grid representations are used without extracting the regional features.
Scaling Up Vision-Language Pre-training for Image Captioning
In this paper, we present LEMON, a LargE-scale iMage captiONer, and provide the first empirical study on the scaling behavior of VLP for image captioning.
Ranked #3 on
Image Captioning
on nocaps-XD entire
(using extra training data)
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling
On grounded captioning, UniTAB presents a simpler solution with a single output head, and significantly outperforms state of the art in both grounding and captioning evaluations.
Florence: A New Foundation Model for Computer Vision
Computer vision foundation models, which are trained on diverse, large-scale dataset and can be adapted to a wide range of downstream tasks, are critical for this mission to solve real-world computer vision applications.
Ranked #1 on
Action Recognition In Videos
on Kinetics-600
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning
In this paper, we propose a single UniFied transfOrmer (UFO), which is capable of processing either unimodal inputs (e. g., image or language) or multimodal inputs (e. g., the concatenation of the image and the question), for vision-language (VL) representation learning.
An Empirical Study of Training End-to-End Vision-and-Language Transformers
Vision-and-language (VL) pre-training has proven to be highly effective on various VL downstream tasks.
Ranked #15 on
Cross-Modal Retrieval
on COCO 2014
Edge Prior Augmented Networks for Motion Deblurring on Naturally Blurry Images
Motion deblurring has witnessed rapid development in recent years, and most of the recent methods address it by using deep learning techniques, with the help of different kinds of prior knowledge.
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA
To address this challenge, we propose PICa, a simple yet effective method that Prompts GPT3 via the use of Image Captions, for knowledge-based VQA.
Ranked #10 on
Visual Question Answering
on OK-VQA
Is Object Detection Necessary for Human-Object Interaction Recognition?
This paper revisits human-object interaction (HOI) recognition at image level without using supervisions of object location and human pose.
RSG: A Simple but Effective Module for Learning Imbalanced Datasets
Imbalanced datasets widely exist in practice and area great challenge for training deep neural models with agood generalization on infrequent classes.
Ranked #15 on
Long-tail Learning
on Places-LT
End-to-End Semi-Supervised Object Detection with Soft Teacher
This paper presents an end-to-end semi-supervised object detection approach, in contrast to previous more complex multi-stage methods.
Ranked #4 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
Convolutional Neural Networks with Gated Recurrent Connections
The critical element of RCNN is the recurrent convolutional layer (RCL), which incorporates recurrent connections between neurons in the standard convolutional layer.
Compressing Visual-linguistic Model via Knowledge Distillation
In this paper, we study knowledge distillation (KD) to effectively compress a transformer-based large VL model into a small VL model.
DAP: Detection-Aware Pre-training with Weak Supervision
This paper presents a detection-aware pre-training (DAP) approach, which leverages only weakly-labeled classification-style datasets (e. g., ImageNet) for pre-training, but is specifically tailored to benefit object detection tasks.
Adversarial Feature Augmentation and Normalization for Visual Recognition
Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models.
On graphs with exactly one anti-adjacency eigenvalue and beyond
This matrix can be interpreted as the opposite of the adjacency matrix, which is instead constructed from the distance matrix of a graph by keeping in each row and each column only the distances equal to 1.
Combinatorics 05C50
LLA: Loss-aware Label Assignment for Dense Pedestrian Detection
A joint loss is then defined as the weighted summation of cls and reg losses as the assigning indicator.
SEED: Self-supervised Distillation For Visual Representation
This paper is concerned with self-supervised learning for small models.
Orthogonal Subspace Decomposition: A New Perspective of Learning Discriminative Features for Face Clustering
Learning discriminative node features is the key to further improve the performance of graph-based face clustering.
The Hoffman program of graphs: old and new
The Hoffman program with respect to any real or complex square matrix $M$ associated to a graph $G$ stems from A. J. Hoffman's pioneering work on the limit points for the spectral radius of adjacency matrices of graphs less than $\sqrt{2+\sqrt{5}}$.
Combinatorics 05C50
MiniVLM: A Smaller and Faster Vision-Language Model
We design a Two-stage Efficient feature Extractor (TEE), inspired by the one-stage EfficientDet network, to significantly reduce the time cost of visual feature extraction by $95\%$, compared to a baseline model.
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption
Due to this aligned representation learning, even pre-trained on the same downstream task dataset, TAP already boosts the absolute accuracy on the TextVQA dataset by +5. 4%, compared with a non-TAP baseline.
End-to-End Object Detection with Fully Convolutional Network
Mainstream object detectors based on the fully convolutional network has achieved impressive performance.
Hashing-based Non-Maximum Suppression for Crowded Object Detection
Considering the intersection-over-union (IoU) as the metric, we propose a simple yet effective hashing algorithm, named IoUHash, which guarantees that the boxes within the same cell are close enough by a lower IoU bound.
Learning to Count Objects with Few Exemplar Annotations
In this paper, we study the problem of object counting with incomplete annotations.
SFace: An Efficient Network for Face Detection in Large Scale Variations
A new dataset called 4K-Face is also introduced to evaluate the performance of face detection with extreme large scale variations.
Gated Recurrent Convolution Neural Network for OCR
Its critical component, Gated Recurrent Convolution Layer (GRCL), is constructed by adding a gate to the Recurrent Convolution Layer (RCL), the critical component of RCNN.
Face Attention Network: An Effective Face Detector for the Occluded Faces
The performance of face detection has been largely improved with the development of convolutional neural network.
Ranked #1 on
Occluded Face Detection
on MAFA
Group $K$-Means
We study how to learn multiple dictionaries from a dataset, and approximate any data point by the sum of the codewords each chosen from the corresponding dictionary.
Optimized Cartesian $K$-Means
In OCKM, multiple sub codewords are used to encode the subvector of a data point in a subspace.
