Search Results for author:
Found 145 papers, 64 papers with code
MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation
Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures.
Mutual Consistency Learning for Semi-supervised Medical Image Segmentation
In this paper, we propose a novel mutual consistency network (MC-Net+) to effectively exploit the unlabeled data for semi-supervised medical image segmentation.
Exploring Smoothness and Class-Separation for Semi-supervised Medical Image Segmentation
The pixel-level smoothness forces the model to generate invariant results under adversarial perturbations.
Scalable Vision Transformers with Hierarchical Pooling
However, the routine of the current ViT model is to maintain a full-length patch sequence during inference, which is redundant and lacks hierarchical representation.
Ranked #22 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
GMFlow: Learning Optical Flow via Global Matching
Learning-based optical flow estimation has been dominated with the pipeline of cost volume with convolutions for flow regression, which is inherently limited to local correlations and thus is hard to address the long-standing challenge of large displacements.
Ranked #8 on
Optical Flow Estimation
on Spring
Unifying Flow, Stereo and Depth Estimation
We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images.
Ranked #1 on
Optical Flow Estimation
on Sintel-clean
Pluralistic Image Completion
In this paper, we present an approach for \textbf{pluralistic image completion} -- the task of generating multiple and diverse plausible solutions for image completion.
3D Hand Shape and Pose Estimation from a Single RGB Image
This work addresses a novel and challenging problem of estimating the full 3D hand shape and pose from a single RGB image.
Fast Vision Transformers with HiLo Attention
Therefore, we propose to disentangle the high/low frequency patterns in an attention layer by separating the heads into two groups, where one group encodes high frequencies via self-attention within each local window, and another group encodes low frequencies by performing global attention between the average-pooled low-frequency keys and values from each window and each query position in the input feature map.
Ranked #281 on
Image Classification
on ImageNet
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
We propose MVSplat, an efficient feed-forward 3D Gaussian Splatting model learned from sparse multi-view images.
Ranked #1 on
Generalizable Novel View Synthesis
on ACID
Semi-supervised Left Atrium Segmentation with Mutual Consistency Training
Such mutual consistency encourages the two decoders to have consistent and low-entropy predictions and enables the model to gradually capture generalized features from these unlabeled challenging regions.
Stitchable Neural Networks
As each model family consists of pretrained models with diverse scales (e. g., DeiT-Ti/S/B), it naturally arises a fundamental question of how to efficiently assemble these readily available models in a family for dynamic accuracy-efficiency trade-offs at runtime.
Auto-Encoding Scene Graphs for Image Captioning
We propose Scene Graph Auto-Encoder (SGAE) that incorporates the language inductive bias into the encoder-decoder image captioning framework for more human-like captions.
MARLIN: Masked Autoencoder for facial video Representation LearnINg
This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS).
Ranked #1 on
Emotion Classification
on CMU-MOSEI
Object-Compositional Neural Implicit Surfaces
This paper proposes a novel framework, ObjectSDF, to build an object-compositional neural implicit representation with high fidelity in 3D reconstruction and object representation.
T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks
Current methods for single-image depth estimation use training datasets with real image-depth pairs or stereo pairs, which are not easy to acquire.
Ranked #3 on
Depth Estimation
on eBDtheque
Learning Object-Language Alignments for Open-Vocabulary Object Detection
In this paper, we propose a novel open-vocabulary object detection framework directly learning from image-text pair data.
Explicit Correspondence Matching for Generalizable Neural Radiance Fields
The key to our approach lies in the explicitly modeled correspondence matching information, so as to provide the geometry prior to the prediction of NeRF color and density for volume rendering.
Bridging Global Context Interactions for High-Fidelity Image Completion
Bridging global context interactions correctly is important for high-fidelity image completion with large masks.
Ranked #2 on
Image Inpainting
on FFHQ 512 x 512
ObjectSDF++: Improved Object-Compositional Neural Implicit Surfaces
Unlike traditional multi-view stereo approaches, the neural implicit surface-based methods leverage neural networks to represent 3D scenes as signed distance functions (SDFs).
Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fields
In light of recent advances in NeRF-based 3D-aware generative models, we introduce a new task, Semantic-to-NeRF translation, that aims to reconstruct a 3D scene modelled by NeRF, conditioned on one single-view semantic mask as input.
Ranked #1 on
3D-Aware Image Synthesis
on CelebAMask-HQ
RSG: A Simple but Effective Module for Learning Imbalanced Datasets
Imbalanced datasets widely exist in practice and area great challenge for training deep neural models with agood generalization on infrequent classes.
Ranked #17 on
Long-tail Learning
on Places-LT
Mesa: A Memory-saving Training Framework for Transformers
While Transformers have delivered significant performance improvements, training such networks is extremely memory intensive owing to storing all intermediate activations that are needed for gradient computation during backpropagation, especially for long sequences.
High-Resolution Optical Flow from 1D Attention and Correlation
Optical flow is inherently a 2D search problem, and thus the computational complexity grows quadratically with respect to the search window, making large displacements matching infeasible for high-resolution images.
Pruning Self-attentions into Convolutional Layers in Single Path
Relying on the single-path space, we introduce learnable binary gates to encode the operation choices in MSA layers.
Ranked #18 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
The Spatially-Correlative Loss for Various Image Translation Tasks
We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency while supporting large appearance changes during unpaired image-to-image (I2I) translation.
Facial Motion Prior Networks for Facial Expression Recognition
Deep learning based facial expression recognition (FER) has received a lot of attention in the past few years.
Ranked #2 on
Facial Expression Recognition (FER)
on MMI
Facial Expression Recognition
Facial Expression Recognition (FER)
J$\hat{\text{A}}$A-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention
Moreover, to extract precise local features, we propose an adaptive attention learning module to refine the attention map of each AU adaptively.
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
By "agnostic", we mean that the feature is less likely biased to the classes of paired objects.
Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
Facial action unit (AU) detection and face alignment are two highly correlated tasks since facial landmarks can provide precise AU locations to facilitate the extraction of meaningful local features for AU detection.
Ranked #5 on
Facial Action Unit Detection
on DISFA
Less is More: Pay Less Attention in Vision Transformers
Transformers have become one of the dominant architectures in deep learning, particularly as a powerful alternative to convolutional neural networks (CNNs) in computer vision.
HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression
3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis, boasting rapid rendering speed with high fidelity.
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model.
Generative Region-Language Pretraining for Open-Ended Object Detection
To address it, we formulate object detection as a generative problem and propose a simple framework named GenerateU, which can detect dense objects and generate their names in a free-form way.
End-to-end One-shot Human Parsing
To address three main challenges in OSHP, i. e., small sizes, testing bias, and similar parts, we devise an End-to-end One-shot human Parsing Network (EOP-Net).
EcoFormer: Energy-Saving Attention with Linear Complexity
To this end, we propose a new binarization paradigm customized to high-dimensional softmax attention via kernelized hashing, called EcoFormer, to map the original queries and keys into low-dimensional binary codes in Hamming space.
Alive Caricature from 2D to 3D
Caricature is an art form that expresses subjects in abstract, simple and exaggerated view.
Dynamic Focus-aware Positional Queries for Semantic Segmentation
In this paper, we propose a simple yet effective query design for semantic segmentation termed Dynamic Focus-aware Positional Queries (DFPQ), which dynamically generates positional queries conditioned on the cross-attention scores from the preceding decoder block and the positional encodings for the corresponding image features, simultaneously.
Ranked #21 on
Semantic Segmentation
on ADE20K
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
Visual Parameter-Efficient Fine-Tuning (PEFT) has become a powerful alternative for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only tunes a small number of parameters while freezing the vast majority ones to ease storage burden and optimization difficulty.
CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing
Generating an interpretable and compact representation of 3D shapes from point clouds is an important and challenging problem.
Disentangled Human Body Embedding Based on Deep Hierarchical Neural Network
Human bodies exhibit various shapes for different identities or poses, but the body shape has certain similarities in structure and thus can be embedded in a low-dimensional space.
Sharpness-aware Quantization for Deep Neural Networks
However, the abrupt changes in quantized weights during training often lead to severe loss fluctuations and result in a sharp loss landscape, making the gradients unstable and thus degrading the performance.
Ranked #196 on
Image Classification
on CIFAR-100
Taming Stable Diffusion for Text to 360° Panorama Image Generation
Generative models, e. g., Stable Diffusion, have enabled the creation of photorealistic images from text prompts.
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning
On the other hand, multi-stage image caption model is hard to train due to the vanishing gradient problem.
An Empirical Study of Language CNN for Image Captioning
Language Models based on recurrent neural networks have dominated recent image caption generation tasks.
Region Deformer Networks for Unsupervised Depth Estimation from Unconstrained Monocular Videos
In this paper, we propose a new learning based method consisting of DepthNet, PoseNet and Region Deformer Networks (RDN) to estimate depth from unconstrained monocular videos without ground truth supervision.
Unconstrained Facial Action Unit Detection via Latent Feature Domain
Due to the combination of source AU-related information and target AU-free information, the latent feature domain with transferred source label can be learned by maximizing the target-domain AU detection performance.
Diversified and Personalized Multi-rater Medical Image Segmentation
To address it, the common practice is to gather multiple annotations from different experts, leading to the setting of multi-rater medical image segmentation.
CoactSeg: Learning from Heterogeneous Data for New Multiple Sclerosis Lesion Segmentation
Moreover, a simple and effective relation regularization is proposed to ensure the longitudinal relations among the three outputs to improve the model learning.
ExtrudeNet: Unsupervised Inverse Sketch-and-Extrude for Shape Parsing
This paper studies the problem of learning the shape given in the form of point clouds by inverse sketch-and-extrude.
Learning Meta-class Memory for Few-Shot Semantic Segmentation
To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage.
Accurate and Real-time 3D Pedestrian Detection Using an Efficient Attentive Pillar Network
Efficiently and accurately detecting people from 3D point cloud data is of great importance in many robotic and autonomous driving applications.
Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition
In this work, we propose a higher-level scene understanding system to tackle both visible and invisible parts of objects and backgrounds in a given scene.
Image Captioning In the Transformer Age
This drawback inspires the researchers to develop a homogeneous architecture that facilitates end-to-end training, for which Transformer is the perfect one that has proven its huge potential in both vision and language domains and thus can be used as the basic component of the visual encoder and language decoder in an IC pipeline.
Stitched ViTs are Flexible Vision Backbones
With extensive experiments on ImageNet-1K, ADE20K, COCO-Stuff-10K and NYUv2, SN-Netv2 demonstrates superior performance over SN-Netv1 on downstream dense predictions and shows strong ability as a flexible vision backbone, achieving great advantages in both training efficiency and deployment flexibility.
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Additionally, an adaptive strategy is designed to autonomously determine the optimal number of sub-channels for channel disassembly.
Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a task that an agent is required to follow a language instruction to navigate to the goal position, which relies on the ongoing interactions with the environment during moving.
Towards Unbiased Visual Emotion Recognition via Causal Intervention
In this work, we scrutinize this problem from the perspective of causal inference, where such dataset characteristic is termed as a confounder which misleads the system to learn the spurious correlation.
Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression
In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts.
Reliability-Adaptive Consistency Regularization for Weakly-Supervised Point Cloud Segmentation
Weakly-supervised point cloud segmentation with extremely limited labels is highly desirable to alleviate the expensive costs of collecting densely annotated 3D points.
Learning to Collocate Visual-Linguistic Neural Modules for Image Captioning
This is because the language is only partially observable, for which we need to dynamically collocate the modules during the process of image captioning.
Efficient Stitchable Task Adaptation
In this work, we present a novel framework, Efficient Stitchable Task Adaptation (ESTA), to efficiently produce a palette of fine-tuned models that adhere to diverse resource constraints.
Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
Gaussian process classification (GPC) provides a flexible and powerful statistical framework describing joint distributions over function space.
Improving Deep Lesion Detection Using 3D Contextual and Spatial Attention
Secondly, in CT scans, the lesions are often indistinguishable from the background since the lesion and non-lesion areas may have very similar appearances.
Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models
Textual-visual cross-modal retrieval has been a hot research topic in both computer vision and natural language processing communities.
CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images
With the powerfulness of convolution neural networks (CNN), CNN based face reconstruction has recently shown promising performance in reconstructing detailed face shape from 2D face images.
VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions
Most existing works in visual question answering (VQA) are dedicated to improving the accuracy of predicted answers, while disregarding the explanations.
Ranked #4 on
Explanatory Visual Question Answering
on GQA-REX
Explanatory Visual Question Answering
Multi-Task Learning
+1
Conditional Adversarial Synthesis of 3D Facial Action Units
Employing deep learning-based approaches for fine-grained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training.
Unpaired Image Captioning by Language Pivoting
Image captioning is a multimodal task involving computer vision and natural language processing, where the goal is to learn a mapping from the image to its natural language description.
Decoupled Spatial Neural Attention for Weakly Supervised Semantic Segmentation
In our work, we focus on the weakly supervised semantic segmentation with image label annotations.
Zero-Shot Learning via Category-Specific Visual-Semantic Mapping
Zero-Shot Learning (ZSL) aims to classify a test instance from an unseen category based on the training instances from seen categories, in which the gap between seen categories and unseen categories is generally bridged via visual-semantic mapping between the low-level visual feature space and the intermediate semantic space.
Zero-Annotation Object Detection with Web Knowledge Transfer
Object detection is one of the major problems in computer vision, and has been extensively studied.
Recent Advances in Convolutional Neural Networks
In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing.
Exploiting Web Images for Weakly Supervised Object Detection
Object detection without bounding box annotations, i. e, weakly supervised detection methods, are still lagging far behind.
Ranked #17 on
Weakly Supervised Object Detection
on PASCAL VOC 2012 test
(using extra training data)
Enriched Deep Recurrent Visual Attention Model for Multiple Object Recognition
We design an Enriched Deep Recurrent Visual Attention Model (EDRAM) - an improved attention-based architecture for multiple object recognition.
WordFence: Text Detection in Natural Images with Border Awareness
In recent years, text recognition has achieved remarkable success in recognizing scanned document text.
MIML-FCN+: Multi-instance Multi-label Learning via Fully Convolutional Networks with Privileged Information
As the proposed PI loss is convex and SGD compatible and the framework itself is a fully convolutional network, MIML-FCN+ can be easily integrated with state of-the-art deep learning networks.
Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations
Experimental results demonstrate the effectiveness of the proposed semantic descriptor and the usefulness of incorporating the structured semantic correlations.
Exploit Bounding Box Annotations for Multi-label Object Recognition
With strong labels, our framework is able to achieve state-of-the-art results in both datasets.
Ranked #16 on
Multi-Label Classification
on PASCAL VOC 2007
Diagnosing State-Of-The-Art Object Proposal Methods
Recently, Hosang et al. conduct the first unified study of existing methods' in terms of various image-level degradations.
Robust Performance-driven 3D Face Tracking in Long Range Depth Scenes
We introduce a novel robust hybrid 3D face tracking framework from RGBD video streams, which is capable of tracking head pose and facial actions without pre-calibration or intervention from a user.
Weakly Supervised Fine-Grained Image Categorization
Most existing works heavily rely on object / part detectors to build the correspondence between object parts by using object or object part annotations inside training images.
Beyond Pixels: A Comprehensive Survey from Bottom-up to Semantic Image Segmentation and Cosegmentation
Image segmentation refers to the process to divide an image into nonoverlapping meaningful regions according to human perception, which has become a classic topic since the early ages of computer vision.
When Gaussian Process Meets Big Data: A Review of Scalable GPs
The review of scalable GPs in the GP community is timely and important due to the explosion of data size.
Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction
In this paper, we propose a boundary-aware hierarchical language decoder for video captioning, which consists of a high-level GRU based language decoder, working as a global (caption-level) language model, and a low-level GRU based language decoder, working as a local (phrase-level) language model.
Facial Action Unit Detection Using Attention and Relation Learning
By finding the region of interest of each AU with the attention mechanism, AU-related local features can be captured.
Keypoint Based Weakly Supervised Human Parsing
We develop an iterative learning method to generate pseudo part segmentation masks from keypoint labels.
Large-scale Heteroscedastic Regression via Gaussian Process
To improve the scalability, we first develop a variational sparse inference algorithm, named VSHGP, to handle large-scale datasets.
Understanding and Comparing Scalable Gaussian Process Regression for Big Data
This paper devotes to investigating the methodological characteristics and performance of representative global and local scalable GPs including sparse approximations and local aggregations from four main perspectives: scalability, capability, controllability and robustness.
M2E-Try On Net: Fashion from Model to Everyone
Instead, we present a novel virtual Try-On network, M2E-Try On Net, which transfers the clothes from a model image to a person image without the need of any clean product images.
Weakly-supervised 3D Hand Pose Estimation from Monocular RGB Images
Compared with depth-based 3D hand pose estimation, it is more challenging to infer 3D hand pose from monocular RGB images, due to substantial depth ambiguity and the difficulty of obtaining fully-annotated training data.
Quadtree Convolutional Neural Networks
Specifically, the computational and memory costs in QCNN grow linearly in the number of non-zero pixels, as opposed to traditional CNNs where the costs are quadratic in the number of pixels.
Real-time 3D Face-Eye Performance Capture of a Person Wearing VR Headset
Teleconference or telepresence based on virtual reality (VR) headmount display (HMD) device is a very interesting and promising application since HMD can provide immersive feelings for users.
Compact Representation for Image Classification: To Choose or to Compress?
In spite of the popularity of various feature compression methods, this paper argues that feature selection is a better choice than feature compression.
Recovering Surface Details under General Unknown Illumination Using Shading and Coarse Multi-view Stereo
As a result, our approach is robust, stable and is able to efficiently recover high quality of surface details even starting with a coarse MVS.
Modality and Component Aware Feature Fusion For RGB-D Scene Classification
While convolutional neural networks (CNN) have been excellent for object recognition, the greater spatial variability in scene images typically meant that the standard full-image CNN features are suboptimal for scene classification.
A Generative Model for Depth-Based Robust 3D Facial Pose Tracking
We consider the problem of depth-based robust 3D facial pose tracking under unconstrained scenarios with heavy occlusions and arbitrary facial expression variations.
Object Co-Skeletonization With Co-Segmentation
Recent advances in the joint processing of images have certainly shown its advantages over the individual processing.
MMSS: Multi-Modal Sharable and Specific Feature Learning for RGB-D Object Recognition
We first construct deep CNN layers for color and depth separately, and then connect them with our carefully designed multi-modal layers, which fuse color and depth information by enforcing a common part to be shared by features of different modalities.
Progress Regression RNN for Online Spatial-Temporal Action Localization in Unconstrained Videos
Previous spatial-temporal action localization methods commonly follow the pipeline of object detection to estimate bounding boxes and labels of actions.
Unpaired Image Captioning via Scene Graph Alignments
Most of current image captioning models heavily rely on paired image-caption datasets.
Scene Graph Generation with External Knowledge and Image Reconstruction
Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc.
Learning to Collocate Neural Modules for Image Captioning
To this end, we make the following technical contributions for CNM training: 1) compact module design --- one for function words and three for visual content words (eg, noun, adjective, and verb), 2) soft module fusion and multi-step module execution, robustifying the visual reasoning in partial observation, 3) a linguistic loss for module controller being faithful to part-of-speech collocations (eg, adjective is before noun).
Exploring Uncertainty Measures for Image-Caption Embedding-and-Retrieval Task
With the wide development of black-box machine learning algorithms, particularly deep neural network (DNN), the practical demand for the reliability assessment is rapidly rising.
Visibility Constrained Generative Model for Depth-based 3D Facial Pose Tracking
In this paper, we propose a generative framework that unifies depth-based 3D facial pose tracking and face model adaptation on-the-fly, in the unconstrained scenarios with heavy occlusions and arbitrary facial expression variations.
Watch It Twice: Video Captioning with a Refocused Video Encoder
With the rapid growth of video data and the increasing demands of various applications such as intelligent video search and assistance toward visually-impaired people, video captioning task has received a lot of attention recently in computer vision and natural language processing fields.
Recovering Facial Reflectance and Geometry from Multi-view Images
While the problem of estimating shapes and diffuse reflectances of human faces from images has been extensively studied, there is relatively less work done on recovering the specular albedo.
Facial Action Unit Detection via Adaptive Attention and Relation
Specifically, we propose an adaptive attention regression network to regress the global attention map of each AU under the constraint of attention predefinition and the guidance of AU detection, which is beneficial for capturing both specified dependencies by landmarks in strongly correlated regions and facial globally distributed dependencies in weakly correlated regions.
GeoConv: Geodesic Guided Convolution for Facial Action Unit Recognition
Automatic facial action unit (AU) recognition has attracted great attention but still remains a challenging task, as subtle changes of local facial muscles are difficult to thoroughly capture.
Deconfounded Image Captioning: A Causal Retrospect
Dataset bias in vision-language tasks is becoming one of the main problems which hinders the progress of our community.
Exploring Bottom-up and Top-down Cues with Attentive Learning for Webly Supervised Object Detection
To reduce the human labeling effort, we propose a novel webly supervised object detection (WebSOD) method for novel classes which only requires the web images without further annotations.
Image Co-skeletonization via Co-segmentation
Object skeletonization in a single natural image is a challenging problem because there is hardly any prior knowledge about the object.
Learning from the Scene and Borrowing from the Rich: Tackling the Long Tail in Scene Graph Generation
Despite the huge progress in scene graph generation in recent years, its long-tail distribution in object relationships remains a challenging and pestering issue.
Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
Some works in the optimization of deep neural networks have shown that a better arrangement of training data can make the classifier converge faster and perform better.
MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models
To interpret the teacher model and assist the learning of the student, an explainer module is introduced to highlight the regions of an input that are important for the predictions of the teacher model.
Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture
The problem of deforming an artist-drawn caricature according to a given normal face expression is of interest in applications such as social media, animation and entertainment.
Learning Progressive Joint Propagation for Human Motion Prediction
Last, in order to incorporate a general motion space for high-quality prediction, we build a memory-based dictionary, which aims to preserve the global motion patterns in training data to guide the predictions.
Splitting vs. Merging: Mining Object Regions with Discrepancy and Intersection Loss for Weakly Supervised Semantic Segmentation
Finally, by training the segmentation model with the masks generated by our Splitting vs Merging strategy, we achieve the state-of-the-art weakly-supervised segmentation results on the Pascal VOC 2012 benchmark.
Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning
In this paper, we propose a novel visual encoder to explicitly distinguish viewpoint changes from semantic changes in the change captioning task.
Self-Supervised Relationship Probing
Structured representations of images that model visual relationships are beneficial for many vision and vision-language applications.
Single-path Bit Sharing for Automatic Loss-aware Model Compression
By jointly training the binary gates in conjunction with network parameters, the compression configurations of each layer can be automatically determined.
Causal Attention for Vision-Language Tasks
Specifically, CATT is implemented as a combination of 1) In-Sample Attention (IS-ATT) and 2) Cross-Sample Attention (CS-ATT), where the latter forcibly brings other samples into every IS-ATT, mimicking the causal intervention.
Multi-Label Image Classification with Contrastive Learning
Recently, as an effective way of learning latent representations, contrastive learning has been increasingly popular and successful in various domains.
Remember What You have drawn: Semantic Image Manipulation with Memory
Therefore, in this paper, we propose a memory-based Image Manipulation Network (MIM-Net), where a set of memories learned from images is introduced to synthesize the texture information with the guidance of the textual description.
Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints
More critically, EAS is able to find compact architectures within 0. 1 second for 50 deployment scenarios.
Semantic Compositional Learning for Low-shot Scene Graph Generation
Scene graphs provide valuable information to many downstream tasks.
Auto-Parsing Network for Image Captioning and Visual Question Answering
We propose an Auto-Parsing Network (APN) to discover and exploit the input data's hidden tree structures for improving the effectiveness of the Transformer-based vision-language systems.
A Unified 3D Human Motion Synthesis Model via Conditional Variational Auto-Encoder
Notably, by considering this problem as a conditional generation process, we estimate a parametric distribution of the missing regions based on the input conditions, from which to sample and synthesize the full motion series.
Domain-Invariant Disentangled Network for Generalizable Object Detection
By disentangling representations on both image and instance levels, DIDN is able to learn domain-invariant representations that are suitable for generalized object detection.
Contrastively Enforcing Distinctiveness for Multi-Label Classification
Recently, as an effective way of learning latent representations, contrastive learning has been increasingly popular and successful in various domains.
ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cues
Firstly, we analyze CLIP for unsupervised open-category proposal generation and design an objectness score based on our empirical analysis on proposal selection.
Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching
To avoid the laborious annotation in conventional referring grounding, unpaired referring grounding is introduced, where the training data only contains a number of images and queries without correspondences.
High-Quality Pluralistic Image Completion via Code Shared VQGAN
In this work, we present a novel framework for pluralistic image completion that can achieve both high quality and diversity at much faster inference speed.
Transformer Scale Gate for Semantic Segmentation
Effectively encoding multi-scale contextual information is crucial for accurate semantic segmentation.
Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation
Weakly supervised point cloud segmentation, i. e. semantically segmenting a point cloud with only a few labeled points in the whole 3D scene, is highly desirable due to the heavy burden of collecting abundant dense annotations for the model training.
FocusFormer: Focusing on What We Need via Architecture Sampler
During architecture search, these methods focus on finding architectures on the Pareto frontier of performance and resource consumption, which forms a gap between training and deployment.
JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and Tracking
In crowded human scenes with close-up human-robot interaction and robot navigation, a deep understanding requires reasoning about human motion and body dynamics over time with human body pose estimation and tracking.
Multi-Person Pose Estimation
Multi-Person Pose Estimation and Tracking
+1
Class Enhancement Losses with Pseudo Labels for Zero-shot Semantic Segmentation
Recent mask proposal models have significantly improved the performance of zero-shot semantic segmentation.
Vector Quantized Wasserstein Auto-Encoder
Learning deep discrete latent presentations offers a promise of better symbolic and summarized abstractions that are more useful to subsequent downstream tasks.
Open-Vocabulary Object Detection via Scene Graph Discovery
However, they only use pairs of nouns and individual objects in VL data, while these data usually contain much more information, such as scene graphs, which are also crucial for OV detection.
Unified Open-Vocabulary Dense Visual Prediction
We present a UOVN training mechanism to reduce such gaps.
Cross-adversarial local distribution regularization for semi-supervised medical image segmentation
Medical semi-supervised segmentation is a technique where a model is trained to segment objects of interest in medical images with limited annotated data.
GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answering
Therefore, we propose GeReA, a generate-reason framework that prompts a MLLM like InstructBLIP with question relevant vision and language information to generate knowledge-relevant descriptions and reasons those descriptions for knowledge-based VQA.
JRDB-PanoTrack: An Open-world Panoptic Segmentation and Tracking Robotic Dataset in Crowded Human Environments
JRDB-PanoTrack includes (1) various data involving indoor and outdoor crowded scenes, as well as comprehensive 2D and 3D synchronized data modalities; (2) high-quality 2D spatial panoptic segmentation and temporal tracking annotations, with additional 3D label projections for further spatial understanding; (3) diverse object classes for closed- and open-world recognition benchmarks, with OSPA-based metrics for evaluation.
DifFUSER: Diffusion Model for Robust Multi-Sensor Fusion in 3D Object Detection and BEV Segmentation
Diffusion models have recently gained prominence as powerful deep generative models, demonstrating unmatched performance across various domains.
