Search Results for author:
Found 84 papers, 44 papers with code
Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence
In this work, we first show that feature aggregation and cost aggregation exhibit distinct characteristics and reveal the potential for substantial benefits stemming from the judicious use of both aggregation processes.
Collaboratively Self-supervised Video Representation Learning for Action Recognition
Considering the close connection between action recognition and human pose estimation, we design a Collaboratively Self-supervised Video Representation (CSVR) learning framework specific to action recognition by jointly considering generative pose prediction and discriminative context matching as pretext tasks.
Exploring Transferability for Randomized Smoothing
However, the model robustness, which is a critical aspect for safety, is often optimized for each specific task rather than at the pretraining stage.
NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time Series Pretraining
In this work, we make key technical contributions that are tailored to the numerical properties of time-series data and allow the model to scale to large datasets, e. g., millions of temporal sequences.
Associative Transformer
Existing studies such as the Coordination method employ iterative cross-attention mechanisms with a bottleneck to enable the sparse association of inputs.
GMConv: Modulating Effective Receptive Fields for Convolutional Kernels
Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv) in this work.
Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning
Self-supervised representation learning follows a paradigm of withholding some part of the data and tasking the network to predict it from the remaining part.
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Image cropping has progressed tremendously under the data-driven paradigm.
Local Magnification for Data and Feature Augmentation
And further combination with our feature augmentation techniques, termed LOMA_IF&FO, can continue to strengthen the model and outperform advanced intensity transformation methods for data augmentation.
Could Giant Pretrained Image Models Extract Universal Representations?
In this paper, we present a study of frozen pretrained models when applied to diverse and representative computer vision tasks, including object detection, semantic segmentation and video action recognition.
Ranked #3 on
Action Recognition In Videos
on Kinetics-400
Integrative Feature and Cost Aggregation with Transformers for Dense Correspondence
The current state-of-the-art are Transformer-based approaches that focus on either feature descriptors or cost volume aggregation.
Ranked #1 on
Geometric Matching
on HPatches
Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation
However, the tokenization of a correlation map for transformer processing can be detrimental, because the discontinuity at token boundaries reduces the local context available near the token edges and decreases inductive bias.
Ranked #2 on
Semantic correspondence
on PF-WILLOW
Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance
Existing solutions to this problem estimate a single image sequence without considering the motion ambiguity for each region.
Extreme Masking for Learning Instance and Distributed Visual Representations
The paper presents a scalable approach for learning spatially distributed visual representations over individual tokens and a holistic instance representation simultaneously.
Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations
We propose an unsupervised method for 3D geometry-aware representation learning of articulated objects, in which no image-pose pairs or foreground masks are used for training.
Region Rebalance for Long-Tailed Semantic Segmentation
In this paper, we study the problem of class imbalance in semantic segmentation.
Ranked #21 on
Semantic Segmentation
on ADE20K
Bringing Rolling Shutter Images Alive with Dual Reversed Distortion
In this paper, instead of two consecutive frames, we propose to exploit a pair of images captured by dual RS cameras with reversed RS directions for this highly challenging task.
A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation
Concretely, we pretrain the sign-to-gloss visual network on the general domain of human actions and the within-domain of a sign-to-gloss dataset, and pretrain the gloss-to-text translation network on the general domain of a multilingual corpus and the within-domain of a gloss-to-text corpus.
Ranked #2 on
Sign Language Translation
on CSL-Daily
Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition
Typically in recent work, the pseudo-labels are obtained by training a model on the labeled data, and then using confident predictions from the model to teach itself.
Towards Tokenized Human Dynamics Representation
For human action understanding, a popular research direction is to analyze short video clips with unambiguous semantic content, such as jumping and drinking.
The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos
Our model starts with two separate pathways: an appearance pathway that outputs feature-based region segmentation for a single image, and a motion pathway that outputs motion features for a pair of images.
Ranked #7 on
Unsupervised Object Segmentation
on FBMS-59
Bootstrap Your Object Detector via Mixed Training
We introduce MixTraining, a new training paradigm for object detection that can improve the performance of existing detectors for free.
Self-supervised Discovery of Human Actons from Long Kinematic Videos
However, methods for understanding short semantic actions cannot be directly translated to long kinematic sequences such as dancing, where it becomes challenging even to semantically label the human movements.
ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection
A common problem in the task of human-object interaction (HOI) detection is that numerous HOI classes have only a small number of labeled examples, resulting in training sets with a long-tailed distribution.
Ranked #41 on
Human-Object Interaction Detection
on HICO-DET
Video Swin Transformer
The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks.
Ranked #28 on
Action Classification
on Kinetics-600
(using extra training data)
Aligning Pretraining for Detection via Object-Level Contrastive Learning
Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning.
Neural Articulated Radiance Field
We present Neural Articulated Radiance Field (NARF), a novel deformable 3D representation for articulated objects learned from images.
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision.
Ranked #2 on
Image Classification
on OmniBenchmark
Instance Localization for Self-supervised Detection Pretraining
The pretext task is to predict the instance category given the composited images as well as the foreground bounding boxes.
Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency
We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision.
Ranked #2 on
Monocular Depth Estimation
on Cityscapes
Global Context Networks
The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies within an image, via aggregating query-specific global context to each query position.
Ranked #40 on
Instance Segmentation
on COCO test-dev
Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning
We argue that the power of contrastive learning has yet to be fully unleashed, as current methods are trained only on instance-level pretext tasks, leading to representations that may be sub-optimal for downstream tasks requiring dense pixel predictions.
Object-based Illumination Estimation with Rendering-aware Neural Networks
With the estimated lighting, virtual objects can be rendered in AR scenarios with shading that is consistent to the real scene, leading to improved realism.
SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approach
With the reduced dimensionality of less relevant body areas, the training set distribution within network branches more closely reflects the statistics of local poses instead of global body poses, without sacrificing information important for joint inference.
Ranked #20 on
Monocular 3D Human Pose Estimation
on Human3.6M
Detecting Human-Object Interactions with Action Co-occurrence Priors
A common problem in human-object interaction (HOI) detection task is that numerous HOI classes have only a small number of labeled examples, resulting in training sets with a long-tailed distribution.
RepPoints V2: Verification Meets Regression for Object Detection
Though RepPoints provides high performance, we find that its heavy reliance on regression for object localization leaves room for improvement.
Ranked #27 on
Object Detection
on COCO-O
Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation
A recent approach for object detection and human pose estimation is to regress bounding boxes or human keypoints from a central point on the object or person.
What makes instance discrimination good for transfer learning?
Contrastive visual pretraining based on the instance discrimination pretext task has made significant progress.
Disentangled Non-Local Neural Networks
This paper first studies the non-local block in depth, where we find that its attention computation can be split into two terms, a whitened pairwise term accounting for the relationship between two pixels and a unary term representing the saliency of every pixel.
Ranked #20 on
Semantic Segmentation
on Cityscapes test
(using extra training data)
A Transductive Approach for Video Object Segmentation
Semi-supervised video object segmentation aims to separate a target object from a video sequence, given the mask in the first frame.
Distilling Localization for Self-Supervised Representation Learning
To address this problem, we propose a data-driven approach for learning invariance to backgrounds.
Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation
In the feature maps of CNNs, there commonly exists considerable spatial redundancy that leads to much repetitive processing.
Cross-Iteration Batch Normalization
We thus compensate for the network weight changes via a proposed technique based on Taylor polynomials, so that the statistics can be accurately estimated and batch normalization can be effectively applied.
Ranked #180 on
Object Detection
on COCO test-dev
Dense RepPoints: Representing Visual Objects with Dense Point Sets
We present a new object representation, called Dense RepPoints, that utilizes a large set of points to describe an object at multiple levels, including both box level and pixel level.
Instance-wise Depth and Motion Learning from Monocular Videos
We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision.
Leveraging Multi-view Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation
We present an unsupervised approach for factorizing object appearance into highlight, shading, and albedo layers, trained by multi-view real images.
Single Image Reflection Removal through Cascaded Refinement
IBCLN is a cascaded network that iteratively refines the estimates of transmission and reflection layers in a manner that they can boost the prediction quality to each other, and information across steps of the cascade is transferred using an LSTM.
Ranked #2 on
Reflection Removal
on SIR^2(Postcard)
Visuomotor Understanding for Representation Learning of Driving Scenes
We postulate that success on this task requires the network to learn semantic and geometric knowledge in the ego-centric view.
Learning Residual Flow as Dynamic Motion from Stereo Videos
Based on rigid projective geometry, the estimated stereo depth is used to guide the camera motion estimation, and the depth and camera motion are used to guide the residual flow estimation.
DPSNet: End-to-end Deep Plane Sweep Stereo
The cost volume is constructed using a differentiable warping process that allows for end-to-end training of the network.
Local Relation Networks for Image Recognition
The convolution layer has been the dominant feature extractor in computer vision for years.
Ranked #867 on
Image Classification
on ImageNet
RepPoints: Point Set Representation for Object Detection
They furthermore do not require the use of anchors to sample a space of bounding boxes.
Ranked #95 on
Object Detection
on COCO minival
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation.
Ranked #25 on
Object Detection
on COCO-O
An Empirical Study of Spatial Attention Mechanisms in Deep Networks
Attention mechanisms have become a popular component in deep neural networks, yet there has been little examination of how different influencing factors and methods for computing attention from these factors affect performance.
Mimicking the In-Camera Color Pipeline for Camera-Aware Object Compositing
We present a method for compositing virtual objects into a photograph such that the object colors appear to have been processed by the photo's camera imaging pipeline.
Angle-Closure Detection in Anterior Segment OCT based on Multi-Level Deep Network
A Multi-Level Deep Network (MLDN) is proposed to formulate this learning, which utilizes three particular AS-OCT regions based on clinical priors: the global anterior segment structure, local iris region, and anterior chamber angle (ACA) patch.
Deep Metric Transfer for Label Propagation with Limited Annotated Data
In this paper, we propose a generic framework that utilizes unlabeled data to aid generalization for all three tasks.
Integrated Object Detection and Tracking with Tracklet-Conditioned Detection
Accurate detection and tracking of objects is vital for effective video understanding.
Ranked #19 on
Video Object Detection
on ImageNet VID
Deformable ConvNets v2: More Deformable, Better Results
The superior performance of Deformable Convolutional Networks arises from its ability to adapt to the geometric variations of objects.
Ranked #116 on
Object Detection
on COCO test-dev
Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose
We present a method for human pose tracking that is based on learning spatiotemporal relationships among joints.
Recurrent Transformer Networks for Semantic Correspondence
Our networks accomplish this through an iterative process of estimating spatial transformations between the input images and using these transformations to generate aligned convolutional activations.
An Integral Pose Regression System for the ECCV2018 PoseTrack Challenge
For the ECCV 2018 PoseTrack Challenge, we present a 3D human pose estimation system based mainly on the integral human pose regression method.
Ranked #1 on
3D Human Pose Estimation
on CHALL H80K
Multi-Context Deep Network for Angle-Closure Glaucoma Screening in Anterior Segment OCT
A major cause of irreversible visual impairment is angle-closure glaucoma, which can be screened through imagery from Anterior Segment Optical Coherence Tomography (AS-OCT).
A High-Quality Denoising Dataset for Smartphone Cameras
We propose a systematic procedure for estimating ground truth for noisy images that can be used to benchmark denoising performance for smartphone cameras.
Faces as Lighting Probes via Unsupervised Deep Highlight Extraction
We present a method for estimating detailed scene illumination using human faces in a single image.
Exposure: A White-Box Photo Post-Processing Framework
Retouching can significantly elevate the visual appeal of photos, but many casual photographers lack the expertise to do this well.
DCTM: Discrete-Continuous Transformation Matching for Semantic Flow
In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor.
FC4: Fully Convolutional Color Constancy With Confidence-Weighted Pooling
However, the patch-based CNNs that exist for this problem are faced with the issue of estimation ambiguity, where a patch may contain insufficient information to establish a unique or even a limited possible range of illumination colors.
Radiometric Calibration From Faces in Images
We present a method for radiometric calibration of cameras from a single image that contains a human face.
Specular Highlight Removal in Facial Images
An important practical feature of the proposed method is that the skin color model is utilized in a way that does not require color calibration of the camera.
FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence
The sampling patterns of local structure and the self-similarity measure are jointly learned within the proposed network in an end-to-end and multi-scale manner.
Image Deblurring Using Smartphone Inertial Sensors
Removing image blur caused by camera shake is an ill-posed problem, as both the latent image and the point spread function (PSF) are unknown.
Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields
We present a method for jointly predicting a depth map and intrinsic images from single-image input.
Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence
We present a novel descriptor, called deep self-convolutional activations (DeSCA), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions.
Automatic Thumbnail Generation Based on Visual Representativeness and Foreground Recognizability
We present an automatic thumbnail generation technique based on two essential considerations: how well they visually represent the original photograph, and how well the foreground can be recognized after the cropping and downsizing steps of thumbnailing.
Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders
With the growing popularity of short-form video sharing platforms such as \em{Instagram} and \em{Vine}, there has been an increasing need for techniques that automatically extract highlights from video.
Data-Driven Depth Map Refinement via Multi-Scale Sparse Representation
Depth maps captured by consumer-level depth cameras such as Kinect are usually degraded by noise, missing values, and quantization.
Simulating Makeup Through Physics-Based Manipulation of Intrinsic Image Layers
We present a method for simulating makeup in a face image.
Object-Based RGBD Image Co-Segmentation With Mutex Constraint
We present an object-based co-segmentation method that takes advantage of depth data and is able to correctly handle noisy images in which the common foreground object is missing.
Object-based Multiple Foreground Video Co-segmentation
We present a video co-segmentation method that uses category-independent object proposals as its basic element and can extract multiple foreground objects in a video set.
A Learning-to-Rank Approach for Image Color Enhancement
We present a machine-learned ranking approach for automatically enhancing the color of a photograph.
Bayesian Depth-from-Defocus with Shading Constraints
We present a method that enhances the performance of depth-from-defocus (DFD) through the use of shading information.
Shading-Based Shape Refinement of RGB-D Images
We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading.
Learning the Change for Automatic Image Cropping
Image cropping is a common operation used to improve the visual quality of photographs.
