Search Results for author:
Found 65 papers, 22 papers with code
Learning Object Placement by Inpainting for Compositional Data Augmentation
We study the problem of common sense placement of visual objects in an image.
ForkGAN: Seeing into the Rainy Night
We present a ForkGAN for task-agnostic image translation that can boost multiple vision tasks in adverse weather conditions.
Detecting Image Attribution for Text-to-Image Diffusion Models in RGB and Beyond
Modern text-to-image (T2I) diffusion models can generate images with remarkable realism and creativity.
Amodal Completion via Progressive Mixed Context Diffusion
We propose to sidestep many of the difficulties of existing approaches, which typically involve a two-step process of predicting amodal masks and then generating pixels.
Brain Decodes Deep Nets
This mapping method, FactorTopy, is plug-and-play for any deep-network; with it, one can paint a picture of the network onto the brain (literally!).
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
Perceptual Artifacts Localization for Image Synthesis Tasks
Recent advancements in deep generative models have facilitated the creation of photo-realistic images across various tasks.
Retinotopy Inspired Brain Encoding Model and the All-for-One Training Recipe
We use this diversity to our advantage by introducing the All-for-One training recipe, which divides the challenging one-big-model problem into multiple small models, with the small models aggregating the knowledge while preserving the distinction between the different functional regions.
Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
Ranked #1 on
Training-free 3D Part Segmentation
on ShapeNet-Part
3D Point Cloud Classification
Training-free 3D Part Segmentation
+1
Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis
Performances on standard 3D point cloud benchmarks have plateaued, resulting in oversized models and complex network design to make a fractional improvement.
Starting From Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
iQuery: Instruments as Queries for Audio-Visual Sound Separation
To generalize to a new instrument or event class, drawing inspiration from the text-prompt design, we insert an additional query as an audio prompt while freezing the attention mechanism.
HashEncoding: Autoencoding with Multiscale Coordinate Hashing
We present HashEncoding, a novel autoencoding architecture that leverages a non-parametric multiscale coordinate hash function to facilitate a per-pixel decoder without convolutions.
Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications
Egocentric videos offer fine-grained information for high-fidelity modeling of human behaviors.
Inpainting at Modern Camera Resolution by Guided PatchMatch with Auto-Curation
Recently, deep models have established SOTA performance for low-resolution image inpainting, but they lack fidelity at resolutions associated with modern cameras such as 4K or more, and for large holes.
Perceptual Artifacts Localization for Inpainting
Inspired by this workflow, we propose a new learning task of automatic segmentation of inpainting perceptual artifacts, and apply the model for inpainting model evaluation and iterative refinement.
Beyond mAP: Towards better evaluation of instance segmentation
Correctness of instance segmentation constitutes counting the number of objects, correctly localizing all predictions and classifying each localized prediction.
DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion
We reverse the conventional design of applying convolution on voxels and attention to points.
Ranked #36 on
3D Part Segmentation
on ShapeNet-Part
SoGCN: Second-Order Graph Convolutional Networks
Graph Convolutional Networks (GCN) with multi-hop aggregation is more expressive than one-hop GCN but suffers from higher model complexity.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
Visual recognition is currently one of the most important and active research areas in computer vision, pattern recognition, and even the general field of artificial intelligence.
Convolutional Ordinal Regression Forest for Image Ordinal Estimation
We propose a novel ordinal regression approach, termed Convolutional Ordinal Regression Forest or CORF, for image ordinal estimation, which can integrate ordinal regression and differentiable decision trees with a convolutional neural network for obtaining precise and stable global ordinal relationships.
Nested Scale Editing for Conditional Image Synthesis
We propose an image synthesis approach that provides stratified navigation in the latent code space.
FoveaBox: Beyound Anchor-based Object Detection
While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors.
Potential Field: Interpretable and Unified Representation for Trajectory Prediction
From the generated potential fields, we further estimate future motion direction and speed, which are modeled as Gaussian distributions to account for the multi-modal nature of the problem.
Multimodal Image Outpainting With Regularized Normalized Diversification
In this paper, we study the problem of generating a set ofrealistic and diverse backgrounds when given only a smallforeground region.
Deep Image Blending
In addition, we jointly optimize the proposed Poisson blending loss as well as the style and content loss computed from a deep network, and reconstruct the blending region by iteratively updating the pixels using the L-BFGS solver.
SegSort: Segmentation by Discriminative Sorting of Segments
The proposed SegSort further produces an interpretable result, as each choice of label can be easily understood from the retrieved nearest segments.
Ranked #10 on
Unsupervised Semantic Segmentation
on PASCAL VOC 2012 val
(using extra training data)
Image-based marker tracking and registration for intraoperative 3D image-guided interventions using augmented reality
The location of the markers in the CT scan are extracted and the CT scan is converted into a 3D surface object.
Neural Embedding for Physical Manipulations
In common real-world robotic operations, action and state spaces can be vast and sometimes unknown, and observations are often relatively sparse.
Enhanced generative adversarial network for 3D brain MRI super-resolution
Single image super-resolution (SISR) reconstruction for magnetic resonance imaging (MRI) has generated significant interest because of its potential to not only speed up imaging but to improve quantitative processing and analysis of available image data.
Learning Temporal Pose Estimation from Sparsely-Labeled Videos
To reduce the need for dense annotations, we propose a PoseWarper network that leverages training videos with sparse annotations (every k frames) to learn to perform dense temporal pose propagation and estimation.
Ranked #2 on
Multi-Person Pose Estimation
on PoseTrack2018
(using extra training data)
FoveaBox: Beyond Anchor-based Object Detector
In FoveaBox, an instance is assigned to adjacent feature levels to make the model more accurate. We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis.
Ranked #89 on
Object Detection
on COCO test-dev
(APM metric)
Normalized Diversification
We introduce the concept of normalized diversity which force the model to preserve the normalized pairwise distance between the sparse samples from a latent parametric distribution and their corresponding high-dimensional outputs.
Conditional Image Generation
Generative Adversarial Network
+2
Trajectory Normalized Gradients for Distributed Optimization
Recently, researchers proposed various low-precision gradient compression, for efficient communication in large-scale distributed optimization.
Consistent Optimization for Single-Shot Object Detection
We present consistent optimization for single stage object detection.
Monocular 3D Pose Recovery via Nonconvex Sparsity with Theoretical Analysis
For recovering 3D object poses from 2D images, a prevalent method is to pre-train an over-complete dictionary $\mathcal D=\{B_i\}_i^D$ of 3D basis poses.
Learning Discriminative Motion Features Through Detection
Our network learns to spatially sample features from Frame B in order to maximize pose detection accuracy in Frame A.
Zoom-In-to-Check: Boosting Video Interpolation via Instance-level Discrimination
We propose a light-weight video frame interpolation algorithm.
Adversarial Structure Matching for Structured Prediction Tasks
The structure analyzer is trained to maximize the ASM loss, or to emphasize recurring multi-scale hard negative structural mistakes among co-occurring patterns.
Object Detection in Video with Spatiotemporal Sampling Networks
We propose a Spatiotemporal Sampling Network (STSN) that uses deformable convolutions across time for object detection in videos.
Egocentric Basketball Motion Planning from a Single First-Person Image
We present a model that uses a single first-person image to generate an egocentric basketball motion sequence in the form of a 12D camera configuration trajectory, which encodes a player's 3D location and 3D head orientation throughout the sequence.
Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention
We present a first-person method for cooperative basketball intention prediction: we predict with whom the camera wearer will cooperate in the near future from unlabeled first-person images.
Predicting Behaviors of Basketball Players From First Person Videos
This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos.
Customizing First Person Image Through Desired Actions
Our conjecture is that the spatial arrangement of a first person visual scene is deployed to afford an action, and therefore, the action can be inversely used to synthesize a new scene such that the action is feasible.
Social Behavior Prediction from First Person Videos
This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos.
Unsupervised Learning of Important Objects from First-Person Videos
In this work, we show that we can detect important objects in first-person images without the supervision by the camera wearer or even third-person labelers.
Am I a Baller? Basketball Performance Assessment from First-Person Videos
Finally, we use this feature to learn a basketball assessment model from pairs of labeled first-person basketball videos, for which a basketball expert indicates, which of the two players is better.
Force From Motion: Decoding Physical Sensation in a First Person Video
In this paper, we focus on a problem of Force from Motion---decoding the sensation of 1) passive forces such as the gravity, 2) the physical scale of the motion (speed) and space, and 3) active forces exerted by the observer such as pedaling a bike or banking on a ski turn.
Egocentric Future Localization
We refine them by minimizing a cost function that describes compatibility between the obstacles in the EgoRetinal map and trajectories.
Local Perturb-and-MAP for Structured Prediction
In this work, we present a new Local Perturb-and-MAP (locPMAP) framework that replaces the global optimization with a local optimization by exploiting our observed connection between locPMAP and the pseudolikelihood of the original CRF model.
Convolutional Random Walk Networks for Semantic Image Segmentation
It combines these two objectives via a novel random walk layer that enforces consistent spatial grouping in the deep layers of the network.
First Person Action-Object Detection with EgoNet
Unlike traditional third-person cameras mounted on robots, a first-person camera, captures a person's visual sensorimotor object interactions from up close.
Exploiting Egocentric Object Prior for 3D Saliency Detection
We empirically show that this representation can accurately characterize the egocentric object prior by testing it on an egocentric RGBD dataset for three tasks: the 3D saliency detection, future saliency prediction, and interaction classification.
Semantic Segmentation with Boundary Neural Fields
To overcome these problems, we introduce a Boundary Neural Field (BNF), which is a global energy model integrating FCN predictions with boundary cues.
Future Localization from an Egocentric Depth Image
As a byproduct of the predicted trajectories of ego-motion, we discover in the image the empty space occluded by foreground objects.
High-for-Low and Low-for-High: Efficient Boundary Detection from Deep Object Features and its Applications to High-Level Vision
We can view this process as a "Low-for-High" scheme, where low-level boundaries aid high-level vision tasks.
DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection
This section of the network is applied to four different scales of the image input.
Pulling Things out of Perspective
The limitations of current state-of-the-art methods for single-view depth estimation and semantic segmentations are closely tied to the property of perspective geometry, that the perceived size of the objects scales inversely with the distance.
Pose from Flow and Flow from Pose
The pose labeled segments and corresponding articulated joints are used to improve the motion flow fields by proposing kinematically constrained affine displacements on body parts.
Machine learning of hierarchical clustering to segment 2D and 3D images
We aim to improve segmentation through the use of machine learning tools during region agglomeration.
A ``Shape Aware'' Model for semi-supervised Learning of Objects and its Context
Using an analogous reasoning, we present an approach that combines bag-of-words and spatial models to perform semantic and syntactic analysis for recognition of an object based on its internal appearance and its context.
Grouping Contours Via a Related Image
We present a method for further grouping of contours in an image using their relationship to the contours of a second, related image.
