Search Results for author:
Found 107 papers, 50 papers with code
In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation
We present lazy visual grounding, a two-stage approach of unsupervised object mask discovery followed by object grounding, for open-vocabulary semantic segmentation.
Ranked #2 on
Open Vocabulary Semantic Segmentation
on COCO-Stuff-171
(mIoU metric)
Online Temporal Action Localization with Memory-Augmented Transformer
We also propose a novel action localization method that observes the current input segment to predict the end time of the ongoing action and accesses the memory queue to estimate the start time of the action.
Classification Matters: Improving Video Action Detection with Class-Specific Attention
Video action detection (VAD) aims to detect actors and classify their actions in a video.
3D Geometric Shape Assembly via Efficient Point Cloud Matching
Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task.
Burst Image Super-Resolution with Base Frame Selection
In this work, we explore using burst shots with non-uniform exposures to confront real-world practical scenarios by introducing a new benchmark dataset, dubbed Non-uniformly Exposed Burst Image (NEBI), that includes the burst frames at varying exposure times to obtain a broader range of irradiance and motion characteristics within a scene.
Multi-view Image Prompted Multi-view Diffusion for Improved 3D Generation
Using image as prompts for 3D generation demonstrate particularly strong performances compared to using text prompts alone, for images provide a more intuitive guidance for the 3D generation process.
Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform
Establishing accurate 3D correspondences between shapes stands as a pivotal challenge with profound implications for computer vision and robotics.
Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences
Leveraging multi-view diffusion models as priors for 3D optimization have alleviated the problem of 3D consistency, e. g., the Janus face problem or the content drift problem, in zero-shot text-to-3D models.
Contrastive Mean-Shift Learning for Generalized Category Discovery
We address the problem of generalized category discovery (GCD) that aims to partition a partially labeled collection of images; only a small part of the collection is labeled and the total number of target classes is unknown.
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering
This paper addresses the task of video question answering (videoQA) via a decomposed multi-stage, modular reasoning framework.
Ranked #6 on
Zero-Shot Video Question Answer
on NExT-QA
Learning Correlation Structures for Vision Transformers
We introduce a new attention mechanism, dubbed structural self-attention (StructSA), that leverages rich correlation patterns naturally emerging in key-query interactions of attention.
Ranked #4 on
Action Recognition
on Diving-48
NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Image
Transfer learning of large-scale Text-to-Image (T2I) models has recently shown impressive potential for Novel View Synthesis (NVS) of diverse objects from a single image.
Activity Grammars for Temporal Action Segmentation
Sequence prediction on temporal data requires the ability to understand compositional structures of multi-level semantics beyond individual and contextual properties.
Towards More Practical Group Activity Detection: A New Benchmark and Model
Group activity detection (GAD) is the task of identifying members of each group and classifying the activity of the group at the same time in a video.
Efficient Semantic Matching with Hypercolumn Correlation
Recent studies show that leveraging the match-wise relationships within the 4D correlation map yields significant improvements in establishing semantic correspondences - but at the cost of increased computation and latency.
Generalized Neural Sorting Networks with Error-Free Differentiable Swap Functions
Sorting is a fundamental operation of all computer systems, having been a long-standing significant research topic.
PriViT: Vision Transformers for Fast Private Inference
The Vision Transformer (ViT) architecture has emerged as the backbone of choice for state-of-the-art deep models for computer vision applications.
Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation
For this mixed setup, we propose to improve the pseudo-labels using a pseudo-label enhancer that was trained using the available ground-truth pixel-level labels.
Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learning
Learning to predict reliable characteristic orientations of 3D point clouds is an important yet challenging problem, as different point clouds of the same class may have largely varying appearances.
Relational Context Learning for Human-Object Interaction Detection
Recent state-of-the-art methods for HOI detection typically build on transformer architectures with two decoder branches, one for human-object pair detection and the other for interaction classification.
Ranked #2 on
Human-Object Interaction Detection
on V-COCO
Devil's on the Edges: Selective Quad Attention for Scene Graph Generation
The edge selection module selects relevant object pairs, i. e., edges in the scene graph, which helps contextual reasoning, and the quad attention module then updates the edge features using both edge-to-node and edge-to-edge cross-attentions to capture contextual information between objects and object pairs.
Learning Rotation-Equivariant Features for Visual Correspondence
The resultant features and their orientations are further processed by group aligning, a novel invariant mapping technique that shifts the group-equivariant features by their orientations along the group dimension.
Generalizable Implicit Neural Representations via Instance Pattern Composers
Despite recent advances in implicit neural representations (INRs), it remains challenging for a coordinate-based multi-layer perceptron (MLP) of INRs to learn a common representation across data instances and generalize it for unseen instances.
Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval
Metric learning aims to build a distance metric typically by learning an effective embedding function that maps similar objects into nearby points in its embedding space.
Soft-Landing Strategy for Alleviating the Task Discrepancy Problem in Temporal Action Localization Tasks
Temporal Action Localization (TAL) methods typically operate on top of feature sequences from a frozen snippet encoder that is pretrained with the Trimmed Action Classification (TAC) tasks, resulting in a task discrepancy problem.
Sequential Brick Assembly with Efficient Constraint Satisfaction
The assembly problem is challenging since the number of possible structures increases exponentially with the number of available bricks, complicating the physical constraints to satisfy across bricks.
PeRFception: Perception using Radiance Fields
The recent progress in implicit 3D representation, i. e., Neural Radiance Fields (NeRFs), has made accurate and photorealistic 3D reconstruction possible in a differentiable manner.
Towards Sequence-Level Training for Visual Tracking
Despite the extensive adoption of machine learning on the task of visual object tracking, recent learning-based approaches have largely overlooked the fact that visual tracking is a sequence-level task in its nature; they rely heavily on frame-level training, which inevitably induces inconsistency between training and testing in terms of both data distributions and task objectives.
Ranked #19 on
Visual Object Tracking
on TrackingNet
Revisiting Self-Distillation
We first show that even with a highly accurate teacher, self-distillation allows a student to surpass the teacher in all cases.
Self-Supervised Learning of Image Scale and Orientation
We study the problem of learning to assign a characteristic pose, i. e., scale and orientation, for an image region of interest.
Peripheral Vision Transformer
We propose to incorporate peripheral position encoding to the multi-head self-attention layers to let the network learn to partition the visual field into diverse peripheral regions given training data.
Draft-and-Revise: Effective Image Generation with Contextual RQ-Transformer
After code stacks in the sequence are randomly masked, Contextual RQ-Transformer is trained to infill the masked code stacks based on the unmasked contexts of the image.
Ranked #1 on
Text-to-Image Generation
on Conceptual Captions
Future Transformer for Long-term Action Anticipation
The task of predicting future actions from a video is crucial for a real-world agent interacting with others.
Learning to Assemble Geometric Shapes
Assembling parts into an object is a combinatorial problem that arises in a variety of contexts in the real world and involves numerous applications in science and engineering.
TransforMatcher: Match-to-Match Attention for Semantic Correspondence
Establishing correspondences between images remains a challenging task, especially under large appearance changes due to different viewpoints or intra-class variations.
Ranked #10 on
Semantic correspondence
on SPair-71k
Smooth-Reduce: Leveraging Patches for Improved Certified Robustness
Randomized smoothing (RS) has been shown to be a fast, scalable technique for certifying the robustness of deep neural network classifiers.
Self-Taught Metric Learning without Labels
At the heart of our framework lies an algorithm that investigates contexts of data on the embedding space to predict their class-equivalence relations as pseudo labels.
Self-Supervised Equivariant Learning for Oriented Keypoint Detection
Detecting robust keypoints from an image is an integral part of many computer vision problems, and the characteristic orientation and scale of keypoints play an important role for keypoint description and matching.
Detector-Free Weakly Supervised Group Activity Recognition
Group activity recognition is the task of understanding the activity conducted by a group of people as a whole in a multi-person video.
Reflection and Rotation Symmetry Detection via Equivariant Learning
The inherent challenge of detecting symmetries stems from arbitrary orientations of symmetry patterns; a reflection symmetry mirrors itself against an axis with a specific orientation while a rotation symmetry matches its rotated copy with a specific orientation.
Integrative Few-Shot Learning for Classification and Segmentation
We introduce the integrative task of few-shot classification and segmentation (FS-CS) that aims to both classify and segment target objects in a query image when the target classes are given with a few examples.
Autoregressive Image Generation using Residual Quantization
However, we postulate that previous VQ cannot shorten the code sequence and generate high-fidelity images together in terms of the rate-distortion trade-off.
Ranked #2 on
Text-to-Image Generation
on Conceptual Captions
Selective Network Linearization for Efficient Private Inference
To reduce PI latency we propose a gradient-based algorithm that selectively linearizes ReLUs while maintaining prediction accuracy.
Contrastive Regularization for Semi-Supervised Learning
Consistency regularization on label predictions becomes a fundamental technique in semi-supervised learning, but it still requires a large number of training iterations for high performance.
Fast Point Transformer
The recent success of neural networks enables a better interpretation of 3D point clouds, but processing a large-scale 3D scene remains a challenging problem.
Ranked #24 on
Semantic Segmentation
on S3DIS
Semi-supervised Domain Adaptation via Sample-to-Sample Self-Distillation
Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain.
Relational Self-Attention: What's Missing in Attention for Video Understanding
Convolution has been arguably the most important feature transform for modern neural networks, leading to the advance of deep learning.
Ranked #11 on
Action Recognition
on Diving-48
Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training
On this foundation, we propose the Rebooted Auxiliary Classifier Generative Adversarial Network (ReACGAN).
Ranked #1 on
Image Generation
on CIFAR-10
(NFE metric)
Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning
Discovering a solution in a combinatorial space is prevalent in many real-world problems but it is also challenging due to diverse complex constraints and the vast number of possible combinations.
Differentiable Spline Approximations
Overall, we show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications such as image segmentation, 3D point cloud reconstruction, and finite element analysis.
Efficient Point Transformer for Large-scale 3D Scene Understanding
Although sparse convolution is efficient and scalable for large 3D scenes, the quantization artifacts impair geometric details and degrade prediction accuracy.
Rotation-Equivariant Keypoint Detection
Therefore, we propose a rotation-invariant keypoint detection method using rotation-equivariant CNNs.
Visual TransforMatcher: Efficient Match-to-Match Attention for Visual Correspondence
Establishing correspondences between images remains a challenging task, especially under large appearance changes due to different viewpoints and intra-class variations.
Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence
To validate the proposed techniques, we develop the neural network with CHM layers that perform convolutional matching in the space of translation and scaling.
Deep Hough Voting for Robust Global Registration
We then construct a set of triplets of correspondences to cast votes on the 6D Hough space, representing the transformation parameters in sparse tensors.
Self-Calibrating Neural Radiance Fields
We also propose a new geometric loss function, viz., projected ray distance loss, to incorporate geometric consistency for complex non-linear camera models.
Learning to Discover Reflection Symmetry via Polar Matching Convolution
The task of reflection symmetry detection remains challenging due to significant variations and ambiguities of symmetry patterns in the wild.
Relational Embedding for Few-Shot Classification
We propose to address the problem of few-shot classification by meta-learning "what to observe" and "where to attend" in a relational perspective.
Ranked #17 on
Few-Shot Image Classification
on CUB 200 5-way 5-shot
Sphynx: ReLU-Efficient Network Design for Private Inference
The emergence of deep learning has been accompanied by privacy concerns surrounding users' data and service providers' models.
Hypercorrelation Squeeze for Few-Shot Segmentation
Few-shot semantic segmentation aims at learning to segment a target object from a query image using only a few annotated support images of the target class.
Ranked #13 on
Few-Shot Semantic Segmentation
on FSS-1000 (5-shot)
Convolutional Hough Matching Networks
Despite advances in feature representation, leveraging geometric relations is crucial for establishing reliable visual correspondences under large variations of images.
Ranked #4 on
Semantic correspondence
on PF-WILLOW
Embedding Transfer with Label Relaxation for Improved Metric Learning
Our method exploits pairwise similarities between samples in the source embedding space as the knowledge, and transfers them through a loss used for learning target embedding models.
Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognition
With a sufficient volume of the neighborhood in space and time, it effectively captures long-term interaction and fast motion in the video, leading to robust action recognition.
Ranked #19 on
Action Recognition
on Something-Something V1
(using extra training data)
Embedding Transfer via Smooth Contrastive Loss
To this end, we design a new loss called smooth contrastive loss, which pulls together or pushes apart a pair of samples in a target embedding space with strength determined by their semantic similarity in the source embedding space; an analysis of the loss reveals that this property enables more important pairs to contribute more to learning the target embedding space.
Hypercorrelation Squeeze for Few-Shot Segmenation
Few-shot semantic segmentation aims at learning to segment a target object from a query image using only a few annotated support images of the target class.
Differentiable Programming for Piecewise Polynomial Functions
The paradigm of differentiable programming has considerably enhanced the scope of machine learning via the judicious use of gradient-based optimization.
Learning Self-Similarity in Space and Time as a Generalized Motion for Action Recognition
We leverage the whole volume of STSS and let our model learn to extract an effective motion representation from it.
Pair-based Self-Distillation for Semi-supervised Domain Adaptation
Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain.
Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes
The main idea is to use a random mapping which embeds the combinatorial space into a convex polytope in a continuous space, on which all essential process is performed to determine a solution to the black-box optimization in the combinatorial space.
CircleGAN: Generative Adversarial Learning across Spherical Circles
We present a novel discriminator for GANs that improves realness and diversity of generated samples by learning a structured hypersphere embedding space using spherical circles.
Fragment Relation Networks for Geometric Shape Assembly
Our model processes the candidate fragments in a permutation-equivariant manner and can generalize to cases with an arbitrary number of fragments and even with a different target object.
Diversified Mutual Learning for Deep Metric Learning
Mutual learning is an ensemble training strategy to improve generalization by transferring individual knowledge to each other while simultaneously training multiple models.
Learning to Compose Hypercolumns for Visual Correspondence
Feature representation plays a crucial role in visual correspondence, and recent methods for image matching resort to deeply stacked convolutional layers.
Ranked #2 on
Semantic correspondence
on Caltech-101
MotionSqueeze: Neural Motion Feature Learning for Video Understanding
As the frame-by-frame optical flows require heavy computation, incorporating motion information has remained a major computational bottleneck for video understanding.
Ranked #1 on
Video Classification
on Something-Something V2
IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos
Most current action recognition methods heavily rely on appearance information by taking an RGB sequence of entire image regions as input.
Hyperparameter Optimization in Neural Networks via Structured Sparse Recovery
In this paper, we study two important problems in the automated design of neural networks -- Hyper-parameter Optimization (HPO), and Neural Architecture Search (NAS) -- through the lens of sparse recovery methods.
ESPN: Extremely Sparse Pruned Networks
Deep neural networks are often highly overparameterized, prohibiting their use in compute-limited systems.
Combinatorial 3D Shape Generation via Sequential Assembly
To alleviate this consequence induced by a huge number of feasible combinations, we propose a combinatorial 3D shape generation framework.
Local-Global Video-Text Interactions for Temporal Grounding
This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query.
Proxy Anchor Loss for Deep Metric Learning
The former class can leverage fine-grained semantic relations between data points, but slows convergence in general due to its high training complexity.
Ranked #10 on
Metric Learning
on CUB-200-2011
(using extra training data)
Fine-Grained Image Classification
Fine-Grained Vehicle Classification
+1
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
Generative adversarial networks (GANs) have shown outstanding performance on a wide range of problems in computer vision, graphics, and machine learning, but often require numerous training data and heavy computational resources.
Ranked #5 on
10-shot image generation
on Babies
Real-Time Object Tracking via Meta-Learning: Efficient Model Adaptation and One-Shot Channel Pruning
We propose a novel meta-learning framework for real-time object tracking with efficient model adaptation and channel pruning.
Mining GOLD Samples for Conditional GANs
Conditional generative adversarial networks (cGANs) have gained a considerable attention in recent years due to its class-wise controllability and superior quality for complex generation tasks.
Regularizing Neural Networks via Stochastic Branch Layers
We introduce a novel stochastic regularization technique for deep neural networks, which decomposes a layer into multiple branches with different parameters and merges stochastically sampled combinations of the outputs from the branches during training.
SPair-71k: A Large-scale Benchmark for Semantic Correspondence
In this paper, we present a new large-scale benchmark dataset of semantically paired images, SPair-71k, which contains 70, 958 image pairs with diverse variations in viewpoint and scale.
Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Features
Establishing visual correspondences under large intra-class variations requires analyzing images at different levels, from features linked to semantics and context to local patterns, while being invariant to instance-specific details.
Ranked #1 on
Semantic correspondence
on Caltech-101
One-Shot Neural Architecture Search via Compressive Sensing
Neural Architecture Search remains a very challenging meta-learning problem.
Instance-aware Image-to-Image Translation
Unsupervised image-to-image translation has gained considerable attention due to the recent impressive progress based on generative adversarial networks (GANs).
Reducing The Search Space For Hyperparameter Optimization Using Group Sparsity
We propose a new algorithm for hyperparameter selection in machine learning algorithms.
Deep Metric Learning Beyond Binary Supervision
Metric Learning for visual similarity has mostly adopted binary supervision indicating whether a pair of images are of the same class or not.
Universal Bounding Box Regression and Its Applications
Bounding-box regression is a popular technique to refine or predict localization boxes in recent object detection approaches.
Relational Knowledge Distillation
Knowledge distillation aims at transferring knowledge acquired in one model (a teacher) to another model (a student) that is typically smaller.
Unsupervised Image Matching and Object Discovery as Optimization
Learning with complete or partial supervision is powerful but relies on ever-growing human annotation efforts.
Ranked #2 on
Single-object colocalization
on Object Discovery
InstaGAN: Instance-aware Image-to-Image Translation
Our comparative evaluation demonstrates the effectiveness of the proposed method on different image datasets, in particular, in the aforementioned challenging cases.
Attentive Semantic Alignment with Offset-Aware Correlation Kernels
Semantic correspondence is the problem of establishing correspondences across images depicting different instances of the same object or scene class.
Multi-Object Tracking With Quadruplet Convolutional Neural Networks
We propose Quadruplet Convolutional Neural Networks (Quad-CNN) for multi-object tracking, which learn to associate object detections across frames using quadruplet losses.
SCNet: Learning Semantic Correspondence
This paper addresses the problem of establishing semantic correspondences between images depicting different instances of the same object or scene category.
Proposal Flow: Semantic Correspondences from Object Proposals
Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout.
Text-guided Attention Model for Image Captioning
Visual attention plays an important role to understand images and demonstrates its effectiveness in generating natural language descriptions of images.
ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization
The additive model encourages the predicted object region to be supported by its surrounding context region.
Ranked #4 on
Weakly Supervised Object Detection
on Charades
Thin-Slicing for Pose: Learning to Understand Pose Without Explicit Pose Estimation
We address the problem of learning a pose-aware, compact embedding that projects images with similar human poses to be placed close-by in the embedding space.
Proposal Flow
Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout.~Semantic flow methods are designed to handle images depicting different instances of the same object or scene category.
Robust Image Filtering Using Joint Static and Dynamic Guidance
Regularizing images under a guidance signal has been used in various tasks in computer vision and computational photography, particularly for noise reduction and joint upsampling.
Unsupervised Object Discovery and Tracking in Video Collections
This paper addresses the problem of automatically localizing dominant objects as spatio-temporal tubes in a noisy collection of videos with minimal or even no supervision.
A General Multi-Graph Matching Approach via Graduated Consistency-regularized Boosting
We propose multi-graph matching methods to incorporate the two aspects by boosting the affinity score, meanwhile gradually infusing the consistency as a regularizer.
Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals
This paper addresses unsupervised discovery and localization of dominant objects from a noisy image collection with multiple object classes.
Finding Matches in a Haystack: A Max-Pooling Strategy for Graph Matching in the Presence of Outliers
A major challenge in real-world feature matching problems is to tolerate the numerous outliers arising in typical visual tasks.
