Search Results for author:
Found 80 papers, 34 papers with code
Generate, Transduct, Adapt: Iterative Transduction with VLMs
Transductive zero-shot learning with vision-language models leverages image-image similarities within the dataset to achieve better classification accuracy compared to the inductive setting.
WildSAT: Learning Satellite Image Representations from Wildlife Observations
WildSAT uses a contrastive learning framework to combine information from species distribution maps with text descriptions that capture habitat and range details, alongside satellite images, to train or fine-tune models.
Improving Satellite Imagery Masking using Multi-task and Transfer Learning
We experiment with recent advances in deep network architectures and show that masking models can benefit from these, especially when combined with pre-training on large satellite imagery datasets.
Combining Observational Data and Language for Species Range Estimation
Species range maps (SRMs) are essential tools for research and policy-making in ecology, conservation, and environmental management.
OSLO: One-Shot Label-Only Membership Inference Attacks
The core idea is that a member sample exhibits more resistance to adversarial perturbations than a non-member.
Task2Box: Box Embeddings for Modeling Asymmetric Task Relationships
Modeling and visualizing relationships between tasks or datasets is an important step towards solving various meta-tasks such as dataset discovery, multi-tasking, and transfer learning.
Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions
By prompting LLMs in various ways, we generate descriptions that capture visual appearance, habitat, and geographic regions and pair them with existing attributes such as the taxonomic structure of the categories.
Human-in-the-Loop Visual Re-ID for Population Size Estimation
We propose a human-in-the-loop approach for estimating population size driven by a pairwise similarity derived from an off-the-shelf Re-ID system.
PARTICLE: Part Discovery and Contrastive Learning for Fine-grained Recognition
For example, under a linear-evaluation scheme, the classification accuracy of a ResNet50 trained on ImageNet using DetCon, a self-supervised learning approach, improves from 35. 4% to 42. 0% on the Caltech-UCSD Birds, from 35. 5% to 44. 1% on the FGVC Aircraft, and from 29. 7% to 37. 4% on the Stanford Cars.
COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification
We present a set of metrics that utilize vision priors to effectively assess the performance of saliency methods on image classification tasks.
LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs
Consequently, there is growing interest in extending NeRF models to jointly optimize camera poses and scene representation, which offers an alternative to off-the-shelf SfM pipelines which have well-understood failure modes.
DISCount: Counting in Large Image Collections with Detector-Based Importance Sampling
Many modern applications use computer vision to detect and count objects in massive image collections.
Accidental Turntables: Learning 3D Pose by Watching Objects Turn
We propose a technique for learning single-view 3D object pose estimation models by utilizing a new source of data -- in-the-wild videos where objects turn.
MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentation
As a result, the learned 2D representations are view-invariant and geometrically consistent, leading to better generalization when trained on a limited number of labeled shapes compared to alternatives that utilize self-supervision in 2D or 3D alone.
Cross-Modal 3D Shape Generation and Manipulation
In this paper, we propose a generic multi-modal generative model that couples the 2D modalities and implicit 3D representations through shared latent spaces.
Improving Few-Shot Part Segmentation using Coarse Supervision
A significant bottleneck in training deep networks for part segmentation is the cost of obtaining detailed annotations.
How well does CLIP understand texture?
We investigate how well CLIP understands texture in natural images described by natural language.
PriFit: Learning to Fit Primitives Improves Few Shot Point Cloud Segmentation
We present PriFit, a semi-supervised approach for label-efficient learning of 3D point cloud segmentation networks.
GANORCON: Are Generative Models Useful for Few-shot Segmentation?
Motivated by this we present an alternative approach based on contrastive learning and compare their performance on standard few-shot part segmentation benchmarks.
Semi-Supervised Learning with Taxonomic Labels
We propose techniques to incorporate coarse taxonomic labels to train image classifiers in fine-grained domains.
Domain Adaptor Networks for Hyperspectral Image Recognition
We consider the problem of adapting a network trained on three-channel color images to a hyperspectral domain with a large number of channels.
On Measuring and Controlling the Spectral Bias of the Deep Image Prior
The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it's parameters to reconstruct a single degraded image.
The Semi-Supervised iNaturalist Challenge at the FGVC8 Workshop
Semi-iNat is a challenging dataset for semi-supervised classification with a long-tailed distribution of classes, fine-grained categories, and domain shifts between labeled and unlabeled data.
A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification
We evaluate the effectiveness of semi-supervised learning (SSL) on a realistic benchmark where data exhibits considerable class imbalance and contains images from novel classes.
The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshop
From this collection, we sample a subset of classes and their labels, while adding the images from the remaining classes to the unlabeled set of images.
Supervised Momentum Contrastive Learning for Few-Shot Classification
In this work we investigate the complementary roles of these two sources of information by combining instance-discriminative contrastive learning and supervised learning in a single framework called Supervised Momentum Contrastive learning (SUPMOCO).
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
We present a plug-in replacement for batch normalization (BN) called exponential moving average normalization (EMAN), which improves the performance of existing student-teacher based self- and semi-supervised learning techniques.
Self-Supervised Learning
Semi-Supervised Image Classification
StarcNet: Machine Learning for Star Cluster Identification
We present a machine learning (ML) pipeline to identify star clusters in the multi{color images of nearby galaxies, from observations obtained with the Hubble Space Telescope as part of the Treasury Project LEGUS (Legacy ExtraGalactic Ultraviolet Survey).
Shot in the Dark: Few-Shot Learning with No Base-Class Labels
To alleviate problems caused by the distribution shift, previous research has explored the use of unlabeled examples from the novel classes, in addition to labeled examples of the base classes, which is known as the transductive setting.
PhraseCut: Language-based Image Segmentation in the Wild
We consider the problem of segmenting image regions given a natural language phrase, and study it on a novel dataset of 77, 262 images and 345, 486 phrase-region pairs.
Ranked #4 on
Referring Expression Segmentation
on PhraseCut
Describing Textures using Natural Language
Textures in natural images can be characterized by color, shape, periodicity of elements within them, and other attributes that can be described using natural language.
On Equivariant and Invariant Learning of Object Landmark Representations
Given a collection of images, humans are able to discover landmarks by modeling the shared geometric structure across instances.
Exploring and Predicting Transferability across NLP Tasks
We also develop task embeddings that can be used to predict the most transferable source tasks for a given target task, and we validate their effectiveness in experiments controlled for source and target data size.
Detecting and Tracking Communal Bird Roosts in Weather Radar Data
The US weather radar archive holds detailed information about biological phenomena in the atmosphere over the last 20 years.
Deep Manifold Prior
We present a prior for manifold structured data, such as surfaces of 3D shapes, where deep neural networks are adopted to reconstruct a target shape using gradient descent starting from a random initialization.
Learning Generative Models of Shape Handles
We present a generative model to synthesize 3D shapes as sets of handles -- lightweight proxies that approximate the original 3D shape -- for applications in interactive editing, shape parsing, and building compact 3D representations.
Label-Efficient Learning on Point Clouds using Approximate Convex Decompositions
The problems of shape classification and part segmentation from 3D point clouds have garnered increasing attention in the last few years.
ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds
We propose a novel, end-to-end trainable, deep network called ParSeNet that decomposes a 3D point cloud into parametric surface patches, including B-spline patches as well as basic geometric primitives.
Neural Shape Parsers for Constructive Solid Geometry
We investigate two architectures for this task --- a vanilla encoder (CNN) - decoder (RNN) and another architecture that augments the encoder with an explicit memory module based on the program execution stack.
The Spectral Bias of the Deep Image Prior
The "deep image prior" proposed by Ulyanov et al. is an intriguing property of neural nets: a convolutional encoder-decoder network can be used as a prior for natural images.
When Does Self-supervision Improve Few-shot Learning?
We investigate the role of self-supervised learning (SSL) in the context of few-shot learning.
Learning Point Embeddings from Shape Repositories for Few-Shot Segmentation
We present a framework for learning representations of 3D shapes that reflect the information present in this meta data and show that it leads to improved generalization for semantic segmentation tasks.
Visualizing and Describing Fine-grained Categories as Textures
We analyze how categories from recent FGVC challenges can be described by their textural content.
Boosting Supervision with Self-Supervision for Few-shot Learning
We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions.
Inferring 3D Shapes from Image Collections using Adversarial Networks
To this end, we present new differentiable projection operators that can be used by PrGAN to learn better 3D generative models.
A Bayesian Perspective on the Deep Image Prior
The deep image prior was recently introduced as a prior for natural images.
Active Adversarial Domain Adaptation
Our approach, active adversarial domain adaptation (AADA), explores a duality between two related problems: adversarial domain alignment and importance sampling for adapting models across domains.
Meta-Learning with Differentiable Convex Optimization
We propose to use these predictors as base learners to learn representations for few-shot learning and show they offer better tradeoffs between feature size and performance across a range of few-shot recognition benchmarks.
Ranked #12 on
Few-Shot Image Classification
on FC100 5-way (1-shot)
Task2Vec: Task Embedding for Meta-Learning
We demonstrate that this embedding is capable of predicting task similarities that match our intuition about semantic and taxonomic relations between different visual tasks (e. g., tasks based on classifying different types of plants are similar) We also demonstrate the practical value of this framework for the meta-task of selecting a pre-trained feature extractor for a new task.
A Deeper Look at 3D Shape Classifiers
We investigate the role of representations and architectures for classifying 3D shapes in terms of their computational efficiency, generalization, and robustness to adversarial transformations.
Second-order Democratic Aggregation
In this paper, we study a class of orderless aggregation functions designed to minimize interference or equalize contributions in the context of second-order features and we show that they can be computed just as efficiently as their first-order counterparts and they have favorable properties over aggregation by summation.
Multiresolution Tree Networks for 3D Point Cloud Processing
We present multiresolution tree-structured networks to process point clouds for 3D shape understanding and generation tasks.
VisemeNet: Audio-Driven Animator-Centric Speech Animation
We present a novel deep-learning based approach to producing animator-centric speech motion curves that drive a JALI or standard FACS-based production face-rig, directly from input audio.
Graphics
SPLATNet: Sparse Lattice Networks for Point Cloud Processing
We present a network architecture for processing point clouds that directly operates on a collection of points represented as a sparse set of samples in a high-dimensional lattice.
Ranked #33 on
Semantic Segmentation
on ScanNet
(test mIoU metric)
CSGNet: Neural Shape Parser for Constructive Solid Geometry
In contrast, our model uses a recurrent neural network that parses the input shape in a top-down manner, which is significantly faster and yields a compact and easy-to-interpret sequence of modeling instructions.
Reasoning about Fine-grained Attribute Phrases using Reference Games
We collect a large dataset of such phrases by asking annotators to describe several visual differences between a pair of instances within a category.
Improved Bilinear Pooling with CNNs
We present an alternative scheme for computing gradients that is faster and yet it offers improvements over the baseline model.
3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks
The decoder converts this representation into depth and normal maps capturing the underlying surface from several output viewpoints.
Shape Generation using Spatially Partitioned Point Clouds
We propose to use the expressive power of neural networks to learn a distribution over the shape coefficients in a generative-adversarial framework.
3D Shape Induction from 2D Views of Multiple Objects
In this paper we investigate the problem of inducing a distribution over three-dimensional structures given two-dimensional views of multiple objects taken from unknown viewpoints.
3D Shape Segmentation with Projective Convolutional Networks
Our architecture combines image-based Fully Convolutional Networks (FCNs) and surface-based Conditional Random Fields (CRFs) to yield coherent segmentations of 3D shapes.
Adapting Models to Signal Degradation using Distillation
Model compression and knowledge distillation have been successfully applied for cross-architecture and cross-domain transfer learning.
High Dimensional Inference with Random Maximum A-Posteriori Perturbations
This paper shows that the expected value of perturb-max inference with low dimensional perturbations can be used sequentially to generate unbiased samples from the Gibbs distribution.
Bilinear CNN Models for Fine-Grained Visual Recognition
We propose bilinear models, a recognition architecture that consists of two feature extractors whose outputs are multiplied using outer product at each location of the image and pooled to obtain an image descriptor.
Ranked #68 on
Fine-Grained Image Classification
on CUB-200-2011
Fine-Grained Image Classification
Fine-Grained Visual Recognition
Visualizing and Understanding Deep Texture Representations
A number of recent approaches have used deep convolutional neural networks (CNNs) to build texture representations.
Deep filter banks for texture recognition, description, and segmentation
Visual textures have played a key role in image understanding because they convey important semantics of images, and because texture representations that pool local image descriptors in an orderless manner have had a tremendous impact in diverse applications.
One-to-many face recognition with bilinear CNNs
We demonstrate the performance of the B-CNN model beginning from an AlexNet-style network pre-trained on ImageNet.
Deep Filter Banks for Texture Recognition and Segmentation
Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications.
Multi-view Convolutional Neural Networks for 3D Shape Recognition
A longstanding question in computer vision concerns the representation of 3D shapes for recognition: should 3D shapes be represented with descriptors operating on their native 3D formats, such as voxel grid or polygon mesh, or can they be effectively represented with view-based descriptors?
Ranked #103 on
3D Point Cloud Classification
on ModelNet40
Bilinear CNNs for Fine-grained Visual Recognition
We then present a systematic analysis of these networks and show that (1) the bilinear features are highly redundant and can be reduced by an order of magnitude in size without significant loss in accuracy, (2) are also effective for other image classification tasks such as texture and scene recognition, and (3) can be trained from scratch on the ImageNet dataset offering consistent improvements over the baseline architecture.
Ranked #27 on
Fine-Grained Image Classification
on NABirds
Fine-Grained Image Classification
Fine-Grained Visual Recognition
+1
Jointly Learning Multiple Measures of Similarities from Triplet Comparisons
Similarity between objects is multi-faceted and it can be easier for human annotators to measure it when the focus is on a specific aspect.
Deep convolutional filter banks for texture recognition and segmentation
Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications.
Parsing World's Skylines using Shape-Constrained MRFs
We propose an approach for segmenting the individual buildings in typical skyline images.
Understanding Objects in Detail with Fine-Grained Attributes
We show that the collected data can be used to study the relation between part detection and attribute prediction by diagnosing the performance of classifiers that pool information from different parts of an object.
Similarity Comparisons for Interactive Fine-Grained Categorization
Current human-in-the-loop fine-grained visual categorization systems depend on a predefined vocabulary of attributes and parts, usually determined by experts.
Learning Efficient Random Maximum A-Posteriori Predictors with Non-Decomposable Loss Functions
In this work we develop efficient methods for learning random MAP predictors for structured label problems.
Describing Textures in the Wild
Patterns and textures are defining characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly.
On Sampling from the Gibbs Distribution with Random Maximum A-Posteriori Perturbations
In this paper we describe how MAP inference can be used to sample efficiently from Gibbs distributions.
Fine-Grained Visual Classification of Aircraft
This paper introduces FGVC-Aircraft, a new dataset containing 10, 000 images of aircraft spanning 100 aircraft models, organised in a three-level hierarchy.
Part Discovery from Partial Correspondence
We study the problem of part discovery when partial correspondence between instances of a category are available.
