Search Results for author:
Found 77 papers, 26 papers with code
Align and Distill: Unifying and Improving Domain Adaptive Object Detection
We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, enabling evaluation on diverse real-world data, and (4) A new method, ALDI++, that achieves state-of-the-art results by a large margin.
Ranked #1 on
Unsupervised Domain Adaptation
on CFC-DAOD
Diversified Ensembling: An Experiment in Crowdsourced Machine Learning
There, unlike in classical crowdsourced ML, participants deliberately specialize their efforts by working on subproblems, such as demographic subgroups in the service of fairness.
On the Effect of Image Resolution on Semantic Segmentation
We have rigorously tested our method using leading-edge semantic segmentation datasets.
A Foundation Model for Cell Segmentation
Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive.
Text-image Alignment for Diffusion-based Perception
Our cross-domain segmentation method, trained on Cityscapes, achieves SOTA results on Dark Zurich-val and Nighttime Driving.
Ranked #1 on
Semantic Segmentation
on Nighttime Driving
Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluation
We validate our method quantitatively by evaluating race and gender biases of three research-grade face recognition models.
Spatial Implicit Neural Representations for Global-Scale Species Mapping
Estimating the geographical range of a species from sparse observations is a challenging and important geospatial prediction problem.
Confidence Intervals for Error Rates in 1:1 Matching Tasks: Critical Statistical Analysis and Recommendations
Matching algorithms are commonly used to predict matches between items in a collection.
Understanding Label Bias in Single Positive Multi-Label Learning
Annotating data for multi-label classification is prohibitively expensive because every category of interest must be confirmed to be present or absent.
BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos
In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations.
Visual Knowledge Tracing
In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks.
On Label Granularity and Object Localization
Weakly supervised object localization (WSOL) aims to learn representations that encode object location using only image-level category labels.
The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting
We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos.
Near Perfect GAN Inversion
To edit a real photo using Generative Adversarial Networks (GANs), we need a GAN inversion algorithm to identify the latent vector that perfectly reproduces it.
Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer
Text spotting end-to-end methods have recently gained attention in the literature due to the benefits of jointly optimizing the text detection and recognition components.
Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features
We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant.
Multi-Dimensional, Nuanced and Subjective - Measuring the Perception of Facial Expressions
Humans can perceive multiple expressions, each one with varying intensity, in the picture of a face.
Self-Supervised Keypoint Discovery in Behavioral Videos
We propose a method for learning the posture and structure of agents from unlabelled behavioral videos.
Ranked #1 on
Unsupervised Human Pose Estimation
on Human3.6M
Weakly Supervised Keypoint Discovery
In this paper, we propose a method for keypoint discovery from a 2D image using image-level supervision.
Species Distribution Modeling for Machine Learning Practitioners: A Review
How many species live there?
Multi-Label Learning from Single Positive Labels
When the number of potential labels is large, human annotators find it difficult to mention all applicable labels for each training image.
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Multi-agent behavior modeling aims to understand the interactions that occur between agents.
Matched sample selection with GANs for mitigating attribute confounding
Measuring biases of vision systems with respect to protected attributes like gender and age is critical as these systems gain widespread use in society.
A linearized framework and a new benchmark for model selection for fine-tuning
Since all model selection algorithms in the literature have been tested on different use-cases and never compared directly, we introduce a new comprehensive benchmark for model selection comprising of: i) A model zoo of single and multi-domain models, and ii) Many target tasks.
Sequence-to-Sequence Contrastive Learning for Text Recognition
We propose a framework for sequence-to-sequence contrastive learning (SeqCLR) of visual representations, which we apply to text recognition.
A Number Sense as an Emergent Property of the Manipulating Brain
As a result, our model acquires the ability to estimate numerosity, i. e. the number of objects in the scene, as well as subitization, i. e. the ability to recognize at a glance the exact number of objects in small scenes.
Task Programming: Learning Data Efficient Behavior Representations
The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts.
Towards causal benchmarking of bias in face analysis algorithms
To address this problem we develop an experimental method for measuring algorithmic bias of face analysis algorithms, which manipulates directly the attributes of interest, e. g., gender and skin tone, in order to reveal causal links between attribute variation and performance change.
Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications
Our training procedure builds on insights from recent video classification literature and uses a trainable 3D CNN to learn the visual features.
Ranked #4 on
Zero-Shot Action Recognition
on ActivityNet
Geocoding of trees from street addresses and street-level images
We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching.
HMM-guided frame querying for bandwidth-constrained video search
We design an agent to search for frames of interest in video stored on a remote server, under bandwidth constraints.
PanDA: Panoptic Data Augmentation
The recently proposed panoptic segmentation task presents a significant challenge of image understanding with computer vision by unifying semantic segmentation and instance segmentation tasks.
From Google Maps to a Fine-Grained Catalog of Street trees
We believe this is the first work to exploit publicly available image data for fine-grained tree mapping at city-scale, respectively over many thousands of trees.
Anchor Loss: Modulating Loss Scale based on Prediction Difficulty
We propose a novel loss function that dynamically rescales the cross entropy based on prediction difficulty regarding a sample.
The iWildCam 2019 Challenge Dataset
We use the Caltech Camera Traps dataset, collected from the American Southwest, as training data.
Presence-Only Geographical Priors for Fine-Grained Image Classification
Appearance information alone is often not sufficient to accurately differentiate between fine-grained visual categories.
The iWildCam 2018 Challenge Dataset
Camera traps are a valuable tool for studying biodiversity, but research using this data is limited by the speed of human annotation.
Synthetic Examples Improve Generalization for Rare Classes
The ability to detect and classify rare occurrences in images has important applications - for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars.
Task2Vec: Task Embedding for Meta-Learning
We demonstrate that this embedding is capable of predicting task similarities that match our intuition about semantic and taxonomic relations between different visual tasks (e. g., tasks based on classifying different types of plants are similar) We also demonstrate the practical value of this framework for the meta-task of selecting a pre-trained feature extractor for a new task.
Recognition in Terra Incognita
The challenge is learning recognition in a handful of locations, and generalizing animal detection and classification to new locations where no training data is available.
Lean Multiclass Crowdsourcing
We introduce a method for efficiently crowdsourcing multiclass annotations in challenging, real world image datasets.
Teaching Multiple Concepts to a Forgetful Learner
Our framework is both generic, allowing the design of teaching schedules for different memory models, and also interactive, allowing the teacher to adapt the schedule to the underlying forgetting mechanisms of the learner.
It's all Relative: Monocular 3D Human Pose Estimation from Weakly Supervised Data
We address the problem of 3D human pose estimation from 2D input images using only weakly supervised training data.
Fast Conditional Independence Test for Vector Variables with Large Sample Sizes
The test is based on the idea that when $P(X \mid Y, Z) = P(X \mid Y)$, $Z$ is not useful as a feature to predict $X$, as long as $Y$ is also a regressor.
Teaching Categories to Human Learners with Visual Explanations
We study the problem of computer-assisted teaching with explanations.
Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners
We highlight that adaptivity does not speed up the teaching process when considering existing models of version space learners, such as "worst-case" (the learner picks the next hypothesis randomly from the version space) and "preference-based" (the learner picks hypothesis according to some global preference).
Context Embedding Networks
Low dimensional embeddings that capture the main variations of interest in collections of data are important for many applications.
The Devil is in the Tails: Fine-grained Classification in the Wild
We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods.
A Simple Multi-Class Boosting Framework with Theoretical Guarantees and Empirical Proficiency
There is a need for simple yet accurate white-box learning systems that train quickly and with little data.
The iNaturalist Species Classification and Detection Dataset
Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories.
Ranked #8 on
Image Classification
on iNaturalist
Benchmarking and Error Diagnosis in Multi-Instance Pose Estimation
We propose a new method to analyze the impact of errors in algorithms for multi-instance pose estimation and a principled benchmark that can be used to compare them.
Lean Crowdsourcing: Combining Humans and Machines in an Online System
We develop specialized models and algorithms for binary annotation, part keypoint annotation, and sets of bounding box annotations.
Deciding How to Decide: Dynamic Routing in Artificial Neural Networks
We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths.
Estimating Causal Direction and Confounding of Two Discrete Variables
We propose a method to classify the causal relationship between two discrete variables given only the joint distribution of the variables, acknowledging that the method is subject to an inherent baseline error.
Learning recurrent representations for hierarchical behavior modeling
We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network.
Seeing into Darkness: Scotopic Visual Recognition
Images are formed by counting how many photons traveling from a given set of directions hit an image sensor during a given time interval.
Improved Multi-Class Cost-Sensitive Boosting via Estimation of the Minimum-Risk Class
We present a simple unified framework for multi-class cost-sensitive boosting.
Cataloging Public Objects Using Aerial and Street-Level Images - Urban Trees
The main technical challenge is combining test time information from multiple views of each geographic location (e. g., aerial and street views).
Unsupervised Discovery of El Nino Using Causal Feature Learning on Microlevel Climate Data
We show that the climate phenomena of El Nino and La Nina arise naturally as states of macro-variables when our recent causal feature learning framework (Chalupka 2015, Chalupka 2016) is applied to micro-level measures of zonal wind (ZW) and sea surface temperatures (SST) taken over the equatorial band of the Pacific Ocean.
Fine-Grained Classification of Pedestrians in Video: Benchmark and State of the Art
This dataset is designed to train and test algorithms for fine-grained categorisation of people, it is also useful for benchmarking tracking, detection and pose estimation of pedestrians.
Multi-Level Cause-Effect Systems
We formalize the connection between micro- and macro-variables in such situations and provide a coherent framework describing causal relations at multiple levels of analysis.
Generalized Regressive Motion: a Visual Cue to Collision
Looming has been proposed as the main monocular visual cue for detecting the approach of other animals and avoiding collisions with stationary obstacles.
Describing Common Human Visual Actions in Images
We address these questions by exploring the actions and interactions that are detectable in the images of the MS COCO dataset.
Building a Bird Recognition App and Large Scale Dataset With Citizen Scientists: The Fine Print in Fine-Grained Dataset Collection
We worked with bird experts to measure the quality of popular datasets like CUB-200-2011 and ImageNet and found class label error rates of at least 4%.
Visual Causal Feature Learning
We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems.
Reconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling
We frame the task of predicting a semantic labeling as a sparse reconstruction procedure that applies a target-specific learned transfer function to a generic deep sparse code representation of an image.
Bird Species Categorization Using Pose Normalized Deep Convolutional Nets
We perform a detailed investigation of state-of-the-art deep convolutional feature implementations and fine-tuning feature learning for fine-grained classification.
Similarity Comparisons for Interactive Fine-Grained Categorization
Current human-in-the-loop fine-grained visual categorization systems depend on a predefined vocabulary of attributes and parts, usually determined by experts.
Active Annotation Translation
Human annotators may be thought of as helping translate the old annotations into the new ones.
From Categories to Individuals in Real Time -- A Unified Boosting Approach
A method for online, real-time learning of individual-object detectors is presented.
Microsoft COCO: Common Objects in Context
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding.
A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration
How many labeled examples are needed to estimate a classifier's performance on a new dataset?
Predicting response time and error rates in visual search
A model of human visual search is proposed.
Discriminative Clustering by Regularized Information Maximization
We present a framework that simultaneously clusters the data and trains a discriminative classifier.
Graph-Based Visual Saliency
A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed.
