Search Results for author: Pietro Perona

Found 83 papers, 31 papers with code

Unsupervised Representation Learning from Sparse Transformation Analysis

1 code implementation7 Oct 2024 Yue Song, Thomas Anderson Keller, Yisong Yue, Pietro Perona, Max Welling

In this paper we propose to learn representations from sequence data by factorizing the transformations of the latent variables into sparse components.

Representation Learning

Social perception of faces in a vision-language model

1 code implementation26 Aug 2024 Carina I. Hausladen, Manuel Knott, Colin F. Camerer, Pietro Perona

To this end, we compare the similarity in CLIP embeddings between different textual prompts and a set of face images.

Language Modelling

A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification

no code implementations16 Jul 2024 Markus Marks, Manuel Knott, Neehar Kondapaneni, Elijah Cole, Thijs Defraeye, Fernando Perez-Cruz, Pietro Perona

However, it is not well understood how well these evaluation protocols estimate the representation quality of a pre-trained model for different downstream tasks under different conditions, such as dataset, metric, and model architecture.

Benchmarking Few-Shot Learning +2

A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimation

1 code implementation11 Jun 2024 Riccardo Fogliato, Pratik Patil, Mathew Monfort, Pietro Perona

Model performance evaluation is a critical and expensive task in machine learning and computer vision.

Align and Distill: Unifying and Improving Domain Adaptive Object Detection

1 code implementation18 Mar 2024 Justin Kay, Timm Haucke, Suzanne Stathatos, Siqi Deng, Erik Young, Pietro Perona, Sara Beery, Grant van Horn

We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, enabling evaluation on diverse real-world data, and (4) A new method, ALDI++, that achieves state-of-the-art results by a large margin.

Benchmarking object-detection +2

Diversified Ensembling: An Experiment in Crowdsourced Machine Learning

no code implementations16 Feb 2024 Ira Globus-Harris, Declan Harrison, Michael Kearns, Pietro Perona, Aaron Roth

There, unlike in classical crowdsourced ML, participants deliberately specialize their efforts by working on subproblems, such as demographic subgroups in the service of fairness.

Fairness Holdout Set +1

A Foundation Model for Cell Segmentation

no code implementations18 Nov 2023 Uriah Israel, Markus Marks, Rohit Dilip, Qilin Li, Morgan Schwartz, Elora Pradhan, Edward Pao, Shenyi Li, Alexander Pearson-Goulart, Pietro Perona, Georgia Gkioxari, Ross Barnowski, Yisong Yue, David Van Valen

Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive.

Cell Segmentation Prompt Engineering +1

Spatial Implicit Neural Representations for Global-Scale Species Mapping

2 code implementations5 Jun 2023 Elijah Cole, Grant van Horn, Christian Lange, Alexander Shepard, Patrick Leary, Pietro Perona, Scott Loarie, Oisin Mac Aodha

Estimating the geographical range of a species from sparse observations is a challenging and important geospatial prediction problem.

Representation Learning

Understanding Label Bias in Single Positive Multi-Label Learning

no code implementations24 May 2023 Julio Arroyo, Pietro Perona, Elijah Cole

Annotating data for multi-label classification is prohibitively expensive because every category of interest must be confirmed to be present or absent.

Classification Multi-Label Classification +1

BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos

1 code implementation CVPR 2023 Jennifer J. Sun, Lili Karashchuk, Amil Dravid, Serim Ryou, Sonia Fereidooni, John Tuthill, Aggelos Katsaggelos, Bingni W. Brunton, Georgia Gkioxari, Ann Kennedy, Yisong Yue, Pietro Perona

In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.

Decoder

On Label Granularity and Object Localization

1 code implementation20 Jul 2022 Elijah Cole, Kimberly Wilber, Grant van Horn, Xuan Yang, Marco Fornoni, Pietro Perona, Serge Belongie, Andrew Howard, Oisin Mac Aodha

Weakly supervised object localization (WSOL) aims to learn representations that encode object location using only image-level category labels.

Object Weakly-Supervised Object Localization

Visual Knowledge Tracing

1 code implementation20 Jul 2022 Neehar Kondapaneni, Pietro Perona, Oisin Mac Aodha

In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks.

Autonomous Driving General Classification +1

Near Perfect GAN Inversion

no code implementations23 Feb 2022 Qianli Feng, Viraj Shah, Raghudeep Gadde, Pietro Perona, Aleix Martinez

To edit a real photo using Generative Adversarial Networks (GANs), we need a GAN inversion algorithm to identify the latent vector that perfectly reproduces it.

Diversity

Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer

no code implementations CVPR 2022 Yair Kittenplon, Inbal Lavi, Sharon Fogel, Yarin Bar, R. Manmatha, Pietro Perona

Text spotting end-to-end methods have recently gained attention in the literature due to the benefits of jointly optimizing the text detection and recognition components.

Text Detection Text Spotting

Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features

no code implementations25 Jan 2022 Guha Balakrishnan, Raghudeep Gadde, Aleix Martinez, Pietro Perona

We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant.

Weakly Supervised Keypoint Discovery

no code implementations28 Sep 2021 Serim Ryou, Pietro Perona

In this paper, we propose a method for keypoint discovery from a 2D image using image-level supervision.

Conditional Image Generation Keypoint Estimation +1

Multi-Label Learning from Single Positive Labels

2 code implementations CVPR 2021 Elijah Cole, Oisin Mac Aodha, Titouan Lorieul, Pietro Perona, Dan Morris, Nebojsa Jojic

When the number of potential labels is large, human annotators find it difficult to mention all applicable labels for each training image.

Missing Labels Multi-Label Image Classification

Matched sample selection with GANs for mitigating attribute confounding

1 code implementation24 Mar 2021 Chandan Singh, Guha Balakrishnan, Pietro Perona

Measuring biases of vision systems with respect to protected attributes like gender and age is critical as these systems gain widespread use in society.

Attribute Face Generation +3

A linearized framework and a new benchmark for model selection for fine-tuning

no code implementations29 Jan 2021 Aditya Deshpande, Alessandro Achille, Avinash Ravichandran, Hao Li, Luca Zancato, Charless Fowlkes, Rahul Bhotika, Stefano Soatto, Pietro Perona

Since all model selection algorithms in the literature have been tested on different use-cases and never compared directly, we introduce a new comprehensive benchmark for model selection comprising of: i) A model zoo of single and multi-domain models, and ii) Many target tasks.

Feature Correlation Model Selection

Sequence-to-Sequence Contrastive Learning for Text Recognition

2 code implementations CVPR 2021 Aviad Aberdam, Ron Litman, Shahar Tsiper, Oron Anschel, Ron Slossberg, Shai Mazor, R. Manmatha, Pietro Perona

We propose a framework for sequence-to-sequence contrastive learning (SeqCLR) of visual representations, which we apply to text recognition.

Contrastive Learning Decoder +1

A Number Sense as an Emergent Property of the Manipulating Brain

1 code implementation8 Dec 2020 Neehar Kondapaneni, Pietro Perona

As a result, our model acquires the ability to estimate numerosity, i. e. the number of objects in the scene, as well as subitization, i. e. the ability to recognize at a glance the exact number of objects in small scenes.

Task Programming: Learning Data Efficient Behavior Representations

1 code implementation CVPR 2021 Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Yue, Pietro Perona

The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts.

Self-Supervised Learning

Towards causal benchmarking of bias in face analysis algorithms

1 code implementation ECCV 2020 Guha Balakrishnan, Yuanjun Xiong, Wei Xia, Pietro Perona

To address this problem we develop an experimental method for measuring algorithmic bias of face analysis algorithms, which manipulates directly the attributes of interest, e. g., gender and skin tone, in order to reveal causal links between attribute variation and performance change.

Attribute Benchmarking +2

Geocoding of trees from street addresses and street-level images

no code implementations5 Feb 2020 Daniel Laumer, Nico Lang, Natalie van Doorn, Oisin Mac Aodha, Pietro Perona, Jan Dirk Wegner

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching.

HMM-guided frame querying for bandwidth-constrained video search

no code implementations31 Dec 2019 Bhairav Chidambaram, Mason McGill, Pietro Perona

We design an agent to search for frames of interest in video stored on a remote server, under bandwidth constraints.

PanDA: Panoptic Data Augmentation

no code implementations27 Nov 2019 Yang Liu, Pietro Perona, Markus Meister

The recently proposed panoptic segmentation task presents a significant challenge of image understanding with computer vision by unifying semantic segmentation and instance segmentation tasks.

Data Augmentation Instance Segmentation +2

From Google Maps to a Fine-Grained Catalog of Street trees

no code implementations7 Oct 2019 Steve Branson, Jan Dirk Wegner, David Hall, Nico Lang, Konrad Schindler, Pietro Perona

We believe this is the first work to exploit publicly available image data for fine-grained tree mapping at city-scale, respectively over many thousands of trees.

Anchor Loss: Modulating Loss Scale based on Prediction Difficulty

1 code implementation ICCV 2019 Serim Ryou, Seong-Gyun Jeong, Pietro Perona

We propose a novel loss function that dynamically rescales the cross entropy based on prediction difficulty regarding a sample.

General Classification Image Classification +1

The iWildCam 2019 Challenge Dataset

no code implementations15 Jul 2019 Sara Beery, Dan Morris, Pietro Perona

We use the Caltech Camera Traps dataset, collected from the American Southwest, as training data.

Transfer Learning

Synthetic Examples Improve Generalization for Rare Classes

no code implementations11 Apr 2019 Sara Beery, Yang Liu, Dan Morris, Jim Piavis, Ashish Kapoor, Markus Meister, Neel Joshi, Pietro Perona

The ability to detect and classify rare occurrences in images has important applications - for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars.

Few-Shot Learning Self-Driving Cars

The iWildCam 2018 Challenge Dataset

no code implementations11 Apr 2019 Sara Beery, Grant van Horn, Oisin Mac Aodha, Pietro Perona

Camera traps are a valuable tool for studying biodiversity, but research using this data is limited by the speed of human annotation.

Task2Vec: Task Embedding for Meta-Learning

1 code implementation ICCV 2019 Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Stefano Soatto, Pietro Perona

We demonstrate that this embedding is capable of predicting task similarities that match our intuition about semantic and taxonomic relations between different visual tasks (e. g., tasks based on classifying different types of plants are similar) We also demonstrate the practical value of this framework for the meta-task of selecting a pre-trained feature extractor for a new task.

Meta-Learning

Recognition in Terra Incognita

3 code implementations ECCV 2018 Sara Beery, Grant van Horn, Pietro Perona

The challenge is learning recognition in a handful of locations, and generalizing animal detection and classification to new locations where no training data is available.

Classification General Classification

Lean Multiclass Crowdsourcing

no code implementations CVPR 2018 Grant Van Horn, Steve Branson, Scott Loarie, Serge Belongie, Pietro Perona

We introduce a method for efficiently crowdsourcing multiclass annotations in challenging, real world image datasets.

Teaching Multiple Concepts to a Forgetful Learner

no code implementations NeurIPS 2019 Anette Hunziker, Yuxin Chen, Oisin Mac Aodha, Manuel Gomez Rodriguez, Andreas Krause, Pietro Perona, Yisong Yue, Adish Singla

Our framework is both generic, allowing the design of teaching schedules for different memory models, and also interactive, allowing the teacher to adapt the schedule to the underlying forgetting mechanisms of the learner.

Scheduling

Fast Conditional Independence Test for Vector Variables with Large Sample Sizes

1 code implementation8 Apr 2018 Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt

The test is based on the idea that when $P(X \mid Y, Z) = P(X \mid Y)$, $Z$ is not useful as a feature to predict $X$, as long as $Y$ is also a regressor.

Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners

no code implementations NeurIPS 2018 Yuxin Chen, Adish Singla, Oisin Mac Aodha, Pietro Perona, Yisong Yue

We highlight that adaptivity does not speed up the teaching process when considering existing models of version space learners, such as "worst-case" (the learner picks the next hypothesis randomly from the version space) and "preference-based" (the learner picks hypothesis according to some global preference).

Context Embedding Networks

no code implementations CVPR 2018 Kun Ho Kim, Oisin Mac Aodha, Pietro Perona

Low dimensional embeddings that capture the main variations of interest in collections of data are important for many applications.

The Devil is in the Tails: Fine-grained Classification in the Wild

no code implementations5 Sep 2017 Grant Van Horn, Pietro Perona

We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods.

Classification General Classification +1

A Simple Multi-Class Boosting Framework with Theoretical Guarantees and Empirical Proficiency

no code implementations ICML 2017 Ron Appel, Pietro Perona

There is a need for simple yet accurate white-box learning systems that train quickly and with little data.

The iNaturalist Species Classification and Detection Dataset

19 code implementations CVPR 2018 Grant Van Horn, Oisin Mac Aodha, Yang song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, Serge Belongie

Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories.

General Classification Image Classification

Benchmarking and Error Diagnosis in Multi-Instance Pose Estimation

no code implementations ICCV 2017 Matteo Ruggero Ronchi, Pietro Perona

We propose a new method to analyze the impact of errors in algorithms for multi-instance pose estimation and a principled benchmark that can be used to compare them.

Benchmarking Pose Estimation

Lean Crowdsourcing: Combining Humans and Machines in an Online System

no code implementations CVPR 2017 Steve Branson, Grant van Horn, Pietro Perona

We develop specialized models and algorithms for binary annotation, part keypoint annotation, and sets of bounding box annotations.

Deciding How to Decide: Dynamic Routing in Artificial Neural Networks

1 code implementation ICML 2017 Mason McGill, Pietro Perona

We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths.

Estimating Causal Direction and Confounding of Two Discrete Variables

no code implementations4 Nov 2016 Krzysztof Chalupka, Frederick Eberhardt, Pietro Perona

We propose a method to classify the causal relationship between two discrete variables given only the joint distribution of the variables, acknowledging that the method is subject to an inherent baseline error.

Vocal Bursts Valence Prediction

Learning recurrent representations for hierarchical behavior modeling

no code implementations1 Nov 2016 Eyrun Eyjolfsdottir, Kristin Branson, Yisong Yue, Pietro Perona

We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network.

Action Detection motion prediction

Seeing into Darkness: Scotopic Visual Recognition

1 code implementation CVPR 2017 Bo Chen, Pietro Perona

Images are formed by counting how many photons traveling from a given set of directions hit an image sensor during a given time interval.

Astronomy General Classification

Cataloging Public Objects Using Aerial and Street-Level Images - Urban Trees

no code implementations CVPR 2016 Jan D. Wegner, Steven Branson, David Hall, Konrad Schindler, Pietro Perona

The main technical challenge is combining test time information from multiple views of each geographic location (e. g., aerial and street views).

Unsupervised Discovery of El Nino Using Causal Feature Learning on Microlevel Climate Data

no code implementations30 May 2016 Krzysztof Chalupka, Tobias Bischoff, Pietro Perona, Frederick Eberhardt

We show that the climate phenomena of El Nino and La Nina arise naturally as states of macro-variables when our recent causal feature learning framework (Chalupka 2015, Chalupka 2016) is applied to micro-level measures of zonal wind (ZW) and sea surface temperatures (SST) taken over the equatorial band of the Pacific Ocean.

Clustering

Fine-Grained Classification of Pedestrians in Video: Benchmark and State of the Art

no code implementations CVPR 2015 David Hall, Pietro Perona

This dataset is designed to train and test algorithms for fine-grained categorisation of people, it is also useful for benchmarking tracking, detection and pose estimation of pedestrians.

Benchmarking General Classification +1

Multi-Level Cause-Effect Systems

no code implementations25 Dec 2015 Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt

We formalize the connection between micro- and macro-variables in such situations and provide a coherent framework describing causal relations at multiple levels of analysis.

Generalized Regressive Motion: a Visual Cue to Collision

no code implementations26 Oct 2015 Krzysztof Chalupka, Michael Dickinson, Pietro Perona

Looming has been proposed as the main monocular visual cue for detecting the approach of other animals and avoiding collisions with stationary obstacles.

Describing Common Human Visual Actions in Images

no code implementations7 Jun 2015 Matteo Ruggero Ronchi, Pietro Perona

We address these questions by exploring the actions and interactions that are detectable in the images of the MS COCO dataset.

Building a Bird Recognition App and Large Scale Dataset With Citizen Scientists: The Fine Print in Fine-Grained Dataset Collection

no code implementations CVPR 2015 Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, Serge Belongie

We worked with bird experts to measure the quality of popular datasets like CUB-200-2011 and ImageNet and found class label error rates of at least 4%.

Visual Causal Feature Learning

no code implementations7 Dec 2014 Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt

We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems.

Active Learning

Reconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling

no code implementations16 Oct 2014 Michael Maire, Stella X. Yu, Pietro Perona

We frame the task of predicting a semantic labeling as a sparse reconstruction procedure that applies a target-specific learned transfer function to a generic deep sparse code representation of an image.

Contour Detection Dictionary Learning

Bird Species Categorization Using Pose Normalized Deep Convolutional Nets

no code implementations11 Jun 2014 Steve Branson, Grant van Horn, Serge Belongie, Pietro Perona

We perform a detailed investigation of state-of-the-art deep convolutional feature implementations and fine-tuning feature learning for fine-grained classification.

Classification Clustering +2

Similarity Comparisons for Interactive Fine-Grained Categorization

no code implementations CVPR 2014 Catherine Wah, Grant van Horn, Steve Branson, Subhransu Maji, Pietro Perona, Serge Belongie

Current human-in-the-loop fine-grained visual categorization systems depend on a predefined vocabulary of attributes and parts, usually determined by experts.

Attribute Fine-Grained Visual Categorization +3

Active Annotation Translation

no code implementations CVPR 2014 Steve Branson, Kristjan Eldjarn Hjorleifsson, Pietro Perona

Human annotators may be thought of as helping translate the old annotations into the new ones.

Attribute Translation

Microsoft COCO: Common Objects in Context

36 code implementations1 May 2014 Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollár

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding.

Instance Segmentation Object +5

Cannot find the paper you are looking for? You can Submit a new open access paper.