1 code implementation • 7 Oct 2024 • Yue Song, Thomas Anderson Keller, Yisong Yue, Pietro Perona, Max Welling
In this paper we propose to learn representations from sequence data by factorizing the transformations of the latent variables into sparse components.
no code implementations • 14 Sep 2024 • Daniel Khalil, Christina Liu, Pietro Perona, Jennifer J. Sun, Markus Marks
The study of social interactions and collective behaviors through multi-agent video analysis is crucial in biology.
1 code implementation • 26 Aug 2024 • Carina I. Hausladen, Manuel Knott, Colin F. Camerer, Pietro Perona
To this end, we compare the similarity in CLIP embeddings between different textual prompts and a set of face images.
no code implementations • 16 Jul 2024 • Markus Marks, Manuel Knott, Neehar Kondapaneni, Elijah Cole, Thijs Defraeye, Fernando Perez-Cruz, Pietro Perona
However, it is not well understood how well these evaluation protocols estimate the representation quality of a pre-trained model for different downstream tasks under different conditions, such as dataset, metric, and model architecture.
1 code implementation • 11 Jun 2024 • Riccardo Fogliato, Pratik Patil, Mathew Monfort, Pietro Perona
Model performance evaluation is a critical and expensive task in machine learning and computer vision.
1 code implementation • 24 May 2024 • Neehar Kondapaneni, Markus Marks, Oisin Mac Aodha, Pietro Perona
It produces 1/30 of CRP's explanations while only resulting in a slight reduction in explanation quality.
Explainable artificial intelligence Explainable Artificial Intelligence (XAI)
1 code implementation • 18 Mar 2024 • Justin Kay, Timm Haucke, Suzanne Stathatos, Siqi Deng, Erik Young, Pietro Perona, Sara Beery, Grant van Horn
We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, enabling evaluation on diverse real-world data, and (4) A new method, ALDI++, that achieves state-of-the-art results by a large margin.
Ranked #1 on Unsupervised Domain Adaptation on CFC-DAOD
no code implementations • 16 Feb 2024 • Ira Globus-Harris, Declan Harrison, Michael Kearns, Pietro Perona, Aaron Roth
There, unlike in classical crowdsourced ML, participants deliberately specialize their efforts by working on subproblems, such as demographic subgroups in the service of fairness.
no code implementations • 8 Feb 2024 • Ritambhara Singh, Abhishek Jain, Pietro Perona, Shivani Agarwal, Junfeng Yang
We have rigorously tested our method using leading-edge semantic segmentation datasets.
no code implementations • 18 Nov 2023 • Uriah Israel, Markus Marks, Rohit Dilip, Qilin Li, Morgan Schwartz, Elora Pradhan, Edward Pao, Shenyi Li, Alexander Pearson-Goulart, Pietro Perona, Georgia Gkioxari, Ross Barnowski, Yisong Yue, David Van Valen
Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive.
1 code implementation • CVPR 2024 • Neehar Kondapaneni, Markus Marks, Manuel Knott, Rogerio Guimaraes, Pietro Perona
Our cross-domain segmentation method, trained on Cityscapes, achieves SOTA results on Dark Zurich-val and Nighttime Driving.
Ranked #1 on Semantic Segmentation on Nighttime Driving
no code implementations • ICCV 2023 • Hao Liang, Pietro Perona, Guha Balakrishnan
We validate our method quantitatively by evaluating race and gender biases of three research-grade face recognition models.
2 code implementations • 5 Jun 2023 • Elijah Cole, Grant van Horn, Christian Lange, Alexander Shepard, Patrick Leary, Pietro Perona, Scott Loarie, Oisin Mac Aodha
Estimating the geographical range of a species from sparse observations is a challenging and important geospatial prediction problem.
1 code implementation • 1 Jun 2023 • Riccardo Fogliato, Pratik Patil, Pietro Perona
Matching algorithms are commonly used to predict matches between items in a collection.
no code implementations • 24 May 2023 • Julio Arroyo, Pietro Perona, Elijah Cole
Annotating data for multi-label classification is prohibitively expensive because every category of interest must be confirmed to be present or absent.
1 code implementation • CVPR 2023 • Jennifer J. Sun, Lili Karashchuk, Amil Dravid, Serim Ryou, Sonia Fereidooni, John Tuthill, Aggelos Katsaggelos, Bingni W. Brunton, Georgia Gkioxari, Ann Kennedy, Yisong Yue, Pietro Perona
In this way, we discover keypoints without requiring manual supervision in videos of humans and rats, demonstrating the potential of 3D keypoint discovery for studying behavior.
1 code implementation • 21 Jul 2022 • Jennifer J. Sun, Markus Marks, Andrew Ulmer, Dipam Chakraborty, Brian Geuther, Edward Hayes, Heng Jia, Vivek Kumar, Sebastian Oleszko, Zachary Partridge, Milan Peelman, Alice Robie, Catherine E. Schretter, Keith Sheppard, Chao Sun, Param Uttarwar, Julian M. Wagner, Eric Werner, Joseph Parker, Pietro Perona, Yisong Yue, Kristin Branson, Ann Kennedy
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations.
1 code implementation • 20 Jul 2022 • Elijah Cole, Kimberly Wilber, Grant van Horn, Xuan Yang, Marco Fornoni, Pietro Perona, Serge Belongie, Andrew Howard, Oisin Mac Aodha
Weakly supervised object localization (WSOL) aims to learn representations that encode object location using only image-level category labels.
1 code implementation • 20 Jul 2022 • Neehar Kondapaneni, Pietro Perona, Oisin Mac Aodha
In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks.
1 code implementation • 19 Jul 2022 • Justin Kay, Peter Kulits, Suzanne Stathatos, Siqi Deng, Erik Young, Sara Beery, Grant van Horn, Pietro Perona
We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos.
no code implementations • 23 Feb 2022 • Qianli Feng, Viraj Shah, Raghudeep Gadde, Pietro Perona, Aleix Martinez
To edit a real photo using Generative Adversarial Networks (GANs), we need a GAN inversion algorithm to identify the latent vector that perfectly reproduces it.
no code implementations • CVPR 2022 • Yair Kittenplon, Inbal Lavi, Sharon Fogel, Yarin Bar, R. Manmatha, Pietro Perona
Text spotting end-to-end methods have recently gained attention in the literature due to the benefits of jointly optimizing the text detection and recognition components.
no code implementations • 25 Jan 2022 • Guha Balakrishnan, Raghudeep Gadde, Aleix Martinez, Pietro Perona
We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant.
no code implementations • CVPR 2022 • De'Aira Bryant, Siqi Deng, Nashlie Sephus, Wei Xia, Pietro Perona
Humans can perceive multiple expressions, each one with varying intensity, in the picture of a face.
1 code implementation • CVPR 2022 • Jennifer J. Sun, Serim Ryou, Roni Goldshmid, Brandon Weissbourd, John Dabiri, David J. Anderson, Ann Kennedy, Yisong Yue, Pietro Perona
We propose a method for learning the posture and structure of agents from unlabelled behavioral videos.
Ranked #1 on Unsupervised Human Pose Estimation on Human3.6M
no code implementations • 28 Sep 2021 • Serim Ryou, Pietro Perona
In this paper, we propose a method for keypoint discovery from a 2D image using image-level supervision.
no code implementations • 3 Jul 2021 • Sara Beery, Elijah Cole, Joseph Parker, Pietro Perona, Kevin Winner
How many species live there?
2 code implementations • CVPR 2021 • Elijah Cole, Oisin Mac Aodha, Titouan Lorieul, Pietro Perona, Dan Morris, Nebojsa Jojic
When the number of potential labels is large, human annotators find it difficult to mention all applicable labels for each training image.
1 code implementation • 6 Apr 2021 • Jennifer J. Sun, Tomomi Karigo, Dipam Chakraborty, Sharada P. Mohanty, Benjamin Wild, Quan Sun, Chen Chen, David J. Anderson, Pietro Perona, Yisong Yue, Ann Kennedy
Multi-agent behavior modeling aims to understand the interactions that occur between agents.
1 code implementation • 24 Mar 2021 • Chandan Singh, Guha Balakrishnan, Pietro Perona
Measuring biases of vision systems with respect to protected attributes like gender and age is critical as these systems gain widespread use in society.
no code implementations • 29 Jan 2021 • Aditya Deshpande, Alessandro Achille, Avinash Ravichandran, Hao Li, Luca Zancato, Charless Fowlkes, Rahul Bhotika, Stefano Soatto, Pietro Perona
Since all model selection algorithms in the literature have been tested on different use-cases and never compared directly, we introduce a new comprehensive benchmark for model selection comprising of: i) A model zoo of single and multi-domain models, and ii) Many target tasks.
2 code implementations • CVPR 2021 • Aviad Aberdam, Ron Litman, Shahar Tsiper, Oron Anschel, Ron Slossberg, Shai Mazor, R. Manmatha, Pietro Perona
We propose a framework for sequence-to-sequence contrastive learning (SeqCLR) of visual representations, which we apply to text recognition.
1 code implementation • 8 Dec 2020 • Neehar Kondapaneni, Pietro Perona
As a result, our model acquires the ability to estimate numerosity, i. e. the number of objects in the scene, as well as subitization, i. e. the ability to recognize at a glance the exact number of objects in small scenes.
1 code implementation • CVPR 2021 • Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Yue, Pietro Perona
The tasks in our method can be efficiently engineered by domain experts through a process we call "task programming", which uses programs to explicitly encode structured knowledge from domain experts.
1 code implementation • ECCV 2020 • Guha Balakrishnan, Yuanjun Xiong, Wei Xia, Pietro Perona
To address this problem we develop an experimental method for measuring algorithmic bias of face analysis algorithms, which manipulates directly the attributes of interest, e. g., gender and skin tone, in order to reveal causal links between attribute variation and performance change.
1 code implementation • CVPR 2020 • Biagio Brattoli, Joseph Tighe, Fedor Zhdanov, Pietro Perona, Krzysztof Chalupka
Our training procedure builds on insights from recent video classification literature and uses a trainable 3D CNN to learn the visual features.
Ranked #4 on Zero-Shot Action Recognition on ActivityNet
no code implementations • 5 Feb 2020 • Daniel Laumer, Nico Lang, Natalie van Doorn, Oisin Mac Aodha, Pietro Perona, Jan Dirk Wegner
We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching.
no code implementations • 31 Dec 2019 • Bhairav Chidambaram, Mason McGill, Pietro Perona
We design an agent to search for frames of interest in video stored on a remote server, under bandwidth constraints.
no code implementations • 27 Nov 2019 • Yang Liu, Pietro Perona, Markus Meister
The recently proposed panoptic segmentation task presents a significant challenge of image understanding with computer vision by unifying semantic segmentation and instance segmentation tasks.
no code implementations • 7 Oct 2019 • Steve Branson, Jan Dirk Wegner, David Hall, Nico Lang, Konrad Schindler, Pietro Perona
We believe this is the first work to exploit publicly available image data for fine-grained tree mapping at city-scale, respectively over many thousands of trees.
1 code implementation • ICCV 2019 • Serim Ryou, Seong-Gyun Jeong, Pietro Perona
We propose a novel loss function that dynamically rescales the cross entropy based on prediction difficulty regarding a sample.
no code implementations • 15 Jul 2019 • Sara Beery, Dan Morris, Pietro Perona
We use the Caltech Camera Traps dataset, collected from the American Southwest, as training data.
4 code implementations • ICCV 2019 • Oisin Mac Aodha, Elijah Cole, Pietro Perona
Appearance information alone is often not sufficient to accurately differentiate between fine-grained visual categories.
no code implementations • 11 Apr 2019 • Sara Beery, Yang Liu, Dan Morris, Jim Piavis, Ashish Kapoor, Markus Meister, Neel Joshi, Pietro Perona
The ability to detect and classify rare occurrences in images has important applications - for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars.
no code implementations • 11 Apr 2019 • Sara Beery, Grant van Horn, Oisin Mac Aodha, Pietro Perona
Camera traps are a valuable tool for studying biodiversity, but research using this data is limited by the speed of human annotation.
1 code implementation • ICCV 2019 • Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Stefano Soatto, Pietro Perona
We demonstrate that this embedding is capable of predicting task similarities that match our intuition about semantic and taxonomic relations between different visual tasks (e. g., tasks based on classifying different types of plants are similar) We also demonstrate the practical value of this framework for the meta-task of selecting a pre-trained feature extractor for a new task.
3 code implementations • ECCV 2018 • Sara Beery, Grant van Horn, Pietro Perona
The challenge is learning recognition in a handful of locations, and generalizing animal detection and classification to new locations where no training data is available.
no code implementations • CVPR 2018 • Grant Van Horn, Steve Branson, Scott Loarie, Serge Belongie, Pietro Perona
We introduce a method for efficiently crowdsourcing multiclass annotations in challenging, real world image datasets.
no code implementations • NeurIPS 2019 • Anette Hunziker, Yuxin Chen, Oisin Mac Aodha, Manuel Gomez Rodriguez, Andreas Krause, Pietro Perona, Yisong Yue, Adish Singla
Our framework is both generic, allowing the design of teaching schedules for different memory models, and also interactive, allowing the teacher to adapt the schedule to the underlying forgetting mechanisms of the learner.
no code implementations • 17 May 2018 • Matteo Ruggero Ronchi, Oisin Mac Aodha, Robert Eng, Pietro Perona
We address the problem of 3D human pose estimation from 2D input images using only weakly supervised training data.
1 code implementation • 8 Apr 2018 • Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt
The test is based on the idea that when $P(X \mid Y, Z) = P(X \mid Y)$, $Z$ is not useful as a feature to predict $X$, as long as $Y$ is also a regressor.
no code implementations • CVPR 2018 • Oisin Mac Aodha, Shih-An Su, Yuxin Chen, Pietro Perona, Yisong Yue
We study the problem of computer-assisted teaching with explanations.
no code implementations • NeurIPS 2018 • Yuxin Chen, Adish Singla, Oisin Mac Aodha, Pietro Perona, Yisong Yue
We highlight that adaptivity does not speed up the teaching process when considering existing models of version space learners, such as "worst-case" (the learner picks the next hypothesis randomly from the version space) and "preference-based" (the learner picks hypothesis according to some global preference).
no code implementations • CVPR 2018 • Kun Ho Kim, Oisin Mac Aodha, Pietro Perona
Low dimensional embeddings that capture the main variations of interest in collections of data are important for many applications.
no code implementations • 5 Sep 2017 • Grant Van Horn, Pietro Perona
We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods.
no code implementations • ICML 2017 • Ron Appel, Pietro Perona
There is a need for simple yet accurate white-box learning systems that train quickly and with little data.
19 code implementations • CVPR 2018 • Grant Van Horn, Oisin Mac Aodha, Yang song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, Serge Belongie
Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories.
Ranked #8 on Image Classification on iNaturalist
no code implementations • ICCV 2017 • Matteo Ruggero Ronchi, Pietro Perona
We propose a new method to analyze the impact of errors in algorithms for multi-instance pose estimation and a principled benchmark that can be used to compare them.
no code implementations • CVPR 2017 • Steve Branson, Grant van Horn, Pietro Perona
We develop specialized models and algorithms for binary annotation, part keypoint annotation, and sets of bounding box annotations.
1 code implementation • ICML 2017 • Mason McGill, Pietro Perona
We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths.
no code implementations • 4 Nov 2016 • Krzysztof Chalupka, Frederick Eberhardt, Pietro Perona
We propose a method to classify the causal relationship between two discrete variables given only the joint distribution of the variables, acknowledging that the method is subject to an inherent baseline error.
no code implementations • 1 Nov 2016 • Eyrun Eyjolfsdottir, Kristin Branson, Yisong Yue, Pietro Perona
We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network.
1 code implementation • CVPR 2017 • Bo Chen, Pietro Perona
Images are formed by counting how many photons traveling from a given set of directions hit an image sensor during a given time interval.
no code implementations • 12 Jul 2016 • Ron Appel, Xavier Burgos-Artizzu, Pietro Perona
We present a simple unified framework for multi-class cost-sensitive boosting.
no code implementations • CVPR 2016 • Jan D. Wegner, Steven Branson, David Hall, Konrad Schindler, Pietro Perona
The main technical challenge is combining test time information from multiple views of each geographic location (e. g., aerial and street views).
no code implementations • 30 May 2016 • Krzysztof Chalupka, Tobias Bischoff, Pietro Perona, Frederick Eberhardt
We show that the climate phenomena of El Nino and La Nina arise naturally as states of macro-variables when our recent causal feature learning framework (Chalupka 2015, Chalupka 2016) is applied to micro-level measures of zonal wind (ZW) and sea surface temperatures (SST) taken over the equatorial band of the Pacific Ocean.
no code implementations • CVPR 2015 • David Hall, Pietro Perona
This dataset is designed to train and test algorithms for fine-grained categorisation of people, it is also useful for benchmarking tracking, detection and pose estimation of pedestrians.
no code implementations • 25 Dec 2015 • Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt
We formalize the connection between micro- and macro-variables in such situations and provide a coherent framework describing causal relations at multiple levels of analysis.
no code implementations • 26 Oct 2015 • Krzysztof Chalupka, Michael Dickinson, Pietro Perona
Looming has been proposed as the main monocular visual cue for detecting the approach of other animals and avoiding collisions with stationary obstacles.
no code implementations • 7 Jun 2015 • Matteo Ruggero Ronchi, Pietro Perona
We address these questions by exploring the actions and interactions that are detectable in the images of the MS COCO dataset.
no code implementations • CVPR 2015 • Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, Serge Belongie
We worked with bird experts to measure the quality of popular datasets like CUB-200-2011 and ImageNet and found class label error rates of at least 4%.
no code implementations • 7 Dec 2014 • Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt
We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems.
no code implementations • 16 Oct 2014 • Michael Maire, Stella X. Yu, Pietro Perona
We frame the task of predicting a semantic labeling as a sparse reconstruction procedure that applies a target-specific learned transfer function to a generic deep sparse code representation of an image.
no code implementations • 11 Jun 2014 • Steve Branson, Grant van Horn, Serge Belongie, Pietro Perona
We perform a detailed investigation of state-of-the-art deep convolutional feature implementations and fine-tuning feature learning for fine-grained classification.
no code implementations • CVPR 2014 • Catherine Wah, Grant van Horn, Steve Branson, Subhransu Maji, Pietro Perona, Serge Belongie
Current human-in-the-loop fine-grained visual categorization systems depend on a predefined vocabulary of attributes and parts, usually determined by experts.
no code implementations • CVPR 2014 • Steve Branson, Kristjan Eldjarn Hjorleifsson, Pietro Perona
Human annotators may be thought of as helping translate the old annotations into the new ones.
no code implementations • CVPR 2014 • David Hall, Pietro Perona
A method for online, real-time learning of individual-object detectors is presented.
36 code implementations • 1 May 2014 • Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollár
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding.
no code implementations • CVPR 2013 • Peter Welinder, Max Welling, Pietro Perona
How many labeled examples are needed to estimate a classifier's performance on a new dataset?
no code implementations • NeurIPS 2011 • Ryan G. Gomes, Peter Welinder, Andreas Krause, Pietro Perona
Is it possible to crowdsource categorization?
no code implementations • NeurIPS 2011 • Bo Chen, Vidhya Navalpakkam, Pietro Perona
A model of human visual search is proposed.
no code implementations • NeurIPS 2010 • Andreas Krause, Pietro Perona, Ryan G. Gomes
We present a framework that simultaneously clusters the data and trains a discriminative classifier.
no code implementations • Advances in Neural Information Processing Systems 19 2006 • Jonathan Harel, Christof Koch, Pietro Perona
A new bottom-up visual saliency model, Graph-Based Visual Saliency (GBVS), is proposed.