Search Results for author:
Found 29 papers, 12 papers with code
Perception Encoder: The best visual embeddings are not at the output of the network
Together with the core contrastive checkpoint, our PE family of models achieves state-of-the-art performance on a wide variety of tasks, including zero-shot image and video classification and retrieval; document, image, and video Q&A; and spatial tasks such as detection, depth estimation, and tracking.
Ranked #1 on
Object Detection
on COCO minival
(using extra training data)
Poly-Autoregressive Prediction for Modeling Interactions
At its core, PAR represents the behavior of all agents as a sequence of tokens, each representing an agent's state at a specific timestep.
An Empirical Study of Autoregressive Pre-training from Videos
Our models are pre-trained on a diverse dataset of videos and images comprising over 1 trillion visual tokens.
Gaussian Masked Autoencoders
Our approach, named Gaussian Masked Autoencoder, or GMAE, aims to learn semantic abstractions and spatial understanding jointly.
Scaling Properties of Diffusion Models for Perceptual Tasks
In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks.
Synergy and Synchrony in Couple Dances
Our contributions are a demonstration of the advantages of socially conditioned future motion prediction and an in-the-wild, couple dance video dataset to enable future research in this direction.
FewShotNeRF: Meta-Learning-based Novel View Synthesis for Rapid Scene-Specific Adaptation
In this paper, we address the challenge of generating novel views of real-world objects with limited multi-view images through our proposed approach, FewShotNeRF.
EgoPet: Egomotion and Interaction Data from an Animal's Perspective
Animals perceive the world to plan their actions and interact with other agents to accomplish complex tasks, demonstrating capabilities that are still unmatched by AI systems.
Humanoid Locomotion as Next Token Prediction
We cast real-world humanoid control as a next token prediction problem, akin to predicting the next word in language.
Synthesizing Moving People with 3D Control
This disentangled approach allows our method to generate a sequence of images that are faithful to the target motion in the 3D pose and, to the input image in terms of visual similarity.
Humans in 4D: Reconstructing and Tracking Humans with Transformers
To analyze video, we use 3D reconstructions from HMR 2. 0 as input to a tracking system that operates in 3D.
Ranked #3 on
Pose Tracking
on PoseTrack2018
On the Benefits of 3D Pose and Tracking for Human Action Recognition
Subsequently, we propose a Lagrangian Action Recognition model by fusing 3D pose and contextualized appearance over tracklets.
Ranked #1 on
Action Recognition
on AVA v2.2
(using extra training data)
Fully Online Meta-Learning Without Task Boundaries
In this paper, we study how meta-learning can be applied to tackle online problems of this nature, simultaneously adapting to changing tasks and input distributions and meta-training the model in order to adapt more quickly in the future.
Tracking People by Predicting 3D Appearance, Location and Pose
For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.
Tracking People by Predicting 3D Appearance, Location & Pose
For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner.
Tracking People with 3D Representations
We find that 3D representations are more effective than 2D representations for tracking in these settings, and we obtain state-of-the-art performance.
Mitigating Mode Collapse by Sidestepping Catastrophic Forgetting
Generative Adversarial Networks (GANs) are a class of generative models used for various applications, but they have been known to suffer from the mode collapse problem, in which some modes of the target distribution are ignored by the generator.
Meta-learning the Learning Trends Shared Across Tasks
This demonstrates their ability to acquire transferable knowledge, a capability that is central to human learning.
Self-supervised Knowledge Distillation for Few-shot Learning
Our experiments show that, even in the first stage, self-supervision can outperform current state-of-the-art methods, with further gains achieved by our second stage distillation process.
Ranked #13 on
Few-Shot Image Classification
on FC100 5-way (5-shot)
A Multi-modal Neural Embeddings Approach for Detecting Mobile Counterfeit Apps: A Case Study on Google Play Store
We show that a novel approach of combining content embeddings and style embeddings outperforms the baseline methods for image similarity such as SIFT, SURF, and various image hashing methods.
iTAML: An Incremental Task-Agnostic Meta-learning Approach
In this paper, we hypothesize this problem can be avoided by learning a set of generalized parameters, that are neither specific to old nor new tasks.
Incremental Object Detection via Meta-Learning
In a real-world setting, object instances from new classes can be continuously encountered by object detectors.
Random Path Selection for Continual Learning
In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity.
Ranked #7 on
Continual Learning
on F-CelebA (10 tasks)
TimeCaps: Capturing Time Series Data With Capsule Networks
Capsule networks excel in understanding spatial relationships in 2D data for vision related tasks.
An Adaptive Random Path Selection Approach for Incremental Learning
In a conventional supervised learning setting, a machine learning model has access to examples of all object classes that are desired to be recognized during the inference stage.
DeepCaps: Going Deeper with Capsule Networks
Capsule Network is a promising concept in deep learning, yet its true potential is not fully realized thus far, providing sub-par performance on several key benchmark datasets with complex data.
TextCaps : Handwritten Character Recognition with Very Small Datasets
Our system is useful in character recognition for localized languages that lack much labeled training data and even in other related more general contexts such as object recognition.
Ranked #6 on
Image Classification
on EMNIST-Letters
Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos
We propose three schemas for combining static and motion components: based on a variance ratio, principal components, and Cholesky decomposition.
A Neural Embeddings Approach for Detecting Mobile Counterfeit Apps
We show that further performance increases can be achieved by combining style embeddings with content embeddings.
