Search Results for author:
Found 20 papers, 9 papers with code
EgoPet: Egomotion and Interaction Data from an Animal's Perspective
Animals perceive the world to plan their actions and interact with other agents to accomplish complex tasks, demonstrating capabilities that are still unmatched by AI systems.
Finding Visual Task Vectors
In this work, we analyze the activations of MAE-VQGAN, a recent Visual Prompting model, and find task vectors, activations that encode task-specific information.
IMProv: Inpainting-based Multimodal Prompting for Computer Vision Tasks
Given a textual description of a visual task (e. g. "Left: input image, Right: foreground segmentation"), a few input-output visual examples, or both, the model in-context learns to solve it for a new test input.
Sequential Modeling Enables Scalable Learning for Large Vision Models
We introduce a novel sequential modeling approach which enables learning a Large Vision Model (LVM) without making use of any linguistic data.
Object-based (yet Class-agnostic) Video Domain Adaptation
Existing video-based action recognition systems typically require dense annotation and struggle in environments when there is significant distribution shift relative to the training data.
Stochastic positional embeddings improve masked image modeling
Masked Image Modeling (MIM) is a promising self-supervised learning approach that enables learning from unlabeled images.
A Cookbook of Self-Supervised Learning
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning.
Visual Prompting via Image Inpainting
How does one adapt a pre-trained visual model to novel downstream tasks without task-specific finetuning or any model modification?
Ranked #5 on
Personalized Segmentation
on PerSeg
Structured Video Tokens @ Ego4D PNR Temporal Localization Challenge 2022
First, as both images and videos contain structured information, we enrich a transformer model with a set of \emph{object tokens} that can be used across images and videos.
Point- of-no-return (PNR) temporal localization
Temporal Localization
Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens
We explore a particular instantiation of scene structure, namely a \emph{Hand-Object Graph}, consisting of hands and objects with their locations as nodes, and physical relations of contact/no-contact as edges.
Object-Region Video Transformers
In this work, we present Object-Region Video Transformers (ORViT), an \emph{object-centric} approach that extends video transformer layers with a block that directly incorporates object representations.
Ranked #6 on
Action Recognition
on Diving-48
DETReg: Unsupervised Pretraining with Region Priors for Object Detection
Recent self-supervised pretraining methods for object detection largely focus on pretraining the backbone of the object detector, neglecting key parts of detection architecture.
Ranked #1 on
Few-Shot Object Detection
on COCO 2017
3D Convolutional Sequence to Sequence Model for Vertebral Compression Fractures Identification in CT
An osteoporosis-related fracture occurs every three seconds worldwide, affecting one in three women and one in five men aged over 50.
Compositional Video Synthesis with Action Graphs
Our generative model for this task (AG2Vid) disentangles motion and appearance features, and by incorporating a scheduling mechanism for actions facilitates a timely and coordinated video generation.
Learning Canonical Representations for Scene Graph to Image Generation
Generating realistic images of complex visual scenes becomes challenging when one wishes to control the structure of the generated images.
Ranked #3 on
Layout-to-Image Generation
on Visual Genome 256x256
Improved ICH classification using task-dependent learning
Head CT is one of the most commonly performed imaging studied in the Emergency Department setting and Intracranial hemorrhage (ICH) is among the most critical and timesensitive findings to be detected on Head CT. We present BloodNet, a deep learning architecture designed for optimal triaging of Head CTs, with the goal of decreasing the time from CT acquisition to accurate ICH detection.
Learning Individual Styles of Conversational Gesture
Specifically, we perform cross-modal translation from "in-the-wild'' monologue speech of a single speaker to their hand and arm motion.
Ranked #4 on
Gesture Generation
on BEAT
PHT-bot: Deep-Learning based system for automatic risk stratification of COPD patients based upon signs of Pulmonary Hypertension
Secondary pulmonary hypertension is a manifestation of advanced COPD, which can be reliably diagnosed by the main Pulmonary Artery (PA) to Ascending Aorta (Ao) ratio.
Compression Fractures Detection on CT
Finally a Recurrent Neural Network (RNN) is utilized to predict whether a vertebral fracture is present in the series of patches.
Language Generation with Recurrent Generative Adversarial Networks without Pre-training
Generative Adversarial Networks (GANs) have shown great promise recently in image generation.
