Search Results for author:
Found 38 papers, 19 papers with code
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
We contribute Synthetic Visual Concepts (SyViC) - a million-scale synthetic dataset and data generation codebase allowing to generate additional suitable data to improve VLC understanding and compositional reasoning of VL models.
Leveraging Temporal Context in Low Representational Power Regimes
We show that including information from the ETM during training improves action recognition and anticipation performance on various egocentric video datasets.
Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines
Second, they allow us to generate novel "Deepfake Caricatures": transformations of the deepfake that exacerbate artifacts to improve human detection.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Dynamic Network Quantization for Efficient Video Inference
Deep convolutional networks have recently achieved great success in video recognition, yet their practical realization remains a challenge due to the large amount of computational resources required to achieve robust recognition.
IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers
The self-attention-based model, transformer, is recently becoming the leading backbone in the field of computer vision.
Cross-Modal Discrete Representation Learning
Recent advances in representation learning have demonstrated an ability to represent information from different modalities such as video, text, and audio in a single high-level embedding vector.
AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition
Specifically, given a video segment, a multi-modal policy network is used to decide what modalities should be used for processing by the recognition model, with the goal of improving both accuracy and efficiency.
Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions
With this in mind, the descriptions people generate for videos of different dynamic events can greatly improve our understanding of the key information of interest in each video.
Memorability: An image-computable measure of information utility
The pixels in an image, and the objects, scenes, and actions that they compose, determine whether an image will be memorable or forgettable.
Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
Second, to effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network.
VA-RED$^2$: Video Adaptive Redundancy Reduction
An inherent property of real-world videos is the high correlation of information across frames which can translate into redundancy in either temporal or spatial feature maps of the models, or both.
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
Temporal modelling is the key for efficient video action recognition.
Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition
In recent years, a number of approaches based on 2D or 3D convolutional neural networks (CNN) have emerged for video action recognition, achieving state-of-the-art results on several large-scale benchmark datasets.
Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability
Based on our findings we propose a new mathematical formulation of memorability decay, resulting in a model that is able to produce the first quantitative estimation of how a video decays in memory over time.
We Have So Much In Common: Modeling Semantic Relational Set Abstractions in Videos
This allows our model to perform cognitive tasks such as set abstraction (which general concept is in common among a set of videos?
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition
Specifically, given a video frame, a policy network is used to decide what input resolution should be used for processing by the action recognition model, with the goal of improving both accuracy and efficiency.
Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding
Videos capture events that typically contain multiple sequential, and simultaneous, actions even in the span of only a few seconds.
Reasoning About Human-Object Interactions Through Dual Attention Networks
The model not only finds when an action is happening and which object is being manipulated, but also identifies which part of the object is being interacted with.
GANalyze: Toward Visual Definitions of Cognitive Image Properties
We introduce a framework that uses Generative Adversarial Networks (GANs) to study cognitive properties like memorability, aesthetics, and emotional valence.
Examining Interpretable Feature Relationships in Deep Networks for Action recognition
A number of recent methods to understand neural networks have focused on quantifying the role of individual features.
The Algonauts Project: A Platform for Communication between the Sciences of Biological and Artificial Intelligence
Recently, researchers of natural intelligence have begun using those AI models to explore how the brain performs such tasks.
Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics
While automatic text extraction works well on infographics, computer vision approaches trained on natural images fail to identify the stand-alone visual elements in infographics, or `icons'.
Moments in Time Dataset: one million videos for event understanding
We present the Moments in Time Dataset, a large-scale human-annotated collection of one million short videos corresponding to dynamic events unfolding within three seconds.
Temporal Relational Reasoning in Videos
Temporal relational reasoning, the ability to link meaningful transformations of objects or entities over time, is a fundamental property of intelligent species.
Ranked #2 on
Hand Gesture Recognition
on Jester test
Interpreting Deep Visual Representations via Network Dissection
In this work, we describe Network Dissection, a method that interprets networks by providing labels for the units of their deep visual representations.
Understanding Infographics through Textual and Visual Tag Prediction
And second, we use these predicted text tags as a supervisory signal to localize the most diagnostic visual elements from within the infographic i. e. visual hashtags.
Network Dissection: Quantifying Interpretability of Deep Visual Representations
Given any CNN model, the proposed method draws on a broad data set of visual concepts to score the semantics of hidden units at each intermediate convolutional layer.
BubbleView: an interface for crowdsourcing image importance maps and tracking visual attention
In this paper, we present BubbleView, an alternative methodology for eye tracking using discrete mouse clicks to measure which information people consciously choose to examine.
Places: An Image Database for Deep Scene Understanding
The rise of multi-million-item dataset initiatives has enabled data-hungry machine learning algorithms to reach near-human semantic classification at tasks such as object and scene recognition.
What do different evaluation metrics tell us about saliency models?
How best to evaluate a saliency model's ability to predict where humans look in images is an open research question.
Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition
The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans.
Learning Deep Features for Discriminative Localization
In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network to have remarkable localization ability despite being trained on image-level labels.
Ranked #2 on
Weakly-Supervised Object Localization
on ILSVRC 2015
Understanding and Predicting Image Memorability at a Large Scale
Progress in estimating visual memorability has been limited by the small scale and lack of variety of benchmark data.
Object Detectors Emerge in Deep Scene CNNs
With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e. g., ImageNet, Places), the state of the art in computer vision is advancing rapidly.
Learning Deep Features for Scene Recognition using Places Database
Whereas the tremendous recent progress in object recognition tasks is due to the availability of large datasets like ImageNet and the rise of Convolutional Neural Networks (CNNs) for learning high-level features, performance at scene recognition has not attained the same level of success.
Learning visual biases from human imagination
Although the human visual system can recognize many concepts under challenging conditions, it still has some biases.
Understanding the Intrinsic Memorability of Images
Artists, advertisers, and photographers are routinely presented with the task of creating an image that a viewer will remember.
