Search Results for author:
Found 31 papers, 13 papers with code
VideoSAGE: Video Summarization with Graph Representation Learning
We propose a graph-based representation learning framework for video summarization.
Action Scene Graphs for Long-Form Understanding of Egocentric Videos
We present Egocentric Action Scene Graphs (EASGs), a new representation for long-form understanding of egocentric videos.
Single-Stage Visual Relationship Learning using Conditional Queries
Research in scene graph generation (SGG) usually considers two-stage models, that is, detecting a set of entities, followed by combining them and labeling all possible relationships.
SViTT: Temporal Learning of Sparse Video-Text Transformers
Do video-text transformers learn to model temporal relationships across frames?
Ranked #4 on
Video Question Answering
on AGQA 2.0 balanced
(Average Accuracy metric)
Unbiased Scene Graph Generation in Videos
The task of dynamic scene graph generation (SGG) from videos is complicated and challenging due to the inherent dynamics of a scene, temporal fluctuation of model predictions, and the long-tailed distribution of the visual relationships in addition to the already existing challenges in image-based SGG.
Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection
Active speaker detection (ASD) in videos with multiple speakers is a challenging task as it requires learning effective audiovisual features and spatial-temporal correlations over long temporal windows.
Ranked #1 on
Node Classification
on AVA
Text Spotting Transformers
In this paper, we present TExt Spotting TRansformers (TESTR), a generic end-to-end text spotting framework using Transformers for text detection and recognition in the wild.
Ranked #6 on
Text Spotting
on ICDAR 2015
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos
To tackle this task, we first provide an automatic way to collect trajectory and hotspots labels on large-scale data.
Exploiting Long-Term Dependencies for Generating Dynamic Scene Graphs
Dynamic scene graph generation from a video is challenging due to the temporal dynamics of the scene and the inherent temporal fluctuations of predictions.
Learning Spatial-Temporal Graphs for Active Speaker Detection
We address the problem of active speaker detection through a new framework, called SPELL, that learns long-range multimodal graphs to encode the inter-modal relationship between audio and visual data.
Towards Panoptic 3D Parsing for Single Image in the Wild
Our proposed panoptic 3D parsing framework points to a promising direction in computer vision.
Learning of Visual Relations: The Devil is in the Tails
Significant effort has been recently devoted to modeling visual relations.
In Defense of Scene Graphs for Image Captioning
Several studies have noted that the naive use of scene graphs from a black-box scene graph generator harms image captioning performance and that scene graph-based captioning models have to incur the overhead of explicit use of image features to generate decent captions.
Structured Query-Based Image Retrieval Using Scene Graphs
A structured query can capture the complexity of object interactions (e. g. 'woman rides motorcycle') unlike single objects (e. g. 'woman' or 'motorcycle').
Triplet-Aware Scene Graph Embeddings
We see a significant performance increase in both metrics that measure the goodness of layout prediction, mean intersection-over-union (mIoU)(52. 3% vs. 49. 2%) and relation score (61. 7% vs. 54. 1%), after the addition of triplet supervision and data augmentation.
Compact Scene Graphs for Layout Composition and Patch Retrieval
Structured representations such as scene graphs serve as an efficient and compact representation that can be used for downstream rendering or retrieval tasks.
Heuristics for Image Generation from Scene Graphs
Existing scene graph to image models have two stages: (1) a scene composition stage, and an (2) image generation stage.
Toward Joint Image Generation and Compression using Generative Adversarial Networks
The proposed method is a promising baseline method for joint image generation and compression using generative adversarial networks.
Using Scene Graph Context to Improve Image Generation
Generating realistic images from scene graphs asks neural networks to be able to reason about object relationships and compositionality.
Image Generation from Scene Graphs
Open-Ended Question Answering
+1
PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding
We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information.
Ranked #3 on
3D Instance Segmentation
on PartNet
Correction by Projection: Denoising Images with Generative Adversarial Networks
In this paper, we propose to denoise corrupted images by finding the nearest point on the GAN manifold, recovering latent vectors by minimizing distances in image space.
LCDet: Low-Complexity Fully-Convolutional Neural Networks for Object Detection in Embedded Systems
Thus the consolidation of a CNN-based object detection for an embedded system is more challenging.
Pose2Instance: Harnessing Keypoints for Person Instance Segmentation
In a more realistic environment, without the oracle keypoints, the proposed deep person instance segmentation model conditioned on human pose achieves 3. 8% to 10. 5% relative improvements comparing with its strongest baseline of a deep network trained only for segmentation.
Precise Recovery of Latent Vectors from Generative Adversarial Networks
Generative adversarial networks (GANs) transform latent vectors into visually plausible images.
A Statistical Approach to Continuous Self-Calibrating Eye Gaze Tracking for Head-Mounted Virtual Reality Systems
This eliminates the need for an explicit calibration step and automatically compensates for small movements of the headset with respect to the head.
Context Matters: Refining Object Detection in Video with Recurrent Neural Networks
Then we train a recurrent neural network that takes as input sequences of pseudo-labeled frames and optimizes an objective that encourages both accuracy on the target frame and consistency across consecutive frames.
Detecting Temporally Consistent Objects in Videos through Object Class Label Propagation
We further propose a clustering of VOPs which can efficiently be used for detecting objects in video in a streaming fashion.
Real-time Sign Language Fingerspelling Recognition using Convolutional Neural Networks from Depth map
We train CNNs for the classification of 31 alphabets and numbers using a subset of collected depth data from multiple subjects.
Semantic Video Segmentation : Exploring Inference Efficiency
We explore the efficiency of the CRF inference beyond image level semantic segmentation and perform joint inference in video frames.
Beyond Semantic Image Segmentation : Exploring Efficient Inference in Video
We explore the efficiency of the CRF inference module beyond image level semantic segmentation.
Improving Streaming Video Segmentation with Early and Mid-Level Visual Processing
Despite recent advances in video segmentation, many opportunities remain to improve it using a variety of low and mid-level visual cues.
