Search Results for author:
Found 33 papers, 19 papers with code
Generating Content for HDR Deghosting from Frequency View
To address this challenge, we propose the Low-Frequency aware Diffusion (LF-Diff) model for ghost-free HDR imaging.
Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework
Medical vision language pre-training (VLP) has emerged as a frontier of research, enabling zero-shot pathological recognition by comparing the query image with the textual descriptions for each disease.
StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing
It contains three main components: (1) A multimodal style adaptor operating at the phoneme level to learn pronunciation style from the reference audio, and generate intermediate representations informed by the facial emotion presented in the video; (2) An utterance-level style learning module, which guides both the mel-spectrogram decoding and the refining processes from the intermediate embeddings to improve the overall style expression; And (3) a phoneme-guided lip aligner to maintain lip sync.
Subject-Oriented Video Captioning
To address this problem, we propose a new video captioning task, subject-oriented video captioning, which allows users to specify the describing target via a bounding box.
Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting
% To provide a more realistic reflection of the underlying practical challenge, we introduce a weakly supervised VIC task, wherein trajectory labels are not provided.
Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection
To address these limitations, we propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection, which learns multi-scale temporal features.
March in Chat: Interactive Prompting for Remote Embodied Referring Expression
Nevertheless, this poses more challenges than other VLN tasks since it requires agents to infer a navigation plan only based on a short instruction.
AerialVLN: Vision-and-Language Navigation for UAVs
Navigating in the sky is more complicated than on the ground because agents need to consider the flying height and more complex spatial relationship reasoning.
Mind the Gap: Improving Success Rate of Vision-and-Language Navigation by Revisiting Oracle Success Routes
Vision-and-Language Navigation (VLN) aims to navigate to the target location by following a given instruction.
Teacher Agent: A Knowledge Distillation-Free Framework for Rehearsal-based Video Incremental Learning
Rehearsal-based video incremental learning often employs knowledge distillation to mitigate catastrophic forgetting of previously learned data.
HOP+: History-enhanced and Order-aware Pre-training for Vision-and-Language Navigation
To address these problems, we present a history-enhanced and order-aware pre-training with the complementing fine-tuning paradigm (HOP+) for VLN.
A Unified Object Counting Network with Object Occupation Prior
In this paper, we build the first evolving object counting dataset and propose a unified object counting network as the first attempt to address this task.
Learning to Dub Movies via Hierarchical Prosody Models
Given a piece of text, a video clip and a reference audio, the movie dubbing (also known as visual voice clone V2C) task aims to generate speeches that match the speaker's emotion presented in the video using the desired speaker voice as reference.
BEVBert: Multimodal Map Pre-training for Language-guided Navigation
Concretely, we build a local metric map to explicitly aggregate incomplete observations and remove duplicates, while modeling navigation dependency in a global topological map.
Ranked #2 on
Visual Navigation
on R2R
Progressive Multi-resolution Loss for Crowd Counting
Crowd counting is usually handled in a density map regression fashion, which is supervised via a L2 loss between the predicted density map and ground truth.
Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection
Weakly supervised video anomaly detection aims to identify abnormal events in videos using only video-level labels.
Consistency-Aware Anchor Pyramid Network for Crowd Localization
Crowd localization aims to predict the spatial position of humans in a crowd scenario.
Multi-Attention Network for Compressed Video Referring Object Segmentation
To address this problem, we propose a multi-attention network which consists of dual-path dual-attention module and a query-based cross-modal Transformer module.
Ranked #5 on
Referring Expression Segmentation
on A2D Sentences
HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation
Pre-training has been adopted in a few of recent works for Vision-and-Language Navigation (VLN).
Ranked #4 on
Visual Navigation
on R2R
V2C: Visual Voice Cloning
Existing Voice Cloning (VC) tasks aim to convert a paragraph text to a speech with desired voice specified by a reference audio.
Hierarchical Modular Network for Video Captioning
(II) Predicate level, which learns the actions conditioned on highlighted objects and is supervised by the predicate in captions.
Neighbor-view Enhanced Model for Vision and Language Navigation
Specifically, our NvEM utilizes a subject module and a reference module to collect contexts from neighbor views.
Ranked #82 on
Vision and Language Navigation
on VLN Challenge
VLN BERT: A Recurrent Vision-and-Language BERT for Navigation
In this paper we propose a recurrent BERT model that is time-aware for use in VLN.
The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation
Vision-and-Language Navigation (VLN) requires an agent to find a path to a remote location on the basis of natural-language instructions and a set of photo-realistic panoramas.
Diagnosing Vision-and-Language Navigation: What Really Matters
Results show that indoor navigation agents refer to both object and direction tokens when making decisions.
A Recurrent Vision-and-Language BERT for Navigation
In this paper we propose a recurrent BERT model that is time-aware for use in VLN.
Ranked #7 on
Visual Navigation
on R2R
Language and Visual Entity Relationship Graph for Agent Navigation
From both the textual and visual perspectives, we find that the relationships among the scene, its objects, and directional clues are essential for the agent to interpret complex instructions and correctly perceive the environment.
Object-and-Action Aware Model for Visual Language Navigation
The first is object description (e. g., 'table', 'door'), each presenting as a tip for the agent to determine the next action by finding the item visible in the environment, and the second is action specification (e. g., 'go straight', 'turn left') which allows the robot to directly predict the next movements without relying on visual perceptions.
Scene Text Recognition via Transformer
Different from previous transformer based models [56, 34], which just use the decoder of the transformer to decode the convolutional attention, the proposed method use a convolutional feature maps as word embedding input into transformer.
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
One of the long-term challenges of robotics is to enable robots to interact with humans in the visual world via natural language, as humans are visual animals that communicate through language.
The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking
Selected from 10 hours raw videos, about 80, 000 representative frames are fully annotated with bounding boxes as well as up to 14 kinds of attributes (e. g., weather condition, flying altitude, camera view, vehicle category, and occlusion) for three fundamental computer vision tasks: object detection, single object tracking, and multiple object tracking.
Ranked #5 on
Object Detection
on UAVDT
Video Object Segmentation with Re-identification
Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module.
Hedged Deep Tracking
In recent years, several methods have been developed to utilize hierarchical features learned from a deep convolutional neural network (CNN) for visual tracking.
