Search Results for author:
Found 16 papers, 9 papers with code
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation
However, talking head translation, converting audio-visual speech (i. e., talking head video) from one language into another, still confronts several challenges compared to audio speech: (1) Existing methods invariably rely on cascading, synthesizing via both audio and text, resulting in delays and cascading errors.
Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding
Thus, to address this issue and enhance model performance on new scenes, we explore the TVG task in an unsupervised domain adaptation (UDA) setting across scenes for the first time, where the video-query pairs in the source scene (domain) are labeled with temporal boundaries, while those in the target scene are not.
Extending Multi-modal Contrastive Representations
Inspired by recent C-MCR, this paper proposes Extending Multimodal Contrastive Representation (Ex-MCR), a training-efficient and paired-data-free method to flexibly learn unified contrastive representation space for more than three modalities by integrating the knowledge of existing MCR spaces.
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes
This paper presents Chat-3D, which combines the 3D visual perceptual ability of pre-trained 3D representations and the impressive reasoning and conversation capabilities of advanced LLMs to achieve the first universal dialogue systems for 3D scenes.
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding
To accomplish this, we design a novel semantic matching model that analyzes the semantic similarity between object proposals and sentences in a coarse-to-fine manner.
Connecting Multi-modal Contrastive Representations
This paper proposes a novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR).
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition
However, despite researchers exploring cross-lingual translation techniques such as machine translation and audio speech translation to overcome language barriers, there is still a shortage of cross-lingual studies on visual speech.
DsMCL: Dual-Level Stochastic Multiple Choice Learning for Multi-Modal Trajectory Prediction
One of the important and inherent factors is the multi-modality of vehicle motion.
Frame Interpolation with Multi-Scale Deep Loss Functions and Generative Adversarial Networks
To improve the quality of synthesised intermediate video frames, our network is jointly supervised at different levels with a perceptual loss function that consists of an adversarial and two content losses.
Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize
Compared to sub-pixel convolution initialized with schemes designed for standard convolution kernels, it is free from checkerboard artifacts immediately after initialization.
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
Convolutional neural networks have enabled accurate image super-resolution in real-time.
Ranked #11 on
Video Super-Resolution
on MSU Video Upscalers: Quality Enhancement
(VMAF metric)
Is the deconvolution layer the same as a convolutional layer?
In this note, we want to focus on aspects related to two questions most people asked us at CVPR about the network we presented.
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
This means that the super-resolution (SR) operation is performed in HR space.
Ranked #1 on
Video Super-Resolution
on Xiph HD - 4x upscaling
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images.
Ranked #3 on
Image Super-Resolution
on VggFace2 - 8x upscaling
