Search Results for author:
Found 21 papers, 7 papers with code
Prior Knowledge and Memory Enriched Transformer for Sign Language Translation
This paper attacks the challenging problem of sign language translation (SLT), which involves not only visual and textual understanding but also additional prior knowledge learning (i. e. performing style, syntax).
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt
Recent singing-voice-synthesis (SVS) methods have achieved remarkable audio quality and naturalness, yet they lack the capability to control the style attributes of the synthesized singing explicitly.
DART: Implicit Doppler Tomography for Radar Novel View Synthesis
Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking.
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation
However, talking head translation, converting audio-visual speech (i. e., talking head video) from one language into another, still confronts several challenges compared to audio speech: (1) Existing methods invariably rely on cascading, synthesizing via both audio and text, resulting in delays and cascading errors.
Extending Multi-modal Contrastive Representations
Inspired by recent C-MCR, this paper proposes Extending Multimodal Contrastive Representation (Ex-MCR), a training-efficient and paired-data-free method to flexibly learn unified contrastive representation space for more than three modalities by integrating the knowledge of existing MCR spaces.
Variance-Aware Regret Bounds for Stochastic Contextual Dueling Bandits
Dueling bandits is a prominent framework for decision-making involving preferential feedback, a valuable feature that fits various applications involving human interaction, such as ranking, information retrieval, and recommendation systems.
Gloss Attention for Gloss-free Sign Language Translation
We find that it can provide two aspects of information for the model, 1) it can help the model implicitly learn the location of semantic boundaries in continuous sign language videos, 2) it can help the model understand the sign language video globally.
Ranked #3 on
Gloss-free Sign Language Translation
on PHOENIX14T
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment
We demonstrate that OpenSR enables modality transfer from one to any in three different settings (zero-, few- and full-shot), and achieves highly competitive zero-shot performance compared to the existing few-shot and full-shot lip-reading methods.
DATE: Domain Adaptive Product Seeker for E-commerce
Then, we present two cooperative seekers to simultaneously search the image for PR and localize the product for PG.
Borda Regret Minimization for Generalized Linear Dueling Bandits
To attain this lower bound, we propose an explore-then-commit type algorithm for the stochastic setting, which has a nearly matching regret upper bound $\tilde{O}(d^{2/3} T^{2/3})$.
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition
However, despite researchers exploring cross-lingual translation techniques such as machine translation and audio speech translation to overcome language barriers, there is still a shortage of cross-lingual studies on visual speech.
Exploring Group Video Captioning with Efficient Relational Approximation
In this study, we propose a new task, group video captioning, which aims to infer the desired content among a group of target videos and describe it with another group of related reference videos.
Generalizable Multi-linear Attention Network
The majority of existing multimodal sequential learning methods focus on how to obtain effective representations and ignore the importance of multimodal fusion.
Adaptive Sampling for Heterogeneous Rank Aggregation from Noisy Pairwise Comparisons
In heterogeneous rank aggregation problems, users often exhibit various accuracy levels when comparing pairs of items.
Dual Low-Rank Multimodal Fusion
Tensor-based fusion methods have been proven effective in multimodal fusion tasks.
SBAT: Video Captioning with Sparse Boundary-Aware Transformer
However, video captioning is a multimodal learning problem, and the video features have much redundancy between different time steps.
Rank Aggregation via Heterogeneous Thurstone Preference Models
By allowing different noise distributions, the proposed HTM model maintains the generality of Thurstone's original framework, and as such, also extends the Bradley-Terry-Luce (BTL) model for pairwise comparisons to heterogeneous populations of users.
Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning
This paper addresses the challenging task of video captioning which aims to generate descriptions for video data.
Inpatient2Vec: Medical Representation Learning for Inpatients
In addition, for unordered medical activity set, existing medical RL methods utilize a simple pooling strategy, which would result in indistinguishable contributions among the activities for learning.
Online Bayesian Collaborative Topic Regression
In this paper, we propose a novel scheme of Online Bayesian Collaborative Topic Regression (OBCTR) which is efficient and scalable for learning from data streams.
