Search Results for author:
Found 13 papers, 5 papers with code
Kimi-Audio Technical Report
We release the codes, model checkpoints, as well as the evaluation toolkits in https://github. com/MoonshotAI/Kimi-Audio.
The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation
Recent advancements in text-to-video (T2V) generation have been driven by two competing paradigms: autoregressive language models and diffusion models.
T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text
In this work, we propose a two-stage sign language production (SLP) paradigm that first encodes sign language sequences into discrete codes and then autoregressively generates sign language from text based on the learned codebook.
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation
However, talking head translation, converting audio-visual speech (i. e., talking head video) from one language into another, still confronts several challenges compared to audio speech: (1) Existing methods invariably rely on cascading, synthesizing via both audio and text, resulting in delays and cascading errors.
Language Model is a Branch Predictor for Simultaneous Machine Translation
Subsequently, we utilize the predicted source words to decode the output in advance.
TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System
We anticipate that our work will contribute to the advancement of research on TrainerAgent in both academic and industry communities, potentially establishing it as a new paradigm for model development in the field of AI.
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding
3D visual grounding aims to localize the target object in a 3D point cloud by a free-form language description.
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding
To accomplish this, we design a novel semantic matching model that analyzes the semantic similarity between object proposals and sentences in a coarse-to-fine manner.
Gloss Attention for Gloss-free Sign Language Translation
We find that it can provide two aspects of information for the model, 1) it can help the model implicitly learn the location of semantic boundaries in continuous sign language videos, 2) it can help the model understand the sign language video globally.
Ranked #7 on
Gloss-free Sign Language Translation
on PHOENIX14T
Connecting Multi-modal Contrastive Representations
This paper proposes a novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR).
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition
However, despite researchers exploring cross-lingual translation techniques such as machine translation and audio speech translation to overcome language barriers, there is still a shortage of cross-lingual studies on visual speech.
MLSLT: Towards Multilingual Sign Language Translation
In addition, we also explore zero-shot translation in sign language and find that our model can achieve comparable performance to the supervised BSLT model on some language pairs.
SimulSLT: End-to-End Simultaneous Sign Language Translation
Sign language translation as a kind of technology with profound social significance has attracted growing researchers' interest in recent years.
