Search Results for author:
Found 39 papers, 13 papers with code
Variational Connectionist Temporal Classification
However, CTC tends to output spiky distributions since it prefers to output blank symbol most of the time.
Mimir: Improving Video Diffusion Models for Precise Text Understanding
Text serves as the key control signal in video generation due to its narrative nature.
Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis
Recent advances in diffusion models have revolutionized audio-driven talking head synthesis.
LokiTalk: Learning Fine-Grained and Generalizable Correspondences to Enhance NeRF-based Talking Head Synthesis
Despite significant progress in talking head synthesis since the introduction of Neural Radiance Fields (NeRF), visual artifacts and high training costs persist as major obstacles to large-scale commercial adoption.
Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts
Multimodal large language models (MLLMs) are closing the gap to human visual perception capability rapidly, while, still lag behind on attending to subtle images details or locating small objects precisely, etc.
Try-On-Adapter: A Simple and Flexible Try-On Paradigm
A key challenge of try-on is to generate realistic images of the model wearing the garments while preserving the details of the garments.
HomoMatcher: Dense Feature Matching Results with Semi-Dense Efficiency by Homography Estimation
To address this limitation, this paper concentrates on enhancing the fine-matching module in the semi-dense matching framework.
LumiSculpt: A Consistency Lighting Control Network for Video Generation
Lighting plays a pivotal role in ensuring the naturalness of video generation, significantly influencing the aesthetic quality of the generated content.
Animate-X: Universal Character Image Animation with Enhanced Motion Representation
Our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character.
StyleTokenizer: Defining Image Style by a Single Instance for Controlling Diffusion Models
To tackle these challenges, we introduce StyleTokenizer, a zero-shot style control image generation method that aligns style representation with text representation using a style tokenizer.
POA: Pre-training Once for Models of All Sizes
Once pre-trained, POA allows the extraction of pre-trained models of diverse sizes for downstream tasks.
Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight
This paper introduces Chain-of-Sight, a vision-language bridge module that accelerates the pre-training of Multimodal Large Language Models (MLLMs).
ViTime: A Visual Intelligence-Based Foundation Model for Time Series Forecasting
The success of large pretrained models in natural language processing (NLP) and computer vision (CV) has opened new avenues for constructing foundation models for time series forecasting (TSF).
SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding
Remote Sensing Large Multi-Modal Models (RSLMMs) are developing rapidly and showcase significant capabilities in remote sensing imagery (RSI) comprehension.
Low algorithmic delay implementation of convolutional beamformer for online joint source separation and dereverberation
Recently, some online algorithms were developed, which achieve separation on a frame-by-frame basis in the short-time-Fourier-transform (STFT) domain and the latency is significantly reduced as compared to those batch methods.
Enhancing DETRs Variants through Improved Content Query and Similar Query Aggregation
The design of the query is crucial for the performance of DETR and its variants.
Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis
Recent works in implicit representations, such as Neural Radiance Fields (NeRF), have advanced the generation of realistic and animatable head avatars from video sequences.
M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining
Vision-language foundation models like CLIP have revolutionized the field of artificial intelligence.
Ranked #1 on
Zero-Shot Transfer Image Classification
on ImageNet
(using extra training data)
Zero-Shot Cross-Modal Retrieval
Zero-shot Image Retrieval
+3
Towards Better Vision-Inspired Vision-Language Models
In this paper we present a vision-inspired vision-language connection module dubbed as VIVL which efficiently exploits the vision cue for VL models.
SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery
Prior studies on Remote Sensing Foundation Model (RSFM) reveal immense potential towards a generic model for Earth Observation.
A computationally efficient semi-blind source separation based approach for nonlinear echo cancellation based on an element-wise iterative source steering
While the semi-blind source separation-based acoustic echo cancellation (SBSS-AEC) has received much research attention due to its promising performance during double-talk compared to the traditional adaptive algorithms, it suffers from system latency and nonlinear distortions.
Large Multimodal Model Compression via Efficient Pruning and Distillation at AntGroup
In our research, we constructed a dataset, the Multimodal Advertisement Audition Dataset (MAAD), from real-world scenarios within Alipay, and conducted experiments to validate the reliability of our proposed strategy.
LogicMP: A Neuro-symbolic Approach for Encoding First-order Logic Constraints
Integrating first-order logic constraints (FOLCs) with neural networks is a crucial but challenging problem since it involves modeling intricate correlations to satisfy the constraints.
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments.
Mapping EEG Signals to Visual Stimuli: A Deep Learning Approach to Match vs. Mismatch Classification
Existing approaches to modeling associations between visual stimuli and brain responses are facing difficulties in handling between-subject variance and model generalization.
Simultaneously Short- and Long-Term Temporal Modeling for Semi-Supervised Video Semantic Segmentation
In this work, we propose a novel feature enhancement network to simultaneously model short- and long-term temporal correlation.
Uncertainty-guided Learning for Improving Image Manipulation Detection
Image manipulation detection (IMD) is of vital importance as faking images and spreading misinformation can be malicious and harm our daily life.
Robust Manifold Nonnegative Tucker Factorization for Tensor Data Representation
Nonnegative Tucker Factorization (NTF) minimizes the euclidean distance or Kullback-Leibler divergence between the original data and its low-rank approximation which often suffers from grossly corruptions or outliers and the neglect of manifold structures of data.
SimAN: Exploring Self-Supervised Representation Learning of Scene Text via Similarity-Aware Normalization
Motivated by this common sense, we augment one image patch and use its neighboring patch as guidance to recover itself.
Hierarchical Memory Learning for Fine-Grained Scene Graph Generation
After the autonomous partition of coarse and fine predicates, the model is first trained on the coarse predicates and then learns the fine predicates.
Training Protocol Matters: Towards Accurate Scene Text Recognition via Training Protocol Searching
The development of scene text recognition (STR) in the era of deep learning has been mainly focused on novel architectures of STR models.
Training Object Detectors From Scratch: An Empirical Study in the Era of Vision Transformer
Instead of proposing a specific vision transformer based detector, in this work, our goal is to reveal the insights of training vision transformer based detectors from scratch.
CBNet: A Composite Backbone Network Architecture for Object Detection
With multi-scale testing, we push the current best single model result to a new record of 60. 1% box AP and 52. 3% mask AP without using extra training data.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(using extra training data)
MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction
Through key-value matching based on relevancy evaluation, the proposed MatchVIE can bypass the recognitions to various semantics, and simply focuses on the strong relevancy between entities.
LPSNet: A Lightweight Solution for Fast Panoptic Segmentation
Panoptic segmentation is a challenging task aiming to simultaneously segment objects (things) at instance level and background contents (stuff) at semantic level.
CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes
Then, we design a two-level perturbation fusion strategy to alleviate the conflict between the adversarial watermarks generated by different facial images and models.
Partial AUC optimization based deep speaker embeddings with class-center learning for text-independent speaker verification
We also propose a class-center based training trial construction method to improve the training efficiency, which is critical for the proposed loss function to be comparable to the identification loss in performance.
End-to-End Model for Speech Enhancement by Consistent Spectrogram Masking
The experiments comparing ourCSM based end-to-end model with other methods are conductedto confirm that the CSM accelerate the model training andhave significant improvements in speech quality.
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages.
