Search Results for author:
Found 6 papers, 1 papers with code
HRVDA: High-Resolution Visual Document Assistant
In addition, we construct a document-oriented visual instruction tuning dataset and apply a multi-stage training strategy to enhance the model's document modeling capabilities.
Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models
It can represent that the contrastive learning between the visual holistic representations and the multimodal fine-grained features of document objects can assist the vision encoder in acquiring more effective visual cues, thereby enhancing the comprehension of text-rich documents in LVLMs.
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes
In our experiments, we treat discrete acoustic codes as textual data and train a masked language model using a cloze-like methodology, ultimately deriving high-quality audio representations.
OS-MSL: One Stage Multimodal Sequential Link Framework for Scene Segmentation and Classification
In this paper, from an alternate perspective to overcome the above challenges, we unite these two tasks into one task by a new form of predicting shots link: a link connects two adjacent shots, indicating that they belong to the same scene or category.
The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training
The self-supervised Masked Image Modeling (MIM) schema, following "mask-and-reconstruct" pipeline of recovering contents from masked image, has recently captured the increasing interest in the multimedia community, owing to the excellent ability of learning visual representation from unlabeled data.
NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognition
For the sake of trade-off between efficiency and performance, a group of works merely perform SA operation within local patches, whereas the global contextual information is abandoned, which would be indispensable for visual recognition tasks.
