Search Results for author:
Found 18 papers, 5 papers with code
LLMs Meet Long Video: Advancing Long Video Comprehension with An Interactive Visual Adapter in LLMs
Long video understanding is a significant and ongoing challenge in the intersection of multimedia and artificial intelligence.
Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment
Evaluating and Rethinking the current landscape of Large Multimodal Models (LMMs), we observe that widely-used visual-language projection approaches (e. g., Q-former or MLP) focus on the alignment of image-text descriptions yet ignore the visual knowledge-dimension alignment, i. e., connecting visuals to their relevant knowledge.
A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation
For this setting, previous methods utilize visual and textual encoders to encode the image and keywords and employ a language model-based decoder to generate the product description.
Frame Structure and Protocol Design for Sensing-Assisted NR-V2X Communications
In this study, we propose novel frame structures that incorporate ISAC signals for three crucial stages in the NR-V2X system: initial access, connected mode, and beam failure and recovery.
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs
These models predominantly map visual information into language representation space, leveraging the vast knowledge and powerful text generation abilities of LLMs to produce multimodal instruction-following responses.
A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answering
The emergence of multimodal large models (MLMs) has significantly advanced the field of visual understanding, offering remarkable capabilities in the realm of visual question answering (VQA).
Sensing as a Service in 6G Perceptive Mobile Networks: Architecture, Advances, and the Road Ahead
Sensing-as-a-service is anticipated to be the core feature of 6G perceptive mobile networks (PMN), where high-precision real-time sensing will become an inherent capability rather than being an auxiliary function as before.
Training Multimedia Event Extraction With Generated Images and Captions
Contemporary news reporting increasingly features multimedia content, motivating research on multimedia event extraction.
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues
This makes the language model well-suitable for such multi-modal reasoning scenario on joint textual and visual clues.
LMEye: An Interactive Perception Network for Large Language Models
LMEye addresses this issue by allowing the LLM to request the desired visual information aligned with various human instructions, which we term as the dynamic visual information interaction.
A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex Text
Inspired by the Divide-and-Conquer algorithm and dual-process theory, in this paper, we regard linguistically complex texts as compound proposition texts composed of multiple simple proposition sentences and propose an end-to-end Neural Divide-and-Conquer Reasoning framework, dubbed NDCR.
Towards ISAC-Empowered Vehicular Networks: Framework, Advances, and Opportunities
Connected and autonomous vehicle (CAV) networks face several challenges, such as low throughput, high latency, and poor localization accuracy.
ISAC-Enabled V2I Networks Based on 5G NR: How Much Can the Overhead Be Reduced?
The emergence of the fifth-generation (5G) New Radio (NR) brings additional possibilities to vehicle-to-everything (V2X) network with improved quality of services.
Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations
CSI), a relation inferrer, and a Lexical Constraint-aware Generator (arr.
MSDF: A General Open-Domain Multi-Skill Dialog Framework
In this paper, we propose a general Multi-Skill Dialog Framework, namely MSDF, which can be applied in different dialog tasks (e. g. knowledge grounded dialog and persona based dialog).
Medical Dialogue Response Generation with Pivotal Information Recalling
To mitigate this problem, we propose a medical response generation model with Pivotal Information Recalling (MedPIR), which is built on two components, i. e., knowledge-aware dialogue graph encoder and recall-enhanced generator.
Sentence-level Online Handwritten Chinese Character Recognition
Single online handwritten Chinese character recognition~(single OLHCCR) has achieved prominent performance.
GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph
Previous works indicate that the glyph of Chinese characters contains rich semantic information and has the potential to enhance the representation of Chinese characters.
