Search Results for author:
Found 27 papers, 9 papers with code
Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA
In this work, we propose a low-cost method for pruning MHA models into GQA models with any compression ratio of key-value heads.
Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation
Automating radiology report generation can significantly reduce the workload of radiologists and enhance the accuracy, consistency, and efficiency of clinical documentation. We propose a novel cross-modal framework that uses MedCLIP as both a vision extractor and a retrieval mechanism to improve the process of medical report generation. By extracting retrieved report features and image features through an attention-based extract module, and integrating them with a fusion module, our method improves the coherence and clinical relevance of generated reports. Experimental results on the widely used IU-Xray dataset demonstrate the effectiveness of our approach, showing improvements over commonly used methods in both report quality and relevance. Additionally, ablation studies provide further validation of the framework, highlighting the importance of accurate report retrieval and feature integration in generating comprehensive medical reports.
ScalingNote: Scaling up Retrievers with Large Language Models for Real-World Dense Retrieval
However, scaling up retrieval models significantly increases online query latency.
SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task
Given that most KGs reside in graph databases accessible solely through specialized query languages (e. g., Cypher), it is critical to connect LLMs with KG databases by automating the translation of natural language into Cypher queries (termed as "Text2Cypher" task).
Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation
Integrating information from various reference databases is a major challenge for Retrieval-Augmented Generation (RAG) systems because each knowledge source adopts a unique data structure and follows different conventions.
From Image to Video, what do we need in multimodal LLMs?
In this paper, we introduce RED-VILLM, a Resource-Efficient Development pipeline which builds robust Video LLMs through leveraging the prior knowledge of Image LLMs.
LogicalDefender: Discovering, Extracting, and Utilizing Common-Sense Knowledge
Experiments show that our model has achieved better logical performance, and the extracted logical knowledge can be effectively applied to other scenarios.
CADReN: Contextual Anchor-Driven Relational Network for Controllable Cross-Graphs Node Importance Estimation
CADReN is also proven to match the performance of previous models on single-graph NIE task.
Boosting Semantic Segmentation from the Perspective of Explicit Class Embeddings
Following this observation, we propose ECENet, a new segmentation paradigm, in which class embeddings are obtained and enhanced explicitly during interacting with multi-stage image features.
Sparse Double Descent: Where Network Pruning Aggravates Overfitting
People usually believe that network pruning not only reduces the computational cost of deep networks, but also prevents overfitting by decreasing model capacity.
Reasoning with Multi-Structure Commonsense Knowledge in Visual Dialog
In our model, the external knowledge is represented with sentence-level facts and graph-level facts, to properly suit the scenario of the composite of dialog history and image.
Can network pruning benefit deep learning under label noise?
Network pruning is a widely-used technique to reduce the computational cost of over-parameterized neural networks.
KBGN: Knowledge-Bridge Graph Network for Adaptive Vision-Text Reasoning in Visual Dialogue
Visual dialogue is a challenging task that needs to extract implicit information from both visual (image) and textual (dialogue history) contexts.
DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue
The ability of generating detailed and non-repetitive responses is crucial for the agent to achieve human-like conversation.
Multi-Level Network for High-Speed Multi-Person Pose Estimation
In multi-person pose estimation, the left/right joint type discrimination is always a hard problem because of the similar appearance.
Multi-Person Pose Estimation
Vocal Bursts Intensity Prediction
FollowMeUp Sports: New Benchmark for 2D Human Keypoint Recognition
Human pose estimation has made significant advancement in recent years.
DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue
More importantly, we can tell which modality (visual or semantic) has more contribution in answering the current question by visualizing the gate values.
Ranked #6 on
Visual Dialog
on VisDial v0.9 val
Scene Graph Reasoning with Prior Visual Relationship for Visual Question Answering
Upon the constructed graph, we propose a Scene Graph Convolutional Network (SceneGCN) to jointly reason the object properties and relational semantics for the correct answer.
A sequential guiding network with attention for image captioning
The recent advances of deep learning in both computer vision (CV) and natural language processing (NLP) provide us a new way of understanding semantics, by which we can deal with more challenging tasks such as automatic description generation from natural images.
Pixel Level Data Augmentation for Semantic Image Segmentation using Generative Adversarial Networks
Semantic segmentation is one of the basic topics in computer vision, it aims to assign semantic labels to every pixel of an image.
Textual Relationship Modeling for Cross-Modal Information Retrieval
A joint neural model is proposed to learn feature representation individually in each modality.
Multimedia
Modeling Text with Graph Convolutional Network for Cross-Modal Information Retrieval
A dual-path neural network model is proposed for couple feature learning in cross-modal information retrieval.
Text Generation Based on Generative Adversarial Nets with Latent Variable
We propose the VGAN model where the generative model is composed of recurrent neural network and VAE.
Data Augmentation in Emotion Classification Using Generative Adversarial Networks
In this paper, we propose a data augmentation method using generative adversarial networks (GAN).
Logical Parsing from Natural Language Based on a Neural Translation Model
Under our learning policy, the Seq2Seq model can learn mappings gradually with noises.
Generative Cooperative Net for Image Generation and Data Augmentation
In this paper, we explore a generative model for the task of generating unseen images with desired features.
Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks
Learning to generate colorful cartoon images from black-and-white sketches is not only an interesting research problem, but also a potential application in digital entertainment.
