Search Results for author:
Found 21 papers, 3 papers with code
DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis
To reduce the workload of radiologists, we propose DeltaNet to generate medical reports automatically.
Aligning Source Visual and Target Language Domains for Unpaired Video Captioning
To this end, we introduce the unpaired video captioning task aiming to train models without coupled video-caption pairs in target language.
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
Most video-and-language representation learning approaches employ contrastive learning, e. g., CLIP, to project the video and text features into a common latent space according to the semantic similarities of text-video pairs.
Ranked #2 on
Video Retrieval
on LSMDC
(text-to-video Mean Rank metric)
DeltaNet:Conditional Medical Report Generation for COVID-19 Diagnosis
To reduce the workload of radiologists, we propose DeltaNet to generate medical reports automatically.
DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention
To enhance the correlation between vision and language in disentangled spaces, we introduce the visual concepts to DiMBERT which represent visual information in textual format.
Generating Accurate and Faithful Discharge Instructions: Task, Dataset, and Model
We build a benchmark clinical dataset and propose the Re3Writer, which imitates the working patterns of physicians to first retrieve related working experience from historical PIs written by physicians, then reason related medical knowledge.
Competence-based Multimodal Curriculum Learning for Medical Report Generation
Medical report generation task, which targets to produce long and coherent descriptions of medical images, has attracted growing research interests recently.
Graph-in-Graph Network for Automatic Gene Ontology Description Generation
This task aims to automatically generate a sentence that describes the function of a GO term belonging to one of the three categories, i. e., molecular function, biological process, and cellular component.
End-to-end Spoken Conversational Question Answering: Task, Dataset and Model
To evaluate the capacity of SCQA systems in a dialogue-style interaction, we assemble a Spoken Conversational Question Answering (Spoken-CoQA) dataset with more than 40k question-answer pairs from 4k conversations.
Ranked #1 on
Spoken Language Understanding
on Spoken-SQuAD
Hazard Detection And Avoidance For The Nova-C Lander
In early 2022, Intuitive Machines' NOVA-C Lander will touch down on the lunar surface becoming the first commercial endeavor to visit a celestial body.
AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation
The acquired disease-grounded visual features can better represent the abnormal regions of the input image, which could alleviate data bias problem; 2) MGT module effectively uses the multi-grained features and Transformer framework to generate the long medical report.
Knowledge Matters: Radiology Report Generation with General and Specific Knowledge
In this paper, we propose a knowledge-enhanced radiology report generation approach introduces two types of medical knowledge: 1) General knowledge, which is input independent and provides the broad knowledge for report generation; 2) Specific knowledge, which is input dependent and provides the fine-grained knowledge for report generation.
Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment
In clinics, a radiology report is crucial for guiding a patient's treatment.
Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation
KGAE consists of a pre-constructed knowledge graph, a knowledge-driven encoder and a knowledge-driven decoder.
Towards Joint Intent Detection and Slot Filling via Higher-order Attention
Intent detection (ID) and Slot filling (SF) are two major tasks in spoken language understanding (SLU).
O2NA: An Object-Oriented Non-Autoregressive Approach for Controllable Video Captioning
Video captioning combines video understanding and language generation.
Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model
As a result, the proposed approach can handle various tasks including: Audio-Oriented Multimodal Machine Comprehension, Machine Reading Comprehension and Machine Listening Comprehension, in a single model, making fair comparisons possible between our model and the existing unimodal MC models.
Contrastive Attention for Automatic Chest X-ray Report Generation
In addition, according to the analysis, the CA model can help existing models better attend to the abnormal regions and provide more accurate descriptions which are crucial for an interpretable diagnosis.
Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation
In detail, PoKE explores the posterior knowledge, which provides explicit abnormal visual regions to alleviate visual data bias; PrKE explores the prior knowledge from the prior medical knowledge graph (medical knowledge) and prior radiology reports (working experience) to alleviate textual data bias.
Prophet Attention: Predicting Attention with Future Attention
Especially for image captioning, the attention based models are expected to ground correct image regions with proper generated words.
On the Generation of Medical Question-Answer Pairs
Question answering (QA) has achieved promising progress recently.
