Search Results for author:
Found 86 papers, 18 papers with code
Eye-gaze Guided Multi-modal Alignment Framework for Radiology
Additionally, we explore the impact of varying amounts of eye-gaze data on model performance, highlighting the feasibility and utility of integrating this auxiliary data into multi-modal pre-training.
Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era
Therefore, in this paper, we introduce Usable XAI in the context of LLMs by analyzing (1) how XAI can benefit LLMs and AI systems, and (2) how LLMs can contribute to the advancement of XAI.
Conditional Score-Based Diffusion Model for Cortical Thickness Trajectory Prediction
Alzheimer's Disease (AD) is a neurodegenerative condition characterized by diverse progression rates among individuals, with changes in cortical thickness (CTh) closely linked to its progression.
Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis
In this study, we leverage LLM to enhance the semantic analysis and develop similarity metrics for texts, addressing the limitations of traditional unsupervised NLP metrics like ROUGE and BLEU.
Revolutionizing Finance with LLMs: An Overview of Applications and Insights
Additionally, we conducted holistic tests on multiple financial tasks through the combination of natural language instructions.
The Radiation Oncology NLP Database
ROND is specifically designed to address this gap in the domain of radiation oncology, a field that offers many opportunities for NLP exploration.
Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understanding
This study is a pioneering endeavor to investigate the capabilities of Large Language Models (LLMs) in addressing conceptual questions within the domain of mechanical engineering with a focus on mechanics.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
Notably, in the realm of robot task planning, LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions.
Understanding LLMs: A Comprehensive Overview from Training to Inference
Low-cost training and deployment of LLMs represent the future development trend.
On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications
The advent of large language models (LLMs) has heightened interest in their potential for multimodal applications that integrate language and vision.
Ophtha-LLaMA2: A Large Language Model for Ophthalmology
In recent years, pre-trained large language models (LLMs) have achieved tremendous success in the field of Natural Language Processing (NLP).
Noisy probing dose facilitated dose prediction for pencil beam scanning proton therapy: physics enhances generalizability
The noisy probing dose method showed better generalizability in the 6 outlier cases than the ROI-based and beam mask-based methods with 3D Gamma passing rates (for prostate cancer, targets: 89. 32%$\pm$1. 45% vs. 93. 48%$\pm$1. 51% vs. 96. 79%$\pm$0. 83%, OARs: 85. 87%$\pm$1. 73% vs. 91. 15%$\pm$1. 13% vs. 94. 29%$\pm$1. 01%).
Holistic Evaluation of GPT-4V for Biomedical Imaging
GPT-4V represents a breakthrough in artificial general intelligence (AGI) for computer vision, with applications in the biomedical domain.
Evaluating Large Language Models in Ophthalmology
Methods: A 100-item ophthalmology single-choice test was administered to three different LLMs (GPT-3. 5, GPT-4, and PaLM2) and three different professional levels (medical undergraduates, medical masters, and attending physicians), respectively.
Evaluating multiple large language models in pediatric ophthalmology
IMPORTANCE The response effectiveness of different large language models (LLMs) and various individuals, including medical students, graduate students, and practicing physicians, in pediatric ophthalmology consultations, has not been clearly established yet.
Evaluating the Potential of Leading Large Language Models in Reasoning Biology Questions
Recent advances in Large Language Models (LLMs) have presented new opportunities for integrating Artificial General Intelligence (AGI) into biological research and education.
Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities
Recent advances in artificial general intelligence (AGI), particularly large language models and creative image generation systems have demonstrated impressive capabilities on diverse tasks spanning the arts and humanities.
Exploring hyperelastic material model discovery for human brain cortex: multivariate analysis vs. artificial neural network approaches
This study validates the applicability and accuracy of artificial neural network to automatically discover constitutive material models with proper regularization as well as the benefits in model simplification without compromising accuracy for traditional multivariable regression.
ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data
Radiology report generation, as a key step in medical image analysis, is critical to the quantitative analysis of clinically informed decision-making levels.
Benchmarking a foundation LLM on its ability to re-label structure names in accordance with the AAPM TG-263 report
Conclusions: Given the accuracy of GPT-4 in re-labeling structure names of both target volumes and normal tissues as presented in this work, LLMs are poised to be the preferred method for standardizing structure names in radiation oncology, especially considering the rapid advancements in LLM capabilities that are likely to continue.
MedEdit: Model Editing for Medical Question Answering with External Knowledge Bases
Focusing on medical QA using the MedQA-SMILE dataset, we evaluate the impact of different retrieval models and the number of facts provided to the LLM.
MediViSTA-SAM: Zero-shot Medical Video Analysis with Spatio-temporal SAM Adaptation for Echocardiography
The Segmentation Anything Model (SAM) has gained significant attention for its robust generalization capabilities across diverse downstream tasks.
PolicyGPT: Automated Analysis of Privacy Policies with Large Language Models
In this study, we investigate a privacy policy text analysis framework PolicyGPT based on the LLM.
NoisyNN: Exploring the Influence of Information Entropy Change in Learning Systems
Herein, we use the information entropy to define the complexity of the task.
Ranked #267 on
Image Classification
on ImageNet
RadOnc-GPT: A Large Language Model for Radiation Oncology
This paper presents RadOnc-GPT, a large language model specialized for radiation oncology through advanced tuning methods.
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation tasks.
Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges
Artificial General Intelligence (AGI), possessing the capacity to comprehend, learn, and execute tasks with human cognitive abilities, engenders significant anticipation and intrigue across scientific, commercial, and societal arenas.
Chat2Brain: A Method for Mapping Open-Ended Semantic Queries to Brain Activation Maps
In this study, we propose a method called Chat2Brain that combines LLMs to basic text-2-image model, known as Text2Brain, to map open-ended semantic queries to brain activation maps in data-scarce and complex query environments.
Radiology-Llama2: Best-in-Class Large Language Model for Radiology
This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning.
Robust Core-Periphery Constrained Transformer for Domain Adaptation
Unsupervised domain adaptation (UDA) aims to learn transferable representation across domains.
Evaluating Large Language Models for Radiology Natural Language Processing
The rise of large language models (LLMs) has marked a pivotal shift in the field of natural language processing (NLP).
PharmacyGPT: The AI Pharmacist
In this study, we introduce PharmacyGPT, a novel framework to assess the capabilities of large language models (LLMs) such as ChatGPT and GPT-4 in emulating the role of clinical pharmacists.
Hierarchical Semantic Tree Concept Whitening for Interpretable Image Classification
With the popularity of deep neural networks (DNNs), model interpretability is becoming a critical concern.
Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data
As AD impairs the patient's language understanding and expression ability, the speech of AD patients can serve as an indicator of this disease.
Identification of Causal Relationship between Amyloid-beta Accumulation and Alzheimer's Disease Progression via Counterfactual Inference
Alzheimer's disease (AD) is a neurodegenerative disorder that is beginning with amyloidosis, followed by neuronal loss and deterioration in structure, function, and cognition.
Review of Large Vision Models and Visual Prompt Engineering
This review aims to summarize the methods employed in the computer vision domain for large vision models and visual prompt engineering, exploring the latest advancements in visual prompt engineering.
Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applications
ChatGPT has shown to be a strong baseline in many NLP tasks, and we believe it has the potential to improve our model in the task of semantic matching and enhance our model's understanding of food-related concepts and relationships.
Segment Anything Model (SAM) for Radiation Oncology
Given that SAM, a model pre-trained purely on natural images, can handle the delineation of OARs from medical images with clinically acceptable accuracy, these results highlight SAM's robust generalization capabilities with consistent accuracy in automatic segmentation for radiotherapy.
AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology
In this pioneering study, inspired by AutoGPT, the state-of-the-art open-source application based on the GPT-4 large language model, we develop a novel tool called AD-AutoGPT which can conduct data collection, processing, and analysis about complex health narratives of Alzheimer's Disease in an autonomous manner via users' textual prompts.
Radiology-GPT: A Large Language Model for Radiology
We introduce Radiology-GPT, a large language model for radiology.
Artificial General Intelligence for Medical Imaging
In this review, we explore the potential applications of Artificial General Intelligence (AGI) models in healthcare, focusing on foundational Large Language Models (LLMs), Large Vision Models, and Large Multimodal Models.
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
Conventional task- and modality-specific artificial intelligence (AI) models are inflexible in real-world deployment and maintenance for biomedicine.
Ranked #1 on
Text Summarization
on MeQSum
SAM for Poultry Science
This study aims to assess the zero-shot segmentation performance of SAM on representative chicken segmentation tasks, including part-based segmentation and the use of infrared thermal images, and to explore chicken-tracking tasks by using SAM as a segmentation tool.
Learning Better Contrastive View from Radiologist's Gaze
Then, we propose a novel augmentation method, i. e., FocusContrast, to learn from radiologists' gaze in diagnosis and generate contrastive views for medical images with guidance from radiologists' visual attention.
ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs
To tackle these limitations, we propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification, while improving its interpretability.
Instruction-ViT: Multi-Modal Prompts for Instruction Learning in ViT
Prompts have been proven to play a crucial role in large language models, and in recent years, vision models have also been using prompts to improve scalability for multiple downstream tasks.
Prompt Engineering for Healthcare: Methodologies and Applications
Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks.
AGI: Artificial General Intelligence for Education
AGI, driven by the recent large pre-trained models, represents a significant leap in the capability of machines to perform tasks that require human-level intelligence, such as reasoning, problem-solving, decision-making, and even understanding human emotions and social interactions.
Differentiate ChatGPT-generated and Human-written Medical Texts
We focus on analyzing the differences between medical texts written by human experts and generated by ChatGPT, and designing machine learning workflows to effectively detect and differentiate medical texts generated by ChatGPT.
ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT
The proposed method uses the strengths of LLMs' understanding and logical reasoning to correct the incomplete logical facts for optimizing the performance of perceptual module, by summarizing and reorganizing reasoning rules represented in natural language format.
Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
To this end, in this study, we evaluate the performance of ChatGPT/GPT-4 on a radiology NLI task and compare it to other models fine-tuned specifically on task-related data samples.
ImpressionGPT: An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT
The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians, and it is typically written by radiologists based on the 'Findings' section.
On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence
In this work, we explore the promises and challenges of developing multimodal foundation models for GeoAI.
AGI for Agriculture
Artificial General Intelligence (AGI) is poised to revolutionize a variety of sectors, including healthcare, finance, transportation, and education.
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
This paper presents a comprehensive survey of ChatGPT-related (GPT-3. 5 and GPT-4) research, state-of-the-art large language models (LLM) from the GPT series, and their prospective applications across diverse domains.
Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics
We present the first study to investigate Large Language Models (LLMs) in answering radiation oncology physics questions.
When Brain-inspired AI Meets AGI
Artificial General Intelligence (AGI) has been a long-standing goal of humanity, with the aim of creating machines capable of performing any intellectual task that humans can do.
CP-CNN: Core-Periphery Principle Guided Convolutional Neural Network
In this study, We explore a novel brain-inspired design principle based on the core-periphery property of the human brain network to guide the design of CNNs.
Core-Periphery Principle Guided Redesign of Self-Attention in Transformers
Designing more efficient, reliable, and explainable neural network architectures is critical to studies that are based on artificial intelligence (AI) techniques.
Coupling Artificial Neurons in BERT and Biological Neurons in the Human Brain
Linking computational natural language processing (NLP) models and neural responses to language in the human brain on the one hand facilitates the effort towards disentangling the neural representations underpinning language perception, on the other hand provides neurolinguistics evidence to evaluate and improve NLP models.
DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4
The digitization of healthcare has facilitated the sharing and re-using of medical data but has also raised concerns about confidentiality and privacy.
AugGPT: Leveraging ChatGPT for Text Data Augmentation
Text data augmentation is an effective strategy for overcoming the challenge of limited sample sizes in many natural language processing (NLP) tasks.
Mask-guided BERT for Few Shot Text Classification
The main challenge of FSL is the difficulty of training robust models on small amounts of samples, which frequently leads to overfitting.
Gyri vs. Sulci: Disentangling Brain Core-Periphery Functional Networks via Twin-Transformer
To solve this fundamental problem, we design a novel Twin-Transformer framework to unveil the unique functional roles of gyri and sulci as well as their relationship in the whole brain function.
Context Matters: A Strategy to Pre-train Language Model for Science Education
However, the ideal type of data to contextualize pre-trained language model and improve the performance in automatically scoring student written responses remains unclear.
Matching Exemplar as Next Sentence Prediction (MeNSP): Zero-shot Prompt Learning for Automatic Scoring in Science Education
Developing models to automatically score students' written responses to science problems is critical for science education.
Coarse-to-fine Knowledge Graph Domain Adaptation based on Distantly-supervised Iterative Training
In this framework, we apply distant-supervision on cross-domain knowledge graph adaptation.
Spatial-Temporal Convolutional Attention for Mapping Functional Brain Networks
To validate the performance of the proposed method, we evaluate the approach on HCP-rest dataset.
BI AVAN: Brain inspired Adversarial Visual Attention Network
Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks.
Coupling Visual Semantics of Artificial Neural Networks and Human Brain Function via Synchronized Activations
However, whether there exists semantic correlations/connections between the visual representations in ANNs and those in BNNs remains largely unexplored due to both the lack of an effective tool to link and couple two different domains, and the lack of a general and effective framework of representing the visual semantics in BNNs such as human functional brain networks (FBNs).
Rectify ViT Shortcut Learning by Visual Saliency
In this work, we propose a novel and effective saliency-guided vision transformer (SGT) model to rectify shortcut learning in ViT with the absence of eye-gaze data.
Representing Brain Anatomical Regularity and Variability by Few-Shot Embedding
Our experimental results show that: 1) the learned embedding vectors can quantitatively encode the commonality and individuality of cortical folding patterns; 2) with the embeddings we can robustly infer the complicated many-to-many anatomical correspondences among different brains and 3) our model can be successfully transferred to new populations with very limited training samples.
Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning
To address this problem, we propose to infuse human experts' intelligence and domain knowledge into the training of deep neural networks.
Brain Cortical Functional Gradients Predict Cortical Folding Patterns via Attention Mesh Convolution
The convolution on mesh considers the spatial organization of functional gradients and folding patterns on a cortical sheet and the newly designed channel attention block enhances the interpretability of the contribution of different functional gradients to cortical folding prediction.
Mask-guided Vision Transformer (MG-ViT) for Few-Shot Learning
However, for data-intensive models such as vision transformer (ViT), current fine-tuning based FSL approaches are inefficient in knowledge generalization and thus degenerate the downstream task performances.
A Unified and Biologically-Plausible Relational Graph Representation of Vision Transformers
The key characteristic of these ViT models is to adopt different aggregation strategies of spatial patch information within the artificial neural networks (ANNs).
Discovering Dynamic Functional Brain Networks via Spatial and Channel-wise Attention
More importantly, our proposed hybrid attention modules (SA and CA) do not enforce assumptions of linearity and independence as previous methods, and thus provide a novel approach to better understanding dynamic functional brain networks.
Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers
In this work, we propose a novel Twin-Transformers framework to simultaneously infer common and individual functional networks in both spatial and temporal space, in a self-supervised manner.
Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis
In this paper, we demonstrate that the eye movement of radiologists reading medical images can be a new form of supervision to train the DNN-based computer-aided diagnosis (CAD) system.
Disease2Vec: Representing Alzheimer's Progression via Disease Embedding Tree
By disease em-bedding, the framework generates a disease embedding tree (DETree) which effectively represents different clinical stages as a tree trajectory reflecting AD progression and thus can be used to predict clinical status by projecting individuals onto this continuous trajectory.
Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN)
Simultaneous modeling of the spatio-temporal variation patterns of brain functional network from 4D fMRI data has been an important yet challenging problem for the field of cognitive neuroscience and medical image analysis.
Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics
In recent years, analyzing task-based fMRI (tfMRI) data has become an essential tool for understanding brain function and networks.
Learning Coarse-to-Fine Sparselets for Efficient Object Detection and Scene Classification
Part model-based methods have been successfully applied to object detection and scene classification and have achieved state-of-the-art results.
Predicting Eye Fixations Using Convolutional Neural Networks
It is believed that eye movements in free-viewing of natural scenes are directed by both bottom-up visual saliency and top-down visual factors.
Individualized ROI Optimization via Maximization of Group-wise Consistency of Structural and Functional Profiles
Our strategy is to formulate the individual ROI optimization as a group variance minimization problem, in which group-wise functional and structural connectivity patterns, and anatomic profiles are defined as optimization constraints.
