In this paper, we critically evaluate the capabilities of the state-of-the-art multimodal large language model, i. e., GPT-4 with Vision (GPT-4V), on Visual Question Answering (VQA) task.
Second, the storage and usage of data from multiple clients in a distributed environment can lead to incompleteness of multi-view data.
In this work, we propose a novel unsupervised Skin Lesion sEgmentation framework based on structural entropy and isolation forest outlier Detection, namely SLED.
Brain extraction, registration and segmentation are indispensable preprocessing steps in neuroimaging studies.
1 code implementation • 26 May 2023 • Kai Zhang, Jun Yu, Zhiling Yan, Yixin Liu, Eashan Adhikarla, Sunyang Fu, Xun Chen, Chen Chen, Yuyin Zhou, Xiang Li, Lifang He, Brian D. Davison, Quanzheng Li, Yong Chen, Hongfang Liu, Lichao Sun
In this paper, we introduce a unified and generalist Biomedical Generative Pre-trained Transformer (BiomedGPT) model, which leverages self-supervision on large and diverse datasets to accept multi-modal inputs and perform a range of downstream tasks.
Ranked #1 on Text Summarization on MeQSum
State abstraction optimizes decision-making by ignoring irrelevant environmental information in reinforcement learning with rich observations.
In this paper, to investigate the applicability of existing denoising techniques, we compare a variety of denoising methods on both synthetic and real-world datasets for different applications.
In order to better learn these neutral expression-disentangled features (NDFs) and to alleviate the non-convex optimization problem, a self-paced learning (SPL) strategy based on NDFs is proposed in the training stage.
no code implementations • 18 Feb 2023 • Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, JianXin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, Lichao Sun
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
Recent research on deformable image registration is mainly focused on improving the registration accuracy using multi-stage alignment methods, where the source image is repeatedly deformed in stages by a same neural network until it is well-aligned with the target image.
Normative modeling is an emerging and promising approach to effectively study disorder heterogeneity in individual participants.
The critical point of MGC is to better utilize the view-specific and view-common information in features and graphs of multiple views.
Finally, we discuss the open challenges and potential future opportunities in different fields of deep clustering.
The assessment of Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI) associated with brain changes remains a challenging task.
Mapping the connections of the human brain as a network is one of the most pervasive paradigms in neuroscience.
Specifically, we propose to meta-train the model on datasets of large sample sizes and transfer the knowledge to small datasets.
To address this issue, in this paper we propose Deep Embedded Multi-view Clustering via Jointly Learning Latent Representations and Graphs (DMVCJ), which utilizes the latent graphs to promote the performance of deep embedded MVC models from two aspects.
Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies.
no code implementations • 18 Mar 2022 • Xusheng Zhao, Jia Wu, Hao Peng, Amin Beheshti, Jessica J. M. Monaghan, David Mcalpine, Heivet Hernandez-Perez, Mark Dras, Qiong Dai, Yangyang Li, Philip S. Yu, Lifang He
Modern neuroimaging techniques, such as diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI), enable us to model the human brain as a brain network or connectome.
To bridge this gap, we present BrainGB, a benchmark for brain network analysis with GNNs.
The ability of an agent to change its objectives in response to unexpected events is desirable in dynamic environments.
This paper presents a novel graph-based kernel learning approach for connectome analysis.
To capture the semantic graph structure from raw text, most existing summarization approaches are built on GNNs with a pre-trained model.
In this paper, we propose MGNet, a simple and effective multiplex graph convolutional network (GCN) model for multimodal brain network analysis.
Interpretable brain network models for disease prediction are of great value for the advancement of neuroscience.
Multimodal brain networks characterize complex connectivities among different brain regions from both structural and functional aspects and provide a new means for mental disease analysis.
Our method learns different levels of features from the raw features, including low-level features, high-level features, and semantic labels/features in a fusion-free manner, so that it can effectively achieve the reconstruction objective and the consistency objectives in different feature spaces.
The prior of view-common variable obeys approximately discrete Gumbel Softmax distribution, which is introduced to extract the common cluster factor of multiple views.
With the prevalence of graph data in real-world applications, many methods have been proposed in recent years to learn high-quality graph embedding vectors various types of graphs.
Based on them, we design MinGE to directly calculate the ideal node embedding dimension for any graph.
This paper aims to provide a generic Federated Multi-View Learning (FedMV) framework for multi-view data leakage prevention, which is based on different types of local data availability and enables to accommodate two types of problems: Vertical Federated Multi-View Learning (V-FedMV) and Horizontal Federated Multi-View Learning (H-FedMV).
Furthermore, they cannot fully capture the content-based correlations between nodes, as they either do not use the self-attention mechanism or only use it to consider the immediate neighbors of each node, ignoring the higher-order neighbors.
1 code implementation • 14 Apr 2021 • Chaoyang He, Keshav Balasubramanian, Emir Ceyani, Carl Yang, Han Xie, Lichao Sun, Lifang He, Liangwei Yang, Philip S. Yu, Yu Rong, Peilin Zhao, Junzhou Huang, Murali Annavaram, Salman Avestimehr
FedGraphNN is built on a unified formulation of graph FL and contains a wide range of datasets from different domains, popular GNN models, and FL algorithms, with secure and efficient system support.
Third, we propose a streaming social event detection and evolution discovery framework for HINs based on meta-path similarity search, historical information about meta-paths, and heterogeneous DBSCAN clustering method.
Depression is one of the most common mental illness problems, and the symptoms shown by patients are not consistent, making it difficult to diagnose in the process of clinical practice and pathological research.
Graph representation learning has attracted increasing research attention.
In this paper, we propose a new interpretable graph pooling framework - CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph representation learning process.
Leveraging the nonlocal self-similarity (NLSS) characteristic of images and sparse representation in the transform domain, the block-matching and 3D filtering (BM3D) based methods show powerful denoising performance.
Review rating prediction of text reviews is a rapidly growing technology with a wide range of applications in natural language processing.
Our studies show that mixup is a domain-independent data augmentation technique to pre-trained language models, resulting in significant performance improvement for transformer-based models.
To promote the ability of commonsense reasoning for text generation, we propose a novel knowledge graph augmented pre-trained language generation model KG-BART, which encompasses the complex relations of concepts through the knowledge graph and produces more logical and natural sentences as output.
Although these attributes may change, an author's co-authors and research topics do not change frequently with time, which means that papers within a period have similar text and relation information in the academic network.
As a departure from prior work, Luce organizes the house data in a heterogeneous information network (HIN) where graph nodes are house entities and attributes that are important for house price valuation.
To capture the directed edges between nodes, existing methods mostly learn two embedding vectors for each node, source vector and target vector.
The last decade has seen a surge of research in this area due to the unprecedented success of deep learning.
Recently, graph-based algorithms have drawn much attention because of their impressive success in semi-supervised setups.
In this paper, we propose a novel hierarchical taxonomy-aware and attentional graph capsule recurrent CNNs framework for large-scale multi-label text classification.
Therefore, this review is intended to provide an overall landscape of more than 100 papers on adversarial attack and defense strategies for graph data, and establish a unified formulation encompassing most graph adversarial learning models.
We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse.
In many real-world applications, data are often unlabeled and comprised of different representations/views which often provide information complementary to each other.
Then, it is possible to utilize unlabeled data that have a potential of failure to further improve the performance of the model.
Meta-graph is currently the most powerful tool for similarity search on heterogeneous information networks, where a meta-graph is a composition of meta-paths that captures the complex structural information.
Network analysis of human brain connectivity is critically important for understanding brain function and disease states.
Parkinson's Disease (PD) is one of the most prevalent neurodegenerative diseases that affects tens of millions of Americans.
The results indicate that once we remove contaminants from the datasets, we can significantly improve both malware detection rate and detection accuracy
Cryptography and Security
Specifically, we propose an auto-weighted framework of Multi-view Graph Embedding with Hub Detection (MVGE-HD) for brain network analysis.
In the context of supervised tensor learning, preserving the structural information and exploiting the discriminative nonlinear relationships of tensor data are crucial for improving the performance of learning tasks.
Owing to prominence as a diagnostic tool for probing the neural correlates of cognition, neuroimaging tensor data has been the focus of intense investigation.
We model the multi-view clustering problem as a joint weighted nonnegative matrix factorization problem and process the multi-view data chunk by chunk to reduce the memory requirement.
Third, how to leverage the consistent and complementary information from different views to improve the feature selection in the situation when the data are too big or come in as streams?
In many real-world applications, information can be gathered from multiple sources, while each source can contain multiple views, which are more cohesive for learning.
The dual-tensorial mapping function can map each tensor instance in the input space to another tensor in the feature space while preserving the tensorial structure.