pubmed.ncbi.nlm.nih.gov

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks - PubMed

  • ️Mon Jan 01 2024

[Preprint]. 2024 Aug 26:arXiv:2408.14254v1.

Affiliations

Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

Gang Qu et al. ArXiv. 2024.

Abstract

Multimodal neuroimaging modeling has become a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural MRI (sMRI) into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging-derived features from various modalities-functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI-within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function. This study not only advances multi-modal neuroimaging analytics by offering a novel method for the integrated analysis of diverse imaging modalities but also improves the understanding of intricate relationship between the brain's structural and functional networks and cognitive development.

Keywords: Brain Connectivity; Cognitive Development; Cognitive Neuroscience; Diffusion Tensor Imaging (DTI); Functional MRI (fMRI); Graph deep learning; Human Connectome Project; Multimodal Neuroimaging Integration; Structural MRI (sMRI).

PubMed Disclaimer

Conflict of interest statement

Financial Dissclosures Section All authors declare that they have no conflicts of interest.

Figures

Fig. 1.
Fig. 1.

Depiction of the Proposed Framework: Functional connectivity (FC) and structural connectivity (SC) obtained from fMRI and DTI, respectively, are amalgamated at the nodal level and subsequently fed into the MaskGNN for predictive analysis. In the latent space, embeddings of nodal features are integrated with anatomical statistics (AS) from sMRI, alongside a computation of structure-functional coupling using the FC and SC matrices. The aggregated features are then subjected to MaskGNN embedding, graph pooling, and readout processes. After post-training, visualization of the uniform mask across MaskGNN layers is achieved, and a posthoc approach elucidates the contribution of AS.

Fig. 2.
Fig. 2.

The age distribution of selected subjects.

Fig. 3.
Fig. 3.

The distribution of intelligence metrics.

Fig. 4.
Fig. 4.

The use of Grad-CAM and Grad-RAM scores for model explainability: (a) Grad-RAM scores for simultaneous prediction of CCC and FCC; (b) Discrimination of groups using Grad-CAM scores across distinct TCC levels

Fig. 5.
Fig. 5.

A comparative analysis of predictive performance showing the individual and combined effects of the manifold regularization term (Lmanifold) and mask penalty (Lmask) on the proposed model, with a baseline scenario for reference. All comparisons are supported by pair-wise t-tests, with p-values less than 1e − 3 relative to the fully proposed model, highlighting significant differences.

Fig. 6.
Fig. 6.

The model interpretability through learned masks with 0.52 as threshold: (a) mask derived from the simultaneous prediction task for CCC and FCC; (b) Chord diagram from the simultaneous CCC and FCC prediction task, showing inter-network connections among brain functional networks, excluding intra-network links;(c) mask generated for the classification task across distinct TCC levels;(d) Chord diagram from the classification task across distinct TCC levels, showing inter-network connections among brain functional networks, excluding intra-network links. Vis-Visual, SM-Somatomotor, CO-Cingulo-Opercular, DA-Dorsal-Attention, Lang-Language, FP-Frontoparietal , Aud-Auditory, Def-Default, Oth-Others.

Fig. 7.
Fig. 7.

The visualization of Identified Brain Connectivities: Enhanced clarity is achieved by setting the visualization threshold to 0.53. (a) Connectivity patterns identified via the mask generated from the prediction task for CCC and FCC scores. (b) Connectivity patterns identified via the mask generated from the classification task for groups with high and low TCC scores.

Similar articles

References

    1. Baum G.L., et al., 2020. Development of structure–function coupling in human brain networks during youth. Proceedings of the National Academy of Sciences 117, 771–778. - PMC - PubMed
    1. Chattopadhay A., et al., 2018. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks, in: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE. pp. 839–847.
    1. Chok N.S., 2010. Pearson’s versus Spearman’s and Kendall’s correlation coefficients for continuous data. Ph.D. thesis. University of Pittsburgh.
    1. Cruces R.R., et al., 2022. Micapipe: A pipeline for multimodal neuroimaging and connectome analysis. NeuroImage 263, 119612. - PMC - PubMed
    1. Dahnke R., et al., 2013. Cortical thickness and central surface estimation. Neuroimage 65, 336–348. - PubMed

Publication types