Advertisement

Prediction of Obsessive-Compulsive Disorder: Importance of Neurobiology-Aided Feature Design and Cross-Diagnosis Transfer Learning

  • Author Footnotes
    1 SVK and AKP contributed equally to this work.
    Sunil Vasu Kalmady
    Correspondence
    Address correspondence to Sunil Vasu Kalmady, Ph.D.
    Footnotes
    1 SVK and AKP contributed equally to this work.
    Affiliations
    Alberta Machine Intelligence Institute, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada

    Canadian VIGOUR Centre, University of Alberta, Edmonton, Alberta, Canada
    Search for articles by this author
  • Author Footnotes
    1 SVK and AKP contributed equally to this work.
    Animesh Kumar Paul
    Footnotes
    1 SVK and AKP contributed equally to this work.
    Affiliations
    Alberta Machine Intelligence Institute, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada
    Search for articles by this author
  • Janardhanan C. Narayanaswamy
    Affiliations
    OCD Clinic, Department of Psychiatry, National Institute of Mental Health and Neuro Sciences, Bangalore, India

    Translational Psychiatry Laboratory, Neurobiology Research Centre, National Institute of Mental Health and Neuro Sciences, Bangalore, India
    Search for articles by this author
  • Rimjhim Agrawal
    Affiliations
    Translational Psychiatry Laboratory, Neurobiology Research Centre, National Institute of Mental Health and Neuro Sciences, Bangalore, India
    Search for articles by this author
  • Venkataram Shivakumar
    Affiliations
    OCD Clinic, Department of Psychiatry, National Institute of Mental Health and Neuro Sciences, Bangalore, India

    Translational Psychiatry Laboratory, Neurobiology Research Centre, National Institute of Mental Health and Neuro Sciences, Bangalore, India
    Search for articles by this author
  • Andrew J. Greenshaw
    Affiliations
    Department of Psychiatry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
    Search for articles by this author
  • Serdar M. Dursun
    Affiliations
    Department of Psychiatry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
    Search for articles by this author
  • Russell Greiner
    Affiliations
    Alberta Machine Intelligence Institute, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada

    Department of Psychiatry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
    Search for articles by this author
  • Ganesan Venkatasubramanian
    Correspondence
    Ganesan Venkatasubramanian, M.D., Ph.D.
    Affiliations
    OCD Clinic, Department of Psychiatry, National Institute of Mental Health and Neuro Sciences, Bangalore, India

    Translational Psychiatry Laboratory, Neurobiology Research Centre, National Institute of Mental Health and Neuro Sciences, Bangalore, India
    Search for articles by this author
  • Y.C. Janardhan Reddy
    Affiliations
    OCD Clinic, Department of Psychiatry, National Institute of Mental Health and Neuro Sciences, Bangalore, India

    Translational Psychiatry Laboratory, Neurobiology Research Centre, National Institute of Mental Health and Neuro Sciences, Bangalore, India
    Search for articles by this author
  • Author Footnotes
    1 SVK and AKP contributed equally to this work.
Open AccessPublished:December 17, 2021DOI:https://doi.org/10.1016/j.bpsc.2021.12.003

      Abstract

      Background

      Machine learning applications using neuroimaging provide a multidimensional, data-driven approach that captures the level of complexity necessary for objectively aiding diagnosis and prognosis in psychiatry. However, models learned from small training samples often have limited generalizability, which continues to be a problem with automated diagnosis of mental illnesses such as obsessive-compulsive disorder (OCD). Earlier studies have shown that features incorporating prior neurobiological knowledge of brain function and combining brain parcellations from various sources can potentially improve the overall prediction. However, it is unknown whether such knowledge-driven methods can provide a performance that is comparable to state-of-the-art approaches based on neural networks.

      Methods

      In this study, we apply a transparent and explainable multiparcellation ensemble learning framework EMPaSchiz (Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction) to the task of predicting OCD, based on a resting-state functional magnetic resonance imaging dataset of 350 subjects. Furthermore, we apply transfer learning using the features found effective for schizophrenia to OCD to leverage the commonality in brain alterations across these psychiatric diagnoses.

      Results

      We show that our knowledge-based approach leads to a prediction performance of 80.3% accuracy for OCD diagnosis that is better than domain-agnostic and automated feature design using neural networks. Furthermore, we show that a selection of reduced feature sets can be transferred from schizophrenia to the OCD prediction model without significant loss in prediction performance.

      Conclusions

      This study presents a machine learning framework for OCD prediction with neurobiology-aided feature design using resting-state functional magnetic resonance imaging that is generalizable and reasonably interpretable.

      Keywords

      Obsessive-compulsive disorder (OCD) is a debilitating condition—the fourth most common psychiatric illness—that affects millions of people worldwide (
      • Ruscio A.M.
      • Stein D.J.
      • Chiu W.T.
      • Kessler R.C.
      The epidemiology of obsessive-compulsive disorder in the National Comorbidity Survey Replication.
      ). Evidence shows that a reliable diagnosis of the condition is crucial for timely intervention and improvement in patients’ overall quality of life (
      • Janardhan Reddy Y.C.
      • Sundar A.S.
      • Narayanaswamy J.C.
      • Math S.B.
      Clinical practice guidelines for obsessive-compulsive disorder.
      ). However, owing to heterogeneity in the clinical presentation (
      • do Rosário M.C.
      • Batistutto M.
      • Ferrao Y.
      Symptom heterogeneity in OCD: A dimensional approach.
      ) and comorbidity with other psychiatric disorders (
      • Viswanath B.
      • Narayanaswamy J.C.
      • Rajkumar R.P.
      • Cherian A.V.
      • Kandavel T.
      • Math S.B.
      • Reddy Y.C.J.
      Impact of depressive and anxiety disorder comorbidity on the clinical expression of obsessive-compulsive disorder.
      ), including bipolar disorder (
      • Amerio A.
      • Odone A.
      • Liapis C.C.
      • Ghaemi S.N.
      Diagnostic validity of comorbid bipolar disorder and obsessive-compulsive disorder: A systematic review.
      ) and schizophrenia (SCZ) (
      • Schirmbeck F.
      • Zink M.
      Comorbid obsessive-compulsive symptoms in schizophrenia: Contributions of pharmacological and genetic factors.
      ), precise diagnosis continues to be a challenge.
      In the past few years, there has been an increased adaptation of machine learning methods in psychiatric applications, often to produce models that predict the diagnosis of novel subjects (
      • Rutledge R.B.
      • Chekroud A.M.
      • Huys Q.J.
      Machine learning and big data in psychiatry: Toward clinical applications.
      ). Many studies have demonstrated that functional magnetic resonance imaging (fMRI) of the brain has sufficient information to generate models that can discriminate healthy control (HC) subjects from patients with various psychiatric illnesses, such as autism (
      • Hyde K.K.
      • Novack M.N.
      • LaHaye N.
      • Parlett-Pelleriti C.
      • Anden R.
      • Dixon D.R.
      • Linstead E.
      Applications of supervised machine learning in autism spectrum disorder research: A review.
      ), SCZ (
      • de Filippis R.
      • Carbone E.A.
      • Gaetano R.
      • Bruni A.
      • Pugliese V.
      • Segura-Garcia C.
      • De Fazio P.
      Machine learning techniques in a structural and functional MRI diagnostic approach in schizophrenia: A systematic review.
      ), depression (
      • Gao S.
      • Calhoun V.D.
      • Sui J.
      Machine learning in major depression: From classification to treatment outcome prediction.
      ), and OCD (

      Shenas SK, Halici U, Cicek M (2013): Detection of obsessive compulsive disorder using resting-state functional connectivity data. Presented at the 6th International Conference on Biomedical Engineering and Informatics, December 16–18, 2013, Hangzhou, China.

      ,
      • Gruner P.
      • Vo A.
      • Argyelan M.
      • Ikuta T.
      • Degnan A.J.
      • John M.
      • et al.
      Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder.
      ,
      • Sen B.
      • Bernstein G.A.
      • Xu T.
      • Mueller B.A.
      • Schreiner M.W.
      • Cullen K.R.
      • Parhi K.K.
      Classification of obsessive-compulsive disorder from resting-state fMRI.
      ,
      • Takagi Y.
      • Sakai Y.
      • Lisi G.
      • Yahata N.
      • Abe Y.
      • Nishida S.
      • et al.
      A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
      ,
      • Yang X.
      • Hu X.
      • Tang W.
      • Li B.
      • Yang Y.
      • Gong Q.
      • Huang X.
      Multivariate classification of drug-naive obsessive-compulsive disorder patients and healthy controls by applying an SVM to resting-state functional MRI data.
      ,
      • Hu X.
      • Zhang L.
      • Bu X.
      • Li H.
      • Li B.
      • Tang W.
      • et al.
      Localized connectivity in obsessive-compulsive disorder: An investigation combining univariate and multivariate pattern analyses.
      ,
      • Bu X.
      • Hu X.
      • Zhang L.
      • Li B.
      • Zhou M.
      • Lu L.
      • et al.
      Investigating the predictive value of different resting-state functional MRI parameters in obsessive-compulsive disorder.
      ) (see list of OCD studies in Table S2 in Supplement 1). However, most of these studies were conducted on small datasets that used fewer than 100 subjects for both learning and cross-validation of the learned model (

      Shenas SK, Halici U, Cicek M (2013): Detection of obsessive compulsive disorder using resting-state functional connectivity data. Presented at the 6th International Conference on Biomedical Engineering and Informatics, December 16–18, 2013, Hangzhou, China.

      ,
      • Gruner P.
      • Vo A.
      • Argyelan M.
      • Ikuta T.
      • Degnan A.J.
      • John M.
      • et al.
      Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder.
      ,
      • Sen B.
      • Bernstein G.A.
      • Xu T.
      • Mueller B.A.
      • Schreiner M.W.
      • Cullen K.R.
      • Parhi K.K.
      Classification of obsessive-compulsive disorder from resting-state fMRI.
      ,
      • Bu X.
      • Hu X.
      • Zhang L.
      • Li B.
      • Zhou M.
      • Lu L.
      • et al.
      Investigating the predictive value of different resting-state functional MRI parameters in obsessive-compulsive disorder.
      ). Limited samples can severely limit the generalizability of such models, and in fact, many have noted that small sample sizes can lead to overestimated measures of performance (
      • Schnack H.G.
      • Kahn R.S.
      Detecting neuroimaging biomarkers for psychiatric disorders: Sample size matters.
      ). Studies on larger sample sizes (>100) have reported OCD prediction performance ranging from 72% to 79% (
      • Takagi Y.
      • Sakai Y.
      • Lisi G.
      • Yahata N.
      • Abe Y.
      • Nishida S.
      • et al.
      A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
      ,
      • Yang X.
      • Hu X.
      • Tang W.
      • Li B.
      • Yang Y.
      • Gong Q.
      • Huang X.
      Multivariate classification of drug-naive obsessive-compulsive disorder patients and healthy controls by applying an SVM to resting-state functional MRI data.
      ,
      • Hu X.
      • Zhang L.
      • Bu X.
      • Li H.
      • Li B.
      • Tang W.
      • et al.
      Localized connectivity in obsessive-compulsive disorder: An investigation combining univariate and multivariate pattern analyses.
      ).
      While it is essential to have sufficient information to learn a generalizable model, pragmatically it is challenging to conduct studies that sample thousands of psychiatric patients. Therefore, it is critical to extract as much relevant information as possible from the limited data at hand to build a generalizable model and, in particular, to reduce the chance of overfitting. Our earlier research explored a way to design features—each a combination of one of several regional connectivity-based measures and one of the various parcellation maps (each incorporating a unique source of prior neurobiological knowledge)—and then learn a classifier based on these features. We found that this framework, called EMPaSchiz (Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction, read as “Emphasis”), could effectively discriminate patients with drug-naïve SCZ from HC subjects (
      • Kalmady S.V.
      • Greiner R.
      • Agrawal R.
      • Shivakumar V.
      • Narayanaswamy J.C.
      • Brown M.R.G.
      • et al.
      Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning.
      ). However, this approach has not been explored to distinguish patients with OCD from HC subjects; none of the earlier machine learning studies in OCD (

      Shenas SK, Halici U, Cicek M (2013): Detection of obsessive compulsive disorder using resting-state functional connectivity data. Presented at the 6th International Conference on Biomedical Engineering and Informatics, December 16–18, 2013, Hangzhou, China.

      ,
      • Gruner P.
      • Vo A.
      • Argyelan M.
      • Ikuta T.
      • Degnan A.J.
      • John M.
      • et al.
      Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder.
      ,
      • Sen B.
      • Bernstein G.A.
      • Xu T.
      • Mueller B.A.
      • Schreiner M.W.
      • Cullen K.R.
      • Parhi K.K.
      Classification of obsessive-compulsive disorder from resting-state fMRI.
      ,
      • Takagi Y.
      • Sakai Y.
      • Lisi G.
      • Yahata N.
      • Abe Y.
      • Nishida S.
      • et al.
      A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
      ,
      • Yang X.
      • Hu X.
      • Tang W.
      • Li B.
      • Yang Y.
      • Gong Q.
      • Huang X.
      Multivariate classification of drug-naive obsessive-compulsive disorder patients and healthy controls by applying an SVM to resting-state functional MRI data.
      ,
      • Hu X.
      • Zhang L.
      • Bu X.
      • Li H.
      • Li B.
      • Tang W.
      • et al.
      Localized connectivity in obsessive-compulsive disorder: An investigation combining univariate and multivariate pattern analyses.
      ,
      • Bu X.
      • Hu X.
      • Zhang L.
      • Li B.
      • Zhou M.
      • Lu L.
      • et al.
      Investigating the predictive value of different resting-state functional MRI parameters in obsessive-compulsive disorder.
      ) have combined the regional and connectivity features, nor have any used an ensemble approach to learn from multiple parcellations jointly. Moreover, earlier studies have not explored whether such knowledge-driven methods can provide performance comparable to the popular machine learning approach of applying neural networks (NNs), which are generally known to provide state-of-the-art results (
      • Shahid N.
      • Rappon T.
      • Berta W.
      Applications of artificial neural networks in health care organizational decision-making: A scoping review.
      ), albeit less transparent simpler models (
      • Chen D.
      • Liu S.
      • Kingsbury P.
      • Sohn S.
      • Storlie C.B.
      • Habermann E.B.
      • et al.
      Deep learning and alternative learning strategies for retrospective real-world clinical data.
      ) such as EMPaSchiz.
      Today, psychiatric disorders are classified based on a consensus about clusters of symptoms that patients experience currently or in the past. Classificatory systems, such as the DSM, have provided clinicians worldwide with a standard framework to identify, treat, and manage these conditions (
      American Psychiatric Association
      Diagnostic and Statistical Manual of Mental Disorders (DSM-5®).
      ). However, while most scholars are convinced that pathology underlying symptoms of psychiatric illnesses is attributable to features of neural pathways and their interfacing axes, most clinical decisions in psychiatric practice are based neither on the etiologic mechanisms (
      • Nelson B.
      • McGorry P.D.
      • Wichers M.
      • Wigman J.T.W.
      • Hartmann J.A.
      Moving from static to dynamic models of the onset of mental disorder: A review.
      ) nor on dynamic aspects of brain structure or function (
      • Hyman S.E.
      Can neuroscience be integrated into the DSM-V?.
      ). Decades of research toward the understanding of neuropathology underlying these illnesses have shown only weak and unclear correspondence between brain measures and clinical categorization (
      • Cuthbert B.N.
      • Insel T.R.
      Toward the future of psychiatric diagnosis: The seven pillars of RDoC.
      ). Many brain regions and networks implicated in one psychiatric disorder seem to be involved in other disorders (or at least share a common subset) (
      • Goodkind M.
      • Eickhoff S.B.
      • Oathes D.J.
      • Jiang Y.
      • Chang A.
      • Jones-Hagata L.B.
      • et al.
      Identification of a common neurobiological substrate for mental illness.
      ). This observation can be potentially leveraged when learning models by transferring knowledge gained with a dataset of one condition to help diagnose another condition, where features relevant for predicting one psychiatric disorder can be used to predict another; this is considered a type of transfer learning (
      • Caruana R.
      Multitask learning.
      ). In this study, we chose to transfer information obtained by learning models for SCZ diagnosis to OCD diagnosis. The two motivating factors for our choice were 1) SCZ and OCD were shown to have shared resting-state features (
      • Zhang Y.
      • Liao J.
      • Li Q.
      • Zhang X.
      • Liu L.
      • Yan J.
      • et al.
      Altered resting-state brain activity in schizophrenia and obsessive-compulsive disorder compared with non-psychiatric controls: Commonalities and distinctions across disorders.
      ) and 2) patients with SCZ were observed to show comorbid obsessive-compulsive symptoms in previous studies (
      • Ongür D.
      • Goff D.C.
      Obsessive-compulsive symptoms in schizophrenia: Associated clinical features, cognitive function and medication status.
      ).
      In addition to generalizability, it is critical that machine learning models in health care demonstrate robustness and trustworthiness (
      • Holzinger A.
      • Langs G.
      • Denk H.
      • Zatloukal K.
      • Müller H.
      Causability and explainability of artificial intelligence in medicine.
      ). Hence, it is valuable to build models that can provide interpretability in terms of providing 1) auxiliary arguments in favor of the model’s correctness, 2) transparency with the usage of features explainable by domain experts such as psychiatrists, and 3) reproducibility to ensure that the model can be trusted in high-stakes situations such as medical diagnosis.
      In this study, we applied the EMPaSchiz learning approach to an OCD versus control dataset to produce a model that can diagnose OCD and show empirically that this use of EMPaSchiz demonstrates some of the points mentioned above. We believe that these factors are relevant to the future of neuroimage-based machine learning methods for psychiatric diagnosis and medical diagnosis in general.
      This research explores whether 1) this EMPaSchiz approach can produce models that can predict OCD accurately in an interpretable way; 2) customized feature design, based on prior neurobiological knowledge (parcellations), can produce models that are comparable to knowledge-agnostic automated methods (neural nets); and 3) one can transfer information obtained by learning models for SCZ diagnosis to OCD diagnosis by using the high-level features selected from the model learned for predicting SCZ for the task of predicting OCD.

      Methods and Materials

      Dataset

      Our study sample contained 188 patients attending the OCD Clinic of the National Institute of Mental Health & Neuro Sciences, India, who fulfilled DSM-IV criteria for OCD. The diagnosis of OCD was established using the Mini-International Neuropsychiatric Interview Plus (
      • Sheehan D.V.
      • Lecrubier Y.
      • Sheehan K.H.
      • Amorim P.
      • Janavs J.
      • Weiller E.
      • et al.
      The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.
      ), which was confirmed by another psychiatrist through an independent clinical interview (YCJR or JCN). We used the Yale-Brown Obsessive Compulsive Scale (
      • Goodman W.K.
      • Price L.H.
      • Rasmussen S.A.
      • Mazure C.
      • Fleischmann R.L.
      • Hill C.L.
      • et al.
      The Yale-Brown Obsessive Compulsive Scale. I. Development, use, and reliability.
      ) to measure symptoms.
      HC subjects were recruited from among consenting healthy volunteers from the same locale to match for age and sex. We used 200 age- and sex-matched HC subjects, who were screened to rule out any psychiatric diagnosis using the Mini-International Neuropsychiatric Interview. For both patients and control subjects, we recruited only right-handed subjects to avoid the potential confounds of differential handedness. None of the study subjects had contraindications to MRI or medical illnesses that could significantly influence central nervous system function or structure, such as seizure disorder, cerebral palsy, or history suggestive of delayed developmental milestones. There was no lifetime history suggestive of DSM-IV psychoactive substance dependence or of head injury associated with loss of consciousness longer than 10 minutes. No subjects had abnormal movements as assessed by the Abnormal Involuntary Movements Scale (
      • Guy W.
      ECDEU Assessment Manual for Psychopharmacology.
      ). Pregnant or postpartum women were excluded. Table S3 in Supplement 1 provides details of the demographic and clinical profiles of subjects who qualified to be included in the study.
      In addition to patients with OCD and HC subjects, we also used a dataset of patients with drug-naïve SCZ for transfer learning analyses in this study. This cohort has been analyzed in our previous study (
      • Kalmady S.V.
      • Greiner R.
      • Agrawal R.
      • Shivakumar V.
      • Narayanaswamy J.C.
      • Brown M.R.G.
      • et al.
      Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning.
      ), and basic demographic information of these subjects is provided in Table S4 in Supplement 1.
      Methodological details related to image acquisition and preprocessing are provided in Section F in Supplement 1. We excluded images for 8 patients and 7 control subjects from the study based on excessive head movement (translational >2.0 mm and/or rotational >2°) (
      • Chang X.
      • Xi Y.B.
      • Cui L.B.
      • Wang H.N.
      • Sun J.B.
      • Zhu Y.Q.
      • et al.
      Distinct inter-hemispheric dysconnectivity in schizophrenia patients with and without auditory verbal hallucinations.
      ) to avoid class differences in head motion. In addition, 5 patients and 18 control subjects were excluded due to incomplete imaging or clinical data. This yielded a total of 350 subjects: 175 control subjects and 175 patients.
      We obtained written informed consent after providing a complete description of the study to all subjects. The National Institute of Mental Health & Neuro Sciences ethics committee reviewed and approved the original research protocol. The Research Ethics Board at the University of Alberta, Edmonton, approved the secondary analysis of deidentified, preprocessed data. All methods were performed in accordance with the relevant guidelines and regulations.

      EMPaSchiz

      We used EMPaSchiz to learn a model to predict the diagnosis of OCD. A detailed description of the EMPaSchiz system is provided in the original paper (
      • Kalmady S.V.
      • Greiner R.
      • Agrawal R.
      • Shivakumar V.
      • Narayanaswamy J.C.
      • Brown M.R.G.
      • et al.
      Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning.
      ). Briefly, EMPaSchiz extracts six resting-state brain fMRI features, including three regional-based features (amplitude of low-frequency fluctuations [ALFF], fractional ALFFs [fALFFs], regional homogeneity [ReHo]) and three connectivity-based features (using functional connectivity [FC] matrices between each pair of regions [see parcellations below] using one of three statistical measures: FC-Pearson correlation, FC-partial correlation, or FC-precision). ALFF was calculated as total power within the frequency range between 0.01 and 0.08 Hz to estimate the strength of low-frequency oscillations (
      • Zang Y.F.
      • He Y.
      • Zhu C.Z.
      • Cao Q.J.
      • Sui M.Q.
      • Liang M.
      • et al.
      Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI.
      ). fALFF was calculated as the power within the low-frequency range (0.01–0.08 Hz) divided by the total power in the entire detectable frequency range (
      • Zou Q.H.
      • Zhu C.Z.
      • Yang Y.
      • Zuo X.N.
      • Long X.Y.
      • Cao Q.J.
      • et al.
      An improved approach to detection of amplitude of low-frequency fluctuation (ALFF) for resting-state fMRI: Fractional ALFF.
      ). Finally, ReHo was calculated using Kendall’s coefficient of concordance (
      • Kendall M.G.
      Rank correlation methods.
      ) as a measure of the similarity between the time series of a given voxel and its nearest neighbors (
      • Kendall M.G.
      Rank correlation methods.
      ,
      • Zang Y.
      • Jiang T.
      • Lu Y.
      • He Y.
      • Tian L.
      Regional homogeneity approach to fMRI data analysis.
      ). To obtain neurobiologically relevant feature compressions, EMPaSchiz projected each feature extraction onto 14 different parcellation schemes, each based on a specific predefined brain atlas or set of regions of interest. These schemes varied widely in principle: 1) predefined ontology of brain structures such as postmortem cytoarchitecture (
      • Tzourio-Mazoyer N.
      • Landeau B.
      • Papathanassiou D.
      • Crivello F.
      • Etard O.
      • Delcroix N.
      • et al.
      Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain.
      ,
      • Zilles K.
      • Amunts K.
      Receptor mapping: Architecture of the human cerebral cortex.
      ) and sulco-gyral anatomy (
      • Destrieux C.
      • Fischl B.
      • Dale A.M.
      • Halgren E.
      A sulcal depth-based anatomical parcellation of the cerebral cortex.
      ,
      • Talairach J.
      • Tournoux P.
      Co-Planar Stereotaxic Atlas of the Human Brain: 3-D Proportional System: An Approach to Cerebral Imaging.
      ), 2) data-driven modeling of the functional features from resting-state (
      • Yeo B.T.T.
      • Krienen F.M.
      • Sepulcre J.
      • Sabuncu M.R.
      • Lashkari D.
      • Hollinshead M.
      • et al.
      The organization of the human cerebral cortex estimated by intrinsic functional connectivity.
      ,
      • Varoquaux G.
      • Gramfort A.
      • Pedregosa F.
      • Michel V.
      • Thirion B.
      Multi-subject dictionary learning to segment an atlas of brain spontaneous activity.
      ,
      • Power J.D.
      • Cohen A.L.
      • Nelson S.M.
      • Wig G.S.
      • Barnes K.A.
      • Church J.A.
      • et al.
      Functional network organization of the human brain.
      ) or task-based (
      • Smith S.M.
      • Fox P.T.
      • Miller K.L.
      • Glahn D.C.
      • Fox P.M.
      • Mackay C.E.
      • et al.
      Correspondence of the brain’s functional architecture during activation and rest.
      ) fMRI, or 3) meta-analyses (
      • Dosenbach N.U.F.
      • Nardos B.
      • Cohen A.L.
      • Fair D.A.
      • Power J.D.
      • Church J.A.
      • et al.
      Prediction of individual brain maturity using fMRI [published correction appears in Science 2010; 330:756].
      ) using analytical techniques such as hierarchical clustering (
      • Bellec P.
      • Rosa-Neto P.
      • Lyttelton O.C.
      • Benali H.
      • Evans A.C.
      Multi-level bootstrap analysis of stable clusters in resting-state fMRI.
      ) or independent components analysis (
      • McKeown M.J.
      • Makeig S.
      • Brown G.G.
      • Jung T.P.
      • Kindermann S.S.
      • Bell A.J.
      • Sejnowski T.J.
      Analysis of fMRI data by blind separation into independent spatial components.
      ). The 14 predefined brain parcellation schemes were yeo (
      • Yeo B.T.T.
      • Krienen F.M.
      • Sepulcre J.
      • Sabuncu M.R.
      • Lashkari D.
      • Hollinshead M.
      • et al.
      The organization of the human cerebral cortex estimated by intrinsic functional connectivity.
      ), smith20, smith70 (
      • Smith S.M.
      • Fox P.T.
      • Miller K.L.
      • Glahn D.C.
      • Fox P.M.
      • Mackay C.E.
      • et al.
      Correspondence of the brain’s functional architecture during activation and rest.
      ), harvard_cort_25, harvard_sub_25 (http://www.cma.mgh.harvard.edu/fsl_atlas.html), MSDL (
      • Varoquaux G.
      • Gramfort A.
      • Pedregosa F.
      • Michel V.
      • Thirion B.
      Multi-subject dictionary learning to segment an atlas of brain spontaneous activity.
      ), aal (
      • Tzourio-Mazoyer N.
      • Landeau B.
      • Papathanassiou D.
      • Crivello F.
      • Etard O.
      • Delcroix N.
      • et al.
      Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain.
      ), basc_multiscale_122, basc_multiscale_197, basc_multiscale_325, basc_multiscale_444 (
      • Bellec P.
      • Rosa-Neto P.
      • Lyttelton O.C.
      • Benali H.
      • Evans A.C.
      Multi-level bootstrap analysis of stable clusters in resting-state fMRI.
      ), destrieux (
      • Destrieux C.
      • Fischl B.
      • Dale A.M.
      • Halgren E.
      A sulcal depth-based anatomical parcellation of the cerebral cortex.
      ), dosenbach (
      • Dosenbach N.U.F.
      • Nardos B.
      • Cohen A.L.
      • Fair D.A.
      • Power J.D.
      • Church J.A.
      • et al.
      Prediction of individual brain maturity using fMRI [published correction appears in Science 2010; 330:756].
      ), and power (
      • Power J.D.
      • Cohen A.L.
      • Nelson S.M.
      • Wig G.S.
      • Barnes K.A.
      • Church J.A.
      • et al.
      Functional network organization of the human brain.
      ). EMPaSchiz first learns 84 (14 parcellation schemes × [3 + 3] feature types) single-source models (SSMs), each applying L2-regularized logistic regression to learn a classifier, for one (feature type, parcellation) description of the data. It then applies L2-regularized logistic regression to the 84 prediction probabilities produced by each of these learned models over the training set to learn a final ensemble system. At performance time, given a new instance, the learned system will first produce the 84 descriptions of that new instance, run those 84 SSMs to produce 84 responses, and feed those values into the final learned function to output the final OCD-versus-HC label.

      Neural Networks

      To examine the importance of incorporating prior neuroanatomical and neurofunctional knowledge in the form of feature design and brain parcellations, we compared the performance of EMPaSchiz with NNs, which are one of the most effective approaches today. Recent projects (

      Zafar R, Malik AS, Shuaibu AN, Javvad ur Rehman M, Dass SC (2017): Classification of fMRI data using support vector machine and convolutional neural network. Presented at the IEEE International Conference on Signal and Image Processing Applications (ICSIPA), September 12–14, 2017, Kuching, Malaysia.

      ,
      • Yan W.
      • Calhoun V.
      • Song M.
      • Cui Y.
      • Yan H.
      • Liu S.
      • et al.
      Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site FMRI data.
      ,
      • Tahmassebi A.
      • Gandomi A.H.
      • McCann I.
      • Schulte M.H.J.
      • Goudriaan A.E.
      • Meyer-Baese A.
      Deep learning in medical imaging: fMRI big data analysis via convolutional neural networks.
      ,
      • Huang H.
      • Hu X.
      • Zhao Y.
      • Makkie M.
      • Dong Q.
      • Zhao S.
      • et al.
      Modeling task fMRI data via deep convolutional autoencoder.
      ,
      • Zhao Y.
      • Dong Q.
      • Zhang S.
      • Zhang W.
      • Chen H.
      • Jiang X.
      • et al.
      Automatic recognition of fMRI-derived functional networks using 3-D convolutional neural networks.
      ,
      • Zou L.
      • Zheng J.
      • Miao C.
      • Mckeown M.J.
      • Wang Z.J.
      3D CNN based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural MRI.
      ,
      • Nie D.
      • Zhang H.
      • Adeli E.
      • Liu L.
      • Shen D.
      3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients.
      ,
      • Meszlényi R.J.
      • Buza K.
      • Vidnyánszky Z.
      Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture.
      ,
      • Yang H.
      • Zhang J.
      • Liu Q.
      • Wang Y.
      Multimodal MRI-based classification of migraine: Using deep learning convolutional neural network.
      ,
      • Vakli P.
      • Deák-Meszlényi R.J.
      • Hermann P.
      • Vidnyánszky Z.
      Transfer learning improves resting-state functional connectivity pattern analysis using convolutional neural networks.
      ,
      • Li X.
      • Hect J.
      • Thomason M.
      • Zhu D.
      Interpreting age effects of human fetal brain from spontaneous fMRI using deep 3D convolutional neural networks.
      ,
      • Gao Y.
      • Zhang Y.
      • Wang H.
      • Guo X.
      • Zhang J.
      Decoding behavior tasks from brain activity using deep transfer learning.
      ,
      • Thomas A.W.
      • Heekeren H.R.
      • Müller K.R.
      • Samek W.
      Analyzing neuroimaging data through recurrent deep learning models.
      ,
      • Thomas A.W.
      • Müller K.-R.
      • Samek W.
      Deep transfer learning for whole-brain fMRI analyses.
      ) using NN models with fMRI data for various psychiatric and neurologic tasks have used several consecutive convolutions and max-pooling layers (where the specific network architecture is not fixed but can depend on the specific task) to extract high-level features. Most of these studies (
      • Tahmassebi A.
      • Gandomi A.H.
      • McCann I.
      • Schulte M.H.J.
      • Goudriaan A.E.
      • Meyer-Baese A.
      Deep learning in medical imaging: fMRI big data analysis via convolutional neural networks.
      ,
      • Meszlényi R.J.
      • Buza K.
      • Vidnyánszky Z.
      Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture.
      ,
      • Yang H.
      • Zhang J.
      • Liu Q.
      • Wang Y.
      Multimodal MRI-based classification of migraine: Using deep learning convolutional neural network.
      ,
      • Vakli P.
      • Deák-Meszlényi R.J.
      • Hermann P.
      • Vidnyánszky Z.
      Transfer learning improves resting-state functional connectivity pattern analysis using convolutional neural networks.
      ,
      • Gao Y.
      • Zhang Y.
      • Wang H.
      • Guo X.
      • Zhang J.
      Decoding behavior tasks from brain activity using deep transfer learning.
      ,
      • Thomas A.W.
      • Heekeren H.R.
      • Müller K.R.
      • Samek W.
      Analyzing neuroimaging data through recurrent deep learning models.
      ,
      • Thomas A.W.
      • Müller K.-R.
      • Samek W.
      Deep transfer learning for whole-brain fMRI analyses.
      ,
      • Sarraf S.
      • Tofighi G.
      Classification of Alzheimer’s disease structural MRI data by deep learning convolutional neural networks.
      ) ran two-dimensional convolutions on the three-dimensional (3D) or two-dimensional data generated from the original fMRI data. Still, few studies (
      • Zhao Y.
      • Dong Q.
      • Zhang S.
      • Zhang W.
      • Chen H.
      • Jiang X.
      • et al.
      Automatic recognition of fMRI-derived functional networks using 3-D convolutional neural networks.
      ,
      • Zou L.
      • Zheng J.
      • Miao C.
      • Mckeown M.J.
      • Wang Z.J.
      3D CNN based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural MRI.
      ,
      • Nie D.
      • Zhang H.
      • Adeli E.
      • Liu L.
      • Shen D.
      3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients.
      ,
      • Li X.
      • Hect J.
      • Thomason M.
      • Zhu D.
      Interpreting age effects of human fetal brain from spontaneous fMRI using deep 3D convolutional neural networks.
      ) show that applying 3D convolution on fMRI data (here, entire brains rather than slices) is more effective than using two-dimensional convolution to produce accurate predictors. We performed this analysis with three levels, adding more domain knowledge in each step (Figure 1).
      Figure thumbnail gr1
      Figure 1The architecture of obsessive-compulsive disorder classification using neural network (NN)-1, NN-2, and NN-3. (A) NN-1: two convolution layers with four parameters depicted in order—kernel size, padding, input channel numbers, and number of filters. This is followed by a max-pooling layer with two parameters, kernel size and stride. Then, the linearized output of the max-pooling layer before the layers of long short-term memory (LSTM) has two parameters, number of layers and number of units in each layer. (B) NN-2: model parameters are depicted in a similar format as NN-1. After max pooling, there are two fully connected layers with one parameter, number of units. (C) NN-3: four fully connected layers with one parameter, number of units. For all methods, the final output layer has two nodes, obsessive-compulsive disorder vs. healthy control subjects (for each test subject, the learned system will return the argmax of the values computed here). 4D, four-dimensional; ALFF, amplitude of low-frequency fluctuations; fALFF, fractional ALFF; ReHo, regional homogeneity; Relu, rectified linear unit.

      NN-1: NN Using Preprocessed fMRI Images Without Manual Feature Design

      Here, we input four-dimensional fMRI images that are completely preprocessed and let NN-1 automatically learn relevant features from the data. The learned NN-1 model takes a fixed input size of 143 × 61 × 73 × 61 for each subject, where 143 is the time dimension and the other three dimensions represent the 3D brain volume. This model consists of two 3D convolution layers with one max-pooling layer, three layers of long short-term memory, and two fully connected layers.

      NN-2: NN Using Designed Features but Without Parcellation-Based Aggregation

      Here, we extracted the three different feature types (ALFF, fALFF, and ReHo features) using the EMPaSchiz framework but did not aggregate the features based on the brain parcellations. Instead, NN-2 uses a convolutional NN to predict the OCD label for each of the three feature types individually. The learned NN-2 model takes a fixed input size of 61 × 73 × 61. It consists of two convolution layers with one max-pooling layer and two fully connected layers.

      NN-3: NN Using Designed Features and Brain Parcellations

      Here, we used the feature extractions and parcellation schemes similar to those of EMPaSchiz. Instead of learning with the simplistic and explainable logistic regression framework, NN-3 instead uses a more complex NN method for the final step. The critical difference is that while NNs can update the weights in lower layers using backpropagation, EMPaSchiz cannot. For each subject, the learned NN-3 model takes a concatenated vector of 84 feature sets derived from the EMPaSchiz framework and consists of four fully connected layers of dimensions of sizes: 1000, 100, 40, and 2.
      Figure 1 shows the architectures of these four NN models. For each NN model, we used rectified linear unit as an activation function for each layer and cross-entropy as a loss function. To avoid overfitting, 50% of layers were dropped out during the training time. We used a maximum of 1000 epochs to train our models with early-stopping criteria for 100 epochs, i.e., we calculated the validation error after each training epoch, and if the error was found to be not decreasing for a span of 100 epochs, then the training state was reverted by 100 epochs. Models were implemented in PyTorch (v.1.0.1) (
      • Paszke A.
      • Gross S.
      • Chintala S.
      • Chanan G.
      • Yang E.
      • DeVito Z.
      • et al.
      Automatic differentiation in PyTorch.
      ) and trained on a computer with Intel Xeon(R) CPU E5-1660, 16 GB RAM, and a 12 GB NVIDIA TITAN Xp GPU. We chose a simplistic version of VGG-16 (
      • Simonyan K.
      • Zisserman A.
      Very deep convolutional networks for large-scale image recognition.
      ) because the full architecture was extremely memory constraining because it involved training on hundreds of 3D and four-dimensional tensors (

      Siu K, Stuart DM, Mahmoud M, Moshovos A (2018): Memory requirements for convolutional neural network hardware accelerators. Presented at the IEEE International Symposium on Workload Characterization (IISWC), September 30–October 2, 2018, Raleigh, North Carolina.

      ).

      Cross-Diagnosis Transfer Learning

      As mentioned above, we earlier used EMPaSchiz to distinguish patients with SCZ from HC subjects; this used a relatively small set of SSMs. Here, we ask whether these SSMs (that were sufficient for SCZ diagnosis) are adequate for diagnosing another psychiatric disorder, OCD. In particular, we limited the SSMs of EMPaSchiz while learning to predict OCD to only those selected in the learned model of SCZ prediction. That earlier model (
      • Kalmady S.V.
      • Greiner R.
      • Agrawal R.
      • Shivakumar V.
      • Narayanaswamy J.C.
      • Brown M.R.G.
      • et al.
      Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning.
      ) was based on 81 patients with SCZ and 93 HC subjects. Our OCD dataset was also collected from the same site, which initially included 175 patients with OCD and 175 HC subjects. To avoid any bias raising due to overlapping subjects in the HC group, we specifically excluded the control subjects who were included in the SCZ study, leaving 88 HC subjects. Running the EMPaSchiz learner on 81 patients with SCZ and 93 HC subjects [results for SCZ prediction model are available elsewhere (
      • Kalmady S.V.
      • Greiner R.
      • Agrawal R.
      • Shivakumar V.
      • Narayanaswamy J.C.
      • Brown M.R.G.
      • et al.
      Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning.
      )] used L1-regularization techniques to select only 10 of the 84 SSMs; we then ran that learner using only those selected 10 SSMs to learn an OCD prediction model, based on a dataset of 175 patients with OCD and 88 control subjects, disjoint from the HC subjects used for producing the SCZ model (SCZ_to_OCD transfer model). We also present the results of the transfer model that includes all HC subjects (SCZ_to_OCD_CommonHC); the sample distributions of these analyses are provided in Table S8 in Supplement 1.

      Model Evaluation

      For evaluation of learned models, we performed fivefold balanced cross-validation in five shuffled iterations (80% training set, 20% test set, for a total of 25 train-test splits). The models’ generalization performance on the held-out fold is estimated using accuracy, sensitivity, specificity, and precision. We report the mean and standard errors for these metrics and elements of confusion matrices over all 25 train-test splits for each variant.

      Results

      OCD Prediction

      The EMPaSchiz algorithm was able to predict OCD with 80.3% accuracy using the 5 times fivefold cross-validation. The model was 82.7% sensitive, 79.2% precise, and 77.8% specific. Table 1 shows the results for the performance of the EMPaSchiz algorithm and also for submodels that stack SSMs only from specific feature extractions. EMPaSchiz’s performance was significantly better than any of those subset-stacked models (compared with the best subset-stacked model, stacked-FC-precision, at 77.9%, t test, p = .018). Figure 2 shows a comparative profile of accuracies for various SSM predictors, parcellation-wise stacked models, and EMPaSchiz.
      Table 1EMPaSchiz Results for OCD Diagnosis Prediction
      ModelsAccuracy, %Precision, %Sensitivity, %Specificity, %True PositiveTrue NegativeFalse PositiveFalse Negative
      EMPaSchiz80.3 (0.7)79.2 (1.0)82.7 (0.9)77.8 (1.4)144.8 (1.5)136.2 (1.0)38.8 (1.0)30.2 (1.5)
      Stacked-ALFF70.5 (1.2)70.3 (1.2)71.2 (1.7)69.7 (1.6)124.6 (2.8)122.0 (2.8)53.0 (2.8)50.4 (2.8)
      Stacked-ReHo62.9 (1.0)63.6 (1.2)61.7 (1.3)64.0 (1.9)108.0 (1.5)112.0 (0.9)63.0 (0.9)67.0 (1.5)
      Stacked-fALFF61.4 (1.1)62.2 (1.2)59.2 (1.6)63.5 (1.8)103.6 (2.0)111.2 (2.2)63.8 (2.2)71.4 (2.0)
      Stacked-FC-Corr.73.5 (0.8)72.4 (1.0)76.6 (1.1)70.4 (1.4)134.0 (1.4)123.2 (1.3)51.8 (1.3)41.0 (1.4)
      Stacked-FC-Part.75.9 (1.0)74.4 (1.2)80.2 (1.3)71.5 (1.9)140.4 (1.6)125.2 (1.6)49.8 (1.6)34.6 (1.6)
      Stacked-FC-Prec.77.9 (1.0)75.0 (1.1)84.2 (1.5)71.5 (1.5)147.4 (1.3)125.2 (0.4)49.8 (0.4)27.6 (1.3)
      Baseline50.00
      Data are presented as mean (standard error).
      ALFF, amplitude of low-frequency fluctuations; Corr., Pearson correlation; EMPaSchiz, Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction; fALFF, fractional ALFF; FC, functional connectivity; OCD, obsessive-compulsive disorder; Part., partial correlation; Prec., precision; ReHo, regional homogeneity.
      Figure thumbnail gr2
      Figure 2Comparison of 5 × 5-fold cross-validation prediction accuracies for single-source models and EMPaSchiz (error bar corresponds to the standard error of the mean). ALFF, amplitude of low-frequency fluctuations; fALFF, fractional ALFF; FC_corr, functional connectivity with Pearson correlation; FC_part, FC with partial correlation; FC_prec, FC precision; ReHo, regional homogeneity.
      Patients with OCD in our sample ranged widely in their symptom severity, which was measured using the Yale-Brown Obsessive Compulsive Scale for obsessions (integer values from 0 to 20) and compulsions (integer values from 0 to 20). For each, we used the first and last quartile of these scales to categorize the least versus the most severely symptomatic patients. We then used EMPaSchiz in a leave-one-out cross-validation setup to predict the high-symptomatic patients against low-symptomatic ones (majority class baseline accuracy was close to 50%). We used leave-one-out cross-validation (rather than fivefold) to deal with a low number of available subjects for this analysis. Prediction accuracy was 58.6% for obsessions and 64.1% for compulsions of OCD psychopathology. In view of subpar performance, we compared this performance to the model that focused on features derived only from the brain regions that are consistently implicated in OCD by meta-analysis of functional and structural neuroimaging studies (
      • Milad M.R.
      • Rauch S.L.
      Obsessive-compulsive disorder: Beyond segregated cortico-striatal pathways.
      ,
      • de Wit S.J.
      • Alonso P.
      • Schweren L.
      • Mataix-Cols D.
      • Lochner C.
      • Menchón J.M.
      • et al.
      Multicenter voxel-based morphometry mega-analysis of structural brain scans in obsessive-compulsive disorder.
      ,
      • Radua J.
      • van den Heuvel O.A.
      • Surguladze S.
      • Mataix-Cols D.
      Meta-analytical comparison of voxel-based morphometry studies in obsessive-compulsive disorder vs other anxiety disorders.
      ), which included areas from the cortico-striato-thalamo-cortical (CSTC) circuit, namely, the orbitofrontal cortex, anterior cingulate cortex, prefrontal cortex, and ventral striatum (specific regions are listed in Table S6 and Figure S1 in Supplement 1). Prediction accuracy with CSTC was 58.6% for obsessions and 52.6% for compulsions. Details are provided in Tables S5 and S7 in Supplement 1.

      Ante Hoc Model Interpretability

      To delineate essential pathological alterations in OCD, we estimated the reliability of a feature’s importance for diagnostic prediction by sorting features by their respective mean logistic regression weight divided by the standard error for each feature in a particular learned SSM generated during 25 folds of cross-validation. Figure 3 (respectively, Figure 4) highlights some of the topmost (>98th or 99th percentile) reliable features using representative atlases for regional resting-state measures (respective connectivity). However, because our ensemble model is composed of 84 SSMs, these depictions should be considered representative in nature and cannot be claimed as the only important features for OCD prediction.
      Figure thumbnail gr3
      Figure 3Key pathological alterations in obsessive-compulsive disorder suggested by topmost reliable features—elevated (red/orange) and suppressed (blue/purple) changes in regional activity. The panels show the top 98th percentile of top regional features overlaid on glass brains. (A) Lower amplitude of low-frequency fluctuations (ALFF), fractional ALFF (fALFF) in left (L) anterior cingulate, L rectus, and right (R) paracentral lobule (destrieux). (B) Higher fALFF in bilateral inferior parietal lobule and lower fALFF in L middle frontal gyrus, R insula, and L paracentral lobule (basc_multiscale_197). (C) Higher regional homogeneity (ReHo) in L middle frontal gyrus and lower ReHo in bilateral putamen and L middle temporal gyrus (basc_multiscale_122).
      Figure thumbnail gr4
      Figure 4Key pathological alterations in obsessive-compulsive disorder suggested by topmost reliable features using the Dosenbach (top) and MSDL (bottom) atlases. Network edges show elevated (red) and suppressed (blue) changes in functional connectivity. aPFC, anterior prefrontal cortex; Ins, insula; IPS, intraparietal sulcus; L, left; R, right; TPJ, temporoparietal junction.
      Figure 4 shows representative alterations in patients with OCD compared with HC subjects that are suggested by the topmost reliable features. Network edges show elevated (red) and suppressed (blue) changes in FC. Panels show the top 99th percentile of top FC features using the Dosenbach and MSDL atlases. Within the Dosenbach atlas, we found decreased FC between the following pairs of regions: right (R) anterior prefrontal cortex and left (L) angular gyrus, R occipital gyrus and L posterior occipital gyrus, and R frontal gyrus and both R intraparietal sulcus and R superior parietal lobule. Increased FC was found between the following pairs of regions: interhemispheric occipital gyri, R dorsal anterior cingulate cortex and R parietal, and L basal ganglia and L posterior parietal cortex. These differences suggest aberrations in frontoparietal, cingulo-opercular, occipital, and sensorimotor networks. Furthermore, with the MSDL atlas, we found decreased FC between the following regions: L superior frontal sulcus and L as well as medial default mode network, and R temporoparietal junction and R parietal cortex. Increased FC was found between L inferior parietal sulcus and bilateral lateral occipital cortex as well as L visual cortex, R anterior insula and motor cortex, and L insula and L auditory cortex. These alterations might imply aberrations in several distributed networks, such as default mode, language, attention, visual, auditory, motor, and salience networks.

      Comparison of EMPaSchiz to Neural Networks Methods

      We compared the performance of the EMPaSchiz model to NN techniques that do not use their feature types and/or feature compression (parcellations) methods. Results (refer to Figure 5) show that the EMPaSchiz model outperforms these NN-1 methods (paired t test, p < .001), NN-2 methods for each feature type (ALFF: t test, p = .011, fALFF: paired t test, p < .001, ReHo: paired t test, p = .005), and NN-3 methods (paired t test, p < .001). Table 2 presents the 5 × 5-fold cross-validation prediction performance of the different NN models.
      Figure thumbnail gr5
      Figure 5Comparison of performance across EMPaSchiz and neural network (NN) methods. ALFF, amplitude of low-frequency fluctuations; fALFF, fractional ALFF; Reho, regional homogeneity.
      Table 2Comparison of Performance Across EMPaSchiz and NN Methods
      ModelsAccuracy, %Precision, %Sensitivity, %Specificity, %True PositiveTrue NegativeFalse PositiveFalse Negative
      NN-167.5 (1.2)70.2 (3.0)66.7 (1.1)70.4 (2.0)122.8 (4.1)113.4 (2.6)52.2 (4.1)61.6 (2.6)
      NN-2 ALFF76.7 (1.0)76.9 (1.3)77.6 (1.6)75.9 (1.8)135.8 (2.1)132.8 (2.8)42.2 (2.8)39.2 (2.1)
      NN-2 fALFF71.7 (0.9)73.3 (1.2)69.3 (1.3)74.2 (1.6)121.2 (2.1)129.8 (2.4)45.2 (2.4)53.8 (2.1)
      NN-2 ReHo76.3 (0.9)77.0 (1.0)76.0 (1.8)76.7 (1.6)133.0 (3.6)134.2 (1.5)40.8 (1.5)42.0 (3.6)
      NN-375.3 (1.1)75.3 (1.1)75.4 (1.8)75.1 (1.4)132.0 (2.7)131.4 (1.6)43.6 (1.6)43.0 (2.7)
      EMPaSchiz80.3 (0.7)79.2 (1.0)82.7 (0.9)77.8 (1.4)144.8 (1.5)136.2 (1.0)38.8 (1.0)30.2 (1.5)
      Baseline50.00
      Data are presented as mean (standard error).
      ALFF, amplitude of low-frequency fluctuations; EMPaSchiz, Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction; fALFF, fractional ALFF; NN, neural network; ReHo, regional homogeneity.

      Transfer Learning: Feature Selection Based on the SCZ Model

      To deal with the SCZ task, EMPaSchiz’s L1-regularization selected only 10 of 84 SSMs; our SCZ_to_OCD transfer model used only these SSMs when dealing with the OCD task. This included only two feature types, FC-Pearson correlation and FC-precision. Parcellation types included aal, dosenbach, harvard_cort_25, MSDL, power, basc_multiscale_122, basc_multiscale_197, basc_multiscale_325, and basc_multiscale_444. More information is provided in Table S9 in Supplement 1.
      When we restricted our OCD learning model to ensemble only these 10 SSMs, the accuracy of the OCD model was 93.1% (Table 3) (note, however, that we expected higher baseline performance ∼66.5% due to class imbalance: OCD = 175, HC = 88). Interestingly, we found that this SCZ_to_OCD model provided a performance that is comparable to the original EMPaSchiz that is retrained on this dataset (accuracy: 91.8%) within statistical significance margin (two-sided t test, p = .15) in OCD prediction. Table S10 in Supplement 1 provides results for the SCZ_to_OCD_CommonHC transfer model. In addition, for the sake of completion, the results for transfer learning models from OCD to SCZ are also provided (Tables S11–S13 in Supplement 1).
      Table 3EMPaSchiz Transfer Learning Results for OCD Diagnosis Prediction With Feature Selection Based on SCZ Model
      ModelsAccuracy, %Precision, %Sensitivity, %Specificity, %True PositiveTrue NegativeFalse PositiveFalse Negative
      SCZ_to_OCD93.1 (0.6)94.9 (0.6)94.7 (0.5)89.8 (1.2)165.8 (0.7)79.0 (0.8)9.0 (0.8)9.2 (0.7)
      Original EMPaSchiz91.8 (0.6)90.1 (0.7)98.7 (0.5)77.9 (1.8)172.8 (0.4)68.6 (0.8)19.4 (0.8)2.2 (0.4)
      Data are presented as mean (standard error). OCD, n = 175; HC, n = 88; baseline accuracy: 66.53.
      EMPaSchiz, Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction; HC, healthy control; OCD, obsessive-compulsive disorder; SCZ, schizophrenia.

      Discussion

      From the study observations, we conclude the following:
      • 1.
        Our EMPaSchiz algorithm can predict OCD with 80.3% accuracy, which outperforms base models that use any individual feature type or parcellation scheme.
      • 2.
        Feature design based on prior neurobiological knowledge (parcellations) leads to better performance than agnostic and automated feature design (neural nets).
      • 3.
        Selection of single-source feature sets can be transferred from SCZ to OCD prediction models without significant loss in prediction performance.
      • 4.
        EMPaSchiz provides a generalizable yet reasonably interpretable linear model that uses human expert–understandable features and model structure.
      Because many psychiatric disorders usually manifest with a myriad of overlapping symptoms, reliable clinical diagnosis is a challenging task (
      • Aboraya A.
      • Rankin E.
      • France C.
      • El-Missiry A.
      • John C.
      The reliability of psychiatric diagnosis revisited: The clinician’s guide to improve the reliability of psychiatric diagnosis.
      ). Here, we demonstrate a computational framework that builds on prior neurobiological information that is derived from numerous neuroanatomical and neurophysiological studies and provides a diagnostic performance that matches trained psychiatrists about 8 of 10 times. In this cohort, patients were seen by two experienced psychiatrists (YCJR and JCN) who assessed the patients independently and concurred on the diagnosis. Earlier studies have shown that diagnostic tools such as the DSM/ICD are not always reliable, and psychiatrists do not always agree on the diagnosis, with reported joint rater agreement as low as 0.2 to 0.4 (intraclass kappa) in some circumstances (
      • Reed G.M.
      • Sharan P.
      • Rebello T.J.
      • Keeley J.W.
      • Medina-Mora M.E.
      • Gureje O.
      • et al.
      The ICD-11 developmental field study of reliability of diagnoses of high-burden mental disorders: Results among adult patients in mental health settings of 13 countries.
      ,
      • Regier D.A.
      • Narrow W.E.
      • Clarke D.E.
      • Kraemer H.C.
      • Kuramoto S.J.
      • Kuhl E.A.
      • Kupfer D.J.
      DSM-5 field trials in the United States and Canada, Part II: Test-retest reliability of selected categorical diagnoses.
      ). Given this, one might argue that a machine with a predictive accuracy of 80% is close enough for human-like performance. Hence, the technology is worth exploring further with even bigger datasets and more sophisticated algorithms. However, this is not an apples-to-apples comparison because clinician decisions are based on a consensus about clusters of clinical symptomatology, while machine learning models are based on fMRI images, which clinicians are not able to discriminate visually. However, it is desirable to establish a clinical diagnosis on objective laboratory measurements (such as an fMRI image) because it strengthens the reproducibility of disease entities (
      • Cuthbert B.N.
      • Insel T.R.
      Toward the future of psychiatric diagnosis: The seven pillars of RDoC.
      ). Unfortunately, this is difficult here, given the earlier evidence that such brain-derived measurements only weakly correspond with clinical classifications of psychiatric conditions (
      • Hyman S.E.
      Can neuroscience be integrated into the DSM-V?.
      ).
      With a cross-validated prediction performance of 80.3%, EMPaSchiz ranks highest among all earlier OCD prediction studies, either using fMRI or other neuroimaging modalities, with a sample size greater than 100 subjects (
      • Takagi Y.
      • Sakai Y.
      • Lisi G.
      • Yahata N.
      • Abe Y.
      • Nishida S.
      • et al.
      A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
      ,
      • Yang X.
      • Hu X.
      • Tang W.
      • Li B.
      • Yang Y.
      • Gong Q.
      • Huang X.
      Multivariate classification of drug-naive obsessive-compulsive disorder patients and healthy controls by applying an SVM to resting-state functional MRI data.
      ,
      • Hu X.
      • Zhang L.
      • Bu X.
      • Li H.
      • Li B.
      • Tang W.
      • et al.
      Localized connectivity in obsessive-compulsive disorder: An investigation combining univariate and multivariate pattern analyses.
      ,
      • Soriano-Mas C.
      • Pujol J.
      • Alonso P.
      • Cardoner N.
      • Menchón J.M.
      • Harrison B.J.
      • et al.
      Identifying patients with obsessive–compulsive disorder using whole-brain anatomy.
      ,
      • Parrado-Hernández E.
      • Gómez-Verdejo V.
      • Martínez-Ramón M.
      • Shawe-Taylor J.
      • Alonso P.
      • Pujol J.
      • et al.
      Discovering brain regions relevant to obsessive–compulsive disorder identification through bagging and transduction.
      ). The earlier fMRI-based OCD prediction studies have neither combined regional and connectivity features nor used an ensemble approach such as ours to learn from multiple parcellations jointly. Most used a single feature type [such as ALFF, fALFF, ReHo, or FC-Pearson correlation (

      Shenas SK, Halici U, Cicek M (2013): Detection of obsessive compulsive disorder using resting-state functional connectivity data. Presented at the 6th International Conference on Biomedical Engineering and Informatics, December 16–18, 2013, Hangzhou, China.

      ,
      • Gruner P.
      • Vo A.
      • Argyelan M.
      • Ikuta T.
      • Degnan A.J.
      • John M.
      • et al.
      Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder.
      ,
      • Takagi Y.
      • Sakai Y.
      • Lisi G.
      • Yahata N.
      • Abe Y.
      • Nishida S.
      • et al.
      A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
      ,
      • Yang X.
      • Hu X.
      • Tang W.
      • Li B.
      • Yang Y.
      • Gong Q.
      • Huang X.
      Multivariate classification of drug-naive obsessive-compulsive disorder patients and healthy controls by applying an SVM to resting-state functional MRI data.
      ,
      • Hu X.
      • Zhang L.
      • Bu X.
      • Li H.
      • Li B.
      • Tang W.
      • et al.
      Localized connectivity in obsessive-compulsive disorder: An investigation combining univariate and multivariate pattern analyses.
      ,
      • Bu X.
      • Hu X.
      • Zhang L.
      • Li B.
      • Zhou M.
      • Lu L.
      • et al.
      Investigating the predictive value of different resting-state functional MRI parameters in obsessive-compulsive disorder.
      )] and one of the predefined atlases (e.g., BrainVISA Sulci Atlas, Anatomical Automatic labeling atlas, Harvard-Oxford atlas) (
      • Sen B.
      • Bernstein G.A.
      • Xu T.
      • Mueller B.A.
      • Schreiner M.W.
      • Cullen K.R.
      • Parhi K.K.
      Classification of obsessive-compulsive disorder from resting-state fMRI.
      ,
      • Takagi Y.
      • Sakai Y.
      • Lisi G.
      • Yahata N.
      • Abe Y.
      • Nishida S.
      • et al.
      A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
      ). A few other studies extracted the functional networks using group independent component analysis (
      • Gruner P.
      • Vo A.
      • Argyelan M.
      • Ikuta T.
      • Degnan A.J.
      • John M.
      • et al.
      Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder.
      ), graph-based methods (
      • Sen B.
      • Bernstein G.A.
      • Xu T.
      • Mueller B.A.
      • Schreiner M.W.
      • Cullen K.R.
      • Parhi K.K.
      Classification of obsessive-compulsive disorder from resting-state fMRI.
      ), and similarity measures such as dot product and cosine (

      Shenas SK, Halici U, Cicek M (2013): Detection of obsessive compulsive disorder using resting-state functional connectivity data. Presented at the 6th International Conference on Biomedical Engineering and Informatics, December 16–18, 2013, Hangzhou, China.

      ). Others (
      • Soriano-Mas C.
      • Pujol J.
      • Alonso P.
      • Cardoner N.
      • Menchón J.M.
      • Harrison B.J.
      • et al.
      Identifying patients with obsessive–compulsive disorder using whole-brain anatomy.
      ,
      • Parrado-Hernández E.
      • Gómez-Verdejo V.
      • Martínez-Ramón M.
      • Shawe-Taylor J.
      • Alonso P.
      • Pujol J.
      • et al.
      Discovering brain regions relevant to obsessive–compulsive disorder identification through bagging and transduction.
      ,
      • Li F.
      • Huang X.
      • Tang W.
      • Yang Y.
      • Li B.
      • Kemp G.J.
      • et al.
      Multivariate pattern analysis of DTI reveals differential white matter in individuals with obsessive-compulsive disorder.
      ,
      • Hu X.
      • Liu Q.
      • Li B.
      • Tang W.
      • Sun H.
      • Li F.
      • et al.
      Multivariate pattern analysis of obsessive–compulsive disorder using structural neuroanatomy.
      ,
      • Trambaiolli L.R.
      • Biazoli Jr., C.E.
      • Balardin J.B.
      • Hoexter M.Q.
      • Sato J.R.
      The relevance of feature selection methods to the classification of obsessive-compulsive disorder based on volumetric measures.
      ,
      • Zhou C.
      • Cheng Y.
      • Ping L.
      • Xu J.
      • Shen Z.
      • Jiang L.
      • et al.
      Support vector machine classification of obsessive-compulsive disorder based on whole-brain volumetry and diffusion tensor imaging.
      ) used structural MRI or diffusion tensor imaging to focus on whole-brain anatomical alterations and white matter abnormalities, respectively, with gray matter volume, white matter volume, fractional anisotropy, and medial diffusivity as their key features. Commonly used dimensionality reduction methods were PCA, Kernel PCA and MRMR [Minimum Redundancy Maximum Relevance (
      • Peng H.
      • Long F.
      • Ding C.
      Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy.
      )], while classification algorithms were support vector machine, logistic regression, and linear discriminant analysis (see Table S2 for details specific to each study).
      While high performance on diagnostic prediction is encouraging, note that discriminating patients with OCD from healthy individuals is perhaps an easy task for trained psychiatrists, especially with identifying subtypes based on symptom dimensions or severity. The patients in this study did not have comorbidities with psychosis, bipolar disorder, and other obsessive-compulsive–related disorders such as body dysmorphic disorder, hair pulling disorder, or excoriation disorder. Only a small proportion of the sample had depression and anxiety comorbidity (current depression, n = 21 [13.8%]; current anxiety, n = 8 [5.3%]). It may be noted that the proportion of patients with OCD with comorbidities is small, and hence it may not have affected the specificity of the findings. However, our experiments with learning models for identifying severe cases were only mildly successful. While it is expected that focusing the model on CSTC brain regions [which are most commonly implicated in OCD literature (
      • Milad M.R.
      • Rauch S.L.
      Obsessive-compulsive disorder: Beyond segregated cortico-striatal pathways.
      ,
      • de Wit S.J.
      • Alonso P.
      • Schweren L.
      • Mataix-Cols D.
      • Lochner C.
      • Menchón J.M.
      • et al.
      Multicenter voxel-based morphometry mega-analysis of structural brain scans in obsessive-compulsive disorder.
      ,
      • Radua J.
      • van den Heuvel O.A.
      • Surguladze S.
      • Mataix-Cols D.
      Meta-analytical comparison of voxel-based morphometry studies in obsessive-compulsive disorder vs other anxiety disorders.
      )] would potentially increase the prediction of symptom severity, our experiments with CSTC regions showed a performance that is inferior to EMPaSchiz’s. We discuss the possible reasons in Section G in Supplement 1. It is possible that the OCD phenotype that the algorithm learned to differentiate from the healthy brain may be independent of OCD severity. The phenotype identified may represent traits and may not be state dependent. Alternatively, these results could also suggest that fMRI data might not have enough information to perform more complicated psychiatric decisions, at least within the scope of our training size and algorithms.
      To ease the bedside translation of these computational models, it is desirable that they demonstrate trustworthiness. Clinicians are more likely to use models that use features that they know, based on familiar anatomical brain regions and well-known properties, such as FC and regional homogeneity. They also prefer models that use easy-to-interpret algorithms, e.g., a linear combination of features. EMPaSchiz’s basic algorithm is a priori explainable because it uses logistic regression classifiers that are inherently interpretable because they involve linear combinations of features (

      Poulin B, Eisner R, Szafron D, Lu P, Greiner R, Wishart DS, et al. (2006): Visual explanation of evidence with additive classifiers. Presented at the 21st National Conference on Artificial Intelligence, July 16–20, 2006, Boston, Massachusetts.

      ). Furthermore, the model’s decision can be viewed as probabilistic. The learned weights suggest how much each feature contributes toward its final decision, thus helping clinicians and researchers understand the factors leading to the decisions.
      Our study faces the general limitation of most machine learning studies in psychiatry: the ground truth itself might be ill defined in terms of validity of current psychiatric classifications as unitary disease constructs. While our approach does correctly distinguish patients with OCD from HC subjects, we do not know whether it can differentially distinguish OCD from other psychiatric disorders. Results from this study do not actually prove the specificity of diagnostic prediction, which would be very useful for clinicians in aiding their diagnosis; this could be a potential future direction. In addition, our data were collected using a single MRI scanner and from individuals of fairly homogeneous ethnicity. We do not know whether it will perform similarly in multicentric data collected across mixed population groups and comorbidities. Moreover, we acknowledge that our processing pipeline is only one of the many possible sensible pipelines (
      • Parkes L.
      • Fulcher B.
      • Yücel M.
      • Fornito A.
      An evaluation of the efficacy, reliability, and sensitivity of motion correction strategies for resting-state functional MRI.
      ,
      • Kassinopoulos M.
      • Mitsis G.D.
      A multi-measure approach for assessing the performance of fMRI preprocessing strategies in resting-state functional connectivity.
      ). Similar to any other machine learning study, the reported claims about the performance of our model only apply to fMRI images that are processed as described. Nonetheless, the potential effects of different choices of denoising pipelines, such as global signal regression, filtering, and scrubbing, on the prediction performance need systematic evaluation (
      • Ciric R.
      • Rosen A.F.G.
      • Erus G.
      • Cieslak M.
      • Adebimpe A.
      • Cook P.A.
      • et al.
      Mitigating head motion artifact in functional connectivity MRI.
      ). Given the wide parameter space of these processing options, we believe that such a systematic analysis merits a future separate research study.
      Machine learned models produce more accurate predictions as the size of the training dataset increases; therefore, it is critical to have a sufficient sample size (
      • Marinai S.
      • Fujisawa H.
      Machine Learning in Document Analysis and Recognition.
      ). Our study used 350 subjects (175 patients with OCD and 175 HC subjects), which is the largest OCD dataset that has been used in machine learning until now [the previously largest sample size is 204 (102 patients with OCD and 100 HC subjects), which reported a prediction performance of 76.6% (
      • Soriano-Mas C.
      • Pujol J.
      • Alonso P.
      • Cardoner N.
      • Menchón J.M.
      • Harrison B.J.
      • et al.
      Identifying patients with obsessive–compulsive disorder using whole-brain anatomy.
      )]. Our study provides two other novel observations. First, it shows that a simple linear model with neurobiology-informed features outperforms complex NN models, even though those models can automatically design new features that can potentially exploit nonlinear interactions. Even though EMPaSchiz was trained on the same ∼252,000-dimensional data as NN-3, we anticipated that EMPaSchiz would work better for two main reasons: 1) EMPaSchiz exploits prior neurologic information because it pools those extracted feature values into brain regions or networks with coherent structure or function based on known parcellation schemes, while NN-3 (which considers all possible combinations of features) does not use this information, and 2) because NN-3 is fully connected, it must fit roughly 1000 times more weights than EMPaSchiz (∼253 million vs. ∼253,000), which for this dataset appears to lead to overfitting. Hence, the EMPaSchiz model is both more accurate and more interpretable. Second, we demonstrate cross-diagnostic transfer learning in psychiatry applications. We show that this can considerably reduce the complexity of data processing in terms of feature engineering by pretraining the model with subjects affected by a different psychiatric condition—SCZ in our case—and still retain the earlier performance accuracy. Our results suggest that this technique can be used to tackle the issue of limited sample sizes in fMRI-based machine learning research. We hypothesize that such transfer learning works in this context because the underlying brain abnormalities are shared across multiple psychiatric disorders (
      • Goodkind M.
      • Eickhoff S.B.
      • Oathes D.J.
      • Jiang Y.
      • Chang A.
      • Jones-Hagata L.B.
      • et al.
      Identification of a common neurobiological substrate for mental illness.
      ). Future studies could attempt to replicate these findings between other disease groups, e.g., between OCD and other compulsive disorders such as alcohol addiction or eating disorders, as well as SCZ and bipolar disorder.

      Acknowledgments and Disclosures

      This study is supported by the IBM Alberta Centre for Advanced Studies (to SVK), Alberta Machine Intelligence Institute and Alberta Innovates Graduate Student Scholarship grants (to AKP), the Government of India (Grant Nos. DST SR/S0/HS/0016/2011 and BT/PR13334/Med/30/259/2009 [to YCJR], DST INSPIRE faculty Grant No. IFA12-LSBM-26 [to JCN], Grant No. DBT BT/06/IYBA/2012 [to JCN], and Grant No. 500236/Z/11/Z [to GV]), Alberta Machine Intelligence Institute and NSERC grants (to RG), and DBT Wellcome Trust India Alliance (Grant No. IA/CRC/19/1/610005 [to GV], Intermediate Fellowship Grant No. IA/CPHI/16/1/502662 [to JCN], and Early Career Fellowship Grant No. IA/CPHE/18/1/503956 [to VS]).
      GV, JCN, YCJR, and VS collected the clinical and neuroimaging data. Clinical symptom ratings were done by YCJR and JCN. Data were cleaned and processed by RA, SVK, and AKP. SVK and AKP designed, and AKP implemented, the machine learning models, with supervision by SVK and RG. SVK and AKP managed the literature search, prepared figures, and wrote the first draft of the manuscript along with RG. AJG, SMD, and other authors revised and optimized further versions of the manuscript. All authors have contributed to and approved the final manuscript.
      The datasets generated and/or analyzed during this study, as well as relevant computer codes that were used to process the data and to generate the results, are available from corresponding authors on a reasonable request.
      The authors report no biomedical financial interests or potential conflicts of interest.

      Supplementary Material

      References

        • Ruscio A.M.
        • Stein D.J.
        • Chiu W.T.
        • Kessler R.C.
        The epidemiology of obsessive-compulsive disorder in the National Comorbidity Survey Replication.
        Mol Psychiatry. 2010; 15: 53-63
        • Janardhan Reddy Y.C.
        • Sundar A.S.
        • Narayanaswamy J.C.
        • Math S.B.
        Clinical practice guidelines for obsessive-compulsive disorder.
        Indian J Psychiatry. 2017; 59: S74-S90
        • do Rosário M.C.
        • Batistutto M.
        • Ferrao Y.
        Symptom heterogeneity in OCD: A dimensional approach.
        in: Pittenger C. Obsessive-compulsive Disorder: Phenomenology, Pathophysiology, and Treatment. Oxford University Press, Oxford2017: 75-92
        • Viswanath B.
        • Narayanaswamy J.C.
        • Rajkumar R.P.
        • Cherian A.V.
        • Kandavel T.
        • Math S.B.
        • Reddy Y.C.J.
        Impact of depressive and anxiety disorder comorbidity on the clinical expression of obsessive-compulsive disorder.
        Compr Psychiatry. 2012; 53: 775-782
        • Amerio A.
        • Odone A.
        • Liapis C.C.
        • Ghaemi S.N.
        Diagnostic validity of comorbid bipolar disorder and obsessive-compulsive disorder: A systematic review.
        Acta Psychiatr Scand. 2014; 129: 343-358
        • Schirmbeck F.
        • Zink M.
        Comorbid obsessive-compulsive symptoms in schizophrenia: Contributions of pharmacological and genetic factors.
        Front Pharmacol. 2013; 4: 99
        • Rutledge R.B.
        • Chekroud A.M.
        • Huys Q.J.
        Machine learning and big data in psychiatry: Toward clinical applications.
        Curr Opin Neurobiol. 2019; 55: 152-159
        • Hyde K.K.
        • Novack M.N.
        • LaHaye N.
        • Parlett-Pelleriti C.
        • Anden R.
        • Dixon D.R.
        • Linstead E.
        Applications of supervised machine learning in autism spectrum disorder research: A review.
        Rev J Autism Dev Disord. 2019; 6: 128-146
        • de Filippis R.
        • Carbone E.A.
        • Gaetano R.
        • Bruni A.
        • Pugliese V.
        • Segura-Garcia C.
        • De Fazio P.
        Machine learning techniques in a structural and functional MRI diagnostic approach in schizophrenia: A systematic review.
        Neuropsychiatr Dis Treat. 2019; 15: 1605-1627
        • Gao S.
        • Calhoun V.D.
        • Sui J.
        Machine learning in major depression: From classification to treatment outcome prediction.
        CNS Neurosci Ther. 2018; 24: 1037-1052
      1. Shenas SK, Halici U, Cicek M (2013): Detection of obsessive compulsive disorder using resting-state functional connectivity data. Presented at the 6th International Conference on Biomedical Engineering and Informatics, December 16–18, 2013, Hangzhou, China.

        • Gruner P.
        • Vo A.
        • Argyelan M.
        • Ikuta T.
        • Degnan A.J.
        • John M.
        • et al.
        Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder.
        Hum Brain Mapp. 2014; 35: 5306-5315
        • Sen B.
        • Bernstein G.A.
        • Xu T.
        • Mueller B.A.
        • Schreiner M.W.
        • Cullen K.R.
        • Parhi K.K.
        Classification of obsessive-compulsive disorder from resting-state fMRI.
        Annu Int Conf IEEE Eng Med Biol Soc. 2016; 2016: 3606-3609
        • Takagi Y.
        • Sakai Y.
        • Lisi G.
        • Yahata N.
        • Abe Y.
        • Nishida S.
        • et al.
        A neural marker of obsessive-compulsive disorder from whole-brain functional connectivity.
        Sci Rep. 2017; 7: 7538
        • Yang X.
        • Hu X.
        • Tang W.
        • Li B.
        • Yang Y.
        • Gong Q.
        • Huang X.
        Multivariate classification of drug-naive obsessive-compulsive disorder patients and healthy controls by applying an SVM to resting-state functional MRI data.
        BMC Psychiatry. 2019; 19: 210
        • Hu X.
        • Zhang L.
        • Bu X.
        • Li H.
        • Li B.
        • Tang W.
        • et al.
        Localized connectivity in obsessive-compulsive disorder: An investigation combining univariate and multivariate pattern analyses.
        Front Behav Neurosci. 2019; 13: 122
        • Bu X.
        • Hu X.
        • Zhang L.
        • Li B.
        • Zhou M.
        • Lu L.
        • et al.
        Investigating the predictive value of different resting-state functional MRI parameters in obsessive-compulsive disorder.
        Transl Psychiatry. 2019; 9: 17
        • Schnack H.G.
        • Kahn R.S.
        Detecting neuroimaging biomarkers for psychiatric disorders: Sample size matters.
        Front Psychiatry. 2016; 7: 50
        • Kalmady S.V.
        • Greiner R.
        • Agrawal R.
        • Shivakumar V.
        • Narayanaswamy J.C.
        • Brown M.R.G.
        • et al.
        Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning.
        NPJ Schizophr. 2019; 5: 2
        • Shahid N.
        • Rappon T.
        • Berta W.
        Applications of artificial neural networks in health care organizational decision-making: A scoping review.
        PLoS One. 2019; 14e0212356
        • Chen D.
        • Liu S.
        • Kingsbury P.
        • Sohn S.
        • Storlie C.B.
        • Habermann E.B.
        • et al.
        Deep learning and alternative learning strategies for retrospective real-world clinical data.
        NPJ Digit Med. 2019; 2: 43
        • American Psychiatric Association
        Diagnostic and Statistical Manual of Mental Disorders (DSM-5®).
        American Psychiatric Publishing, Washington, DC2013
        • Nelson B.
        • McGorry P.D.
        • Wichers M.
        • Wigman J.T.W.
        • Hartmann J.A.
        Moving from static to dynamic models of the onset of mental disorder: A review.
        JAMA Psychiatry. 2017; 74: 528-534
        • Hyman S.E.
        Can neuroscience be integrated into the DSM-V?.
        Nat Rev Neurosci. 2007; 8: 725-732
        • Cuthbert B.N.
        • Insel T.R.
        Toward the future of psychiatric diagnosis: The seven pillars of RDoC.
        BMC Med. 2013; 11: 126
        • Goodkind M.
        • Eickhoff S.B.
        • Oathes D.J.
        • Jiang Y.
        • Chang A.
        • Jones-Hagata L.B.
        • et al.
        Identification of a common neurobiological substrate for mental illness.
        JAMA Psychiatry. 2015; 72: 305-315
        • Caruana R.
        Multitask learning.
        in: Thrun S. Pratt L. Learning to Learn. Springer, Boston, MA1998: 95-133
        • Zhang Y.
        • Liao J.
        • Li Q.
        • Zhang X.
        • Liu L.
        • Yan J.
        • et al.
        Altered resting-state brain activity in schizophrenia and obsessive-compulsive disorder compared with non-psychiatric controls: Commonalities and distinctions across disorders.
        Front Psychiatry. 2021; 12: 681701
        • Ongür D.
        • Goff D.C.
        Obsessive-compulsive symptoms in schizophrenia: Associated clinical features, cognitive function and medication status.
        Schizophr Res. 2005; 75: 349-362
        • Holzinger A.
        • Langs G.
        • Denk H.
        • Zatloukal K.
        • Müller H.
        Causability and explainability of artificial intelligence in medicine.
        Wiley Interdiscip Rev Data Min Knowl Discov. 2019; 9: e1312
        • Sheehan D.V.
        • Lecrubier Y.
        • Sheehan K.H.
        • Amorim P.
        • Janavs J.
        • Weiller E.
        • et al.
        The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.
        J Clin Psychiatry. 1998; 59 (quiz 34–57): 22-33
        • Goodman W.K.
        • Price L.H.
        • Rasmussen S.A.
        • Mazure C.
        • Fleischmann R.L.
        • Hill C.L.
        • et al.
        The Yale-Brown Obsessive Compulsive Scale. I. Development, use, and reliability.
        Arch Gen Psychiatry. 1989; 46: 1006-1011
        • Guy W.
        ECDEU Assessment Manual for Psychopharmacology.
        US Department of Health, Education, and Welfare, Rockville, MD1976
        • Chang X.
        • Xi Y.B.
        • Cui L.B.
        • Wang H.N.
        • Sun J.B.
        • Zhu Y.Q.
        • et al.
        Distinct inter-hemispheric dysconnectivity in schizophrenia patients with and without auditory verbal hallucinations.
        Sci Rep. 2015; 5: 11218
        • Zang Y.F.
        • He Y.
        • Zhu C.Z.
        • Cao Q.J.
        • Sui M.Q.
        • Liang M.
        • et al.
        Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI.
        Brain Dev. 2007; 29: 83-91
        • Zou Q.H.
        • Zhu C.Z.
        • Yang Y.
        • Zuo X.N.
        • Long X.Y.
        • Cao Q.J.
        • et al.
        An improved approach to detection of amplitude of low-frequency fluctuation (ALFF) for resting-state fMRI: Fractional ALFF.
        J Neurosci Methods. 2008; 172: 137-141
        • Kendall M.G.
        Rank correlation methods.
        Biometrika. 1957; 44: 298
        • Zang Y.
        • Jiang T.
        • Lu Y.
        • He Y.
        • Tian L.
        Regional homogeneity approach to fMRI data analysis.
        Neuroimage. 2004; 22: 394-400
        • Tzourio-Mazoyer N.
        • Landeau B.
        • Papathanassiou D.
        • Crivello F.
        • Etard O.
        • Delcroix N.
        • et al.
        Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain.
        Neuroimage. 2002; 15: 273-289
        • Zilles K.
        • Amunts K.
        Receptor mapping: Architecture of the human cerebral cortex.
        Curr Opin Neurol. 2009; 22: 331-339
        • Destrieux C.
        • Fischl B.
        • Dale A.M.
        • Halgren E.
        A sulcal depth-based anatomical parcellation of the cerebral cortex.
        Neuroimage. 2009; 47: S151
        • Talairach J.
        • Tournoux P.
        Co-Planar Stereotaxic Atlas of the Human Brain: 3-D Proportional System: An Approach to Cerebral Imaging.
        Thieme, New York1988
        • Yeo B.T.T.
        • Krienen F.M.
        • Sepulcre J.
        • Sabuncu M.R.
        • Lashkari D.
        • Hollinshead M.
        • et al.
        The organization of the human cerebral cortex estimated by intrinsic functional connectivity.
        J Neurophysiol. 2011; 106: 1125-1165
        • Varoquaux G.
        • Gramfort A.
        • Pedregosa F.
        • Michel V.
        • Thirion B.
        Multi-subject dictionary learning to segment an atlas of brain spontaneous activity.
        in: Székely G. Hahn H.K. Information Processing in Medical Imaging: 22nd International Conference, IPMI 2011, Kloster Irsee, Germany, July 3-8, 2011. Proceedings. Springer, Berlin, Germany2011: 562-573
        • Power J.D.
        • Cohen A.L.
        • Nelson S.M.
        • Wig G.S.
        • Barnes K.A.
        • Church J.A.
        • et al.
        Functional network organization of the human brain.
        Neuron. 2011; 72: 665-678
        • Smith S.M.
        • Fox P.T.
        • Miller K.L.
        • Glahn D.C.
        • Fox P.M.
        • Mackay C.E.
        • et al.
        Correspondence of the brain’s functional architecture during activation and rest.
        Proc Natl Acad Sci U S A. 2009; 106: 13040-13045
        • Dosenbach N.U.F.
        • Nardos B.
        • Cohen A.L.
        • Fair D.A.
        • Power J.D.
        • Church J.A.
        • et al.
        Prediction of individual brain maturity using fMRI [published correction appears in Science 2010; 330:756].
        Science. 2010; 329: 1358-1361
        • Bellec P.
        • Rosa-Neto P.
        • Lyttelton O.C.
        • Benali H.
        • Evans A.C.
        Multi-level bootstrap analysis of stable clusters in resting-state fMRI.
        Neuroimage. 2010; 51: 1126-1139
        • McKeown M.J.
        • Makeig S.
        • Brown G.G.
        • Jung T.P.
        • Kindermann S.S.
        • Bell A.J.
        • Sejnowski T.J.
        Analysis of fMRI data by blind separation into independent spatial components.
        Hum Brain Mapp. 1998; 6: 160-188
      2. Zafar R, Malik AS, Shuaibu AN, Javvad ur Rehman M, Dass SC (2017): Classification of fMRI data using support vector machine and convolutional neural network. Presented at the IEEE International Conference on Signal and Image Processing Applications (ICSIPA), September 12–14, 2017, Kuching, Malaysia.

        • Yan W.
        • Calhoun V.
        • Song M.
        • Cui Y.
        • Yan H.
        • Liu S.
        • et al.
        Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site FMRI data.
        EBioMedicine. 2019; 47: 543-552
        • Tahmassebi A.
        • Gandomi A.H.
        • McCann I.
        • Schulte M.H.J.
        • Goudriaan A.E.
        • Meyer-Baese A.
        Deep learning in medical imaging: fMRI big data analysis via convolutional neural networks.
        PEARC ’18: Proceedings of the Practice and Experience on Advanced Research Computing. 2018; : 1-4
        • Huang H.
        • Hu X.
        • Zhao Y.
        • Makkie M.
        • Dong Q.
        • Zhao S.
        • et al.
        Modeling task fMRI data via deep convolutional autoencoder.
        IEEE Trans Med Imaging. 2018; 37: 1551-1561
        • Zhao Y.
        • Dong Q.
        • Zhang S.
        • Zhang W.
        • Chen H.
        • Jiang X.
        • et al.
        Automatic recognition of fMRI-derived functional networks using 3-D convolutional neural networks.
        IEEE Trans Biomed Eng. 2018; 65: 1975-1984
        • Zou L.
        • Zheng J.
        • Miao C.
        • Mckeown M.J.
        • Wang Z.J.
        3D CNN based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural MRI.
        IEEE Access. 2017; 5: 23626-23636
        • Nie D.
        • Zhang H.
        • Adeli E.
        • Liu L.
        • Shen D.
        3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients.
        Med Image Comput Assist Interv. 2016; 9901: 212-220
        • Meszlényi R.J.
        • Buza K.
        • Vidnyánszky Z.
        Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture.
        Front Neuroinform. 2017; 11: 61
        • Yang H.
        • Zhang J.
        • Liu Q.
        • Wang Y.
        Multimodal MRI-based classification of migraine: Using deep learning convolutional neural network.
        Biomed Eng Online. 2018; 17: 138
        • Vakli P.
        • Deák-Meszlényi R.J.
        • Hermann P.
        • Vidnyánszky Z.
        Transfer learning improves resting-state functional connectivity pattern analysis using convolutional neural networks.
        Gigascience. 2018; 7: giy130
        • Li X.
        • Hect J.
        • Thomason M.
        • Zhu D.
        Interpreting age effects of human fetal brain from spontaneous fMRI using deep 3D convolutional neural networks.
        arXiv. 2019; (doi: 1906.03691)
        • Gao Y.
        • Zhang Y.
        • Wang H.
        • Guo X.
        • Zhang J.
        Decoding behavior tasks from brain activity using deep transfer learning.
        IEEE Access. 2019; 7: 43222-43232
        • Thomas A.W.
        • Heekeren H.R.
        • Müller K.R.
        • Samek W.
        Analyzing neuroimaging data through recurrent deep learning models.
        Front Neurosci. 2019; 13: 1321
        • Thomas A.W.
        • Müller K.-R.
        • Samek W.
        Deep transfer learning for whole-brain fMRI analyses.
        arXiv. 2019; (doi: 1907.01953)
        • Sarraf S.
        • Tofighi G.
        Classification of Alzheimer’s disease structural MRI data by deep learning convolutional neural networks.
        arXiv. 2016; (doi: 1607.06583)
        • Paszke A.
        • Gross S.
        • Chintala S.
        • Chanan G.
        • Yang E.
        • DeVito Z.
        • et al.
        Automatic differentiation in PyTorch.
        (Available at:)
        https://openreview.net/pdf?id=BJJsrmfCZ
        Date: 2017
        Date accessed: November 13, 2019
        • Simonyan K.
        • Zisserman A.
        Very deep convolutional networks for large-scale image recognition.
        arXiv. 2014; (https://doi.org/1409.1556)
      3. Siu K, Stuart DM, Mahmoud M, Moshovos A (2018): Memory requirements for convolutional neural network hardware accelerators. Presented at the IEEE International Symposium on Workload Characterization (IISWC), September 30–October 2, 2018, Raleigh, North Carolina.

        • Milad M.R.
        • Rauch S.L.
        Obsessive-compulsive disorder: Beyond segregated cortico-striatal pathways.
        Trends Cogn Sci. 2012; 16: 43-51
        • de Wit S.J.
        • Alonso P.
        • Schweren L.
        • Mataix-Cols D.
        • Lochner C.
        • Menchón J.M.
        • et al.
        Multicenter voxel-based morphometry mega-analysis of structural brain scans in obsessive-compulsive disorder.
        Am J Psychiatry. 2014; 171: 340-349
        • Radua J.
        • van den Heuvel O.A.
        • Surguladze S.
        • Mataix-Cols D.
        Meta-analytical comparison of voxel-based morphometry studies in obsessive-compulsive disorder vs other anxiety disorders.
        Arch Gen Psychiatry. 2010; 67: 701-711
        • Aboraya A.
        • Rankin E.
        • France C.
        • El-Missiry A.
        • John C.
        The reliability of psychiatric diagnosis revisited: The clinician’s guide to improve the reliability of psychiatric diagnosis.
        Psychiatry (Edgmont). 2006; 3: 41-50
        • Reed G.M.
        • Sharan P.
        • Rebello T.J.
        • Keeley J.W.
        • Medina-Mora M.E.
        • Gureje O.
        • et al.
        The ICD-11 developmental field study of reliability of diagnoses of high-burden mental disorders: Results among adult patients in mental health settings of 13 countries.
        World Psychiatry. 2018; 17: 174-186
        • Regier D.A.
        • Narrow W.E.
        • Clarke D.E.
        • Kraemer H.C.
        • Kuramoto S.J.
        • Kuhl E.A.
        • Kupfer D.J.
        DSM-5 field trials in the United States and Canada, Part II: Test-retest reliability of selected categorical diagnoses.
        Am J Psychiatry. 2013; 170: 59-70
        • Soriano-Mas C.
        • Pujol J.
        • Alonso P.
        • Cardoner N.
        • Menchón J.M.
        • Harrison B.J.
        • et al.
        Identifying patients with obsessive–compulsive disorder using whole-brain anatomy.
        Neuroimage. 2007; 35: 1028-1037
        • Parrado-Hernández E.
        • Gómez-Verdejo V.
        • Martínez-Ramón M.
        • Shawe-Taylor J.
        • Alonso P.
        • Pujol J.
        • et al.
        Discovering brain regions relevant to obsessive–compulsive disorder identification through bagging and transduction.
        Med Image Anal. 2014; 18: 435-448
        • Li F.
        • Huang X.
        • Tang W.
        • Yang Y.
        • Li B.
        • Kemp G.J.
        • et al.
        Multivariate pattern analysis of DTI reveals differential white matter in individuals with obsessive-compulsive disorder.
        Hum Brain Mapp. 2014; 35: 2643-2651
        • Hu X.
        • Liu Q.
        • Li B.
        • Tang W.
        • Sun H.
        • Li F.
        • et al.
        Multivariate pattern analysis of obsessive–compulsive disorder using structural neuroanatomy.
        Eur Neuropsychopharmacol. 2016; 26: 246-254
        • Trambaiolli L.R.
        • Biazoli Jr., C.E.
        • Balardin J.B.
        • Hoexter M.Q.
        • Sato J.R.
        The relevance of feature selection methods to the classification of obsessive-compulsive disorder based on volumetric measures.
        J Affect Disord. 2017; 222: 49-56
        • Zhou C.
        • Cheng Y.
        • Ping L.
        • Xu J.
        • Shen Z.
        • Jiang L.
        • et al.
        Support vector machine classification of obsessive-compulsive disorder based on whole-brain volumetry and diffusion tensor imaging.
        Front Psychiatry. 2018; 9: 524
        • Peng H.
        • Long F.
        • Ding C.
        Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy.
        IEEE Trans Pattern Anal Mach Intell. 2005; 27: 1226-1238
      4. Poulin B, Eisner R, Szafron D, Lu P, Greiner R, Wishart DS, et al. (2006): Visual explanation of evidence with additive classifiers. Presented at the 21st National Conference on Artificial Intelligence, July 16–20, 2006, Boston, Massachusetts.

        • Parkes L.
        • Fulcher B.
        • Yücel M.
        • Fornito A.
        An evaluation of the efficacy, reliability, and sensitivity of motion correction strategies for resting-state functional MRI.
        Neuroimage. 2018; 171: 415-436
        • Kassinopoulos M.
        • Mitsis G.D.
        A multi-measure approach for assessing the performance of fMRI preprocessing strategies in resting-state functional connectivity.
        Magn Reson Imaging. 2022; 85: 228-250
        • Ciric R.
        • Rosen A.F.G.
        • Erus G.
        • Cieslak M.
        • Adebimpe A.
        • Cook P.A.
        • et al.
        Mitigating head motion artifact in functional connectivity MRI.
        Nat Protoc. 2018; 13: 2801-2826
        • Marinai S.
        • Fujisawa H.
        Machine Learning in Document Analysis and Recognition.
        Springer, Berlin2007