Neurocognitive Measures of Self-blame and Risk Prediction Models of Recurrence in Major Depressive Disorder

  • Andrew J. Lawrence
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom
    Search for articles by this author
  • Daniel Stahl
    Affiliations
    Department of Biostatistics, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
    Search for articles by this author
  • Suqian Duan
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom
    Search for articles by this author
  • Diede Fennema
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom
    Search for articles by this author
  • Tanja Jaeckle
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom
    Search for articles by this author
  • Allan H. Young
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom

    National Service for Affective Disorders, South London and Maudsley NHS Foundation Trust, London, United Kingdom
    Search for articles by this author
  • Paola Dazzan
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom

    National Service for Affective Disorders, South London and Maudsley NHS Foundation Trust, London, United Kingdom
    Search for articles by this author
  • Jorge Moll
    Affiliations
    Cognitive and Behavioral Neuroscience Unit, D’Or Institute for Research and Education, Rio de Janeiro, Brazil
    Search for articles by this author
  • Roland Zahn
    Correspondence
    Address correspondence to Roland Zahn, M.D., Ph.D.
    Affiliations
    Centre for Affective Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, United Kingdom

    National Service for Affective Disorders, South London and Maudsley NHS Foundation Trust, London, United Kingdom

    Cognitive and Behavioral Neuroscience Unit, D’Or Institute for Research and Education, Rio de Janeiro, Brazil
    Search for articles by this author
Open AccessPublished:June 24, 2021DOI:https://doi.org/10.1016/j.bpsc.2021.06.010

      Abstract

      Background

      Overgeneralized self-blaming emotions, such as self-disgust, are core symptoms of major depressive disorder and prompt specific actions (i.e., action tendencies), which are more functionally relevant than the emotions themselves. We have recently shown, using a novel cognitive task, that when feeling self-blaming emotions, maladaptive action tendencies (feeling like hiding and feeling like creating a distance from oneself) and an overgeneralized perception of control are characteristic of major depressive disorder, even after remission of symptoms. Here, we probed the potential of this cognitive signature, and its combination with previously employed functional magnetic resonance imaging (fMRI) measures, to predict individual recurrence risk. For this purpose, we developed a user-friendly hybrid machine/statistical learning tool, which we make freely available.

      Methods

      A total of 52 medication-free patients with remitted major depressive disorder, who had completed the action tendencies task and our self-blame fMRI task at baseline, were followed up clinically over 14 months to determine recurrence. Prospective prediction models included baseline maladaptive self-blame–related action tendencies and anterior temporal fMRI connectivity patterns across a set of frontolimbic a priori regions of interest, as well as including established clinical and standard psychological predictors. Prediction models used elastic net regularized logistic regression with nested 10-fold cross-validation.

      Results

      Cross-validated discrimination was highly promising (area under the receiver-operating characteristic curve ≥ 0.86), and positive predictive values over 80% were achieved when including fMRI in multimodal models, but only up to 71% (area under the receiver-operating characteristic curve ≤ 0.74) when solely relying on cognitive and clinical measures.

      Conclusions

      This study shows the high potential of multimodal signatures of self-blaming biases to predict recurrence risk at an individual level and calls for external validation in an independent sample.

      Keywords

      One central feature of cognitive models of vulnerability to major depressive disorder (MDD) is a tendency to excessively blame oneself (
      • Abramson L.Y.
      • Seligman M.E.P.
      • Teasdale J.D.
      Learned helplessness in humans - Critique and reformulation.
      ). These models have pointed to overgeneralized self-blame as causing worthlessness and hopelessness, core symptoms of MDD that are distinctive compared with other recurrent emotional disorders such as panic disorder (
      • Harris T.
      Recent developments in understanding the psychosocial aspects of depression.
      ). Self-blame in MDD is strongly associated with complex emotions such as guilt, self-disgust, and shame (
      • Zahn R.
      • Lythe K.E.
      • Gethin J.A.
      • Green S.
      • Deakin J.F.
      • Young A.H.
      • et al.
      The role of self-blame and worthlessness in the psychopathology of major depressive disorder.
      ). Social psychologists have highlighted that specific maladaptive action tendencies such as feeling like hiding were associated with self-blaming emotions (
      • Roseman I.J.
      • Wiest C.
      • Swartz T.S.
      Phenomenology, behaviors, and goals differentiate discrete emotions.
      ) and distinguish adaptive from depressogenic self-blame more accurately than the label of the emotion (
      • Tangney J.P.
      • Stuewig J.
      • Mashek D.J.
      Moral emotions and moral behavior.
      ). Indeed, using a novel action tendencies task, we have observed that self-blame–related action tendencies, in particular feeling like creating a distance from oneself and hiding, as well as an overgeneralized perception of control for other people’s wrongdoing, were more frequent in patients with remitted MDD compared with control participants cross-sectionally (
      • Duan S.
      • Lawrence A.
      • Valmaggia L.
      • Moll J.
      • Zahn R.
      The role of blame-related action tendencies in the vulnerability to major depressive disorder.
      ). In this comparison, feeling like hiding was also more common in patients with MDD when their friend was described to act badly toward them, which could be interpreted in different ways but may reflect an overgeneralized sense of responsibility (
      • O'Connor L.E.
      • Berry J.W.
      • Weiss J.
      • Gilbert P.
      Guilt, fear, submission, and empathy in depression.
      ). If measures of maladaptive self-blame–related action tendencies prospectively predicted future recurrence risk in remitted MDD, this would provide critical evidence for their role in MDD vulnerability. Apart from its pathophysiological importance, the identification of novel predictors of recurrence is needed for developing accurate risk prediction tools, because clinical variables are poor predictors at an individual level (
      • Nigatu Y.T.
      • Liu Y.
      • Wang J.
      External validation of the international risk prediction algorithm for major depressive episode in the US general population: The PredictD-US study.
      ) and because there are no established biomarkers (
      • Berwian I.M.
      • Walter H.
      • Seifritz E.
      • Huys Q.J.
      Predicting relapse after antidepressant withdrawal - A systematic review.
      ).
      Over the last decade, the application of machine learning to magnetic resonance imaging (MRI)–based prediction of individual responses to antidepressant treatments (
      • Williams L.M.
      • Korgaonkar M.S.
      • Song Y.C.
      • Paton R.
      • Eagles S.
      • Goldstein-Piekarski A.
      • et al.
      Amygdala reactivity to emotional faces in the prediction of general and medication-specific responses to antidepressant treatment in the randomized iSPOT-D trial.
      ,
      • Schmaal L.
      • Marquand A.F.
      • Rhebergen D.
      • van Tol M.-J.
      • Ruhé H.G.
      • van der Wee N.J.A.
      • et al.
      Predicting the naturalistic course of major depressive disorder using clinical and multimodal neuroimaging information: A multivariate pattern recognition study.
      ,
      • Fu C.H.
      • Mourao-Miranda J.
      • Costafreda S.G.
      • Khanna A.
      • Marquand A.F.
      • Williams S.C.
      • et al.
      Pattern classification of sad facial processing: Toward the development of neurobiological markers in depression.
      ,
      • Marquand A.F.
      • Mourao-Miranda J.
      • Brammer M.J.
      • Cleare A.J.
      • Fu C.H.
      Neuroanatomy of verbal working memory as a diagnostic biomarker for depression.
      ) has raised great promise for developing predictive markers in MDD. Machine learning models are powerful at making personalized predictions because they condense multiple variables, such as MRI signal in different brain regions, or different modalities (e.g., cognitive, structural MRI and functional MRI [fMRI]) (
      • Schmaal L.
      • Marquand A.F.
      • Rhebergen D.
      • van Tol M.-J.
      • Ruhé H.G.
      • van der Wee N.J.A.
      • et al.
      Predicting the naturalistic course of major depressive disorder using clinical and multimodal neuroimaging information: A multivariate pattern recognition study.
      ) into one model used to classify individuals based on complex interactions between the multiple input variables (
      • Sato J.R.
      • Thomaz C.E.
      • Cardoso E.F.
      • Fujita A.
      • Martin Mda G.
      • Amaro Jr., E.
      Hyperplane navigation: A method to set individual scores in fMRI group datasets.
      ). This approach has been used to predict whose symptoms will respond to a treatment (
      • Williams L.M.
      • Korgaonkar M.S.
      • Song Y.C.
      • Paton R.
      • Eagles S.
      • Goldstein-Piekarski A.
      • et al.
      Amygdala reactivity to emotional faces in the prediction of general and medication-specific responses to antidepressant treatment in the randomized iSPOT-D trial.
      ,
      • Schmaal L.
      • Marquand A.F.
      • Rhebergen D.
      • van Tol M.-J.
      • Ruhé H.G.
      • van der Wee N.J.A.
      • et al.
      Predicting the naturalistic course of major depressive disorder using clinical and multimodal neuroimaging information: A multivariate pattern recognition study.
      ,
      • Dunlop B.W.
      • Rajendra J.K.
      • Craighead W.E.
      • Kelley M.E.
      • McGrath C.L.
      • Choi K.S.
      • et al.
      Functional connectivity of the subcallosal cingulate cortex and differential outcomes to treatment with cognitive-behavioral therapy or antidepressant medication for major depressive disorder.
      ,
      • Frässle S.
      • Marquand A.F.
      • Schmaal L.
      • Dinga R.
      • Veltman D.J.
      • van der Wee N.J.A.
      • et al.
      Predicting individual clinical trajectories of depression with generative embedding.
      ). Successful development of novel treatments and stratification algorithms to improve long-term outcomes of recurrent MDD, however, requires predictors that capture the neurocognitive underpinnings of asymptomatic precursors of recurrent symptoms. Using a standard general linear model, we previously identified an fMRI signature of patients with MDD who will develop a recurring episode after recovering from previous depression over 1 year of follow-up (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ), but the machine learning approach we used to show its predictive value at the individual level with a positive predictive value of 74% had two main weaknesses. First, it used a high variance leave-one-out cross-validation method (
      • Kohavi R.
      A study of cross-validation and bootstrap for accuracy estimation and model selection.
      ,
      • Efron B.
      Estimating the error rate of a prediction rule: Improvement on cross-validation.
      ). Second, it used the extracted clusters showing the most significant association with recurrence risk in our SPM general linear model. This technique overestimates model performance owing to selection of voxels and variables on the basis of another model comparing recurring episode and stable remission patients in the same sample (
      • Kriegeskorte N.
      • Simmons W.K.
      • Bellgowan P.S.F.
      • Baker C.I.
      Circular analysis in systems neuroscience: The dangers of double dipping.
      ). Here, we sought to overcome these two weaknesses.
      As recently reviewed (
      • Zahn R.
      • de Oliveira-Souza R.
      • Moll J.
      Moral motivation and the basal forebrain.
      ), guilt proneness has been previously selectively associated with subgenual cingulate cortex (SCC) activation on fMRI (
      • Lythe K.E.
      • Gethin J.A.
      • Workman C.I.
      • Lambon Ralph M.A.
      • Deakin J.F.W.
      • Moll J.
      • et al.
      Subgenual activation and the finger of blame: individual differences and depression vulnerability [published online ahead of print Sep 25].
      ), whereas frontopolar activation was most reproducible for guilt versus other-blaming emotions irrespective of individual differences in guilt proneness (
      • Zahn R.
      • Moll J.
      • Paiva M.
      • Garrido G.
      • Krueger F.
      • Huey E.D.
      • et al.
      The neural basis of human social values: Evidence from functional MRI.
      ,
      • Zahn R.
      • de Oliveira-Souza R.
      • Bramati I.
      • Garrido G.
      • Moll J.
      Subgenual cingulate activity reflects individual differences in empathic concern.
      ,
      • Green S.
      • Lambon Ralph M.A.
      • Moll J.
      • Deakin J.F.W.
      • Zahn R.
      Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
      ). The latter is part of the default mode network, and its connectivity with the SCC has been implied in depressive rumination (
      • Hamilton J.P.
      • Farmer M.
      • Fogelman P.
      • Gotlib I.H.
      Depressive rumination, the default-mode network, and the dark matter of clinical neuroscience.
      ), which is typically of a self-critical nature. Interestingly, the SCC was abnormally active in current MDD, and normalization of its metabolism was associated with remission (
      • Price J.L.
      • Drevets W.C.
      Neurocircuitry of mood disorders.
      ,
      • Ressler K.J.
      • Mayberg H.S.
      Targeting abnormal neural circuits in mood and anxiety disorders: From the laboratory to the clinic.
      ), and its connectivity predicted response to psychological versus antidepressant treatment (
      • Dunlop B.W.
      • Rajendra J.K.
      • Craighead W.E.
      • Kelley M.E.
      • McGrath C.L.
      • Choi K.S.
      • et al.
      Functional connectivity of the subcallosal cingulate cortex and differential outcomes to treatment with cognitive-behavioral therapy or antidepressant medication for major depressive disorder.
      ), suggesting its central importance. The SCC, however, is only part of a network of regions involved in self-blaming feelings. Using fMRI, we demonstrated that proneness to self-blaming feelings in healthy people was associated with increased functional connectivity between the SCC and the right superior anterior temporal lobe (ATL) (
      • Green S.
      • Ralph M.A.L.
      • Moll J.
      • Stamatakis E.A.
      • Grafman J.
      • Zahn R.
      Selective functional integration between anterior temporal and distinct fronto-mesolimbic regions during guilt and indignation.
      ), which we had previously demonstrated to enable differentiated interpretations of the meaning of social behavior (e.g., differentiating actions as impolite or absent-minded, rather than just overgeneralizing them as “bad”) (
      • Zahn R.
      • Moll J.
      • Krueger F.
      • Huey E.D.
      • Garrido G.
      • Grafman J.
      Social concepts are represented in the superior anterior temporal cortex.
      ,
      • Zahn R.
      • Moll J.
      • Iyengar V.
      • Huey E.D.
      • Tierney M.
      • Krueger F.
      • et al.
      Social conceptual impairments in frontotemporal lobar degeneration with right anterior temporal hypometabolism.
      ).
      In an independent previous sample of patients with remitted MDD, we demonstrated abnormal functional connectivity between the right superior ATL and frontolimbic networks when experiencing self-blaming feelings relative to feelings related to blaming others (compared with never-depressed control participants independent of medication status) (
      • Green S.
      • Lambon Ralph M.A.
      • Moll J.
      • Deakin J.F.W.
      • Zahn R.
      Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
      ). These frontolimbic networks included the SCC, frontopolar cortex, hypothalamus, and hippocampus. This concorded with Price and Drevets’ (
      • Price J.L.
      • Drevets W.C.
      Neurocircuitry of mood disorders.
      ) neuroanatomical model of mood disorders, which includes the superior ATL because of its close anatomical connection with the medial prefrontal network including the subgenual cortex and the hypothalamus, thereby providing a crucial and often forgotten link between the limbic forebrain (including the hippocampus) and the frontal cortex. The ATL is probably underreported because fMRI in this region requires optimized imaging parameters owing to the abundance of artifacts (
      • Wastling S.J.
      • Barker G.J.
      Designing hyperbolic secant excitation pulses to reduce signal dropout in gradient-echo echo-planar imaging.
      ). It is notable that the original formulations of the default mode network based on positron emission tomography included the superior ATL (
      • Buckner R.L.
      • Andrews-Hanna J.R.
      • Schacter D.L.
      The brain's default network: Anatomy, function, and relevance to disease.
      ). Our focus on the right superior ATL is based on its predominance for socioemotional functions (
      • Zahn R.
      • Moll J.
      • Paiva M.
      • Garrido G.
      • Krueger F.
      • Huey E.D.
      • et al.
      The neural basis of human social values: Evidence from functional MRI.
      ,
      • Zahn R.
      • Moll J.
      • Iyengar V.
      • Huey E.D.
      • Tierney M.
      • Krueger F.
      • et al.
      Social conceptual impairments in frontotemporal lobar degeneration with right anterior temporal hypometabolism.
      ,
      • Pobric G.
      • Lambon Ralph M.A.
      • Zahn R.
      Hemispheric specialization within the superior anterior temporal cortex for social and nonsocial concepts.
      ,
      • Simmons W.K.
      • Reddish M.
      • Bellgowan P.S.F.
      • Martin A.
      The selectivity and functional connectivity of the anterior temporal lobes.
      ,
      • Olson I.R.
      • McCoy D.
      • Klobusicky E.
      • Ross L.A.
      Social cognition and the anterior temporal lobes: A review and theoretical framework.
      ,
      • Zahn R.
      • Green S.
      • Beaumont H.
      • Burns A.
      • Moll J.
      • Caine D.
      • et al.
      Frontotemporal lobar degeneration and social behaviour: Dissociation between the knowledge of its consequences and its conceptual meaning.
      ) relative to the left ATL. This concords with the efficacy of right temporal electroconvulsive therapy in MDD (
      • Kellner C.H.
      • Knapp R.
      • Husain M.M.
      • Rasmussen K.
      • Sampson S.
      • Cullum M.
      • et al.
      Bifrontal, bitemporal and right unilateral electrode placement in ECT: Randomised trial.
      ), response to which was predicted by right ATL–frontal connectivity including the SCC (
      • Leaver A.M.
      • Wade B.
      • Vasavada M.
      • Hellemann G.
      • Joshi S.H.
      • Espinoza R.
      • et al.
      Fronto-temporal connectivity predicts ECT outcome in major depression.
      ).
      The aim of the current study was to develop a user-friendly prediction tool and use it to provide robust estimates for the potential to predict MDD recurrence risk at the individual level, when using cognitive and fMRI-based signatures of self-blaming biases. Further, as one could argue that fMRI is relatively expensive for widespread use, we also probed the potential to replace it by our novel cognitive task for future clinical decision support systems.

      Methods and Materials

       Participants and Standard Measures

      Participants were recruited as a part of a prospective cohort study (from 2011 to 2014) in a clinical research facility (ethics reference: 07/H1003/194) (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ). All participants provided informed consent (verbal for initial telephone screening and written otherwise). Recruitment has been described previously (
      • Zahn R.
      • Lythe K.E.
      • Gethin J.A.
      • Green S.
      • Deakin J.F.
      • Workman C.
      • et al.
      Negative emotions towards others are diminished in remitted major depression.
      ). Inclusion criteria were MDD according to the DSM-IV-TR (
      American Psychiatric Association
      Diagnostic and Statistical Manual of Mental Disorders.
      ), remitted for ≥6 months, psychotropic medication–free, right-handed, native English speaking, and normal or corrected-to-normal vision. Main exclusion criteria were current Axis I disorders including a history of substance or alcohol abuse, and past comorbid Axis I disorders being the likely cause of depressive symptoms. Patients were subsequently followed up clinically over 14 months using the well-validated Longitudinal Interval Follow-up Evaluation for DSM-IV (
      • Keller M.B.
      • Lavori P.W.
      • Friedman B.
      • Nielsen E.
      • Endicott J.
      • McDonald-Scott P.
      • et al.
      The Longitudinal Interval Follow-Up Evaluation: A comprehensive method for assessing outcome in prospective longitudinal studies.
      ). Raters were blinded to the baseline results; interrater reliability was excellent (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ).
      For our multimodal prediction model, we used a complete-cases analysis (see Supplemental Methods) including 52 patients with MDD with complete follow-up data that we were able to categorize into recurring episode over 14 months (n = 18) versus stable remission with no recurring episode (n = 34).
      The following standard clinical and psychological measures were selected for use in the prediction model as based on previous reports establishing them as reproducible predictors of MDD recurrence at the group level: the number of previous episodes (categorized into nonrecurrent [i.e., 1 previous episode], recurrent [2–4 episodes], and highly recurrent [>5 episodes]) (
      • Buckman J.E.J.
      • Underwood A.
      • Clarke K.
      • Saunders R.
      • Hollon S.D.
      • Fearon P.
      • et al.
      Risk factors for relapse and recurrence of depression in adults and how they operate: A four-phase systematic review and meta-synthesis.
      ); the Beck Depression Inventory (
      • Beck A.T.
      • Steer R.A.
      • Carbin M.G.
      Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation.
      ) as a measure of residual symptoms that are associated with recurrence risk (
      • Buckman J.E.J.
      • Underwood A.
      • Clarke K.
      • Saunders R.
      • Hollon S.D.
      • Fearon P.
      • et al.
      Risk factors for relapse and recurrence of depression in adults and how they operate: A four-phase systematic review and meta-synthesis.
      ) and that loads heavily on self-criticism, also associated with recurrence (
      • LeMoult J.
      • Kircanski K.
      • Prasad G.
      • Gotlib I.H.
      Negative self-referential processing predicts the recurrence of major depressive episodes.
      ); the Global Assessment of Functioning Scale to capture comorbidity and functional impact (
      • First M.B.
      • Spitzer R.L.
      • Gibbon M.
      • Williams J.B.W.
      Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition (SCID-I/P).
      ); the Rosenberg Self-Esteem Scale (
      • Rosenberg M.
      Society and the Adolescent Self-Image.
      ,
      • Ormel J.
      • Oldehinkel A.J.
      • Vollebergh W.
      Vulnerability before, during, and after a major depressive episode - A 3-wave population-based study.
      ); and the Positive and Negative Affect Schedule asking how people feel in general (
      • Watson D.
      • Clark L.A.
      • Tellegen A.
      Development and validation of brief measures of positive and negative affect: The PANAS scales.
      ), in which the negative affect subscale was used as a measure of trait negative emotionality shown to be associated with poor prognosis (
      • Morris B.H.
      • Bylsma L.M.
      • Rottenberg J.
      Does emotion predict the course of major depressive disorder? A review of prospective studies.
      ) and the positive affect subscale was used because of its negative association with subsequent recurrence (
      • de Jonge M.
      • Dekker J.J.M.
      • Kikkert M.J.
      • Peen J.
      • van Rijsbergen G.D.
      • Bockting C.L.H.
      The role of affect in predicting depressive symptomatology in remitted recurrently depressed patients.
      ) in previous studies.

       Experimental Stimuli

      Stimuli for both fMRI and behavioral tasks were written sentences each presenting an abstract hypothetical social behavior contrary to social and moral values. Participants were asked to imagine the situation in each stimulus for two conditions differing by the agency of the participant. In the self-agency condition, the participant was described to act toward their friend, and in the other-agency condition, their friend acted toward them. Stimuli were based on the value-related moral sentiment task (VMST) used previously (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ,
      • Zahn R.
      • Lythe K.E.
      • Gethin J.A.
      • Green S.
      • Deakin J.F.
      • Workman C.
      • et al.
      Negative emotions towards others are diminished in remitted major depression.
      ). The same social behaviors were used in both conditions (90 trials per condition, 50% per se negative social behaviors [e.g., “does act stingily”] and 50% negated positive behaviors [e.g., “does not act generously”]). Participants were asked to provide the name of their best friend of the same gender, with whom they were not related and not romantically involved.
      After the fMRI and behavioral tasks, participants were asked to rate all items in the self- and other-agency conditions for unpleasantness ("How strongly would you feel unpleasant feelings?": scale of 1 [not unpleasant] to 7 [extremely unpleasant]). As in our previous article (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ), we restricted our analyses to the items deemed most emotionally relevant to each participant (defined as items rated equal or higher than the individual’s median unpleasantness rating for each condition: self- and other-agency).

       fMRI Acquisition and Paradigm

      As previously described (Figure S1) (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ), we used an fMRI protocol optimized for detection of ventral brain regions (see Supplemental Methods), and standard fMRI results were previously reported (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ). Participants were presented with hypothetical social actions in self-agency and other-agency conditions (stimuli described above). Stimuli were presented for 5 seconds in three runs in pseudorandom order; runs were counterbalanced across participants and were interspersed with a baseline visual fixation of pattern condition (n = 90). In the scanner, participants were asked to decide whether each situation would feel mildly or very unpleasant to ensure that they paid attention to the task and that they made an emotional decision about the stimuli. We used a jittered (range = 500 ms) intertrial interval with mean duration of 4 seconds.

       fMRI Analysis

      As in our previous analysis (Supplemental Methods) (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ) to measure functional connectivity, we employed the well-established psychophysiological interaction analysis (
      • Friston K.J.
      • Buechel C.
      • Fink G.R.
      • Morris J.
      • Rolls E.
      • Dolan R.J.
      Psychophysiological and modulatory interactions in neuroimaging.
      ), which requires the extraction of the right superior ATL signal time course [physiological variable, as done previously (
      • Green S.
      • Ralph M.A.L.
      • Moll J.
      • Stamatakis E.A.
      • Grafman J.
      • Zahn R.
      Selective functional integration between anterior temporal and distinct fronto-mesolimbic regions during guilt and indignation.
      ); 6-mm-radius sphere; Montreal Neurological Institute coordinates: x = 58, y = 0, z = −12] and the creation of an interaction term with the psychological variable (the contrast between the most highly unpleasant items in the self-agency condition vs. the visual fixation condition and highly unpleasant items in the other-agency condition vs. the visual fixation condition).
      Regression coefficient averages over a priori regions of interest (ROIs) (depicted in Figure 1B and further described below) as defined previously using an independent sample (
      • Green S.
      • Lambon Ralph M.A.
      • Moll J.
      • Deakin J.F.W.
      • Zahn R.
      Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
      ) were extracted as raw values using MarsBaR (
      • Brett M.
      • Anton J.-L.
      • Valabregue R.
      • Poline J.-B.
      Region of interest analysis using an SPM toolbox.
      ) and used to capture self-blame–related fMRI connectivity with our ATL seed region, as well as to capture ATL connectivity with these ROIs irrespective of psychological condition, while covarying for root mean square movement parameters during the scan (obtained from the realignment process). In addition, we modeled standard blood oxygen level–dependent effects following the modeling approach as previously described (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ) for self-blame in the SCC (
      • Ressler K.J.
      • Mayberg H.S.
      Targeting abnormal neural circuits in mood and anxiety disorders: From the laboratory to the clinic.
      ) and ATL seed region. This was in keeping with the psychophysiological interaction approach (
      • Friston K.J.
      • Buechel C.
      • Fink G.R.
      • Morris J.
      • Rolls E.
      • Dolan R.J.
      Psychophysiological and modulatory interactions in neuroimaging.
      ). To limit the number of variables, we modeled blood oxygen level–dependent effects only in our primary ROIs (SCC and ATL).
      Figure thumbnail gr1
      Figure 1(A) Average cross-validated receiver-operating characteristic curves for repeated nested regularized logistic regression models (the same 52 participants with major depressive disorder in all models, 30 nested inner and 100 outer loop repetitions using 10-fold cross-validation, R software). The clinical variables (Clin) model included standard scales and known clinical predictors. For the clinical and cognitive (Clin & Cog) model, maladaptive self-blaming action tendencies measures were added. For the clinical and functional magnetic resonance imaging (fMRI) (Clin & MRI) model, fMRI measures were added. The all modalities (All) model included cognitive, clinical, and fMRI measures. (B) All a priori regions of interest reported in a previous independent article (
      • Green S.
      • Lambon Ralph M.A.
      • Moll J.
      • Deakin J.F.W.
      • Zahn R.
      Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
      ) with close anatomical connections to our anterior temporal lobe (ATL) seed region and of sufficient anatomical specificity for data extraction were included (i.e., regions spanning multiple Brodmann areas such as the dorsolateral frontal cortex were discarded). The model included the midline subgenual cingulate cortex, frontopolar cortex (Brodmann area 10), and hypothalamus, as well as the right hemispheric hippocampus, amygdala, insula, and basal ganglia (striatum/pallidum) regions of interest to match our a priori right hemisphere ATL seed region (
      • Green S.
      • Lambon Ralph M.A.
      • Moll J.
      • Deakin J.F.W.
      • Zahn R.
      Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
      ). R, right.

       fMRI ROIs

      To ensure independence of our ROIs from the current dataset, we used all a priori ROIs from our previous independent publication (
      • Green S.
      • Lambon Ralph M.A.
      • Moll J.
      • Deakin J.F.W.
      • Zahn R.
      Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
      ), but in order to reduce the number of ROIs, we chose only right-sided ROIs reflecting our right-sided ATL seed region, except for the midline regions, in which we used bilateral ROIs (see Supplemental Methods and Table S2).

       Action Tendencies Task

      As previously described (
      • Duan S.
      • Lawrence A.
      • Valmaggia L.
      • Moll J.
      • Zahn R.
      The role of blame-related action tendencies in the vulnerability to major depressive disorder.
      ), for each described social behavior, the participant was asked, “What would you feel like doing in response to this behavior?” “Please select the option that you feel that you would most strongly feel like doing” (see Figure S1): “verbally or physically attacking/punishing your best friend,” “verbally or physically attacking/punishing yourself,” “apologising/fixing what you have done,” “hiding,” “creating a distance from your best friend,” “creating a distance from yourself,” “no action,” or “other action.” They then rated the hypothetical situation regarding how much control they feel they would have (responses ranging from 1 [none at all] to 7 [complete]).
      We have previously reported that self-agency–related hiding and self-distancing, as well as an overgeneralized perception of control for others’ wrongdoing, were characteristic of MDD at baseline compared with control participants (
      • Duan S.
      • Lawrence A.
      • Valmaggia L.
      • Moll J.
      • Zahn R.
      The role of blame-related action tendencies in the vulnerability to major depressive disorder.
      ). In the present article, we report their association with recurrence risk for the first time. As done previously (
      • Duan S.
      • Lawrence A.
      • Valmaggia L.
      • Moll J.
      • Zahn R.
      The role of blame-related action tendencies in the vulnerability to major depressive disorder.
      ), we separately calculated the proportion of trials in which hiding or self-distancing were selected in the self-agency condition for the highly unpleasant items as defined using individual median splits as described above. In order to reduce the number of variables, we z-transformed these proportions across the whole sample to combine hiding and self-distancing into a single average of z scores of maladaptive self-blaming action tendencies. We measured overgeneralized perception of control as done previously by subtracting each subject’s average perceived control rating in the highly unpleasant other-agency condition trials from the same quantity in their highly unpleasant self-agency trials. Perceived control is expected to be higher during self-agency, so a smaller value in the calculated difference measure indicates abnormally higher perception of control in the other-agency condition, which is conceptually related to the notion of omnipotent responsibility guilt (i.e., caused by an exaggerated sense of responsibility for others’ well-being) as shown to be associated with MDD (
      • O'Connor L.E.
      • Berry J.W.
      • Weiss J.
      • Gilbert P.
      Guilt, fear, submission, and empathy in depression.
      ).

       Prediction Modeling

      We developed a software tool (AL) to implement the proposed methods and make this tool available for use by the scientific and clinical community as an R package (https://www.github.com/AndrewLawrence/dCVnet). We adopted a hybrid approach combining statistical and machine learning to balance their benefits and limitations in this type of prediction problem. In contrast to traditional statistical methods, we reduced overfitting and allowed correlated predictors through regularization. We also carefully cross-validated performance by including all modeling decisions within the cross-validation. This hybrid approach can likewise be contrasted to more complex (and data-hungry) (
      • van der Ploeg T.
      • Austin P.C.
      • Steyerberg E.W.
      Modern modelling techniques are data hungry: A simulation study for predicting dichotomous endpoints.
      ) machine learning methods: each model uses relatively few of the available predictors, which are chosen a priori based on previously published results and clinical utility. To further reduce complexity, we used a model that assumes that the effects of the predictors are linear and additive, and any interaction terms or predictor transformations must be prespecified. The software tool is called dCVnet, for “double cross-validation for the elastic net.” It employs elastic net–regularized logistic regression with double (also termed “nested”) cross-validation, consisting of an outer cross-validation of model performance measures with an inner cross-validation to independently tune elastic net hyperparameters.

       Elastic Net Model

      Elastic net–regularized binary logistic regression extends logistic regression with regularization. This acts as a penalty to model complexity and reduces this by shrinking coefficients toward zero (
      • Zou H.
      • Hastie T.
      Regularization and variable selection via the elastic net.
      ). The elastic net regularization penalty comprises two types of penalty with different roles: 1) the L2 (Ridge) penalty allows correlated predictors to jointly enter the model and stabilizes solutions and 2) the L1 (LASSO) penalty encourages variable selection (
      • Hastie T.
      • Tibshirani R.
      • Friedman J.H.
      The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
      ). The amount of regularization (hyperparameter: lambda) and balance of the two types of regularization penalty (hyperparameter: alpha; indicating the fraction of the penalty that is of LASSO type) are hyperparameters of the elastic net procedure in that they determine the learning performance of the algorithm, and tuning alpha and lambda adapts the algorithm to different problem settings.

       Cross-Validation and Tuning

      We implemented double cross-validation in dCVnet to obtain performance estimates without optimistic bias at the same time as tuning hyperparameters based on cross-validated performance (
      • Cawley G.C.
      • Talbot N.L.C.
      On over-fitting in model selection and subsequent selection bias in performance evaluation.
      ). For both inner and outer cross-validation loops, our tool employed repeated k-fold cross-validation (Figure S4), an unbiased estimate of expected (out-of-sample) prediction error with lower variance than bootstrap, hold-out, and leave-one-out cross-validation methods (
      • Hastie T.
      • Tibshirani R.
      • Friedman J.H.
      The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
      ,
      • Kim J.-H.
      Estimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap.
      ).
      For this particular application of dCVnet, stable hyperparameter selection could be obtained (for all models) with 30 repetitions of 10-fold cross-validation, and stable cross-validation results in 100 repetitions of 10-fold cross-validation. For tuning, six logarithmically spaced values of alpha were considered between 0.01 (mostly Ridge) and 1.0 (a LASSO model). For each alpha, 100 lambda values were determined from the data by glmnet (
      • Friedman J.H.
      • Hastie T.
      • Tibshirani R.
      Regularization paths for generalized linear models via coordinate descent.
      ). The tuned alpha and lambda were selected based on the minimum mean square error.

       Software

      dCVnet was programmed in R software version 3.6.2 (R Foundation for Statistical Computing) and makes repeated calls to the glmnet 3.0.2 package’s binomial elastic net fitting function to produce the multiple models required for nested k-fold cross-validation (
      • Friedman J.H.
      • Hastie T.
      • Tibshirani R.
      Regularization paths for generalized linear models via coordinate descent.
      ). The novelty of dCVnet lies in providing a documented and standardized implementation of this particular machine learning pipeline, making it accessible to researchers lacking programming experience required for more general machine learning software environments (e.g., sklearn, caret, tidymodels, or mlr). We make this software including documentation available (https://www.github.com/andrewlawrence/dCVnet) to improve reproducibility and future open source development of clinical decision support systems. We ran permutation tests to rule out biases and information leakage within dCVnet, which was confirmed by a distribution of the balanced accuracy measures from the permuted datasets around the expected mean of 50% (i.e., chance) (Figure S2, Table S1, and Supplemental Methods).

       Model Evaluation

      Predicted probabilities of depression recurrence were compared with the known recurrence status to calculate a variety of cross-validated prediction performance measures (
      • Steyerberg E.W.
      • Vickers A.J.
      • Cook N.R.
      • Gerds T.
      • Gonen M.
      • Obuchowski N.
      • et al.
      Assessing the performance of prediction models: A framework for traditional and novel measures.
      ). These included common classification metrics (such as positive predictive value and negative predictive value) based on a 50% classification cutoff. For positive predictive value and negative predictive value, we adjusted our estimates for the 42% prevalence of recurrence in the wider study sample. We further considered threshold-independent performance measures: the Brier score (mean squared difference between predicted probability and binary outcome) and the area under the receiver-operating characteristic curve (AUC) (also termed the concordance statistic; see Supplemental Methods). Finally, we considered model calibration (i.e., the fidelity of the predicted probability to the observed proportion with the outcome) by calculating the intercept and slope of the calibration graph (
      • Steyerberg E.W.
      • Vickers A.J.
      • Cook N.R.
      • Gerds T.
      • Gonen M.
      • Obuchowski N.
      • et al.
      Assessing the performance of prediction models: A framework for traditional and novel measures.
      ). Consistent with our aims, at this point in model development, the measure of prime interest is the AUC, as this reflects the discriminative potential of a model independent of the classification threshold, or model calibration.

      Results

      As shown in Table 1, there were no demographic differences between groups. As expected, the recurring episode group showed a higher Beck Depression Inventory score at baseline. We obtained an AUC of at least 0.86, ≥80% cross-validated accuracy, and moderate calibration for classifying patients into recurring episode versus stable remission when combining multimodal information from fMRI and standard clinical and psychological measures, with or without our novel action tendencies task (Table 2, Figure 1). We further probed the contributions of the different modalities to predictive performance. Relying solely on standard clinical and psychological measures achieved an AUC of 0.73 and a positive predictive value of 68% (Table 2). Our results showed, however, that the novel cognitive measure, our action tendencies task, did not improve prediction performance to a relevant degree (AUC = 0.74; positive predictive value of 71%) (Table 2). We therefore also ran a prediction model that only contained the action tendencies task variables to demonstrate that they were associated with recurrence risk (AUC = 0.67) (Table 2), but this model performed slightly worse compared with known clinical and standard measures. Likewise, when using fMRI without standard measures, there was an association with recurrence risk (AUC = 0.69), but this was not as strong as in the multimodal model.
      Table 1Demographic Characteristics and Symptoms
      VariableRemitted MDD
      Recurring Group (n = 18)Stable Group (n = 34)P Overall
      Age, Years, Mean (SD)35.9 (13.2)34.9 (13.9).79
      Sex, n (%).98
       Female13 (72.2%)23 (67.6%)
       Male5 (27.8%)11 (32.4%)
      Years of Education, Mean (SD)16.1 (2.08)17.1 (1.88).11
      BDI Score, Mean (SD)5.44 (3.81)2.97 (3.11).03
      Significant group difference at an uncorrected two-sided threshold of p ≤ .05.
      MADRS Score, Mean (SD)1.22 (1.56)1.03 (1.49).67
      GAF Score, Mean (SD)83.6 (6.46)86.5 (4.84).10
      A total of 52 patients with fully remitted MDD had available baseline data for all predictors included in the multimodal prediction model and were followed up clinically over 14 months to determine recurrence. As expected, there were slightly higher residual depressive symptoms on the more sensitive BDI in the group with a subsequent recurrence.
      BDI, Beck Depression Inventory; GAF, Global Assessment of Functioning; MADRS, Montgomery–Åsberg Depression Rating Scale; MDD, major depressive disorder.
      a Significant group difference at an uncorrected two-sided threshold of p ≤ .05.
      Table 2Cross-Validated Prediction Model Results
      MetricfMRI
      Self-blame–related fMRI variables.
      Cognitive
      Self-blame–related action tendencies task variables.
      Clinical
      Standard psychological and clinical measures.
      Clinical
      Standard psychological and clinical measures.
      and Cognitive
      Self-blame–related action tendencies task variables.
      Clinical
      Standard psychological and clinical measures.
      and fMRI
      Self-blame–related fMRI variables.
      All Modalities
      Accuracy0.660.680.690.700.830.80
      Sensitivity0.320.330.280.330.690.61
      Specificity0.840.860.910.900.900.90
      PPV0.590.630.680.710.830.81
      NPV0.630.640.630.650.800.76
      AUC0.690.670.730.740.900.86
      Brier Score0.220.200.200.200.130.15
      Calibration Intercept−0.22−0.12−0.02−0.100.02−0.08
      Calibration Slope0.500.780.990.830.610.55
      Cross-validated (100× repeated 10-fold cross-validation) performance measures from the different prediction models. Both were adjusted to the 42% prevalence found in the full sample (
      • Lythe K.E.
      • Moll J.
      • Gethin J.A.
      • Workman C.
      • Green S.
      • Lambon Ralph M.A.
      • et al.
      Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
      ).
      AUC, area under the receiver-operating characteristic curve; fMRI, functional magnetic resonance imaging; NPV, negative predictive value; PPV, positive predictive value.
      a Self-blame–related fMRI variables.
      b Self-blame–related action tendencies task variables.
      c Standard psychological and clinical measures.
      When comparing unregularized odds ratios from single-predictor logistic regressions with the regularized odds ratios, there was relatively little shrinkage of coefficients (i.e., a small lambda was chosen), reflecting the relatively good cross-validated performance of the model (Table 3). Interestingly, there were predictors that on their own had no relevant association with recurrence risk but contributed to the multivariate prediction. Maladaptive action tendencies were associated with recurrence risk by themselves and in the full regularized model (Table 3), whereas overgeneralized perception of control was not (Table 3).
      Table 3Model Coefficients Predicting Recurrence
      ModalityPredictor VariableAll Modalities βAll Modalities ORUnivariate OR
      Clinical/Standard PsychologicalIntercept−2.260.10
      Global Assessment of Functioning score−0.190.830.91
      Number of episodes categorized1.373.943.19
      Rosenberg Self-Esteem Scale−1.000.370.90
      PANAS negative affect−2.560.080.96
      PANAS positive affect−0.730.480.86
      Beck Depression Inventory1.303.671.23
      fMRIMotion parameters−1.290.280.86
      SCC–ATL connectivity (self-blame)2.4311.362.14
      Basal ganglia–ATL connectivity (self-blame)2.7615.803.49
      Frontopolar–ATL connectivity (self-blame)−1.680.190.73
      Hypothalamus–ATL connectivity (self-blame)−1.820.160.76
      Insula–ATL connectivity (self-blame)−0.140.871.67
      Amygdala–ATL connectivity (self-blame)1.464.311.05
      Hippocampus–ATL connectivity (self-blame)1.153.161.38
      SCC–signal time course1.957.036.62
      Basal ganglia–signal time course−0.130.888.08
      Frontopolar–signal time course0.101.116.69
      Hypothalamus–signal time course1.022.772.23
      Insula–signal time course1.253.494.14
      Amygdala–signal time course0.091.091.62
      Hippocampus–signal time course0.271.315.53
      ATL BOLD effect (self-blame)2.148.501.07
      SCC BOLD effect (self-blame)0.722.051.05
      CognitivePerceived control for self-blame vs. other blame−0.250.781.19
      Maladaptive action tendencies0.912.482.53
      Displayed are the multivariate semi-standardized regularized logistic regression coefficient (β) and OR for each predictor variable from the all modalities model (n = 52), which is generated by dCVnet. Recurrence was coded as the positive outcome, so that positive regression coefficients for any predictor can be interpreted as a positive association with recurrence risk. For reference, univariate unregularized logistic regression ORs are provided. For fMRI measures, self-blame indicates that the measurement is derived from a contrast between self-blame trials and fixation cross trials. Motion parameters were derived from the SPM realignment and used root mean squares of the three translation and three rotation parameters, averaged after z transformation. Figure S3 details the a priori rationale for each predictor.
      ATL, anterior temporal lobe; BOLD, blood oxygen level–dependent; OR, odds ratio; PANAS, Positive and Negative Affect Schedule; SCC, subgenual cingulate cortex.

      Discussion

      We confirmed that combinations of clinical, psychological, and self-blame fMRI measures were highly promising candidates for predicting individual recurrence risk in MDD. Using a principled hybrid statistical/machine learning approach, we observed excellent cross-validated discriminative performance, with an AUC of 0.9 and positive predictive values above the suggested benchmark of 80% for clinically useful markers (
      • Savitz J.B.
      • Rauch S.L.
      • Drevets W.C.
      Clinical application of brain imaging for the diagnosis of mood disorders: The current state of play.
      ). In contrast, our novel cognitive task to capture self-blame–related action tendencies was not sufficient to replace fMRI. We nevertheless obtained AUC values above 0.7, which is considered fair discriminatory value (
      • Fischer J.E.
      • Bachmann L.M.
      • Jaeschke R.
      A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis.
      ), and there was an association of maladaptive action tendencies with subsequent recurrence risk (odds ratio above 2.5). Yet, there was no relevant improvement on known clinical and standard measures regarding its discriminative value. These findings of modest predictive utility are in keeping with the previous literature showing that clinical and standard measures in themselves are unable to make accurate predictions of individual recurrence risk (
      • Nigatu Y.T.
      • Liu Y.
      • Wang J.
      External validation of the international risk prediction algorithm for major depressive episode in the US general population: The PredictD-US study.
      ,
      • Wojnarowski C.
      • Firth N.
      • Finegan M.
      • Delgadillo J.
      Predictors of depression relapse and recurrence after cognitive behavioural therapy: A systematic review and meta-analysis.
      ).
      Despite the lack of evidence for the predictive utility of the action tendencies task in this dataset, there are several strategies that could be pursued to improve the value of experimental cognitive tasks in future prediction models. First, our task was purely text based and lacked immersive features. Immersion could be improved by adopting a virtual reality paradigm (
      • Valmaggia L.R.
      • Latif L.
      • Kempton M.J.
      • Rus-Calafell M.
      Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence.
      ). Second, any single task is unlikely to capture all relevant aspects of vulnerability; thus, future work could integrate a wider range of cognitive measures: for example, a recently developed cognitive task measuring effort and reward decisions showed promising but modest effect sizes for predicting recurrence in MDD (
      • Berwian I.M.
      • Wenzel J.G.
      • Collins A.G.E.
      • Seifritz E.
      • Stephan K.E.
      • Walter H.
      • et al.
      Computational mechanisms of effort and reward decisions in patients with depression and their association with relapse after antidepressant discontinuation.
      ) resulting in similar AUC values of around 0.70 as our action tendencies task, and its combination with action tendencies may outperform the separate models.
      Although a wide variety of clinical and standard measures with known predictive value were assessed, performance of the model using just these features was modest (AUC = 0.73), and one might ask whether important predictors were omitted. Recently, promising results were shown for recurrence risk prediction using a machine learning model that included childhood trauma (
      • Ruhe H.G.
      • Mocking R.J.T.
      • Figueroa C.A.
      • Seeverens P.W.J.
      • Ikani N.
      • Tyborowska A.
      • et al.
      Emotional biases and recurrence in major depressive disorder. Results of 2.5 years follow-up of drug-free cohort vulnerable for recurrence.
      ,
      • Opel N.
      • Redlich R.
      • Dohm K.
      • Zaremba D.
      • Goltermann J.
      • Repple J.
      • et al.
      Mediation of the influence of childhood maltreatment on depression relapse by cortical structure: A 2-year longitudinal observational study.
      ); although this needs further replication, we did not measure childhood trauma, and adding this to our prediction model may improve its accuracy in the future. Despite their theoretical importance (
      • Harris T.
      Recent developments in understanding the psychosocial aspects of depression.
      ), our model did not include stressful life events during the follow-up period because our aim was prospective prediction from baseline data. We note that although 60% of our 93 patients with completed follow-up data in the overall study reported stressful life events during the 14-month follow-up period (as determined by clinical interview and standardized questionnaires), there was no difference in the rate of occurrence of these events between the recurring episode (61%) and stable remission (59%) groups. This is consistent with the hypothesis that life events trigger recurrence not directly, but rather through their interaction with other factors, such as self-blaming biases (
      • Harris T.
      Recent developments in understanding the psychosocial aspects of depression.
      ), a hypothesis that merits further investigation.
      Previous studies using machine learning approaches to imaging data in depression have usually relied on voxel-based methods (
      • Kambeitz J.
      • Cabral C.
      • Sacchet M.D.
      • Gotlib I.H.
      • Zahn R.
      • Serpa M.H.
      • et al.
      Detecting neuroimaging biomarkers for depression: A meta-analysis of multivariate pattern recognition studies.
      ). Yet, these methods do not allow integrating cognitive and clinical variables in a straightforward way. Hence, we decided to employ a simpler approach to the imaging analysis, relying on ROIs, and this approach may be more feasible for future clinical applications. By employing the elastic net as a machine learning extension of logistic regression, we alleviated overfitting by shrinking the regression coefficients toward zero, which allows for automatic variable selection by omitting some predictors. The nested cross-validation is vital to provide realistic estimates of out-of-sample prediction accuracy and thereby estimate internal validity (
      • Steyerberg E.W.
      • Moons K.G.M.
      • van der Windt D.A.
      • Hayden J.A.
      • Perel P.
      • Schroter S.
      • et al.
      Prognosis Research Strategy (PROGRESS) 3: Prognostic model research.
      ), a shortcoming of many studies of clinical predictors that commonly employ shared cross-validation for both model tuning and performance estimation. Although this prognostic model is at an early stage, our approach follows the Medical Research Council Prognosis Research Strategy Partnership (
      • Steyerberg E.W.
      • Moons K.G.M.
      • van der Windt D.A.
      • Hayden J.A.
      • Perel P.
      • Schroter S.
      • et al.
      Prognosis Research Strategy (PROGRESS) 3: Prognostic model research.
      ) guidelines for developing and reporting prognostic models.
      On a more cautionary note, for prognostic model development, our sample size was relatively small (
      • Varoquaux G.
      Cross-validation failure: Small sample sizes lead to large error bars.
      ). Although we deliberately adopted more stable hybrid methods than the voxelwise support vector machine considered by Varoquaux (
      • Varoquaux G.
      Cross-validation failure: Small sample sizes lead to large error bars.
      ), we must nevertheless replicate these findings in a larger independent sample before drawing clinical conclusions. Further, we included patients who were fully remitted from symptoms and had no relevant comorbidity; thus, generalization to patients with partial remission and comorbidity will need to be investigated.

       Conclusions

      Maladaptive self-blame–related action tendencies and fMRI measures predicted subsequent recurrence risk in our MDD sample. When including fMRI, our statistical learning–based risk prediction tool showed promising potential to predict recurrence risk at an individual level. This calls for external validation in an independent sample.

      Acknowledgments and Disclosures

      This work was supported by a Medical Research Council Clinician Scientist Fellowship (Grant No. G0902304 [to RZ]), the LABS-D'Or Hospital Network (Rio de Janeiro, Brazil) (to JM), and a Medical Research Council Doctoral Training Partnership (Grant No. 2064430 [to DF]). RZ and DS were partly funded and AJL was fully funded by the National Institute for Health Research Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London and by NARSAD Independent Investigator Grant No. 24715 from the Brain & Behavior Research Foundation. The views expressed are those of the authors and not necessarily those of the National Health Service, the National Institute for Health Research, or the Department of Health.
      We are grateful to Dr. Karen Lythe for collecting the primary data.
      RZ has collaborations with e-health companies Depsee Ltd, EMIS PLC, and Alloc Modulo Ltd. All other authors report no biomedical financial interests or potential conflicts of interest.

      Supplementary Material

      References

        • Abramson L.Y.
        • Seligman M.E.P.
        • Teasdale J.D.
        Learned helplessness in humans - Critique and reformulation.
        J Abnorm Psychol. 1978; 87: 49-74
        • Harris T.
        Recent developments in understanding the psychosocial aspects of depression.
        Br Med Bull. 2001; 57: 17-32
        • Zahn R.
        • Lythe K.E.
        • Gethin J.A.
        • Green S.
        • Deakin J.F.
        • Young A.H.
        • et al.
        The role of self-blame and worthlessness in the psychopathology of major depressive disorder.
        J Affect Disord. 2015; 186: 337-341
        • Roseman I.J.
        • Wiest C.
        • Swartz T.S.
        Phenomenology, behaviors, and goals differentiate discrete emotions.
        J Pers Soc Psychol. 1994; 67: 206-221
        • Tangney J.P.
        • Stuewig J.
        • Mashek D.J.
        Moral emotions and moral behavior.
        Annu Rev Psychol. 2007; 58: 345-372
        • Duan S.
        • Lawrence A.
        • Valmaggia L.
        • Moll J.
        • Zahn R.
        The role of blame-related action tendencies in the vulnerability to major depressive disorder.
        medRxiv. 2020; https://doi.org/10.1101/2020.11.15.20232058
        • O'Connor L.E.
        • Berry J.W.
        • Weiss J.
        • Gilbert P.
        Guilt, fear, submission, and empathy in depression.
        J Affect Disord. 2002; 71: 19-27
        • Nigatu Y.T.
        • Liu Y.
        • Wang J.
        External validation of the international risk prediction algorithm for major depressive episode in the US general population: The PredictD-US study.
        BMC Psychiatry. 2016; 16: 256
        • Berwian I.M.
        • Walter H.
        • Seifritz E.
        • Huys Q.J.
        Predicting relapse after antidepressant withdrawal - A systematic review.
        Psychol Med. 2017; 47: 426-437
        • Williams L.M.
        • Korgaonkar M.S.
        • Song Y.C.
        • Paton R.
        • Eagles S.
        • Goldstein-Piekarski A.
        • et al.
        Amygdala reactivity to emotional faces in the prediction of general and medication-specific responses to antidepressant treatment in the randomized iSPOT-D trial.
        Neuropsychopharmacology. 2015; 40: 2398-2408
        • Schmaal L.
        • Marquand A.F.
        • Rhebergen D.
        • van Tol M.-J.
        • Ruhé H.G.
        • van der Wee N.J.A.
        • et al.
        Predicting the naturalistic course of major depressive disorder using clinical and multimodal neuroimaging information: A multivariate pattern recognition study.
        Biol Psychol. 2015; 78: 278-286
        • Fu C.H.
        • Mourao-Miranda J.
        • Costafreda S.G.
        • Khanna A.
        • Marquand A.F.
        • Williams S.C.
        • et al.
        Pattern classification of sad facial processing: Toward the development of neurobiological markers in depression.
        Biol Psychol. 2008; 63: 656-662
        • Marquand A.F.
        • Mourao-Miranda J.
        • Brammer M.J.
        • Cleare A.J.
        • Fu C.H.
        Neuroanatomy of verbal working memory as a diagnostic biomarker for depression.
        Neuroreport. 2008; 19: 1507-1511
        • Sato J.R.
        • Thomaz C.E.
        • Cardoso E.F.
        • Fujita A.
        • Martin Mda G.
        • Amaro Jr., E.
        Hyperplane navigation: A method to set individual scores in fMRI group datasets.
        Neuroimage. 2008; 42: 1473-1480
        • Dunlop B.W.
        • Rajendra J.K.
        • Craighead W.E.
        • Kelley M.E.
        • McGrath C.L.
        • Choi K.S.
        • et al.
        Functional connectivity of the subcallosal cingulate cortex and differential outcomes to treatment with cognitive-behavioral therapy or antidepressant medication for major depressive disorder.
        Am J Psychiatry. 2017; 174: 533-545
        • Frässle S.
        • Marquand A.F.
        • Schmaal L.
        • Dinga R.
        • Veltman D.J.
        • van der Wee N.J.A.
        • et al.
        Predicting individual clinical trajectories of depression with generative embedding.
        Neuroimage Clin. 2020; 26: 102213
        • Lythe K.E.
        • Moll J.
        • Gethin J.A.
        • Workman C.
        • Green S.
        • Lambon Ralph M.A.
        • et al.
        Self-blame-selective hyperconnectivity between anterior temporal and subgenual cortices and prediction of recurrent depressive episodes.
        JAMA Psychiatry. 2015; 72: 1119-1126
        • Kohavi R.
        A study of cross-validation and bootstrap for accuracy estimation and model selection.
        in: IJCAI’95: Proceedings of the 14th International Joint Conference on Artificial Intelligence – Volume 2. Canada: Morgan Kaufmann, Montreal, Quebec1995: 1137-1143
        • Efron B.
        Estimating the error rate of a prediction rule: Improvement on cross-validation.
        J Am Stat Assoc. 1983; 78: 316-331
        • Kriegeskorte N.
        • Simmons W.K.
        • Bellgowan P.S.F.
        • Baker C.I.
        Circular analysis in systems neuroscience: The dangers of double dipping.
        Nat Neurosci. 2009; 12: 535-540
        • Zahn R.
        • de Oliveira-Souza R.
        • Moll J.
        Moral motivation and the basal forebrain.
        Neurosci Biobehav Rev. 2020; 108: 207-217
        • Lythe K.E.
        • Gethin J.A.
        • Workman C.I.
        • Lambon Ralph M.A.
        • Deakin J.F.W.
        • Moll J.
        • et al.
        Subgenual activation and the finger of blame: individual differences and depression vulnerability [published online ahead of print Sep 25].
        Psychol Med. 2020;
        • Zahn R.
        • Moll J.
        • Paiva M.
        • Garrido G.
        • Krueger F.
        • Huey E.D.
        • et al.
        The neural basis of human social values: Evidence from functional MRI.
        Cereb Cortex. 2009; 19: 276-283
        • Zahn R.
        • de Oliveira-Souza R.
        • Bramati I.
        • Garrido G.
        • Moll J.
        Subgenual cingulate activity reflects individual differences in empathic concern.
        Neurosci Lett. 2009; 457: 107-110
        • Green S.
        • Lambon Ralph M.A.
        • Moll J.
        • Deakin J.F.W.
        • Zahn R.
        Guilt-selective functional disconnection of anterior temporal and subgenual cortices in major depressive disorder.
        Arch Gen Psychiatry. 2012; 69: 1014-1021
        • Hamilton J.P.
        • Farmer M.
        • Fogelman P.
        • Gotlib I.H.
        Depressive rumination, the default-mode network, and the dark matter of clinical neuroscience.
        Biol Psychiatry. 2015; 78: 224-230
        • Price J.L.
        • Drevets W.C.
        Neurocircuitry of mood disorders.
        Neuropsychopharmacology. 2010; 35: 192-216
        • Ressler K.J.
        • Mayberg H.S.
        Targeting abnormal neural circuits in mood and anxiety disorders: From the laboratory to the clinic.
        Nat Neurosci. 2007; 10: 1116-1124
        • Green S.
        • Ralph M.A.L.
        • Moll J.
        • Stamatakis E.A.
        • Grafman J.
        • Zahn R.
        Selective functional integration between anterior temporal and distinct fronto-mesolimbic regions during guilt and indignation.
        Neuroimage. 2010; 52: 1720-1726
        • Zahn R.
        • Moll J.
        • Krueger F.
        • Huey E.D.
        • Garrido G.
        • Grafman J.
        Social concepts are represented in the superior anterior temporal cortex.
        Proc Natl Acad Sci U S A. 2007; 104: 6430-6435
        • Zahn R.
        • Moll J.
        • Iyengar V.
        • Huey E.D.
        • Tierney M.
        • Krueger F.
        • et al.
        Social conceptual impairments in frontotemporal lobar degeneration with right anterior temporal hypometabolism.
        Brain. 2009; 132: 604-616
        • Wastling S.J.
        • Barker G.J.
        Designing hyperbolic secant excitation pulses to reduce signal dropout in gradient-echo echo-planar imaging.
        Magn Reson Med. 2015; 74: 661-672
        • Buckner R.L.
        • Andrews-Hanna J.R.
        • Schacter D.L.
        The brain's default network: Anatomy, function, and relevance to disease.
        Ann N Y Acad Sci. 2008; 1124: 1-38
        • Pobric G.
        • Lambon Ralph M.A.
        • Zahn R.
        Hemispheric specialization within the superior anterior temporal cortex for social and nonsocial concepts.
        J Cogn Neurosci. 2016; 28: 351-360
        • Simmons W.K.
        • Reddish M.
        • Bellgowan P.S.F.
        • Martin A.
        The selectivity and functional connectivity of the anterior temporal lobes.
        Cereb Cortex. 2009; 20: 813-825
        • Olson I.R.
        • McCoy D.
        • Klobusicky E.
        • Ross L.A.
        Social cognition and the anterior temporal lobes: A review and theoretical framework.
        Soc Cogn Affect Neurosci. 2013; 8: 123-133
        • Zahn R.
        • Green S.
        • Beaumont H.
        • Burns A.
        • Moll J.
        • Caine D.
        • et al.
        Frontotemporal lobar degeneration and social behaviour: Dissociation between the knowledge of its consequences and its conceptual meaning.
        Cortex. 2017; 93: 107-118
        • Kellner C.H.
        • Knapp R.
        • Husain M.M.
        • Rasmussen K.
        • Sampson S.
        • Cullum M.
        • et al.
        Bifrontal, bitemporal and right unilateral electrode placement in ECT: Randomised trial.
        Br J Psychiatry. 2010; 196: 226-234
        • Leaver A.M.
        • Wade B.
        • Vasavada M.
        • Hellemann G.
        • Joshi S.H.
        • Espinoza R.
        • et al.
        Fronto-temporal connectivity predicts ECT outcome in major depression.
        Front Psychiatry. 2018; 9: 11
        • Zahn R.
        • Lythe K.E.
        • Gethin J.A.
        • Green S.
        • Deakin J.F.
        • Workman C.
        • et al.
        Negative emotions towards others are diminished in remitted major depression.
        Eur Psychiatry. 2015; 30: 448-453
        • American Psychiatric Association
        Diagnostic and Statistical Manual of Mental Disorders.
        4th ed text rev. American Psychiatric Press, Washington, DC2000
        • Keller M.B.
        • Lavori P.W.
        • Friedman B.
        • Nielsen E.
        • Endicott J.
        • McDonald-Scott P.
        • et al.
        The Longitudinal Interval Follow-Up Evaluation: A comprehensive method for assessing outcome in prospective longitudinal studies.
        Arch Gen Psychiatry. 1987; 44: 540-548
        • Buckman J.E.J.
        • Underwood A.
        • Clarke K.
        • Saunders R.
        • Hollon S.D.
        • Fearon P.
        • et al.
        Risk factors for relapse and recurrence of depression in adults and how they operate: A four-phase systematic review and meta-synthesis.
        Clin Psychol Rev. 2018; 64: 13-38
        • Beck A.T.
        • Steer R.A.
        • Carbin M.G.
        Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation.
        Clin Psychol Rev. 1988; 8: 77-100
        • LeMoult J.
        • Kircanski K.
        • Prasad G.
        • Gotlib I.H.
        Negative self-referential processing predicts the recurrence of major depressive episodes.
        Clin Psychol Sci. 2017; 5: 174-181
        • First M.B.
        • Spitzer R.L.
        • Gibbon M.
        • Williams J.B.W.
        Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition (SCID-I/P).
        Biometrics Research, New York State Psychiatric Institute, New York, NY2002
        • Rosenberg M.
        Society and the Adolescent Self-Image.
        Wesley University Press, Middleton, CT1989
        • Ormel J.
        • Oldehinkel A.J.
        • Vollebergh W.
        Vulnerability before, during, and after a major depressive episode - A 3-wave population-based study.
        Arch Gen Psychiatry. 2004; 61: 990-996
        • Watson D.
        • Clark L.A.
        • Tellegen A.
        Development and validation of brief measures of positive and negative affect: The PANAS scales.
        J Pers Soc Psychol. 1988; 54: 1063-1070
        • Morris B.H.
        • Bylsma L.M.
        • Rottenberg J.
        Does emotion predict the course of major depressive disorder? A review of prospective studies.
        Br J Clin Psychol. 2009; 48: 255-273
        • de Jonge M.
        • Dekker J.J.M.
        • Kikkert M.J.
        • Peen J.
        • van Rijsbergen G.D.
        • Bockting C.L.H.
        The role of affect in predicting depressive symptomatology in remitted recurrently depressed patients.
        J Affect Disord. 2017; 210: 66-71
        • Friston K.J.
        • Buechel C.
        • Fink G.R.
        • Morris J.
        • Rolls E.
        • Dolan R.J.
        Psychophysiological and modulatory interactions in neuroimaging.
        Neuroimage. 1997; 6: 218-229
        • Brett M.
        • Anton J.-L.
        • Valabregue R.
        • Poline J.-B.
        Region of interest analysis using an SPM toolbox.
        Neuroimage. 2002; 13: 210-217
        • van der Ploeg T.
        • Austin P.C.
        • Steyerberg E.W.
        Modern modelling techniques are data hungry: A simulation study for predicting dichotomous endpoints.
        BMC Med Res Methodol. 2014; 14: 137
        • Zou H.
        • Hastie T.
        Regularization and variable selection via the elastic net.
        J R Stat Soc Series B Stat Methodol. 2005; 67: 301-320
        • Hastie T.
        • Tibshirani R.
        • Friedman J.H.
        The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
        Springer, New York, NY2001
        • Cawley G.C.
        • Talbot N.L.C.
        On over-fitting in model selection and subsequent selection bias in performance evaluation.
        J Mach Learn Res. 2010; 11: 2079-2107
        • Kim J.-H.
        Estimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap.
        Comput Stat Data Anal. 2009; 53: 3735-3745
        • Friedman J.H.
        • Hastie T.
        • Tibshirani R.
        Regularization paths for generalized linear models via coordinate descent.
        J Stat Softw. 2010; 33: 1-22
        • Steyerberg E.W.
        • Vickers A.J.
        • Cook N.R.
        • Gerds T.
        • Gonen M.
        • Obuchowski N.
        • et al.
        Assessing the performance of prediction models: A framework for traditional and novel measures.
        Epidemiology. 2010; 21: 128-138
        • Savitz J.B.
        • Rauch S.L.
        • Drevets W.C.
        Clinical application of brain imaging for the diagnosis of mood disorders: The current state of play.
        Mol Psychiatry. 2013; 18: 528-539
        • Fischer J.E.
        • Bachmann L.M.
        • Jaeschke R.
        A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis.
        Intensive Care Med. 2003; : 1043-1051
        • Wojnarowski C.
        • Firth N.
        • Finegan M.
        • Delgadillo J.
        Predictors of depression relapse and recurrence after cognitive behavioural therapy: A systematic review and meta-analysis.
        Behav Cogn Psychother. 2019; 47: 514-529
        • Valmaggia L.R.
        • Latif L.
        • Kempton M.J.
        • Rus-Calafell M.
        Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence.
        Psychiatry Res. 2016; 236: 189-195
        • Berwian I.M.
        • Wenzel J.G.
        • Collins A.G.E.
        • Seifritz E.
        • Stephan K.E.
        • Walter H.
        • et al.
        Computational mechanisms of effort and reward decisions in patients with depression and their association with relapse after antidepressant discontinuation.
        JAMA Psychiatry. 2020; 77: 513-522
        • Ruhe H.G.
        • Mocking R.J.T.
        • Figueroa C.A.
        • Seeverens P.W.J.
        • Ikani N.
        • Tyborowska A.
        • et al.
        Emotional biases and recurrence in major depressive disorder. Results of 2.5 years follow-up of drug-free cohort vulnerable for recurrence.
        Front Psychiatry. 2019; 10
        • Opel N.
        • Redlich R.
        • Dohm K.
        • Zaremba D.
        • Goltermann J.
        • Repple J.
        • et al.
        Mediation of the influence of childhood maltreatment on depression relapse by cortical structure: A 2-year longitudinal observational study.
        Lancet Psychiatry. 2019; 6: 318-326
        • Kambeitz J.
        • Cabral C.
        • Sacchet M.D.
        • Gotlib I.H.
        • Zahn R.
        • Serpa M.H.
        • et al.
        Detecting neuroimaging biomarkers for depression: A meta-analysis of multivariate pattern recognition studies.
        Biol Psychiatry. 2017; 82: 330-338
        • Steyerberg E.W.
        • Moons K.G.M.
        • van der Windt D.A.
        • Hayden J.A.
        • Perel P.
        • Schroter S.
        • et al.
        Prognosis Research Strategy (PROGRESS) 3: Prognostic model research.
        PLoS Med. 2013; 10e1001381
        • Varoquaux G.
        Cross-validation failure: Small sample sizes lead to large error bars.
        Neuroimage. 2018; 180: 68-77