Accurate prediction of momentary cognition from intensive longitudinal data

Published:December 14, 2022DOI:



      Deficits in cognitive performance are implicated in the development and maintenance of psychopathology. Emerging evidence further suggests that within-person fluctuations in cognitive performance may represent sensitive early markers of neuropsychiatric decline. Incorporating routine cognitive assessments into standard clinical care—to identify between-person differences and monitor within-person fluctuations—has the potential to improve diagnostic screening and treatment planning. In support of these goals, it is critical to understand to what extent cognitive performance varies under routine, remote assessment conditions (i.e., momentary cognition) in relation to a wide range of possible predictors.


      Using data-driven, high-dimensional methods, we ranked strong predictors of momentary cognition and evaluated out-of-sample predictive accuracy. Our approach leveraged innovations in digital technology, including ambulatory assessment of cognition and behavior (
      • Rutter L.A.
      • Vahia I.V.
      • Forester B.P.
      • Ressler K.J.
      • Germine L.
      Heterogeneous Indicators of Cognitive Performance and Performance Variability Across the Lifespan.
      ) at scale (n = 122, n = 94 female), (
      • Singh S.
      • Strong R.W.
      • Jung L.
      • Li F.H.
      • Grinspoon L.
      • Scheuer L.S.
      • et al.
      The TestMyBrain Digital Neuropsychology Toolkit: Development and Psychometric Characteristics.
      ) in naturalistic environments, and (

      Kreitler S, Weissler K, Barak F. Physical health and cognition. In: Cognition and motivation: Forging an interdisciplinary perspective. New York, NY, US: Cambridge University Press; 2013. p. 238–269.

      ) within an intensive longitudinal study design (mean = 25.5 assessments/participant).


      Reaction time (R2 > .70) and accuracy (.56 > R2 > .35) were strongly predicted by age, between-person differences in mean performance, and time of day. Effects of self-reported, intra-individual fluctuations in environmental (e.g., noise) and internal (e.g., stress) states were also observed.


      Results provide robust estimates of effect size to characterize sources of cognitive variability, support the identification of optimal windows for psychosocial interventions, and may inform clinical evaluation under remote neuropsychological assessment conditions.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


        • Rutter L.A.
        • Vahia I.V.
        • Forester B.P.
        • Ressler K.J.
        • Germine L.
        Heterogeneous Indicators of Cognitive Performance and Performance Variability Across the Lifespan.
        Front Aging Neurosci. 2020;12(March; : 1-9
        • Singh S.
        • Strong R.W.
        • Jung L.
        • Li F.H.
        • Grinspoon L.
        • Scheuer L.S.
        • et al.
        The TestMyBrain Digital Neuropsychology Toolkit: Development and Psychometric Characteristics.
        J Clin Exp Neuropsychol [Internet]. 2021; 43 (Available from:): 786-795
      1. Kreitler S, Weissler K, Barak F. Physical health and cognition. In: Cognition and motivation: Forging an interdisciplinary perspective. New York, NY, US: Cambridge University Press; 2013. p. 238–269.

        • Lawton R.
        • Parker D.
        Individual differences in accident liability: A review and integrative approach.
        Hum Factors. 1998; 40: 655-671
        • Ayalon L.
        • Mackin S.
        • Arean P.A.
        • Chen H.
        • McDonel Herr E.C.
        The role of cognitive functioning and distress in suicidal ideation in older adults.
        J Am Geriatr Soc. 2007; 55: 1090-1094
        • Millan M.J.
        • Agid Y.
        • Brüne M.
        • Bullmore E.T.
        • Carter C.S.
        • Clayton N.S.
        • et al.
        Cognitive dysfunction in psychiatric disorders: Characteristics, causes and the quest for improved therapy.
        Nat Rev Drug Discov. 2012; 11: 141-168
        • Pessoa L.
        On the relationship between emotion and cognition.
        Nat Rev Neurosci. 2008; 9: 148-158
        • Chakrabarty T.
        • Hadjipavlou G.
        • Lam R.W.
        Cognitive Dysfunction in Major Depressive Disorder: Assessment, Impact, and Management.
        Focus (Madison). 2016; 14: 194-206
        • Bora E.
        • Harrison B.J.
        • Yücel M.
        • Pantelis C.
        Cognitive impairment in euthymic major depressive disorder: a meta-analysis.
        Psychol Med. 2013; 43: 2017-2026
        • Mohlman J.
        • Gorman J.M.
        The role of executive functioning in CBT: A pilot study with anxious older adults.
        Behav Res Ther. 2005; 43: 447-465
        • Lam R.W.
        • Kennedy S.H.
        • McIntyre R.S.
        • Khullar A.
        Cognitive dysfunction in major depressive disorder: Effects on psychosocial functioning and implications for treatment.
        Can J Psychiatry. 2014; 59: 649-654
        • Sahdra B.K.
        • MacLean K.A.
        • Ferrer E.
        • Shaver P.R.
        • Rosenberg E.L.
        • Jacobs T.L.
        • et al.
        Enhanced response inhibition during intensive meditation training predicts improvements in self-reported adaptive socioemotional functioning.
        Emotion. 2011; 11: 299
        • Rutter L.A.
        • Vahia I.V.
        • Passell E.
        • Forester B.P.
        • Germine L.
        The role of intraindividual cognitive variability in posttraumatic stress syndromes and cognitive aging: A literature search and proposed research agenda.
        Int Psychogeriatrics. 2020;
        • Salthouse T.A.
        • Nesselroade J.R.
        • Berish D.E.
        Short-term variability in cognitive performance and the calibration of longitudinal change.
        Journals Gerontol - Ser B Psychol Sci Soc Sci. 2006; 61: 144-151
      2. Dixon RA, Garrett DD, Lentz TL, MacDonald SWS, Strauss E, Hultsch DF. Neurocognitive markers of cognitive impairment: Exploring the roles of speed and inconsistency. Vol. 21, Neuropsychology. Dixon, Roger A.: Department of Psychology, University of Alberta, P-217 Biological Sciences Building, Edmonton, AB, Canada, T6G 2E9, [email protected]: American Psychological Association; 2007. p. 381–399.

      3. Sliwinski M, Hoffman L, Hofer S. Modeling retest and aging effects in a measurement burst design. Individual pathways of change: Statistical models for analyzing learning and development. Washington, DC, US: American Psychological Association; 2010. p. 37–50.

      4. Smyth, Heron. Is providing mobile interventions “just-in-time” helpful? an experimental proof of concept study of just-in-time intervention for stress management. In: 2016 IEEE Wireless Health (WH). 2016. p. 1–7.

      5. Germine L, Strong R, Singh S, Sliwinski MJ. Toward dynamic phenotypes and the scalable measurement of human behavior. Neuropsychopharmacology [Internet]. 2020;(June):1–8. Available from:

        • Sliwinski M.J.
        • Mogle J.A.
        • Hyun J.
        • Munoz E.
        • Smyth J.M.
        • Lipton R.B.
        Reliability and Validity of Ambulatory Cognitive Assessments.
        Assessment. 2018 Jan 1; 25: 14-30
      6. Nicosia J, Aschenbrenner AJ, Balota D, Sliwinski M, Tahan M, Adams S, et al. Unsupervised high-frequency smartphone-based cognitive assessments are reliable, valid, and feasible in older adults at risk for Alzheimer disease. PsyArXiv [Internet]. 2022;(January 5). Available from:

        • Germine L.
        • Reinecke K.
        • Chaytor N.S.
        Digital neuropsychology: Challenges and opportunities at the intersection of science and software.
        Clin Neuropsychol [Internet]. 2019; 33 (Available from:): 271-286
        • Weizenbaum E.
        • Torous J.
        • Fulford D.
        Cognition in context: Understanding the everyday predictors of cognitive performance in a New Era of measurement.
        JMIR mHealth uHealth. 2020; 8
        • Collins A.L.
        • Kim Y.
        • Sklar P.
        • O’Donovan M.C.
        • Sullivan P.F.
        Hypothesis-driven candidate genes for schizophrenia compared to genome-wide association results.
        Psychol Med. 2012; 42: 607-616
        • Bosker F.J.
        • Hartman C.A.
        • Nolte I.M.
        • Prins B.P.
        • Terpstra P.
        • Posthuma D.
        • et al.
        Poor replication of candidate genes for major depressive disorder using genome-wide association data.
        Mol Psychiatry. 2011; 16: 516-532
        • De Vries B.
        • Anttila V.
        • Freilinger T.
        • Wessman M.
        • Kaunisto M.A.
        • Kallela M.
        • et al.
        Systematic re-evaluation of genes from candidate gene association studies in migraine using a large genome-wide association data set.
        Cephalalgia. 2016; 36: 604-614
        • Kell D.B.
        • Oliver S.G.
        Here is the evidence, now what is the hypothesis? The complementary roles of inductive and hypothesis-driven science in the post-genomic era.
        BioEssays. 2004; 26: 99-105
      7. Prentice DA, Miller DT. When small effects are impressive. 2016;

        • Carey C.
        • Strong R.
        • Huang Y.
        • Aslibekyan S.
        • Gentleman R.
        • Smoller J.
        • et al.
        Shared and distinct genetic influences between cognitive domains and psychiatric disorder risk based on genome-wide data.
        Biol Psychiatry. 2021; 89: S45-S46
        • Allegrini A.G.
        • Selzam S.
        • Rimfeld K.
        • von Stumm S.
        • Pingault J.B.
        • Plomin R.
        Genomic prediction of cognitive traits in childhood and adolescence.
        Mol Psychiatry [Internet]. 2019; 24 (Available from:): 819-827
        • Lee J.J.
        • Wedow R.
        • Okbay A.
        • Kong E.
        • Maghzian O.
        • Zacher M.
        • et al.
        Gene discovery and polygenic prediction from a genome-wide association study of educational attainment in 1.1 million individuals.
        Nat Genet. 2018; 50: 1112-1121
        • Schütze M.
        • de Souza Costa D.
        • de Paula J.J.
        • Malloy-Diniz L.F.
        • Malamut C.
        • Mamede M.
        • et al.
        Use of machine learning to predict cognitive performance based on brain metabolism in Neurofibromatosis type 1.
        PLoS One. 2018 Sep 1; 13
        • Cui Z.
        • Su M.
        • Li L.
        • Shu H.
        • Gong G.
        Individualized Prediction of Reading Comprehension Ability Using Gray Matter Volume.
        Cereb Cortex. 2018; 28: 1656-1672
        • Rosenberg M.D.
        • Finn E.S.
        • Scheinost D.
        • Papademetris X.
        • Shen X.
        • Constable R.T.
        • et al.
        A neuromarker of sustained attention from whole-brain functional connectivity.
        Nat Neurosci. 2015; 19: 165-171
      8. Greene AS, Gao S, Scheinost D, Constable RT. Task-induced brain state manipulation improves prediction of individual traits. Nat Commun [Internet]. 2018;9(1). Available from:

      9. Sui J, Jiang R, Bustillo J, Calhoun V. Neuroimaging-based Individualized Prediction of Cognition and Behavior for Mental Disorders and Health: Methods and Promises. Biol Psychiatry [Internet]. 2020;88(11):818–828. Available from:

      10. Dagum P. Digital biomarkers of cognitive function. npj Digit Med [Internet]. 2018;1(1):26–28. Available from:

        • Cavedoni S.
        • Chirico A.
        • Pedroli E.
        • Cipresso P.
        • Riva G.
        Digital Biomarkers for the Early Detection of Mild Cognitive Impairment: Artificial Intelligence Meets Virtual Reality.
        Front Hum Neurosci. 2020;14(July; : 1-12
      11. Bzdok D, Meyer-Lindenberg A. Machine Learning for Precision Psychiatry: Opportunities and Challenges. Biol Psychiatry Cogn Neurosci Neuroimaging [Internet]. 2018;3(3):223–230. Available from:

      12. Mõttus R. How correlations can (not) be applied to individual people : A tutorial for researchers , students and the public. 2021;

      13. Molenaar PCM. A Manifesto on Psychology as Idiographic Science: Bringing the Person Back Into Scientific Psychology, This Time Forever. Meas Interdiscip Res Perspect [Internet]. 2004;2(4):201–218. Available from:

      14. Hamaker EL. Why researchers should think" within-person": A paradigmatic rationale. 2012;

        • Kanning M.K.
        • Ebner-Priemer U.W.
        • Schlicht W.M.
        How to Investigate Within-Subject Associations between Physical Activity and Momentary Affective States in Everyday Life: A Position Statement Based on a Literature Overview.
        Front Psychol. 2013;4(April; : 1-16
        • de Laat P.B.
        Algorithmic Decision-Making Based on Machine Learning from Big Data: Can Transparency Restore Accountability?.
        Philos Technol. 2018; 31: 525-541
      15. Arnold KD, Rohn KC. Internet-Enabled Experience Sampling: Innovations in Signaling and Data Sources. In: Advancing Educational Research With Emerging Technology. IGI Global; 2020. p. 124–151.

      16. Junghaenel DU, Stone AA. Ecological Momentary Assessment for the Psychosocial Study of Health. Wiley Encycl Heal Psychol. 2020;105–12.

        • Derby C.A.
        • Zhaoyang R.
        • Jiao J.L.
        • Sliwinski M.J.
        • Buxton O.M.
        Within and between person effects of sleep quality on daily cognitive performance: Data from the Einstein Aging Study (EAS).
        Alzheimer’s Dement. 2020 Dec; 16
        • van Erp S.
        • Oberski D.L.
        • Mulder J.
        Shrinkage priors for Bayesian penalized regression.
        J Math Psychol [Internet. 2019; 89 (Available from:): 31-50
        • Bürkner P.C.
        brms: An R package for Bayesian multilevel models using Stan.
        J Stat Softw. 2017; 80
        • Germine L.
        • Nakayama K.
        • Duchaine B.C.
        • Chabris C.F.
        • Chatterjee G.
        • Wilmer J.B.
        Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments.
        Psychon Bull Rev. 2012 Oct; 19: 847-857
        • Mackinnon A.
        • Jorm A.F.
        • Christensen H.
        • Korten A.E.
        • Jacomb P.A.
        • Rodgers B.
        A short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample.
        Pers Individ Dif. 1999; 27: 405-416
        • Almeida D.M.
        • Wethington E.
        • Kessler R.C.
        The daily inventory of stressful events: An interview-based approach for measuring daily stressors.
        Assessment. 2002; 9: 41-55
        • Stawski R.S.
        • Sliwinski M.J.
        • Almeida D.M.
        • Smyth J.M.
        Reported exposure and emotional reactivity to daily stressors: the roles of adult age and global perceived stress.
        Psychol Aging. 2008; 23: 52
        • Granholm E.
        • Ben-Zeev D.
        • Fulford D.
        • Swendsen J.
        Ecological momentary assessment of social functioning in schizophrenia: impact of performance appraisals and affect on social interactions.
        Schizophr Res. 2013; 145: 120-124
        • Sliwinski M.J.
        • Almeida D.M.
        • Smyth J.
        • Stawski R.S.
        Intraindividual change and variability in daily stress processes: findings from two measurement-burst diary studies.
        Psychol Aging. 2009; 24: 828
        • Åkerstedt T.
        • Anund A.
        • Axelsson J.
        • Kecklund G.
        Subjective sleepiness is a sensitive indicator of insufficient sleep and impaired waking function.
        J Sleep Res. 2014; 23: 242-254
      17. Germine LT, Joormann J, Passell E, Rutter LA, Scheuer L, Martini P, et al. Neurocognition after motor vehicle collision and adverse post-traumatic neuropsychiatric sequelae within 8 weeks: Initial findings from the AURORA study. J Affect Disord [Internet]. 2022;298(PB):57–67. Available from:

        • Esterman M.
        • Noonan S.K.
        • Rosenberg M.
        • Degutis J.
        In the zone or zoning out? Tracking behavioral and neural fluctuations during sustained attention.
        Cereb Cortex. 2013; 23: 2712-2723
        • Fortenbaugh F.C.
        • Degutis J.
        • Germine L.
        • Wilmer J.B.
        • Grosso M.
        • Russo K.
        • et al.
        Sustained attention across the life span in a sample of 10,000: Dissociating ability and strategy.
        Psychol Sci. 2015; 26: 1497-1510
        • Storm R.W.
        • Pylyshyn Z.W.
        Tracking multiple independent targets: Evidence for a parallel tracking mechanism.
        Spat Vis. 1988; 3: 179-197
        • Treviño M.
        • Zhu X.
        • Lu Y.Y.
        • Scheuer L.S.
        • Passell E.
        • Huang G.C.
        • et al.
        How do we measure attention? Using factor analysis to establish construct validity of neuropsychological tests.
        Cogn Res Princ Implic. 2021; 6: 1-26
      18. Wechsler D. WAIS‐III administration and scoring manual. The Psychological Corporation, San Antonio, TX. Wechsler, D(1997) WAIS III Adm Scoring Manual Psychol Corp San Antonio, TX. 1997;

        • Chaytor N.S.
        • Barbosa-Leiker C.
        • Germine L.T.
        • Fonseca L.M.
        • McPherson S.M.
        • Tuttle K.R.
        Construct validity, ecological validity and acceptance of self-administered online neuropsychological assessment in adults.
        Clin Neuropsychol [Internet]. 2021; 35 (Available from:): 148-164
        • Hartshorne J.K.
        • Germine L.T.
        When Does Cognitive Functioning Peak? The Asynchronous Rise and Fall of Different Cognitive Abilities Across the Life Span.
        Psychol Sci. 2015 Apr 18; 26: 433-443
        • Raykov T.
        • Marcoulides G.A.
        On Multilevel Model Reliability Estimation From the Perspective of Structural Equation Modeling.
        Struct Equ Model A Multidiscip J [Internet]. 2006 Jan 1; 13 (Available from:): 130-141
      19. Brose A, Lövdén M, Schmiedek F. Daily fluctuations in positive affect positively co-vary with working memory performance. Emotion [Internet]. 2014;14(1):1–6. Available from:

        • Sliwinski M.J.
        • Smyth J.M.
        • Hofer S.M.
        • Stawski R.S.
        Intraindividual coupling of daily stress and cognition.
        Psychol Aging. 2006 Sep; 21: 545-557
      20. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021.

        • Wickham H.
        • Averick M.
        • Bryan J.
        • Chang W.
        • McGowan L.
        • François R.
        • et al.
        Welcome to the Tidyverse.
        J Open Source Softw. 2019; 4: 1686
        • Woltman H.
        • Feldstain A.
        • MacKay C.
        • Rocchi M.
        An introduction to hierarchical linear modeling.
        Tutor Quant Methods Psychol. 2012; 8: 52-69
        • Tso I.F.
        • Taylor S.F.
        • Johnson T.D.
        Applying hierarchical bayesian modeling to experimental psychopathology data: An introduction and tutorial.
        J Abnorm Psychol. 2021; 130: 923-936
        • Van De Maat R.
        • Lataster J.
        • Verboon P.
        Why and How to Deal with Diurnal Cyclic Patterns in Ambulatory Assessment of Emotions: A Practical Guide and Discussion.
        Eur J Psychol Assess. 2020; 36: 471-481
        • Hamaker E.L.
        • Muthén B.
        The fixed versus random effects debate and how it relates to centering in multilevel modeling.
        Psychol Methods. 2020; 25: 365-379
        • Altelbany S.
        Evaluation of Ridge, Elastic Net and Lasso Regression Methods in Precedence of Multicollinearity Problem: A Simulation Study.
        J Appl Econ Bus Stud. 2021; 5: 131-142
      21. Huang YK, Mitchell UA, Conroy LM, Jones RM. Community daytime noise pollution and socioeconomic differences in Chicago, IL. PLoS One [Internet]. 2021;16(8 August):1–12. Available from:

        • Zamora W.
        • Calafate C.T.
        • Cano J.C.
        • Manzoni P.
        Accurate ambient noise assessment using smartphones.
        Sensors (Switzerland). 2017; 17: 1-18
        • Liu J.
        • Ramakrishnan S.
        • Laxminarayan S.
        • Balkin T.J.
        • Reifman J.
        Real-time individualization of the unified model of performance.
        J Sleep Res. 2017; 26: 820-831
        • Bei B.
        • Wiley J.F.
        • Allen N.B.
        • Trinder J.
        A cognitive vulnerability model of sleep and mood in adolescents under naturalistically restricted and extended sleep opportunities.
        Sleep. 2015; 38: 453-461
        • Winslow B.D.
        • Nguyen N.
        • Venta K.E.
        Improved mental acuity forecasting with an individualized quantitative sleep model.
        Front Neurol. 2017; 8: 1-10
        • Lo J.C.
        • Groeger J.A.
        • Cheng G.H.
        • Dijk D.J.
        • Chee M.W.L.
        Self-reported sleep duration and cognitive performance in older adults: A systematic review and meta-analysis.
        Sleep Med [Internet. 2016; 17 (Available from:): 87-98
        • Short M.A.
        • Blunden S.
        • Rigney G.
        • Matricciani L.
        • Coussens S.
        • Reynolds C M.
        • et al.
        Cognition and objectively measured sleep duration in children: a systematic review and meta-analysis.
        Sleep Heal [Internet]. 2018; 4 (Available from:): 292-300
      22. O’Neil ME, Laman-Maharg B, Schnurr PP, Carlson KF, Twamley EW, Peterson C, et al. Objective cognitive impairment and subjective cognitive problems in veterans initiating psychotherapy for posttraumatic stress disorder: An exploratory study. Appl Neuropsychol Adult [Internet]. 2019 May 4;26(3):247–254. Available from:

        • Neubauer A.B.
        • Dirk J.
        • Schmiedek F.
        Momentary working memory performance is coupled with different dimensions of affect for different children: A mixture model analysis of ambulatory assessment data.
        Dev Psychol. 2019; 55: 754-766
      23. von Stumm S. Is day-to-day variability in cognitive function coupled with day-to-day variability in affect? Intelligence [Internet]. 2016;55:1–6. Available from:

        • Hyun J.
        • Sliwinski M.J.
        • Smyth J.M.
        Waking Up on the Wrong Side of the Bed: The Effects of Stress Anticipation on Working Memory in Daily Life.
        Journals Gerontol - Ser B Psychol Sci Soc Sci. 2019 Jan 1; 74: 38-46
        • Sandi C.
        Stress and cognition.
        Wiley Interdiscip Rev Cogn Sci. 2013; 4: 245-261
        • Kyrios M.
        Cognitive-behaviour therapy in medical illness.
        Cogn Behav Ther. 2008; 139–55
        • Reichenberg A.
        • Weiser M.
        • Rapp M.A.
        • Rabinowitz J.
        • Caspi A.
        • Schmeidler J.
        • et al.
        Premorbid intra-individual variability in intellectual performance and risk for schizophrenia: A population-based study.
        Schizophr Res. 2006; 85: 49-57
        • Bangen K.J.
        • Weigand A.J.
        • Thomas K.R.
        • Delano-Wood L.
        • Clark L.R.
        • Eppig J.
        • et al.
        Cognitive dispersion is a sensitive marker for early neurodegenerative changes and functional decline in nondemented older adults.
        Neuropsychology. 2019; 33: 599-608
        • Nesselroade J.R.
        The warp and the woof of the developmental fabric. Visions aesthetics.
        Environ Dev Leg Joachim F Wohlwill. 1991; : 213-240