Mobile toolbox (MTB) remote measures of executive function and processing speed: development and validation.

OBJECTIVE: The ability to remotely monitor cognitive skills is increasing with the ubiquity of smartphones. The Mobile Toolbox (MTB) is a new measurement system that includes measures assessing Executive Functioning (EF) and Processing Speed (PS): Arrow Matching, Shape-Color Sorting, and Number-Symbol Match. The purpose of this study was to assess their psychometric properties. METHOD: MTB measures were developed for smartphone administration based on constructs measured in the NIH Toolbox® (NIHTB). Psychometric properties of the resulting measures were evaluated in three studies with participants ages 18 to 90. In Study 1 (N = 92), participants completed MTB measures in the lab and were administered both equivalent NIH TB measures and other external measures of similar cognitive constructs. In Study 2 (N = 1,021), participants completed the equivalent NIHTB measures in the lab and then took the MTB measures on their own, remotely. In Study 3 (N = 168), participants completed MTB measures twice remotely, two weeks apart. RESULTS: All three measures exhibited very high internal consistency and strong test-retest reliability, as well as moderately high correlations with comparable NIHTB tests and moderate correlations with external measures of similar constructs. Phone operating system (iOS vs. Android) had a significant impact on performance for Arrow Matching and Shape-Color Sorting, but no impact on either validity or reliability. CONCLUSIONS: Results support the reliability and convergent validity of MTB EF and PS measures for use across the adult lifespan in remote, self-administered designs.

Development and validation of an episodic memory measure in the Mobile Toolbox (MTB): Arranging Pictures.

INTRODUCTION: Arranging Pictures is a new episodic memory test based on the NIH Toolbox (NIHTB) Picture Sequence Memory measure and optimized for self-administration on a personal smartphone within the Mobile Toolbox (MTB). We describe evidence from three distinct validation studies. METHOD: In Study 1, 92 participants self-administered Arranging Pictures on study-provided smartphones in the lab and were administered external measures of similar and dissimilar constructs by trained examiners to assess validity under controlled circumstances. In Study 2, 1,021 participants completed the external measures in the lab and self-administered Arranging Pictures remotely on their personal smartphones to assess validity in real-world contexts. In Study 3, 141 participants self-administered Arranging Pictures remotely twice with a two-week delay on personal iOS smartphones to assess test-retest reliability and practice effects. RESULTS: Internal consistency was good across samples (ρxx = .80 to .85, p < .001). Test-retest reliability was marginal (ICC = .49, p < .001) and there were significant practice effects after a two-week delay (ΔM = 3.21 (95% CI [2.56, 3.88]). As expected, correlations with convergent measures were significant and moderate to large in magnitude (ρ = .44 to .76, p < .001), while correlations with discriminant measures were small (ρ = .23 to .27, p < .05) or nonsignificant. Scores demonstrated significant negative correlations with age (ρ = -.32 to -.21, p < .001). Mean performance was slightly higher in the iOS compared to the Android group (MiOS = 18.80, NiOS = 635; MAndroid = 17.11, NAndroid = 386; t(757.73) = 4.17, p < .001), but device type did not significantly influence the psychometric properties of the measure. Indicators of potential cheating were mixed; average scores were significantly higher in the remote samples (F(2, 850) = 11.415, p < .001), but there were not significantly more perfect scores. CONCLUSION: The MTB Arranging Pictures measure demonstrated evidence of reliability and validity when self-administered on personal device. Future research should examine the potential for cheating in remote settings and the properties of the measure in clinical samples.

Development and Validation of a Vocabulary Measure in the Mobile Toolbox.

OBJECTIVE: We describe the development of a new computer adaptive vocabulary test, Mobile Toolbox (MTB) Word Meaning, and validity evidence from 3 studies. METHOD: Word Meaning was designed to be a multiple-choice synonym test optimized for self-administration on a personal smartphone. The items were first calibrated online in a sample of 7,525 participants to create the computer-adaptive test algorithm for the Word Meaning measure within the MTB app. In Study 1, 92 participants self-administered Word Meaning on study-provided smartphones in the lab and were administered external measures by trained examiners. In Study 2, 1,021 participants completed the external measures in the lab and Word Meaning was self-administered remotely on their personal smartphones. In Study 3, 141 participants self-administered Word Meaning remotely twice with a 2-week delay on personal iPhones. RESULTS: The final bank included 1363 items. Internal consistency was adequate to good across samples (ρxx = 0.78 to 0.81, p < .001). Test-retest reliability was good (ICC = 0.65, p < .001), and the mean theta score was not significantly different upon the second administration. Correlations were moderate to large with measures of similar constructs (ρ = 0.67-0.75, p < .001) and non-significant with measures of dissimilar constructs. Scores demonstrated small to moderate correlations with age (ρ = 0.35 to 0.45, p < .001) and education (ρ = 0.26, p < .001). CONCLUSION: The MTB Word Meaning measure demonstrated evidence of reliability and validity in three samples. Further validation studies in clinical samples are necessary.

Using the Mobile Toolbox in child and adolescent samples: A feasibility study.

Cognitive research with developmental samples requires improved methods that support large-scale, diverse, and open science. This paper offers initial evidence to support the Mobile Toolbox (MTB), a self-administered remote smartphone-based cognitive battery, in youth populations, from a pilot sample of 99 children (Mage = 11.79 years; 36% female; 53% White, 33% Black or African American, 9% Asian, and 15% Hispanic). Completion rates (95%-99%), practice performance (96%-100%), internal consistency (0.60-0.98), and correlations with similar NIHTB measures (0.55-0.77) provide the first evidence to support the MTB in a youth sample, although there were some inconsistencies across measures. Preliminary findings provide promising evidence of the MTB in developmental populations, and further studies are encouraged.

Primary care detection of cognitive impairment leveraging health and consumer technologies in underserved US communities: protocol for a pragmatic randomised controlled trial of the MyCog paradigm.

INTRODUCTION: Early identification of cognitive impairment (CI), including Alzheimer's disease and related dementias (ADRD), is a top public health priority. Yet, CI/ADRD is often undetected and underdiagnosed within primary care settings, and in health disparate populations. The MyCog paradigm is an iPad-based, self-administered, validated cognitive assessment based on the National Institutes of Health (NIH) Toolbox Cognition Battery and coupled with clinician decision-support tools that is specifically tailored for CI/ADRD detection within diverse, primary care settings. METHODS AND ANALYSIS: We will conduct a two-arm, primary care practice-randomised (N=24 practices; 45 257 active patients at the proposed practices), pragmatic trial among geographically diverse Oak Street Health sites to test the effectiveness of the MyCog paradigm to improve early detection CI/ADRD among low socioeconomic, black and Hispanic older adults compared with usual care. Participating practices randomised to the intervention arm will impart the MyCog paradigm as a new standard of care over a 3-year implementation period; as the cognitive component for Annual Wellness Visits and for any patient/informant-reported or healthcare provider-suspected cognitive concern. Rates of detected (cognitive test suggesting impairment) and/or diagnosed (relevant International Classification of Diseases-9/10 [ICD-9/10] code) cognitive deficits, impairments or dementias including ADRD will be our primary outcome of study compared between arms. Secondary outcomes will include ADRD severity (ie, mild or later stage), rates of cognitive-related referrals and rates of family member or caregiver involvement in ADRD care planning. We will use generalised linear mixed models to account for clustered study design. Secondary models will adjust for subject, clinic or visit-specific characteristics. We will use mixed-methods approaches to examine fidelity and cost-effectiveness of the MyCog paradigm. ETHICS AND DISSEMINATION: The Institutional Review Board at Advarra has approved the study protocol (Pro00064339). Results will be published in peer-reviewed journals and summaries will be provided to the funders of the study. TRIAL REGISTRATION NUMBER: NCT05607732.

Pilot Testing of the MyCog Assessment: Rapid Detection of Cognitive Impairment in Everyday Clinical Settings.

Cognitive impairment (CI) and dementia can have profound social and emotional effects on older adults. Early detection of CI is imperative both to the identification of potentially treatable conditions and to provide services to minimize the effects of CI in cases of dementia. While primary care settings are ideal for identifying CI, it frequently goes undetected. We tailored a brief, iPad-based, cognitive assessment (MyCog) for primary care settings and piloted it in a sample of older adults. Eighty participants were recruited from an existing cohort study and completed a brief, in-person interview. CI was determined based on a diagnosis of dementia or CI in their medical record or based on a comprehensive cognitive battery performed within the past 18 months. MyCog had a sensitivity of 79% and specificity of 82%, offering a practical, scalable, primary care assessment for the routine case finding of cognitive impairment and dementia.

Network Engagement in Action: Stakeholder Engagement Activities to Enhance Patient-centeredness of Research.

BACKGROUND: Stakeholders (ie, patients, policymakers, clinicians, advocacy groups, health system leaders, payers, and others) offer critical input at various stages in the research continuum, and their contributions are increasingly recognized as an important component of effective translational research. Natural experiments, in particular, may benefit from stakeholder feedback in addressing real-world issues and providing insight into future policy decisions, though best practices for the engagement of stakeholders in observational studies are limited in the literature. METHODS: The Natural Experiments for Translation in Diabetes 2.0 (NEXT-D2) network utilizes rigorous methods to evaluate natural experiments in health policy and program delivery with a focus on diabetes-related outcomes. Each of the 8 partnering institutions incorporates stakeholder engagement throughout multiple study phases to enhance the patient-centeredness of results. NEXT-D2 dedicates a committee to Engagement for resource sharing, enhancing engagement approaches, and advancing network-wide engagement activities. Key stakeholder engagement activities include Study Meetings, Proposal Development, Trainings & Educational Opportunities, Data Analysis, and Results Dissemination. Network-wide patient-centered resources and multimedia have also been developed through the broad expertise of each site's stakeholder group. CONCLUSIONS: This collaboration has created a continuous feedback loop wherein site-level engagement approaches are informed via the network and network-level engagement efforts are shaped by individual sites. Emerging best practices include: incorporating stakeholders in multiple ways throughout the research, building on previous relationships with stakeholders, enhancing capacity through stakeholder and investigator training, involving stakeholders in refining outcome choices and understanding the meaning of variables, and recognizing the power of stakeholders in maximizing dissemination.

Associations between religion-related factors and breast cancer screening among American Muslims.

American Muslims have low rates of mammography utilization, and research suggests that religious values influence their health-seeking behaviors. We assessed associations between religion-related factors and breast cancer screening in this population. A diverse group of Muslim women were recruited from mosques and Muslim organization sites in Greater Chicago to self-administer a survey incorporating measures of fatalism, religiosity, discrimination, and Islamic modesty. 254 surveys were collected of which 240 met age inclusion criteria (40 years of age or older). Of the 240, 72 respondents were Arab, 71 South Asian, 59 African American, and 38 identified with another ethnicity. 77% of respondents had at least one mammogram in their lifetime, yet 37% had not obtained mammography within the past 2 years. In multivariate models, positive religious coping, and perceived religious discrimination in healthcare were negatively associated with having a mammogram in the past 2 years, while having a PCP was positively associated. Ever having a mammogram was positively associated with increasing age and years of US residency, and knowing someone with breast cancer. Promoting biennial mammography among American Muslims may require addressing ideas about religious coping and combating perceived religious discrimination through tailored interventions.

Associations between religion-related factors and cervical cancer screening among Muslims in greater chicago.

OBJECTIVE: This study aimed to assess rates of Papanicolaou (Pap) testing and associations between religion-related factors and these rates among a racially and ethnically diverse sample of American Muslim women. MATERIALS AND METHODS: A community-based participatory research design was used in partnering with the Council of Islamic Organizations of Greater Chicago to recruit Muslim women attending mosque and community events. These participants self-administered surveys incorporating measures of fatalism, religiosity, perceived discrimination, Islamic modesty, and a marker of Pap test use. RESULTS: A total of 254 survey respondents were collected with nearly equal numbers of Arabs, South Asians, and African American respondents. Of these respondents, 84% had obtained a Pap test in their lifetime, with individuals who interpret disease as a manifestation of God's punishment having a lower odds of having had Pap testing after controlling for sociodemographic factors (odds ratio [OR]=0.87, 95% CI=0.77-1.0). In multivariate models, living in the United States for more than 20 years (OR=4.7, 95% CI=1.4-16) and having a primary care physician (OR=7.7, 95% CI=2.5-23.4) were positive predictors of having had a Pap test. Ethnicity, fatalistic beliefs, perceived discrimination, and modesty levels were not significantly associated with Pap testing rates. CONCLUSIONS: To our knowledge, this is the first study to assess Pap testing behaviors among a diverse sample of American Muslim women and to observe that negative religious coping (e.g., viewing health problems as a punishment from God) is associated with a lower odds of obtaining a Pap test. The relationship between religious coping and cancer screening behaviors deserves further study so that religious values can be appropriately addressed through cancer screening programs.