Development and psychometric evaluation of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool: a study protocol.

INTRODUCTION: The need for quantitative criteria to appraise the quality of implementation research has recently been highlighted to improve methodological rigour. The Implementation Science Research development (ImpRes) tool and supplementary guide provide methodological guidance and recommendations on how to design high-quality implementation research. This protocol reports on the development of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool, a quantitative appraisal tool, developed based on the structure and content of the ImpRes tool and supplementary guide, to evaluate the conceptual and methodological quality of implementation research. METHODS AND ANALYSIS: This study employs a three-stage sequential mixed-methods design. During stage 1, the research team will map core domains of the ImpRes tool, guidance and recommendations contained in the supplementary guide and within the literature, to ImpResPAC. In stage 2, an international multidisciplinary expert group, recruited through purposive sampling, will inform the refinement of ImpResPAC, including content, scoring system and user instructions. In stage 3, an extensive psychometric evaluation of ImpResPAC, that was created in stage 1 and refined in stage 2, will be conducted. The scaling assumptions (inter-item and item-total correlations), reliability (internal consistency, inter-rater) and validity (construct and convergent validity) will be investigated by applying ImpResPAC to 50 protocols published in Implementation Science. We envisage developing ImpResPAC in this way will provide implementation research stakeholders, primarily grant reviewers and educators, a comprehensive, transparent and fair appraisal of the conceptual and methodological quality of implementation research, increasing the likelihood of funding research that will generate knowledge and contribute to the advancement of the field. ETHICS AND DISSEMINATION: This study will involve human participants. This study has been registered and minimal risk ethical clearance granted by The Research Ethics Office, King's College London (reference number MRA-20/21-20807). Participants will receive written information on the study via email and will provide e-consent if they wish to participate. We will use traditional academic modalities of dissemination (eg, conferences and publications).

Trends in the Utilization of Teleneurology and Other Healthcare Resources Prior to and During the COVID-19 Pandemic in an Urban, Tertiary Health System.

BACKGROUND: Patient groups traditionally affected by health disparities were less likely to use video teleneurology (TN) care during the initial COVID-19 pandemic surge in the United States. Whether this asymmetry persisted later in the pandemic or was accompanied with a loss of access to care remains unknown. METHODS: We conducted a retrospective cohort study using patient data from a multicenter healthcare system in New York City. We identified all established pediatric or adult neurology patients with at least two prior outpatient visits between June 16th, 2019 and March 15th, 2020 using our electronic medical record. For this established pre-COVID cohort, we identified telephone, in-person, video TN or emergency department visits and hospital admissions for any cause between March 16th and December 15th, 2020 ("COVID period"). We determined clinical, sociodemographic, income, and visit characteristics. Our primary outcome was video TN utilization, and our main secondary outcome was loss to follow-up during the COVID period. We used multivariable logistic regression to model the relationship between patient-level characteristics and both outcomes. RESULTS: We identified 23,714 unique visits during the COVID period, which corresponded to 14,170 established patients from our institutional Neurology clinics during the pre-COVID period. In our cohort, 4,944 (34.9%) utilized TN and 4,997 (35.3%) were entirely lost to follow-up during the COVID period. In the adjusted regression analysis, Black or African-American race [adjusted odds ratio (aOR) 0.60, 97.5%CI 0.52-0.70], non-English preferred language (aOR 0.49, 97.5%CI 0.39-0.61), Medicaid insurance (aOR 0.50, 97.5%CI 0.44-0.57), and Medicare insurance (aOR 0.73, 97.5%CI 0.65-0.83) had decreased odds of TN utilization. Older age (aOR 0.98, 97.5%CI 0.98-0.99), female sex (aOR 0.90 97.5%CI 0.83-0.99), and Medicaid insurance (aOR 0.78, 0.68-0.90) were associated with decreased odds of loss to follow-up. CONCLUSION: In the first 9 months of the COVID-19 pandemic, we found sociodemographic patterns in TN utilization that were similar to those found very early in the pandemic. However, these sociodemographic characteristics were not associated with loss to follow-up, suggesting that lack of TN utilization may not have coincided with loss of access to care.

A retrospective cohort study of clinical factors, visit patterns, and demographic factors associated with use of remote communications in patients with headache.

OBJECTIVE: To identify the characteristics associated with high utilization of remote communications (RCs) in patients with headache. BACKGROUND: Patients with headache frequently communicate with their providers using secure portal messaging and telephone calls. However, clinical and demographic factors as well as visit patterns associated with RC utilization remain poorly characterized. METHODS: We retrospectively analyzed data from patients with headache who were evaluated in the ambulatory neurology faculty practice at the Icahn School of Medicine at Mount Sinai in New York between January 1 and June 30, 2019. We extracted clinical and demographic characteristics, total office visits, secure MyChart portal messages, and telephone encounters from our institutional data warehouse. We defined high RC and MyChart utilization as the top tertile of RC and MyChart message volume, respectively, and assessed the relationship between patient characteristics and high RC (primary outcome), as well as high MyChart utilization (secondary outcome). We characterized the relationship between clinicodemographic characteristics and the ratio of MyChart messages to total RCs (secondary outcome). RESULTS: We identified 1390 patients, of whom 477 (34.3%) were high RC utilizers and 321 (23.1%) were high MyChart utilizers. High RC utilizers generated 3306/3921 (84.3%) RCs. The presence of chronic headache (aOR 2.31, 95% CI 1.75-3.03, p < 0.0001), cluster headache (aOR 18.3, 95% CI 5.0-71.7, p = 0.001), and migraine (aOR 3.82, 95% CI 1.93-9.3, p = 0.011) was associated with high RC utilization. Patients ≥65 years of age were less likely to engage in MyChart messaging as a proportion of RC (191/680, 28.1%) compared with patients 18-30 years of age (243/620, 39.2%, p = 0.049) and 30-64 years of age (1172/2721, 43.1%, p < 0.0001). CONCLUSIONS: A minority of patients with headache (477/1390; 34.3%) generated the majority (3306/3921; 84.3%) of RCs. Our findings should be validated in external patient cohorts with the objective of developing strategies to optimize RC utilization.

Teleneurology Expansion in Response to the COVID-19 Outbreak at a Tertiary Health System in New York City.

OBJECTIVE: To assess the implementation of teleneurology (TN), including patient and clinician experiences, during the coronavirus respiratory disease 2019 (COVID-19) pandemic. METHODS: We studied synchronous (video visit) and asynchronous (store-and-forward, patient-portal evaluation, remote monitoring) TN utilization in the Mount Sinai Health System Neurology Department in New York, 2 months before and after the start of our department's response to the pandemic in mid-March 2020. Weekly division meetings enabled ongoing assessments and analysis of barriers and facilitators according to the Consolidated Framework for Implementation Research and the Expert Recommendations for Implementing Change models. We used postvisit surveys of clinicians (from April 13 to May 15, 2020) and patients (from May 11 to 15, 2020) to determine technology platforms used, and TN experience and acceptability, using Likert scales (1 = very poor/unlikely to 5 = very good/likely). RESULTS: Over the 4-month period, 117 TN clinicians (n = 14 subspecialties) conducted 4,225 TN visits with 3,717 patients (52 pre- vs 4,173 post-COVID-19). No asynchronous TN services were delivered. Post-COVID-19, the number of TN clinicians, subspecialties performing TN, and visits increased by 963%, 133%, and 7,925%, respectively. Mean acceptability among patients and clinicians was 4.7 (SD 0.6) and 3.4 (SD 1.6), respectively. Most video visits were completed using Epic MyChart (78.5%) and Zoom (8.1%). TN implementation facilitators included Medicare geographic restriction waivers, development of clinician educational materials, and MyChart outreach programs for patients experiencing technical difficulties. CONCLUSIONS: A significant expansion of TN utilization accompanied the COVID-19 response. Patients found TN more acceptable than did clinicians. Proactive application of an implementation framework facilitated rapid and effective TN expansion.