Development of a Pilot Literacy Scale to Assess Knowledge, Attitudes, and Behaviors towards Climate Change and Infectious Disease Dynamics in Suriname.

Prior research has shown that climate literacy is sparse among low- and middle-income countries. Additionally, no standardized questionnaire exists for researchers to measure climate literacy among general populations, particularly with regards to climate change effects on vector-borne diseases (VBDs). We developed a comprehensive literacy scale to assess current knowledge, attitudes, and behaviors towards climate change and VBD dynamics among women enrolled in the Caribbean Consortium for Research in Environmental and Occupational Health (CCREOH) cohort in Suriname. Items were generated by our research team and reviewed by a group of six external climate and health experts. After the expert review, a total of 31 climate change and 21 infectious disease items were retained. We estimated our sample size at a 10:1 ratio of participants to items for each scale. In total, 301 women were surveyed. We validated our scales through exploratory (n = 180) and confirmatory factor analyses (n = 121). An exploratory factor analysis for our general Climate Change Scale provided a four-construct solution of 11 items. Our chi-squared value (X2 = 74.32; p = 0.136) indicated that four factors were sufficient. A confirmatory factor analysis reinforced our findings, providing a good model fit (X2 = 39.03; p = 0.23; RMSEA = 0.015). Our Infectious Disease Scale gave a four-construct solution of nine items (X2 = 153.86; p = 0.094). A confirmatory factor analysis confirmed these results, with a chi-squared value of 19.16 (p = 0.575) and an RMSEA of 0.00. This research is vitally important for furthering climate and health education, especially with increases in VBDs spread by Aedes mosquitoes in the Caribbean, South America, and parts of the southern United States.

Inter-rater reliability of the infectious disease modeling reproducibility checklist (IDMRC) as applied to COVID-19 computational modeling research.

BACKGROUND: Infectious disease computational modeling studies have been widely published during the coronavirus disease 2019 (COVID-19) pandemic, yet they have limited reproducibility. Developed through an iterative testing process with multiple reviewers, the Infectious Disease Modeling Reproducibility Checklist (IDMRC) enumerates the minimal elements necessary to support reproducible infectious disease computational modeling publications. The primary objective of this study was to assess the reliability of the IDMRC and to identify which reproducibility elements were unreported in a sample of COVID-19 computational modeling publications. METHODS: Four reviewers used the IDMRC to assess 46 preprint and peer reviewed COVID-19 modeling studies published between March 13th, 2020, and July 30th, 2020. The inter-rater reliability was evaluated by mean percent agreement and Fleiss' kappa coefficients (κ). Papers were ranked based on the average number of reported reproducibility elements, and average proportion of papers that reported each checklist item were tabulated. RESULTS: Questions related to the computational environment (mean κ = 0.90, range = 0.90-0.90), analytical software (mean κ = 0.74, range = 0.68-0.82), model description (mean κ = 0.71, range = 0.58-0.84), model implementation (mean κ = 0.68, range = 0.39-0.86), and experimental protocol (mean κ = 0.63, range = 0.58-0.69) had moderate or greater (κ > 0.41) inter-rater reliability. Questions related to data had the lowest values (mean κ = 0.37, range = 0.23-0.59). Reviewers ranked similar papers in the upper and lower quartiles based on the proportion of reproducibility elements each paper reported. While over 70% of the publications provided data used in their models, less than 30% provided the model implementation. CONCLUSIONS: The IDMRC is the first comprehensive, quality-assessed tool for guiding researchers in reporting reproducible infectious disease computational modeling studies. The inter-rater reliability assessment found that most scores were characterized by moderate or greater agreement. These results suggest that the IDMRC might be used to provide reliable assessments of the potential for reproducibility of published infectious disease modeling publications. Results of this evaluation identified opportunities for improvement to the model implementation and data questions that can further improve the reliability of the checklist.

Inter-rater reliability of the Infectious Disease Modeling Reproducibility Checklist (IDMRC) as applied to COVID-19 computational modeling research.

Background: Infectious disease computational modeling studies have been widely published during the coronavirus disease 2019 (COVID-19) pandemic, yet they have limited reproducibility. Developed through an iterative testing process with multiple reviewers, the Infectious Disease Modeling Reproducibility Checklist (IDMRC) enumerates the minimal elements necessary to support reproducible infectious disease computational modeling publications. The primary objective of this study was to assess the reliability of the IDMRC and to identify which reproducibility elements were unreported in a sample of COVID-19 computational modeling publications. Methods: Four reviewers used the IDMRC to assess 46 preprint and peer reviewed COVID-19 modeling studies published between March 13th, 2020, and July 31st, 2020. The inter-rater reliability was evaluated by mean percent agreement and Fleiss' kappa coefficients (κ). Papers were ranked based on the average number of reported reproducibility elements, and average proportion of papers that reported each checklist item were tabulated. Results: Questions related to the computational environment (mean κ = 0.90, range = 0.90-0.90), analytical software (mean κ = 0.74, range = 0.68-0.82), model description (mean κ = 0.71, range = 0.58-0.84), model implementation (mean κ = 0.68, range = 0.39-0.86), and experimental protocol (mean κ = 0.63, range = 0.58-0.69) had moderate or greater (κ > 0.41) inter-rater reliability. Questions related to data had the lowest values (mean κ = 0.37, range = 0.23-0.59). Reviewers ranked similar papers in the upper and lower quartiles based on the proportion of reproducibility elements each paper reported. While over 70% of the publications provided data used in their models, less than 30% provided the model implementation. Conclusions: The IDMRC is the first comprehensive, quality-assessed tool for guiding researchers in reporting reproducible infectious disease computational modeling studies. The inter-rater reliability assessment found that most scores were characterized by moderate or greater agreement. These results suggests that the IDMRC might be used to provide reliable assessments of the potential for reproducibility of published infectious disease modeling publications. Results of this evaluation identified opportunities for improvement to the model implementation and data questions that can further improve the reliability of the checklist.