Evaluation of the Online Component of a Blended Learning Electroconvulsive Therapy Curriculum for Psychiatry Residents to Treat Depression in Older Adults.

OBJECTIVE: The objective of this study was to evaluate the online component of a blended curriculum for psychiatry residents on the use of electroconvulsive therapy (ECT) to treat depression in older adults. METHODS: Second- and third-year general psychiatry residents completed a blended learning curriculum during their core geriatric psychiatry rotation. The curriculum consisted of didactic seminars, hands-on clinical management, and two online clinical cases focused on the management of late-life depression with ECT. Knowledge acquisition following module completion was measured using a nine-question multiple-choice test. The authors adapted the Medical E-Learning Evaluation Survey (MEES) to measure resident satisfaction, clinical relevance, and instructional design. RESULTS: A total of 37 residents completed both online modules. Of these, 35 residents completed the knowledge test and 23 completed the adapted MEES. Almost all participants (96%) agreed or strongly agreed that the modules were relevant to their clinical work, evidence-based, able to be completed in a reasonable amount of time, and a valuable learning experience. The average score on the knowledge test, after removing one outlier, was 83%. CONCLUSION: Psychiatry residents are very satisfied with the content and delivery of the online component of a blended curriculum for understanding the use of ECT for late-life depression. Future work should examine satisfaction with the remainder of the curricula as well as the impact on longer-term knowledge acquisition and patient care.

Evaluation of an online knowledge translation intervention to promote cancer risk reduction behaviours: findings from a randomized controlled trial.

BACKGROUND: Many cancers are preventable through lifestyle modification; however, few adults engage in behaviors that are in line with cancer prevention guidelines. This may be partly due to the mixed messages on effective cancer prevention strategies in popular media. The goal of the McMaster Optimal Aging Portal (the Portal) is to increase access to trustworthy health information. The purpose of this study was to explore if and how knowledge translation strategies to disseminate cancer prevention evidence using the Portal influence participants' knowledge, intentions and health behaviors related to cancer risk. METHODS: Adults ≥40 years old, with no cancer history were randomized to a 12-week intervention (weekly emails and social media posts) or control group. Quantitative data on knowledge, intentions and behaviors (physical activity, diet, alcohol consumption and use of tobacco products) were collected at baseline, end of study and 3 months later. Participant engagement was assessed using Google Analytics, and participant satisfaction through open-ended survey questions and semi-structured interviews. RESULTS: Participants (n = 557, mean age 64.9) were predominantly retired (72%) females (81%). Knowledge of cancer prevention guidelines was higher in the intervention group at end of study only (+ 0.3, p = 0.01). Intentions to follow cancer prevention guidelines increased in both groups, with no between-group differences. Intervention participants reported greater light-intensity physical activity at end of study (+ 0.7 vs. 0.1, p = 0.03), and reduced alcohol intake at follow u (- 0.2 vs. + 0.3, p < 0.05), but no other between-group differences were found. Overall satisfaction with the Portal and intervention materials was high. CONCLUSIONS: Dissemination of evidence-based cancer prevention information through the Portal results in small increases in knowledge of risk-reduction strategies and with little to no impact on self-reported health behaviours, except in particular groups. Further tailoring of knowledge translation strategies may be needed to see more meaningful change in knowledge and health behaviours. TRIAL REGISTRATION: ClinicalTrials.gov NCT03186703, June 14, 2017.

Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program.

OBJECTIVES: ChatGPT is an artificial intelligence model that can interpret free-text prompts and return detailed, human-like responses across a wide domain of subjects. This study evaluated the extent of the threat posed by ChatGPT to the validity of short-answer assessment problems used to examine pre-clerkship medical students in our undergraduate medical education program. METHODS: Forty problems used in prior student assessments were retrieved and stratified by levels of Bloom's Taxonomy. Thirty of these problems were submitted to ChatGPT-3.5. For the remaining 10 problems, we retrieved past minimally passing student responses. Six tutors graded each of the 40 responses. Comparison of performance between student-generated and ChatGPT-generated answers aggregated as a whole and grouped by Bloom's levels of cognitive reasoning, was done using t-tests, ANOVA, Cronbach's alpha, and Cohen's d. Scores for ChatGPT-generated responses were also compared to historical class average performance. RESULTS: ChatGPT-generated responses received a mean score of 3.29 out of 5 (n = 30, 95% CI 2.93-3.65) compared to 2.38 for a group of students meeting minimum passing marks (n = 10, 95% CI 1.94-2.82), representing higher performance (P = .008, η2 = 0.169), but was outperformed by historical class average scores on the same 30 problems (mean 3.67, P = .018) when including all past responses regardless of student performance level. There was no statistically significant trend in performance across domains of Bloom's Taxonomy. CONCLUSION: While ChatGPT was able to pass short answer assessment problems spanning the pre-clerkship curriculum, it outperformed only underperforming students. We remark that tutors in several cases were convinced that ChatGPT-produced responses were produced by students. Risks to assessment validity include uncertainty in identifying struggling students and inability to intervene in a timely manner. The performance of ChatGPT on problems requiring increasing demands of cognitive reasoning warrants further research.

New directions in e-learning research in health professions education: Report of two symposia.

BACKGROUND: The use of Computer Assisted Instruction (CAI) is rising across health professions education. Research to date is of limited use in guiding the implementation and selection of CAI innovations. AIMS: In the context of two symposia, systemic reviews were discussed that evaluate literature in Internet-based learning, Virtual Patients, and animations. Each session included a debate with the goal of reaching consensus on best current practices and future research. METHODS: Thematic analysis of the discussions was performed to arrange the questions by theme, eliminate redundancy, and craft them into a cohesive narrative. RESULTS: The question analysis revealed that there are clear advantages to the use of CAI, and that established educational theories should certainly inform the future development and selection of CAI tools. Schools adopting CAI need to carefully consider the benefits, cost, available resources, and capacity for teachers and learners to accept change in their practice of education. Potential areas for future research should focus on the effectiveness of CAI instructional features, integration of e-learning into existing curricula and with other modalities like simulation, and the use of CAI in assessment of higher-level outcomes. CONCLUSIONS: There are numerous opportunities for future research and it will be important to achieve consensus on important themes.

Software Architecture for Automated Assessment of Prescription Writing.

Prescribing skills are a crucial competency in medical practice considering the increasing numbers of medications available and the increasingly complex patients with multiple diseases faced in clinical practice. Medical students need to become proficient in these skills during training, as required by medical licensing colleges. Not only is teaching the fundamentals of safe and cost-effective prescribing to medical students challenging but evaluating their prescribing skills by faculty members is difficult and time consuming. The COVID-19 pandemic has accelerated the interest in clinically relevant online exams, including automated assessment of short answer style questions. The goal of this project was to design a software to automate the assessment of learners' prescriptions written during low stakes formative assessments. After establishing the components of a legal prescription with multiple medications, and identifying the sources of errors in prescribing and prescribing assessment, we designed and validated an architecture and developed a prototype for automated parsing of learner prescriptions.

Impact and Perceived Value of iGeriCare e-Learning Among Dementia Care Partners and Others: Pilot Evaluation Using the IAM4all Questionnaire.

BACKGROUND: Care partners of people living with dementia may benefit from web-based education. We developed iGeriCare, an award-winning internet-based platform with 12 multimedia e-learning lessons about dementia. OBJECTIVE: Our objective was to evaluate users' perceptions of impact. METHODS: From March 17, 2021 to May 16, 2022, data were collected upon lesson completion. We used the content-validated Information Assessment Method for all (IAM4all) for patients and the public adapted for dementia care partners. The IAM4all questionnaire assesses outcomes of web-based consumer health information. Responses were collected using SurveyMonkey, and data were analyzed using IBM SPSS Statistics (version 28). RESULTS: A total of 409 responses were collected, with 389 (95.1%) survey respondents completing the survey. Of 409 respondents, 179 (43.8%) identified as a family or friend care partner, 84 (20.5%) identified as an individual concerned they may have mild cognitive impairment or dementia, 380 (92.9%) identified the lesson as relevant or very relevant, and 403 (98.5%) understood the lesson well or very well. Over half of respondents felt they were motivated to learn more, they were taught something new, or they felt validated in what they do, while some felt reassured or felt that the lesson refreshed their memory. Of 409 respondents, 401 (98%) said they would use the information, in particular, to better understand something, discuss the information with someone else, do things differently, or do something. CONCLUSIONS: Users identified iGeriCare as relevant and beneficial and said that they would use the information. To our knowledge, this is the first time the IAM4all questionnaire has been used to assess patient and caregiver feedback on internet-based dementia education resources. A randomized controlled trial to study feasibility and impact on caregiver knowledge, self-efficacy, and burden is in progress.

Method and reporting quality in health professions education research: a systematic review.

CONTEXT: Studies evaluating reporting quality in health professions education (HPE) research have demonstrated deficiencies, but none have used comprehensive reporting standards. Additionally, the relationship between study methods and effect size (ES) in HPE research is unknown. OBJECTIVES: This review aimed to evaluate, in a sample of experimental studies of Internet-based instruction, the quality of reporting, the relationship between reporting and methodological quality, and associations between ES and study methods. METHODS: We conducted a systematic search of databases including MEDLINE, Scopus, CINAHL, EMBASE and ERIC, for articles published during 1990-2008. Studies (in any language) quantifying the effect of Internet-based instruction in HPE compared with no intervention or other instruction were included. Working independently and in duplicate, we coded reporting quality using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, and coded study methods using a modified Newcastle-Ottawa Scale (m-NOS), the Medical Education Research Study Quality Instrument (MERSQI), and the Best Evidence in Medical Education (BEME) global scale. RESULTS: For reporting quality, articles scored a mean±standard deviation (SD) of 51±25% of STROBE elements for the Introduction, 58±20% for the Methods, 50±18% for the Results and 41±26% for the Discussion sections. We found positive associations (all p<0.0001) between reporting quality and MERSQI (ρ=0.64), m-NOS (ρ=0.57) and BEME (ρ=0.58) scores. We explored associations between study methods and knowledge ES by subtracting each study's ES from the pooled ES for studies using that method and comparing these differences between subgroups. Effect sizes in single-group pretest/post-test studies differed from the pooled estimate more than ESs in two-group studies (p=0.013). No difference was found between other study methods (yes/no: representative sample, comparison group from same community, randomised, allocation concealed, participants blinded, assessor blinded, objective assessment, high follow-up). CONCLUSIONS: Information is missing from all sections of reports of HPE experiments. Single-group pre-/post-test studies may overestimate ES compared with two-group designs. Other methodological variations did not bias study results in this sample.

Spiralling Pre-clerkship Concepts into the Clinical Phase: Augmenting Knowledge Transfer Using Innovative Technology-Enhanced Curriculum Activities.

Background: Clerkship is a challenging transition for medical students where they learn to apply functional knowledge and diagnostic reasoning skills learned in the pre-clinical phase into the clinical environment. Rather than a smooth continuum to facilitate application of knowledge, clerkship blocks are discrete, fragmented structures with little integration. Developments in cognitive psychology and increasing attention to the student learning environment are driving more purposeful integration in medical education. We sought to enhance knowledge transfer in the Family Medicine clerkship by developing an e-learning pathway with both asynchronous and synchronous components to integrate pre-clerkship problem-based learning (PBL) cases into more complex clinical scenarios. Methods: A parallel-convergent mixed methods evaluation was conducted, which included comparison of learning outcomes (exit exams) relative to the prior class, knowledge check quiz performance, and qualitative analysis of student and faculty perceptions. Results: Analyses revealed no significant difference between exit exam scores of the intervention and pre-intervention clerks (p = 0.30). There were statistically significant differences in mean quiz scores over the rotation (p = 0.0001). Moreover, learners and faculty each perceived the integration components as facilitating the transfer of pre-clinical learning into clerkship activities. Conclusion: The novel e-learning pathway firmly anchored FM clerkship learning and will continue to ensure learners are ideally primed to optimize their direct clinical learning opportunities.

Online Educational Tools for Caregivers of People with Dementia: A Scoping Literature Review.

BACKGROUND: Informal caregivers of people with dementia provide the majority of health-based care to people with dementia. Providing this care requires knowledge and access to resources, which caregivers often do not receive. We set out to evaluate the effect of online educational tools on informal caregiver self-efficacy, quality of life, burden/stress, depression, and anxiety, and to identify effective processes for online educational tool development. METHODS: We conducted a scoping review of articles on online educational interventions for informal caregivers of people with dementia searching CINAHL, MEDLINE, EMBASE, and PubMed from 1990 to March 2018, with an updated search conducted in 2020. The identified articles were screened and the data were charted. RESULTS: 33 articles that reported on 24 interventions were included. There is some evidence that online interventions improve caregiver-related outcomes such as self-efficacy, depression, dementia knowledge, and quality of life; and decrease caregiver burden. Common findings across the studies included the need for tailored, stage-specific information applicable to the caregiver's situation and the use of psychosocial techniques to develop the knowledge components of the interventions. CONCLUSION: We demonstrate the importance of having caregivers and health-care professionals involved at all stages of tool conceptualization and development. Online tools should be evaluated with robust trials that focus on how increased knowledge and development approaches affect caregiver-related outcomes.

What do we mean by web-based learning? A systematic review of the variability of interventions.

OBJECTIVES: Educators often speak of web-based learning (WBL) as a single entity or a cluster of similar activities with homogeneous effects. Yet a recent systematic review demonstrated large heterogeneity among results from individual studies. Our purpose is to describe the variation in configurations, instructional methods and presentation formats in WBL. METHODS: We systematically searched MEDLINE, EMBASE, ERIC, CINAHL and other databases (last search November 2008) for studies comparing a WBL intervention with no intervention or another educational activity. From eligible studies we abstracted information on course participants, topic, configuration and instructional methods. We summarised this information and then purposively selected and described several WBL interventions that illustrate specific technologies and design features. RESULTS: We identified 266 eligible studies. Nearly all courses (89%) used written text and most (55%) used multimedia. A total of 32% used online communication via e-mail, threaded discussion, chat or videoconferencing, and 9% implemented synchronous components. Overall, 24% blended web-based and non-computer-based instruction. Most web-based courses (77%) employed specific instructional methods, other than text alone, to enhance the learning process. The most common instructional methods (each used in nearly 50% of courses) were patient cases, self-assessment questions and feedback. We describe several studies to illustrate the range of instructional designs. CONCLUSIONS: Educators and researchers cannot treat WBL as a single entity. Many different configurations and instructional methods are available for WBL instructors. Researchers should study when to use specific WBL designs and how to use them effectively.

Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis.

UNLABELLED: Authors have claimed that Internet-based instruction promotes greater learning efficiency than non-computer methods. OBJECTIVES: determine, through a systematic synthesis of evidence in health professions education, how Internet-based instruction compares with non-computer instruction in time spent learning, and what features of Internet-based instruction are associated with improved learning efficiency. DATA SOURCES: we searched databases including MEDLINE, CINAHL, EMBASE, and ERIC from 1990 through November 2008. STUDY SELECTION AND DATA ABSTRACTION we included all studies quantifying learning time for Internet-based instruction for health professionals, compared with other instruction. Reviewers worked independently, in duplicate, to abstract information on interventions, outcomes, and study design. RESULTS: we identified 20 eligible studies. Random effects meta-analysis of 8 studies comparing Internet-based with non-Internet instruction (positive numbers indicating Internet longer) revealed pooled effect size (ES) for time -0.10 (p = 0.63). Among comparisons of two Internet-based interventions, providing feedback adds time (ES 0.67, p =0.003, two studies), and greater interactivity generally takes longer (ES 0.25, p = 0.089, five studies). One study demonstrated that adapting to learner prior knowledge saves time without significantly affecting knowledge scores. Other studies revealed that audio narration, video clips, interactive models, and animations increase learning time but also facilitate higher knowledge and/or satisfaction. Across all studies, time correlated positively with knowledge outcomes (r = 0.53, p = 0.021). CONCLUSIONS: on average, Internet-based instruction and non-computer instruction require similar time. Instructional strategies to enhance feedback and interactivity typically prolong learning time, but in many cases also enhance learning outcomes. Isolated examples suggest potential for improving efficiency in Internet-based instruction.

A Novel Educational Prescription Web-Based Application to Support Education for Caregivers of People Living With Dementia: Development and Usability Study With Clinicians.

BACKGROUND: It is estimated that 564,000 Canadians are currently living with dementia and there are approximately 486,000 to 1.1 million informal family/friend caregivers. Family/friend caregivers often receive little to no education or training about dementia but are expected to provide ongoing support for a complex condition. Web-based family/friend caregiver interventions may be helpful, but little is known about how best to implement them. OBJECTIVE: The objectives of this study were to 1) design and develop a novel education prescription application to help scale and spread web-based dementia education to family/friend caregivers, 2) conduct user testing, and 3) conduct a larger-scale field trial. METHODS: A novel education prescription web-based application was designed and developed. Initial user testing used task completion and the "think aloud" technique with a small sample of representative clinicians who work with people living with dementia and family/friend caregivers. Following iterative incorporation of feedback, a larger field trial was conducted with a convenience sample of clinicians. Account invitations were sent to 55 clinicians and, following a 2-month trial period, surveys were administered to participants including the System Usability Scale and the Net Promoter Score. RESULTS: During the initial user testing phase, participants (N=7) from representative disciplines easily completed associated tasks, and had very positive feedback with respect to the usability of the application. The System Usability Scale score during this phase was 91.4. Suggestions from feedback were incorporated into the application. During the larger field trial phase, participants (total N=55; activated account n=17; did not activate account n=38) were given access to the iGeriCare education prescription application. During this period, 2 participants created educational prescriptions; a total of 3 educational prescriptions were sent. Survey completers who did not activate their account (n=5) identified that their lack of use was due to time constraints, competing priorities, or forgetting to use the application. Survey completers who activated their account (n=5) identified their lower use was due to lack of time, lack of eligible family/friend caregivers during trial period, and competing priorities due to the COVID-19 pandemic. The System Usability Scale score during this phase was 78.75, and the Net Promoter Score was 50. CONCLUSIONS: Study findings indicate a generally positive response for the usability of a web-based application for clinicians to prescribe dementia education to family/friend caregivers. The dissonance between the promising data and widespread enthusiasm for the design and purpose of the education prescription application found in the initial user testing phase and subsequent lack of significant adoption in the field trial represents both an important lesson for other novel health technologies and a potential area for further investigation. Further research is required to better understand factors associated with implementation of this type of intervention and impact on dissemination of education to family/friend caregivers.

Barriers and Facilitators to Implementing Web-Based Dementia Caregiver Education From the Clinician's Perspective: Qualitative Study.

BACKGROUND: Internet-based dementia caregiver interventions have been shown to be effective for a range of caregiver outcomes; however, little is known about how to best implement them. We developed iGeriCare, an evidence-based, multimedia, web-based educational resource for family caregivers of people living with dementia. OBJECTIVE: This study aims to obtain feedback and opinions from experts and clinicians involved in dementia care and caregiver education about 1 iGeriCare and 2 barriers and facilitators to implementing a web-based caregiver program. METHODS: We carried out semistructured interviews with individuals who had a role in dementia care and/or caregiver education in several key stakeholder settings in Southern Ontario, Canada. We queried participants' perceptions of iGeriCare, caregiver education, the implementation process, and their experience with facilitators and barriers. Transcripts were coded and analyzed using a grounded theory approach. The themes that emerged were organized using the Consolidated Framework for Implementation Research. RESULTS: A total of 12 participants from a range of disciplines described their perceptions of iGeriCare and identified barriers and facilitators to the implementation of the intervention. The intervention was generally perceived as a high-quality resource for caregiver education and support, with many stakeholders highlighting the relative advantage of a web-based format. The intervention was seen to meet dementia caregiver needs, partially because of its flexibility, accessibility, and compatibility within existing clinical workflows. In addition, the intervention helps to overcome time constraints for both caregivers and clinicians. CONCLUSIONS: Study findings indicate a generally positive response to the use of internet-based interventions for dementia caregiver education. Results suggest that iGeriCare may be a useful clinical resource to complement traditional face-to-face and print material-based caregiver education. More comprehensive studies are required to identify the effectiveness and longevity of web-based caregiver education interventions and to better understand barriers and facilitators with regard to the implementation of technology-enhanced caregiver educational interventions in various health care settings.

Computer animations in medical education: a critical literature review.

CONTEXT: Animations can depict dynamic changes over time and location, and illustrate phenomena and concepts that might otherwise be difficult to visualise. However, animations may not always be effective and educators who use animations must understand the principles that govern their use. OBJECTIVES: This review aims to illustrate potential applications of animations in medical education, to identify evidence-based principles for their design and use, and to propose an agenda for future research. METHODS: We searched MEDLINE, PsychINFO and EMBASE for articles describing the use of computer animations in medical education. We reviewed and summarised all identified original research studies comparing animations with an alternative computer-based or non-computer-based format. We also selectively reviewed non-medical education research on the use of computer animations. RESULTS: Medical educators have used animations in a variety of computer-assisted learning applications, but few comparative studies have been published and the evidence is inconclusive. Research outside medical education shows conflicting results for studies comparing animations with static images. This may reflect differences in cognitive load induced by animation, or differences in the type of motion being illustrated. The benefits of animations may also vary according to learner characteristics such as prior knowledge and spatial ability. Features of animation that appear to facilitate learning include permitting learner control over the animation's pace, allowing learners to interact with animations and splitting the animation activity into small chunks (segmenting). CONCLUSIONS: Existing medical education research does little to inform the use of animations. Research is needed to confirm and extend non-medicine research to ascertain when to use animations and how to use them effectively.

Internet-based learning in the health professions: a meta-analysis.

CONTEXT: The increasing use of Internet-based learning in health professions education may be informed by a timely, comprehensive synthesis of evidence of effectiveness. OBJECTIVES: To summarize the effect of Internet-based instruction for health professions learners compared with no intervention and with non-Internet interventions. DATA SOURCES: Systematic search of MEDLINE, Scopus, CINAHL, EMBASE, ERIC, TimeLit, Web of Science, Dissertation Abstracts, and the University of Toronto Research and Development Resource Base from 1990 through 2007. STUDY SELECTION: Studies in any language quantifying the association of Internet-based instruction and educational outcomes for practicing and student physicians, nurses, pharmacists, dentists, and other health care professionals compared with a no-intervention or non-Internet control group or a preintervention assessment. DATA EXTRACTION: Two reviewers independently evaluated study quality and abstracted information including characteristics of learners, learning setting, and intervention (including level of interactivity, practice exercises, online discussion, and duration). DATA SYNTHESIS: There were 201 eligible studies. Heterogeneity in results across studies was large (I(2) > or = 79%) in all analyses. Effect sizes were pooled using a random effects model. The pooled effect size in comparison to no intervention favored Internet-based interventions and was 1.00 (95% confidence interval [CI], 0.90-1.10; P < .001; n = 126 studies) for knowledge outcomes, 0.85 (95% CI, 0.49-1.20; P < .001; n = 16) for skills, and 0.82 (95% CI, 0.63-1.02; P < .001; n = 32) for learner behaviors and patient effects. Compared with non-Internet formats, the pooled effect sizes (positive numbers favoring Internet) were 0.10 (95% CI, -0.12 to 0.32; P = .37; n = 43) for satisfaction, 0.12 (95% CI, 0.003 to 0.24; P = .045; n = 63) for knowledge, 0.09 (95% CI, -0.26 to 0.44; P = .61; n = 12) for skills, and 0.51 (95% CI, -0.24 to 1.25; P = .18; n = 6) for behaviors or patient effects. No important treatment-subgroup interactions were identified. CONCLUSIONS: Internet-based learning is associated with large positive effects compared with no intervention. In contrast, effects compared with non-Internet instructional methods are heterogeneous and generally small, suggesting effectiveness similar to traditional methods. Future research should directly compare different Internet-based interventions.

A Tool That Assesses the Evidence, Transparency, and Usability of Online Health Information: Development and Reliability Assessment.

BACKGROUND: The internet is commonly used by older adults to obtain health information and this trend has markedly increased in the past decade. However, studies illustrate that much of the available online health information is not informed by good quality evidence, developed in a transparent way, or easy to use. Furthermore, studies highlight that the general public lacks the skills necessary to distinguish between online products that are credible and trustworthy and those that are not. A number of tools have been developed to assess the evidence, transparency, and usability of online health information; however, many have not been assessed for reliability or ease of use. OBJECTIVE: The first objective of this study was to determine if a tool assessing the evidence, transparency, and usability of online health information exists that is easy and quick to use and has good reliability. No such tool was identified, so the second objective was to develop such a tool and assess it for reliability when used to assess online health information on topics of relevant to optimal aging. METHODS: An electronic database search was conducted between 2002 and 2012 to identify published papers describing tools that assessed the evidence, transparency, and usability of online health information. Papers were retained if the tool described was assessed for reliability, assessed the quality of evidence used to create online health information, and was quick and easy to use. When no one tool met expectations, a new instrument was developed and tested for reliability. Reliability between two raters was assessed using the intraclass correlation coefficient (ICC) for each item at two time points. SPSS Statistics 22 software was used for statistical analyses and a one-way random effects model was used to report the results. The overall ICC was assessed for the instrument as a whole in July 2015. The threshold for retaining items was ICC>0.60 (ie, "good" reliability). RESULTS: All tools identified that evaluated online health information were either too complex, took a long time to complete, had poor reliability, or had not undergone reliability assessment. A new instrument was developed and assessed for reliability in April 2014. Three items had an ICC<0.60 (ie, "good" reliability). One of these items was removed ("minimal scrolling") and two were retained but reworded for clarity. Four new items were added that assessed the level of research evidence that informed the online health information and the tool was retested in July 2015. The total ICC score showed excellent agreement with both single measures (ICC=0.988; CI 0.982-0.992) and average measures (ICC=0.994; CI 0.991-0.996). CONCLUSIONS: The results of this study suggest that this new tool is reliable for assessing the evidence, transparency, and usability of online health information that is relevant to optimal aging.

If You Build It, Who Will Come? A Description of User Characteristics and Experiences With the McMaster Optimal Aging Portal.

Objectives: The McMaster Optimal Aging Portal (the Portal) aims to increase access to evidence-based health information. We would now like to understand who uses the Portal, why, and for what, and elicit feedback and suggestions for future initiatives. Methods: An online survey of users collected data on demographics, eHealth literacy, Internet use, information-seeking behavior, site acceptability and perceived impact on health behaviors, participant satisfaction, and suggestions for improvements using mixed methods. Results: Participants (n = 163, age 69.8 ± 8.6 years) were predominantly female (76%), married (67%), retired (80%), and well-educated with very good/excellent health (55%). The Portal was easy to use (83%) and relevant (80%), with 68% intending to, and 48% having changed behavior after using the Portal. A number of suggestions for improvement were obtained. Discussion: A better understanding of users' characteristics, needs, and preferences will allow us to improve content, target groups who are not engaging with the Portal, and plan future directions.

McMaster Optimal Aging Portal: an evidence-based database for geriatrics-focused health professionals.

OBJECTIVE: The objective of this work was to provide easy access to reliable health information based on good quality research that will help health care professionals to learn what works best for seniors to stay as healthy as possible, manage health conditions and build supportive health systems. This will help meet the demands of our aging population that clinicians provide high quality care for older adults, that public health professionals deliver disease prevention and health promotion strategies across the life span, and that policymakers address the economic and social need to create a robust health system and a healthy society for all ages. RESULTS: The McMaster Optimal Aging Portal's (Portal) professional bibliographic database contains high quality scientific evidence about optimal aging specifically targeted to clinicians, public health professionals and policymakers. The database content comes from three information services: McMaster Premium LiteratUre Service (MacPLUS™), Health Evidence™ and Health Systems Evidence. The Portal is continually updated, freely accessible online, easily searchable, and provides email-based alerts when new records are added. The database is being continually assessed for value, usability and use. A number of improvements are planned, including French language translation of content, increased linkages between related records within the Portal database, and inclusion of additional types of content. While this article focuses on the professional database, the Portal also houses resources for patients, caregivers and the general public, which may also be of interest to geriatric practitioners and researchers.

The value of basic science in clinical diagnosis.

BACKGROUND: The role of basic science knowledge in clinical diagnosis is unclear. There has been no experimental demonstration of its value in helping students recall and organize clinical information. This study examines how causal knowledge may lead to better recall and diagnostic skill over time. METHOD: Undergraduate medical students learned either four neurological or rheumatic disorders. One group learned a basic science explanation for the symptoms. The other learned epidemiological information. Both were then tested with the same set of clinical cases immediately after learning and one week later. RESULTS: On immediate test, there was no difference in accuracy (70% for both groups). However, one week later, performance in the epidemiology group dropped to 51%; the basic science group only dropped to 62%. CONCLUSIONS: Basic science knowledge relating causal knowledge to disease symptoms can improve diagnostic accuracy after a delay.

User Experiences of the McMaster Optimal Aging Portal's Evidence Summaries and Blog Posts: Usability Study.

BACKGROUND: Evidence summaries and blogs can support evidence-informed healthy aging, by presenting high-quality health research evidence in plain language for a nonprofessional (citizen) audience. OBJECTIVE: Our objective was to explore citizens' perceptions about the usability of evidence summaries and blog posts on the Web-based McMaster Optimal Aging Portal. METHODS: Twenty-two citizens (aged 50 years and older) and informal caregivers participated in a qualitative study using a think-aloud method and semistructured interviews. Eleven interviews were conducted in person, 7 over the telephone, and 4 by Skype. RESULTS: We identified themes that fell under 4 user-experience categories: (1) desirability: personal relevance, (2) understandability: language comprehension, grasping the message, dealing with uncertainty, (3) usability: volume of information, use of numbers, and (4) usefulness: intention to use, facility for sharing. CONCLUSIONS: Participants recognized that high-quality evidence on aging was valuable. Their intended use of the information was influenced by how much it applied to their own health circumstances or those of a loved one. Some specific formatting features that were preferred included consistent layout, content organized by subheadings, catchy titles, numerical information summarized in a table, and inclusion of a glossary.