Empowering Health Care Education Through Learning Analytics: In-depth Scoping Review.

BACKGROUND: Digital education has expanded since the COVID-19 pandemic began. A substantial amount of recent data on how students learn has become available for learning analytics (LA). LA denotes the "measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs." OBJECTIVE: This scoping review aimed to examine the use of LA in health care professions education and propose a framework for the LA life cycle. METHODS: We performed a comprehensive literature search of 10 databases: MEDLINE, Embase, Web of Science, ERIC, Cochrane Library, PsycINFO, CINAHL, ICTP, Scopus, and IEEE Explore. In total, 6 reviewers worked in pairs and performed title, abstract, and full-text screening. We resolved disagreements on study selection by consensus and discussion with other reviewers. We included papers if they met the following criteria: papers on health care professions education, papers on digital education, and papers that collected LA data from any type of digital education platform. RESULTS: We retrieved 1238 papers, of which 65 met the inclusion criteria. From those papers, we extracted some typical characteristics of the LA process and proposed a framework for the LA life cycle, including digital education content creation, data collection, data analytics, and the purposes of LA. Assignment materials were the most popular type of digital education content (47/65, 72%), whereas the most commonly collected data types were the number of connections to the learning materials (53/65, 82%). Descriptive statistics was mostly used in data analytics in 89% (58/65) of studies. Finally, among the purposes for LA, understanding learners' interactions with the digital education platform was cited most often in 86% (56/65) of papers and understanding the relationship between interactions and student performance was cited in 63% (41/65) of papers. Far less common were the purposes of optimizing learning: the provision of at-risk intervention, feedback, and adaptive learning was found in 11, 5, and 3 papers, respectively. CONCLUSIONS: We identified gaps for each of the 4 components of the LA life cycle, with the lack of an iterative approach while designing courses for health care professions being the most prevalent. We identified only 1 instance in which the authors used knowledge from a previous course to improve the next course. Only 2 studies reported that LA was used to detect at-risk students during the course's run, compared with the overwhelming majority of other studies in which data analysis was performed only after the course was completed.

Do teaching staff trust stakeholders and tools in learning analytics? A mixed methods study.

Learning analytics (LA) has gained increasing attention for its potential to improve different educational aspects (e.g., students' performance and teaching practice). The existing literature identified some factors that are associated with the adoption of LA in higher education, such as stakeholder engagement and transparency in data use. The broad literature on information systems also emphasizes the importance of trust as a critical predictor of technology adoption. However, the extent to which trust plays a role in the adoption of LA in higher education has not been examined in detail in previous research. To fill this literature gap, we conducted a mixed method (survey and interviews) study aimed to explore how much teaching staff trust LA stakeholders (e.g., higher education institutions or third-parties) and LA technology, as well as the trust factors that could hinder or enable adoption of LA. The findings show that the teaching staff had a high level of trust in the competence of higher education institutions and the usefulness of LA; however, the teaching staff had a low level of trust in third parties that are involved in LA (e.g., external technology vendors) in terms of handling privacy and ethics-related issues. They also had a low level of trust in data accuracy due to issues such as outdated data and lack of data governance. The findings have strategic implications for institutional leaders and third parties in the adoption of LA by providing recommendations to increase trust, such as, improving data accuracy, developing policies for data sharing and ownership, enhancing the consent-seeking process, and establishing data governance guidelines. Therefore, this study contributes to the literature on the adoption of LA in HEIs by integrating trust factors.

Untangling connections between challenges in the adoption of learning analytics in higher education.

Potential benefits of learning analytics (LA) for improving students' performance, predicting students' success, and enhancing teaching and learning practice have increasingly been recognized in higher education. However, the adoption of LA in higher education institutions (HEIs) to date remains sporadic and predominantly small in scale due to several socio-technical challenges. To better understand why HEIs struggle to scale LA adoption, it is needed to untangle adoption challenges and their related factors. This paper presents the findings of a study that sought to investigate the associations of adoption factors with challenges HEIs face in the adoption of LA and how these associations are compared among HEIs at different scopes of adoption. The study was based on a series of semi-structured interviews with senior managers in HEIs. The interview data were thematically analysed to identify the main challenges in LA adoption. The connections between challenges and other factors related to LA adoption were analysed using epistemic network analysis (ENA). From senior managers' viewpoints, ethical issues of informed consent and resistance culture had the strongest links with challenges of learning analytic adoption in HEI; this was especially true for those institutions that had not adopted LA or who were in the initial phase of adoption (i.e., preparing for or partially implementing LA). By contrast, among HEIs that had fully adopted LA, the main challenges were found to be associated with centralized leadership, gaps in the analytic capabilities, external stakeholders, and evaluations of technology. Based on the results, we discuss implications for LA strategy that can be useful for institutions at various stages of LA adoption, from early stages of interest to the full adoption phase.

An exploration into physician and surgeon data sensemaking: a qualitative systematic review using thematic synthesis.

Providing electronic health data to medical practitioners to reflect on their performance can lead to improved clinical performance and quality of care. Understanding the sensemaking process that is enacted when practitioners are presented with such data is vital to ensure an improvement in performance. Thus, the primary objective of this research was to explore physician and surgeon sensemaking when presented with electronic health data associated with their clinical performance. A systematic literature review was conducted to analyse qualitative research that explored physicians and surgeons experiences with electronic health data associated with their clinical performance published between January 2010 and March 2022. Included articles were assessed for quality, thematically synthesised, and discussed from the perspective of sensemaking. The initial search strategy for this review returned 8,829 articles that were screened at title and abstract level. Subsequent screening found 11 articles that met the eligibility criteria and were retained for analyses. Two articles met all of the standards within the chosen quality assessment (Standards for Reporting Qualitative Research, SRQR). Thematic synthesis generated five overarching themes: data communication, performance reflection, infrastructure, data quality, and risks. The confidence of such findings is reported using CERQual (Confidence in the Evidence from Reviews of Qualitative research). The way the data is communicated can impact sensemaking which has implications on what is learned and has impact on future performance. Many factors including data accuracy, validity, infrastructure, culture can also impact sensemaking and have ramifications on future practice. Providing data in order to support performance reflection is not without risks, both behavioural and affective. The latter of which can impact the practitioner's ability to effectively make sense of the data. An important consideration when data is presented with the intent to improve performance.Registration This systematic review was registered with Prospero, registration number: CRD42020197392.

Quality Indicators in the Clinical Specialty of Urology: A Systematic Review.

CONTEXT: In health care, monitoring of quality indicators (QIs) in general urology remains underdeveloped in comparison to other clinical specialties. OBJECTIVE: To identify, synthesise, and appraise QIs that monitor in-hospital care for urology patients. EVIDENCE ACQUISITION: This systematic review included peer-reviewed articles identified via Embase, MEDLINE, Web of Science, CINAHL, Global Health, Google Scholar, and grey literature from 2000 to February 19, 2021. The review was carried out under the Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA) guidelines and used the Appraisal of Indicators through Research and Evaluation (AIRE) tool for quality assessment. EVIDENCE SYNTHESIS: A total of 5111 articles and 62 government agencies were screened for QI sets. There were a total of 57 QI sets included for analysis. Most QIs focused on uro-oncology, with prostate, bladder, and testicular cancers the most represented. The most common QIs were surgical QIs in uro-oncology (positive surgical margin, surgical volume), whereas in non-oncology the QIs most frequently reported were for treatment and diagnosis. Out of 61 articles, only four scored a total of ≥50% on the AIRE tool across four domains. Aside from QIs developed in uro-oncology, general urological QIs are underdeveloped and of poor methodological quality and most lack testing for both content validity and reliability. CONCLUSIONS: There is an urgent need for the development of methodologically robust QIs in the clinical specialty of general urology for patients to enable standardised quality of care monitoring and to improve patient outcomes. PATIENT SUMMARY: We investigated a range of quality indicators (QIs) that provide health care professionals with feedback on the quality of their care for patients with general urological diseases. We found that aside from urological cancers, there is a lack of QIs for general urology. Hence, there is an urgent need for the development of robust and disease-specific QIs in general urology.

Analytics of self-regulated learning scaffolding: effects on learning processes.

Self-regulated learning (SRL) is the ability to regulate cognitive, metacognitive, motivational, and emotional states while learning and is posited to be a strong predictor of academic success. It is therefore important to provide learners with effective instructions to promote more meaningful and effective SRL processes. One way to implement SRL instructions is through providing real-time SRL scaffolding while learners engage with a task. However, previous studies have tended to focus on fixed scaffolding rather than adaptive scaffolding that is tailored to student actions. Studies that have investigated adaptive scaffolding have not adequately distinguished between the effects of adaptive and fixed scaffolding compared to a control condition. Moreover, previous studies have tended to investigate the effects of scaffolding at the task level rather than shorter time segments-obscuring the impact of individual scaffolds on SRL processes. To address these gaps, we (a) collected trace data about student activities while working on a multi-source writing task and (b) analyzed these data using a cutting-edge learning analytic technique- ordered network analysis (ONA)-to model, visualize, and explain how learners' SRL processes changed in relation to the scaffolds. At the task level, our results suggest that learners who received adaptive scaffolding have significantly different patterns of SRL processes compared to the fixed scaffolding and control conditions. While not significantly different, our results at the task segment level suggest that adaptive scaffolding is associated with earlier engagement in SRL processes. At both the task level and task segment level, those who received adaptive scaffolding, compared to the other conditions, exhibited more task-guided learning processes such as referring to task instructions and rubrics in relation to their reading and writing. This study not only deepens our understanding of the effects of scaffolding at different levels of analysis but also demonstrates the use of a contemporary learning analytic technique for evaluating the effects of different kinds of scaffolding on learners' SRL processes.

Transmission of SARS-CoV-2 in a primary school setting with and without public health measures using real-world contact data: A modelling study.

Background: Stringent public health measures have been shown to influence the transmission of SARS-CoV-2 within school environments. We investigated the potential transmission of SARS-CoV-2 in a primary school setting with and without public health measures, using fine-grained physical positioning traces captured before the COVID-19 pandemic. Methods: Approximately 172.63 million position data from 98 students and six teachers from an open-plan primary school were used to predict a potential transmission of SARS-CoV-2 in primary school settings. We first estimated the daily average number of contacts of students and teachers with an infected individual during the incubation period. We then used the Reed-Frost model to estimate the probability of transmission per contact for the SARS-CoV-2 Alpha (B.1.1.7), Delta (B.1.617.2), and Omicron variant (B.1.1.529). Finally, we built a binomial distribution model to estimate the probability of onward transmission in schools with and without public health measures, including face masks and physical distancing. Results: An infectious student would have 49.1 (95% confidence interval (CI) = 46.1-52.1) contacts with their peers and 2.00 (95% CI = 1.82-2.18) contacts with teachers per day. An infectious teacher would have 47.6 (95% CI = 45.1-50.0) contacts with students and 1.70 (95% CI = 1.48-1.92) contacts with their colleague teachers per day. While the probability of onward SARS-CoV-2 transmission was relatively low for the Alpha and Delta variants, the risk increased for the Omicron variant, especially in the absence of public health measures. Onward teacher-to-student transmission (88.9%, 95% CI = 88.6%-89.1%) and teacher-to-teacher SARS-CoV-2 transmission (98.4%, 95% CI = 98.5%-98.6%) were significantly higher for the Omicron variant without public health measures in place. Conclusions: Our findings illustrate that, despite a lower frequency of close contacts, teacher-to-teacher close contacts demonstrated a higher risk of transmission per contact of SARS-CoV-2 compared to student-to-student close contacts. This was especially significant with the Omicron variant, with onward transmission more likely occurring from teacher index cases than student index cases. Public health measures (eg, face masks and physical distance) seem essential in reducing the risk of onward transmission within school environments.

Electronic Health Records That Support Health Professional Reflective Practice: a Missed Opportunity in Digital Health.

A foundational component of digital health involves collecting and leveraging electronic health data to improve health and wellbeing. One of the central technologies for collecting these data are electronic health records (EHRs). In this commentary, the authors explore intersection between digital health and data-driven reflective practice that is described, including an overview of the role of EHRs underpinning technology innovation in healthcare. Subsequently, they argue that EHRs are a rich but under-utilised source of information on the performance of health professionals and healthcare teams that could be harnessed to support reflective practice and behaviour change. EHRs currently act as systems of data collection, not systems of data engagement and reflection by end users such as health professionals and healthcare organisations. Further consideration should be given to supporting reflective practice by health professionals in the design of EHRs and other clinical information systems.

Exploring the Intersection Between Health Professionals' Learning and eHealth Data: Protocol for a Comprehensive Research Program in Practice Analytics in Health Care.

BACKGROUND: There is an increasing amount of electronic data sitting within the health system. These data have untapped potential to improve clinical practice if extracted efficiently and harnessed to change the behavior of health professionals. Furthermore, there is an increasing expectation from the government and peak bodies that both individual health professionals and health care organizations will use electronic data for a range of applications, including improving health service delivery and informing clinical practice and professional accreditation. OBJECTIVE: The aim of this research program is to make eHealth data captured within tertiary health care organizations more actionable to health professionals for use in practice reflection, professional development, and other quality improvement activities. METHODS: A multidisciplinary approach was used to connect academic experts from core disciplines of health and medicine, education and learning sciences, and engineering and information communication technology with government and health service partners to identify key problems preventing the health care industry from using electronic data to support health professional learning. This multidisciplinary approach was used to design a large-scale research program to solve the problem of making eHealth data more accessible to health professionals for practice reflection. The program will be delivered over 5 years by doctoral candidates undertaking research projects with discrete aims that run in parallel to achieving this program's objectives. RESULTS: The process used to develop the research program identified 7 doctoral research projects to answer the program objectives, split across 3 streams. CONCLUSIONS: This research program has the potential to successfully unpack electronic data siloed within clinical sites and enable health professionals to use them to reflect on their practice and deliver informed and improved care. The program will contribute to current practices by fostering stronger connections between industry and academia, interlinking doctoral research projects to solve complex problems, and creating new knowledge for clinical sites on how data can be used to understand and improve performance. Furthermore, the program aims to affect policy by developing insights on how professional development programs may be strengthened to enhance their alignment with clinical practice. The key contributions of this paper include the introduction of a new conceptualized research program, Practice Analytics in Health care, by describing the foundational academic disciplines that the program is formed of and presenting scientific methods for its design and development. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/27984.

Students' experience of online learning during the COVID-19 pandemic: A province-wide survey study.

Online learning is currently adopted by educational institutions worldwide to provide students with ongoing education during the COVID-19 pandemic. Even though online learning research has been advancing in uncovering student experiences in various settings (i.e., tertiary, adult, and professional education), very little progress has been achieved in understanding the experience of the K-12 student population, especially when narrowed down to different school-year segments (i.e., primary and secondary school students). This study explores how students at different stages of their K-12 education reacted to the mandatory full-time online learning during the COVID-19 pandemic. For this purpose, we conducted a province-wide survey study in which the online learning experience of 1,170,769 Chinese students was collected from the Guangdong Province of China. We performed cross-tabulation and Chi-square analysis to compare students' online learning conditions, experiences, and expectations. Results from this survey study provide evidence that students' online learning experiences are significantly different across school years. Foremost, policy implications were made to advise government authorises and schools on improving the delivery of online learning, and potential directions were identified for future research into K-12 online learning.

Applications of Learning Analytics in High Schools: A Systematic Literature Review.

Learning analytics aims to analyze data from students and learning environments to support learning at different levels. Although learning analytics is a recent field, it reached a high level of maturity, especially in its applications for higher education. However, little of the research in learning analytics targets other educational levels, such as high school. This paper reports the results of a systematic literature review (SLR) focused on the adoption of learning analytics in high schools. More specifically, the SLR followed four steps: the search, selection of relevant studies, critical assessment, and the extraction of the relevant field, which included the main goals, approaches, techniques, and challenges of adopting learning analytics in high school. The results show that, in this context, learning analytics applications are focused on small-scale initiatives rather than institutional adoption. Based on the findings of this study, in combination with the literature, this paper proposes future directions of research and development in order to scale up learning analytics applications in high schools.

Temporal Assessment of Self-Regulated Learning by Mining Students' Think-Aloud Protocols.

It has been widely theorized and empirically proven that self-regulated learning (SRL) is related to more desired learning outcomes, e.g., higher performance in transfer tests. Research has shifted to understanding the role of SRL during learning, such as the strategies and learning activities, learners employ and engage in the different SRL phases, which contribute to learning achievement. From a methodological perspective, measuring SRL using think-aloud data has been shown to be more insightful than self-report surveys as it helps better in determining the link between SRL activities and learning achievements. Educational process mining on the basis of think-aloud data enables a deeper understanding and more fine-grained analyses of SRL processes. Although students' SRL is highly contextualized, there are consistent findings of the link between SRL activities and learning outcomes pointing to some consistency of the processes that support learning. However, past studies have utilized differing approaches which make generalization of findings between studies investigating the unfolding of SRL processes during learning a challenge. In the present study with 29 university students, we measured SRL via concurrent think-aloud protocols in a pre-post design using a similar approach from a previous study in an online learning environment during a 45-min learning session, where students learned about three topics and wrote an essay. Results revealed significant learning gain and replication of links between SRL activities and transfer performance, similar to past research. Additionally, temporal structures of successful and less successful students indicated meaningful differences associated with both theoretical assumptions and past research findings. In conclusion, extending prior research by exploring SRL patterns in an online learning setting provides insights to the replicability of previous findings from online learning settings and new findings show that it is important not only to focus on the repertoire of SRL strategies but also on how and when they are used.