Designing digital patient experiences: The digital health design framework.

BACKGROUND: Digital health (DH) brings considerable benefits, but it comes with potential risks. Human Factors (HF) play a critical role in providing high-quality and acceptable DH solutions. Consultation with designers is crucial for reflecting on and improving current DH design practices. OBJECTIVES: We investigated the general DH design processes, challenges, and corresponding strategies that can improve the digital patient experience (PEx). METHODS: A semi-structured interview study with 24 design professionals. All audio recordings were transcribed, deidentified, grammatically corrected, and imported into ATLAS.ti for data analysis. Three coders participated in data coding following the thematic analysis approach. RESULTS: We identified eight DH design stages and grouped them into four phases: preparation, problem-thinking, problem-solving, and implementation. The analysis presented twelve design challenges associated with contextual, practical, managerial, and commercial aspects that can hinder the design process. We identified eight common strategies used by respondents to tackle these challenges. CONCLUSIONS: We propose a Digital Health Design (DHD) framework to improve the digital PEx. It provides an overview of design deliverables, activities, stakeholders, challenges, and corresponding strategies for each design stage.

Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review.

BACKGROUND: The increasing prevalence of DH applications has outpaced research and practice in digital health (DH) evaluations. Patient experience (PEx) was reported as one of the challenges facing the health system by the World Health Organization. To generate evidence on DH and promote the appropriate integration and use of technologies, a standard evaluation of PEx in DH is required. OBJECTIVE: This study aims to systematically identify evaluation timing considerations (ie, when to measure), evaluation indicators (ie, what to measure), and evaluation approaches (ie, how to measure) with regard to digital PEx. The overall aim of this study is to generate an evaluation guide for further improving digital PEx evaluation. METHODS: This is a 2-phase study parallel to our previous study. In phase 1, literature reviews related to PEx in DH were systematically searched from Scopus, PubMed, and Web of Science databases. Two independent raters conducted 2 rounds of paper screening, including title and abstract screening and full-text screening, and assessed the interrater reliability for 20% (round 1: 23/115 and round 2: 12/58) random samples using the Fleiss-Cohen coefficient (round 1: k1=0.88 and round 2: k2=0.80). When reaching interrater reliability (k>0.60), TW conducted the rest of the screening process, leaving any uncertainties for group discussions. Overall, 38% (45/119) of the articles were considered eligible for further thematic analysis. In phase 2, to check if there were any meaningful novel insights that would change our conclusions, we performed an updated literature search in which we collected 294 newly published reviews, of which 102 (34.7%) were identified as eligible articles. We considered them to have no important changes to our original results on the research objectives. Therefore, they were not integrated into the synthesis of this review and were used as supplementary materials. RESULTS: Our review highlights 5 typical evaluation objectives that serve 5 stakeholder groups separately. We identified a set of key evaluation timing considerations and classified them into 3 categories: intervention maturity stages, timing of the evaluation, and timing of data collection. Information on evaluation indicators of digital PEx was identified and summarized into 3 categories (intervention outputs, patient outcomes, and health care system impact), 9 themes, and 22 subthemes. A set of evaluation theories, common study designs, data collection methods and instruments, and data analysis approaches was captured, which can be used or adapted to evaluate digital PEx. CONCLUSIONS: Our findings enabled us to generate an evaluation guide to help DH intervention researchers, designers, developers, and program evaluators evaluate digital PEx. Finally, we propose 6 directions for encouraging further digital PEx evaluation research and practice to address the challenge of poor PEx.

Increased temporal binding during voluntary motor task under local anesthesia.

Temporal binding refers to a systemic bias in the perceived time interval between two related events, most frequently voluntary motor actions and a subsequent sensory effect. An inevitable component of most instrumental motor actions is tactile feedback. Yet, the role of tactile feedback within this phenomenon remains largely unexplored. Here, we used local anesthesia of the index finger to temporarily inhibit incoming sensory input from the finger itself, while participants performed an interval-estimation task in which they estimated the delay between a voluntary motor action (button press) and a second sensory event (click sound). Results were compared to a control condition with intact sensation. While clear binding was present in both conditions, the effect was significantly enhanced when tactile feedback was temporarily removed via local anesthesia. The results are discussed in light of current debates surrounding the underlying mechanisms and function of this temporal bias.

Pressure sensitivity for head, face and neck in relation to soft tissue.

Pressure sensitivity research on the head, face, and neck is critical to develop ways to reduce discomfort caused by pressure in head-related products. The aim of this paper is to provide information for designers to be able to reduce the pressure discomfort by studying the relation between pressure sensitivity and soft tissue in the head, face and neck. We collected pressure discomfort threshold (PDT) and pressure pain threshold (PPT) from 119 landmarks (unilateral) for 36 Chinese subjects. Moreover, soft tissue thickness data on the head, face and neck regions of 50 Chinese people was obtained through CT scanning while tissue deformation data under the PDT and PPT states was obtained from literature. The results of the three-elements correlation analysis revealed that soft tissue thickness is positively correlated with deformation but not an important factor in pressure sensitivity. Our high-precision pressure sensitivity maps confirm earlier findings of more rough pressure sensitivity studies, while also revealing additional fine scale sensitivity differences. Finally, based on the findings, a high-precision "recommended map" of the optimal stress-bearing area of the head, face and neck was generated.

Digital Patient Experience: Umbrella Systematic Review.

BACKGROUND: The adoption and use of technology have significantly changed health care delivery. Patient experience has become a significant factor in the entire spectrum of patient-centered health care delivery. Digital health facilitates further improvement and empowerment of patient experiences. Therefore, the design of digital health is served by insights into the barriers to and facilitators of digital patient experience (PEx). OBJECTIVE: This study aimed to systematically review the influencing factors and design considerations of PEx in digital health from the literature and generate design guidelines for further improvement of PEx in digital health. METHODS: We performed an umbrella systematic review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We searched Scopus, PubMed, and Web of Science databases. Two rounds of small random sampling (20%) were independently reviewed by 2 reviewers who evaluated the eligibility of the articles against the selection criteria. Two-round interrater reliability was assessed using the Fleiss-Cohen coefficient (k1=0.88 and k2=0.80). Thematic analysis was applied to analyze the extracted data based on a small set of a priori categories. RESULTS: The search yielded 173 records, of which 45 (26%) were selected for data analysis. Findings and conclusions showed a great diversity; most studies presented a set of themes (19/45, 42%) or descriptive information only (16/45, 36%). The digital PEx-related influencing factors were classified into 9 categories: patient capability, patient opportunity, patient motivation, intervention technology, intervention functionality, intervention interaction design, organizational environment, physical environment, and social environment. These can have three types of impacts: positive, negative, or double edged. We captured 4 design constructs (personalization, information, navigation, and visualization) and 3 design methods (human-centered or user-centered design, co-design or participatory design, and inclusive design) as design considerations. CONCLUSIONS: We propose the following definition for digital PEx: "Digital patient experience is the sum of all interactions affected by a patient's behavioral determinants, framed by digital technologies, and shaped by organizational culture, that influence patient perceptions across the continuum of care channeling digital health." In this study, we constructed a design and evaluation framework that contains 4 phases-define design, define evaluation, design ideation, and design evaluation-and 9 design guidelines to help digital health designers and developers address digital PEx throughout the entire design process. Finally, our review suggests 6 directions for future digital PEx-related research.

Design-Relevant Factors Affecting the Patient Experience in Digital Health: Preliminary Results of an Umbrella Systematic Review.

Since Covid-19, digital health interventions (DHIs) have been embraced as never before. The pandemic led to many new challenges, including the patient experience in digital health care delivery. In this literature study, we identified and synthesized factors that impact patient experience in digital health (dPEx), and reviewed the methods and strategies relevant to its design and implementation. We conducted an umbrella review including 15 reviews representing 543 studies. Four themes were identified that describe design-relevant factors that impact dPEx: individual context, content, technical issues, and design features. We propose a preliminary framework to explain the relationship between each factor and support user-centered design efforts. Further research is needed to identify which factors have the most impact.

Game Design in Mental Health Care: Case Study-Based Framework for Integrating Game Design Into Therapeutic Content.

While there has been increasing interest in the use of gamification in mental health care, there is a lack of design knowledge on how elements from games could be integrated into existing therapeutic treatment activities in a manner that is balanced and effective. To help address this issue, we propose a design process framework to support the development of mental health gamification. Based on the concept of experienced game versus therapy worlds, we highlight 4 different therapeutic components that could be gamified to increase user engagement. By means of a Dual-Loop model, designers can balance the therapeutic and game design components and design the core elements of a mental health care gamification. To support the proposed framework, 4 cases of game design in mental health care (eg, therapeutic protocols for addiction, anxiety, and low self-esteem) are presented.

Innovating health care: key characteristics of human-centered design.

Human-centered design is about understanding human needs and how design can respond to these needs. With its systemic humane approach and creativity, human-centered design can play an essential role in dealing with today's care challenges. 'Design' refers to both the process of designing and the outcome of that process, which includes physical products, services, procedures, strategies and policies. In this article, we address the three key characteristics of human-centered design, focusing on its implementation in health care: (1) developing an understanding of people and their needs; (2) engaging stakeholders from early on and throughout the design process; (3) adopting a systems approach by systematically addressing interactions between the micro-, meso- and macro-levels of sociotechnical care systems, and the transition from individual interests to collective interests.

The Need for Sustainable Teleconsultation Systems in the Aftermath of the First COVID-19 Wave.

The physical and social distancing measures that have been adopted worldwide because of COVID-19 will probably remain in place for a long time, especially for senior adults, people with chronic conditions, and other at-risk populations. Teleconsultations can be useful in ensuring that patients continue to receive clinical care while reducing physical crowding and avoiding unnecessary exposure of health care staff. Implementation processes that typically take months of planning, budgeting, pilot testing, and education were compressed into days. However, in the urgency to deal with the present crisis, we may be forgetting that the introduction of digital health is not exclusively a technological issue, but part of a complex organizational change problem. This viewpoint offers insight regarding issues that rapidly adopted teleconsultation systems may face in a post-COVID-19 world.

Communication Preferences in Total Joint Arthroplasty: Exploring the Patient Experience Through Generative Research.

BACKGROUND: Improving communication and information services for people receiving a total joint (knee or hip) arthroplasty (TJA) depends on the differences in patient communication needs and personal characteristics. PURPOSE: The purpose of this study was to further examine individual differences in TJA patient preferences regarding communication and information provision. METHODS: Nineteen patients participated in generative research, which meant they actively reflected on their TJA experiences and communication preferences through creative exercises (e.g., collage making). Audio transcripts of their shared reflections were qualitatively analyzed through an inductive approach. RESULTS: Some participants wanted detailed health education, others did not. Participants also reported different support needs (e.g., at hospital discharge or during rehabilitation). Moreover, participant preferences for social connections with care providers differed. CONCLUSIONS: An individual patient's mindset, his or her social support needs, physical condition, and medical history should guide the provision of tailored services.

Mental Health Therapy Protocols and eHealth Design: Focus Group Study.

BACKGROUND: Electronic health (eHealth) programs are often based on protocols developed for the original face-to-face therapies. However, in practice, therapists and patients may not always follow the original therapy protocols. This form of personalization may also interfere with the intended implementation and effects of eHealth interventions if designers do not take these practices into account. OBJECTIVE: The aim of this explorative study was to gain insights into the personalization practices of therapists and patients using cognitive behavioral therapy, one of the most commonly applied types of psychotherapy, in a youth addiction care center as a case context. METHODS: Focus group discussions were conducted asking therapists and patients to estimate the extent to which a therapy protocol was followed and about the type and reasons for personalization of a given therapy protocol. A total of 7 focus group sessions were organized involving therapists and patients. We used a commonly applied protocol for cognitive behavioral therapy as a therapy protocol example in youth mental health care. The first focus group discussions aimed at assessing the extent to which patients (N=5) or therapists (N=6) adapted the protocol. The second focus group discussions aimed at estimating the extent to which the therapy protocol is applied and personalized based on findings from the first focus groups to gain further qualitative insight into the reasons for personalization with groups of therapists and patients together (N=7). Qualitative data were analyzed using thematic analysis. RESULTS: Therapists used the protocol as a "toolbox" comprising different therapy tools, and personalized the protocol to enhance the therapeutic alliance and based on their therapy-provision experiences. Therapists estimated that they strictly follow 48% of the protocol, adapt 30%, and replace 22% by other nonprotocol therapeutic components. Patients personalized their own therapy to conform the assignments to their daily lives and routines, and to reduce their levels of stress and worry. Patients estimated that 29% of the provided therapy had been strictly followed by the therapist, 48% had been adjusted, and 23% had been replaced by other nonprotocol therapeutic components. CONCLUSIONS: A standard cognitive behavioral therapy protocol is not strictly and fully applied but is mainly personalized. Based on these results, the following recommendations for eHealth designers are proposed to enhance alignment of eHealth to therapeutic practice and implementation: (1) study and copy at least the applied parts of a protocol, (2) co-design eHealth with therapists and patients so they can allocate the components that should be open for user customization, and (3) investigate if components of the therapy protocol that are not applied should remain part of the eHealth applied. To best generate this information, we suggest that eHealth designers should collaborate with therapists, patients, protocol developers, and mental health care managers during the development process.

Validation of Motion Tracking Software for Evaluation of Surgical Performance in Laparoscopic Cholecystectomy.

Motion tracking software for assessing laparoscopic surgical proficiency has been proven to be effective in differentiating between expert and novice performances. However, with several indices that can be generated from the software, there is no set threshold that can be used to benchmark performances. The aim of this study was to identify the best possible algorithm that can be used to benchmark expert, intermediate and novice performances for objective evaluation of psychomotor skills. 12 video recordings of various surgeons were collected in a blinded fashion. Data from our previous study of 6 experts and 23 novices was also included in the analysis to determine thresholds for performance. Video recording were analyzed both by the Kinovea 0.8.15 software and a blinded expert observer using the CAT form. Multiple algorithms were tested to accurately identify expert and novice performances. ½ L + [Formula: see text] A + [Formula: see text] J scoring of path length, average movement and jerk index respectively resulted in identifying 23/24 performances. Comparing the algorithm to CAT assessment yielded in a linear regression coefficient R2 of 0.844. The value of motion tracking software in providing objective clinical evaluation and retrospective analysis is evident. Given the prospective use of this tool the algorithm developed in this study proves to be effective in benchmarking performances for psychomotor skills evaluation.

Evaluation of conventional training in Clinical Breast Examination (CBE).

BACKGROUND: Clinical Breast Examination (CBE) is the examination of a women's breasts by a healthcare professional, such as a breast surgeon, family physician or breast-care nurse who is trained to recognise many different types of abnormalities and warning signs in the breast [1]. CBE is particularly important in rural areas and developing countries who have limited access to technology such as mammography. CBE needs to be taught to health professionals like any other clinical skill used by medical professionals in the workplace. CBE in part involves palpation of the breast, that is, determining by touch which breast lumps are normal and which are suspicious in feeling. The gold standard for assessing tactile skills in CBE is seeing whether students can accurately identify and discriminate between different breast lumps also known as masses (IDBM) on actual patients in a clinical setting. However, this is not practical in a medical education setting. Usually the testing methods 'go through the motions' of feeling the breast as part of CBE. So the students' technique is examined either using unrealistic simulation models or using an intimate examination associate (IEA), an actor/volunteer who permits students to examine their intimate body parts such as breast or genitals for teaching purposes. These volunteers do not have any abnormalities so this teaching does not include the actual detection of suspicious lumps. We undertook a study of clinical skill with 10 medical students to examine different methods of assessing novice student clinical skills after a brief training in CBE. OBJECTIVES: This study aims to evaluate the effectiveness of current training and assessment of novice students in CBE and their capacity to identify and discriminate breast masses (IDBM) on actual patients. METHODS: We assessed each student's IDBM ability in an actual clinical situation, a breast clinic with a mixture of eight IEAs and one real patient with a large, easily palpable, putative breast cancer. We recruited 10 clinically inexperienced medical students, who were trained for 30 minutes by two breast surgeons using an IEA. Students were tested in a simulated clinical setting, a breast clinic where each examined 4 IEAs and one patient. The students were blind to who was the real patient and who was an IEA. Patients were examined by a breast surgeon in private prior to the commencement in the study. The breast surgeon recorded any clinical finding on the patients during the initial examination. The surgeon coached each patient on how to mark the students and showed the patient their results so the patients had a benchmark. After each examination was finished the students had four different assessments: 1) patients marked each student, 2) students were independently proctored - that is, marked by an expert, 3) students recorded their clinical findings and 4) students recorded how confident they were that they had the correct findings. Results from different kinds of student assessments were compared.RESULTSA chi-square test for independence between true positive or negative masses versus student-assessed positive or negative masses was not significant at alpha = .05. This means that there was no statistical association in the indication of positive or negative presence of masses versus whether such masses were actually present or absent. By comparison, experts (breast surgeons) were able to detect normal and abnormal breast masses by palpation alone 100% of the time and rate their confidence level as 'certain'. Unlike the experts, student self-reported confidence was unrelated to their competence score (CS). Proctoring was inversely related to the students' CS.CONCLUSIONSThe main conclusion is that novice students do not seem to be able to accurately detect breast masses in a clinical setting even after training. On the basis of these results, we believe that a comprehension component in the current CBE testing is needed in addition to the current methods of testing.

Breast Cancer Detection: The Development and Pilot Study of a "Tactile Landscape" as a Standardized Testing Tool.

INTRODUCTION: There is still a need for competent breast lump detection palpation skills, especially in developing countries. Our goal is to design, develop, and establish a test to determine whether students can, by touch alone, identify and discriminate between a range of different simulated lesions at different adiposity levels. METHODS: Common lesions, breast cancers, and cysts were physically simulated and hidden in a test object referred to as the "tactile landscape" (TL). Ribs, intercostal muscle, and nodularity-normal anatomical features-increased their realistic complexity. Varying depths of features simulated varying degrees of adiposity. A testing protocol was created to determine the testee's ability to identify and discriminate different commonly occurring breast masses using palpation. Five experts (four breast surgeons and one general practitioner) and 20 inexperienced medical students were recruited and tested. Results were compared. RESULTS: The TL has been based on previously verified breast models and has softness similar to 53% of women's breasts and nodularity similar to 60% as assessed in a breast clinic by breast surgeons. The five experts indicated that the simulated lesions felt like those they might encounter in clinical practice and all of them identified the lesions and nonlesions hidden in the TL 100% correctly, thus indicating the value of the model. In contrast, only one student was able to identify all the lesions. One student identified none of them. The remaining students mean score was 65%. CONCLUSIONS: All students but one performed poorly in comparison to the experts. This indicates that the test could be useful to test students' ability to identify and discriminate breast masses. If successful, it will add previously missing capability to the mix of assessment instruments already used, thus potentially improving clinical breast examination training and assessment.

Developing digital applications for tailored communication in orthopaedics using a Research through Design approach.

OBJECTIVE: Tailored communication and information provision is expected to contribute to patient-centred care (PCC) in total hip arthroplasty (THA). In previous research, three subgroups of THA patients were identified that are similar in their clinical, psychological and communication characteristics. Preliminary subgroup-specific design guidelines were also formulated. Using these insights as a starting point, a theoretical framework was developed for tailored information provision and communication using digital applications. This study aims to refine the framework as well as subgroup-specific design guidelines for digital applications. METHODS: This study uses a Research through Design (RtD) approach, generating insights both from the development and evaluation of prototypes in the early design stage. Paper-based prototypes will be made for each subgroup and evaluated with patients and care providers. Semi-structured interviews are held with participants exploring their experiences with the prototype. A quasi-experiment with a non-random control cohort is used to validate the qualitative findings. Post-surgery consultations with and without prototype are videotaped and scored using a structured instrument. RESULTS: A design diary will be used to summarize design decisions and considerations. Feedback from participants is analysed inductively. Adaptations in subgroup-specific guidelines will be based on comparison of verbal feedback and descriptive statistics from consultations with and without prototype. CONCLUSIONS: Although mixed-method feasibility studies of digital health interventions are common, this protocol also considers the utility of the early design process and the designer's perspective for realizing PCC and tailored care.

A software-based tool for video motion tracking in the surgical skills assessment landscape.

BACKGROUND: The use of motion tracking has been proved to provide an objective assessment in surgical skills training. Current systems, however, require the use of additional equipment or specialised laparoscopic instruments and cameras to extract the data. The aim of this study was to determine the possibility of using a software-based solution to extract the data. METHODS: 6 expert and 23 novice participants performed a basic laparoscopic cholecystectomy procedure in the operating room. The recorded videos were analysed using Kinovea 0.8.15 and the following parameters calculated the path length, average instrument movement and number of sudden or extreme movements. RESULTS: The analysed data showed that experts had significantly shorter path length (median 127 cm vs. 187 cm, p = 0.01), smaller average movements (median 0.40 cm vs. 0.32 cm, p = 0.002) and fewer sudden movements (median 14.00 vs. 21.61, p = 0.001) than their novice counterparts. CONCLUSION: The use of software-based video motion tracking of laparoscopic cholecystectomy is a simple and viable method enabling objective assessment of surgical performance. It provides clear discrimination between expert and novice performance.

"Reflection-Before-Practice" Improves Self-Assessment and End-Performance in Laparoscopic Surgical Skills Training.

OBJECTIVE: To establish whether a systematized approach to self-assessment in a laparoscopic surgical skills course improves accordance between expert- and self-assessment. DESIGN: A systematic training course in self-assessment using Competency Assessment Tool was introduced into the normal course of evaluation within a Laparoscopic Surgical Skills training course for the test group (n = 30). Differences between these and a control group (n = 30) who did not receive the additional training were assessed. SETTING: Catharina Hospital, Eindhoven, The Netherlands (n = 27), and GSL Medical College, Rajahmundry, India (n = 33). PARTICIPANTS: Sixty postgraduate year 2 and 3 surgical residents who attended the 2-day Laparoscopic Surgical Skills grade 1 level 1 curriculum were invited to participate. RESULTS: The test group (n = 30) showed better accordance between expert- and self-assessment (difference of 1.5, standard deviation [SD] = 0.2 versus 3.83, SD = 0.6, p = 0.009) as well as half the number (7 versus 14) of cases of overreporting. Furthermore, the test group also showed higher overall mean performance (mean = 38.1, SD = 0.7 versus mean = 31.8, SD = 1.0, p < 0.001) than the control group (n = 30). The systematic approach to self-assessment can be viewed as responsible for this and can be seen as "reflection-before-practice" within the framework of reflective practice as defined by Donald Schon. CONCLUSION: Our results suggest that "reflection-before-practice" in implementing self-assessment is an important step in the development of surgical skills, yielding both better understanding of one's strengths and weaknesses and also improving overall performance.

Setting-related influences on physical inactivity of older adults in residential care settings: a review.

BACKGROUND: Despite the detrimental effects of physical inactivity for older adults, especially aged residents of residential care settings may spend much time in inactive behavior. This may be partly due to their poorer physical condition; however, there may also be other, setting-related factors that influence the amount of inactivity. The aim of this review was to review setting-related factors (including the social and physical environment) that may contribute to the amount of older adults' physical inactivity in a wide range of residential care settings (e.g., nursing homes, assisted care facilities). METHODS: Five databases were systematically searched for eligible studies, using the key words 'inactivity', 'care facilities', and 'older adults', including their synonyms and MeSH terms. Additional studies were selected from references used in articles included from the search. Based on specific eligibility criteria, a total of 12 studies were included. Quality of the included studies was assessed using the Mixed Methods Appraisal Tool (MMAT). RESULTS: Based on studies using different methodologies (e.g., interviews and observations), and of different quality (assessed quality range: 25-100%), we report several aspects related to the physical environment and caregivers. Factors of the physical environment that may be related to physical inactivity included, among others, the environment's compatibility with the abilities of a resident, the presence of equipment, the accessibility, security, comfort, and aesthetics of the environment/corridors, and possibly the presence of some specific areas. Caregiver-related factors included staffing levels, the available time, and the amount and type of care being provided. CONCLUSIONS: Inactivity levels in residential care settings may be reduced by improving several features of the physical environment and with the help of caregivers. Intervention studies could be performed in order to gain more insight into causal effects of improving setting-related factors on physical inactivity of aged residents.

Using a contextualized sensemaking model for interaction design: A case study of tumor contouring.

Sensemaking theories help designers understand the cognitive processes of a user when he/she performs a complicated task. This paper introduces a two-step approach of incorporating sensemaking support within the design of health information systems by: (1) modeling the sensemaking process of physicians while performing a task, and (2) identifying software interaction design requirements that support sensemaking based on this model. The two-step approach is presented based on a case study of the tumor contouring clinical task for radiotherapy planning. In the first step of the approach, a contextualized sensemaking model was developed to describe the sensemaking process based on the goal, the workflow and the context of the task. In the second step, based on a research software prototype, an experiment was conducted where three contouring tasks were performed by eight physicians respectively. Four types of navigation interactions and five types of interaction sequence patterns were identified by analyzing the gathered interaction log data from those twenty-four cases. Further in-depth study on each of the navigation interactions and interaction sequence patterns in relation to the contextualized sensemaking model revealed five main areas for design improvements to increase sensemaking support. Outcomes of the case study indicate that the proposed two-step approach was beneficial for gaining a deeper understanding of the sensemaking process during the task, as well as for identifying design requirements for better sensemaking support.

Self-assessment in laparoscopic surgical skills training: Is it reliable?

BACKGROUND: The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education. METHODS: Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed. RESULTS: A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the 'tissue handling' having a weaker correlation (r s = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03). CONCLUSION: Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.