Using brief reflections to capture and evaluate end-user engagement: a case example using the COMPASS study.

BACKGROUND: Use of participatory research methods is increasing in research trials. Once partnerships are established with end-users, there is less guidance about processes research teams can use to successfully incorporate end-user feedback. The current study describes the use of a brief reflections process to systematically examine and evaluate the impact of end-user feedback on study conduct. METHODS: The Comparative Effectiveness of Trauma-Focused and Non-Trauma- Focused Treatment Strategies for PTSD among those with Co-Occurring SUD (COMPASS) study was a randomized controlled trial to determine the effectiveness of trauma-focused psychotherapy versus non-trauma-focused psychotherapy for Veterans with co-occurring posttraumatic stress disorder and substance use disorder who were entering substance use treatment within the Department of Veterans Affairs. We developed and paired a process of "brief reflections" with our end-user engagement methods as part of a supplemental evaluation of the COMPASS study engagement plan. Brief reflections were 30-minute semi-structured discussions with the COMPASS Team following meetings with three study engagement panels about feedback received regarding study issues. To evaluate the impact of panel feedback, 16 reflections were audio-recorded, transcribed, rapidly analyzed, and integrated with other study data sources. RESULTS: Brief reflections revealed that the engagement panels made recommended changes in eight areas: enhancing recruitment; study assessment completion; creating uniformity across Study Coordinators; building Study Coordinator connection to Veteran participants; mismatch between study procedures and clinical practice; therapist skill with patients with active substance use; therapist burnout; and dissemination of study findings. Some recommendations positively impact study conduct while others had mixed impact. Reflections were iterative and led to emergent processes that included revisiting previously discussed topics, cross-pollination of ideas across panels, and sparking solutions amongst the Team when the panels did not make any recommendations or recommendations were not feasible. CONCLUSIONS: When paired with end-user engagement methods, brief reflections can facilitate systematic examination of end-user input, particularly when the engagement strategy is robust. Reflections offer a forum of accountability for researchers to give careful thought to end-user recommendations and make timely improvements to the study conduct. Reflections can also facilitate evaluation of these recommendations and reveal end-user-driven strategies that can effectively improve study conduct. TRIAL REGISTRATION: ClinicalTrials.gov (NCT04581434) on October 9, 2020; https://clinicaltrials.gov/ct2/show/study/NCT04581434?term=NCT04581434&draw=2&rank=1 .

Methods, Indicators, and End-User Involvement in the Evaluation of Digital Health Interventions for the Public: Scoping Review.

BACKGROUND: Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge. OBJECTIVE: This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview. METHODS: This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias. RESULTS: The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated. CONCLUSIONS: The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.

Challenges and Alternatives to Evaluation Methods and Regulation Approaches for Medical Apps as Mobile Medical Devices: International and Multidisciplinary Focus Group Discussion.

BACKGROUND: The rapid proliferation of medical apps has transformed the health care landscape by giving patients and health care providers unprecedented access to personalized health information and services. However, concerns regarding the effectiveness and safety of medical apps have raised questions regarding the efficacy of randomized controlled trials (RCTs) in the evaluation of such apps and as a requirement for their regulation as mobile medical devices. OBJECTIVE: This study aims to address this issue by investigating alternative methods, apart from RCTs, for evaluating and regulating medical apps. METHODS: Using a qualitative approach, a focus group study with 46 international and multidisciplinary public health experts was conducted at the 17th World Congress on Public Health in May 2023 in Rome, Italy. The group was split into 3 subgroups to gather in-depth insights into alternative approaches for evaluating and regulating medical apps. We conducted a policy analysis on the current regulation of medical apps as mobile medical devices for the 4 most represented countries in the workshop: Italy, Germany, Canada, and Australia. We developed a logic model that combines the evaluation and regulation domains on the basis of these findings. RESULTS: The focus group discussions explored the strengths and limitations of the current evaluation and regulation methods and identified potential alternatives that could enhance the quality and safety of medical apps. Although RCTs were only explicitly mentioned in the German regulatory system as one of many options, an analysis of chosen evaluation methods for German apps on prescription pointed toward a "scientific reflex" where RCTs are always the chosen evaluation method. However, this method has substantial limitations when used to evaluate digital interventions such as medical apps. Comparable results were observed during the focus group discussions, where participants expressed similar experiences with their own evaluation approaches. In addition, the participants highlighted numerous alternatives to RCTs. These alternatives can be used at different points during the life cycle of a digital intervention to assess its efficacy and potential harm to users. CONCLUSIONS: It is crucial to recognize that unlike analog tools, digital interventions constantly evolve, posing challenges to inflexible evaluation methods such as RCTs. Potential risks include high dropout rates, decreased adherence, and nonsignificant results. However, existing regulations do not explicitly advocate for other evaluation methodologies. Our research highlighted the necessity of overcoming the gap between regulatory demands to demonstrate safety and efficacy of medical apps and evolving scientific practices, ensuring that digital health innovation is evaluated and regulated in a way that considers the unique characteristics of mobile medical devices.

Can self-assessment ability predict student performance in a dental anatomy course?

INTRODUCTION: The development of early self-assessment skills and determining its correlation with academic performance could provide evidence to alter dental curricula. The aim of this retrospective study was to examine the relationships between students' early self-assessment ability in waxing and three different evaluation methods (waxing assessment, written examinations, and tooth identification examination) in a dental anatomy course. MATERIALS AND METHODS: The dental anatomy scores during the academic years of 2018-2019 and 2019-2020 from two cohorts of second-year pre-doctoral dental students at the Harvard School of Dental Medicine were analyzed. Regression analyses were completed to assess the relationship between all evaluation methods. RESULTS: There was a statistically significant correlation between self-assessment ability and the waxing assessment but no significant correlation between self-assessment ability and the other evaluation methods. CONCLUSION: Our results showed that the introduction of self-assessments in dental anatomy waxing was correlated with successful waxing skills. Furthermore, a relevant finding is that students who received higher classifications were also capable of doing better self-assessments. These findings provide evidence that impacts dental curricula.

Use of a multi-method approach to rapidly assess the impact of public health policies at the state and local level: a case study of flavored e-cigarette policies.

BACKGROUND: E-cigarettes are the most-commonly used tobacco product by youth since 2014. To prevent youth access and use of e-cigarettes, many U.S. states and localities have enacted policies over a relatively short period of time. The adoption of these policies has necessitated timely data collection to evaluate impacts. METHODS: To assess the impact of flavored e-cigarette policies in select states and local jurisdictions across the United States, a multi-method, complementary approach was implemented from July 2019 to present, which includes analyses of cross-sectional online surveys of young people ages 13-24 years with retail sales data. RESULTS: From February 2020 through February 2023, cross-sectional surveys have been conducted in three cities, one county, and eight states where policy changes have been enacted or are likely to be enacted. Data collection occurred every six months to provide near real-time data and examine trends over time. Additionally, weekly retail sales data were aggregated to showcase monthly sales trends at the national level and for the selected states. DISCUSSION: This rapid and efficient method of coupling online survey data with retail sales data provides a timely and effective approach for monitoring a quickly changing tobacco product landscape, particularly for states and localities where rapidly-available data is often not available. This approach can also be used to monitor other health behaviors and relevant policy impacts.

Evaluation Methods and Measurement Challenges for Industrial Exoskeletons.

In recent years, exoskeleton test methods for industrial exoskeletons have evolved to include simulated laboratory and field environments. Physiological, kinematic, and kinetic metrics, as well as subjective surveys, are used to evaluate exoskeleton usability. In particular, exoskeleton fit and usability can also impact the safety of exoskeletons and their effectiveness at reducing musculoskeletal injuries. This paper surveys the state of the art in measurement methods applied to exoskeleton evaluation. A notional classification of the metrics based on exoskeleton fit, task efficiency, comfort, mobility, and balance is proposed. In addition, the paper describes the test and measurement methods used in supporting the development of exoskeleton and exosuit evaluation methods to assess their fit, usability, and effectiveness in industrial tasks such as peg in hole, load align, and applied force. Finally, the paper includes a discussion of how the metrics can be applied towards a systematic evaluation of industrial exoskeletons, current measurement challenges, and future research directions.

Comparison Research on Characterization and Evaluation Approaches for Paint Coated Corrosion Using Eddy Current Pulsed Thermography.

Paint coated corrosion detection and evaluation is a big challenge for steel performance and structure health. Eddy current pulsed thermography (ECPT) technique is investigated because it can reflect the corrosion physical properties through paint coating by the infrared signal. This paper proposes skewness method, which presents the feature of temperature curve's shape automatically, and compares it with principal component analysis (PCA), phase analysis, and kurtosis feature extraction methods for paint coated corrosion characterization and evaluation. The averaged skewness shows the best sensitivity for 0-6 months corrosion. The normalized second principal component (PC) presents good sensitivity and the best measurement scale for corroded time. Furthermore, the temperature curve analysis proves that the electrical conductivity dominates the induced heating and heat distribution. The corrosion height is utilized to explain why ECPT technique is valid within 10 months corroded time. ECPT technique is proved as a smart sensor system for paint coated corrosion detection and characterization.

The Bioavailability of Drugs-The Current State of Knowledge.

Drug bioavailability is a crucial aspect of pharmacology, affecting the effectiveness of drug therapy. Understanding how drugs are absorbed, distributed, metabolized, and eliminated in patients' bodies is essential to ensure proper and safe treatment. This publication aims to highlight the relevance of drug bioavailability research and its importance in therapy. In addition to biochemical activity, bioavailability also plays a critical role in achieving the desired therapeutic effects. This may seem obvious, but it is worth noting that a drug can only produce the expected effect if the proper level of concentration can be achieved at the desired point in a patient's body. Given the differences between patients, drug dosages, and administration forms, understanding and controlling bioavailability has become a priority in pharmacology. This publication discusses the basic concepts of bioavailability and the factors affecting it. We also looked at various methods of assessing bioavailability, both in the laboratory and in the clinic. Notably, the introduction of new technologies and tools in this field is vital to achieve advances in drug bioavailability research. This publication also discusses cases of drugs with poorly described bioavailability, providing a deeper understanding of the complex challenges they pose to medical researchers and practitioners. Simultaneously, the article focuses on the perspectives and trends that may shape the future of research regarding bioavailability, which is crucial to the development of modern pharmacology and drug therapy. In this context, the publication offers an essential, meaningful contribution toward understanding and highlighting bioavailability's role in reliable patient treatment. The text also identifies areas that require further research and exploration.

Implementation of Overdose Prevention in Maryland: Implications for Resource Allocation, Program Scale-Up, and Evaluation.

Overdose mortality in the United States continues to climb, with Maryland being one of the hardest hit states. We summarized implementation of overdose prevention and response programs in Maryland and identified associations between opioid overdose deaths by jurisdiction in 2019 and implementation of overdose programs by 2021. Data on program implementation are from Maryland's Opioid Operational Command Center (OOCC) Program Inventory. OOCC coordinates the state's response to overdose, and their Program Inventory tracks implementation of 145 programs across 12 domains (e.g., public health, education, and judiciary), including 10 programs designed to broaden naloxone access. The level of program implementation was dichotomized as substantial implementation versus other levels (i.e., partial, planned, and none). We estimated associations between per capita opioid overdose deaths and substantial implementation of: all 145 programs in the Inventory, programs within each of 12 domains, and 10 naloxone programs. Data on program implementation and overdose mortality are summarized at the jurisdiction level. Across jurisdictions, the median proportion of programs with substantial implementation was 51% across all programs and 70% among naloxone programs. Overdose mortality was associated with subsequent substantial implementation of programs within the public health domain (p = .04), but not in the other 11 domains. We did not find evidence that per capita overdose deaths in 2019 spurred overdose program implementation by 2021, with the exception of public health programs. The OOCC Program Inventory is a novel way to track implementation across jurisdictions. Findings can inform the implementation and evaluation of overdose programs in other jurisdictions across the United States.

The Role of Thermodynamic and Informational Entropy in Improving Real Estate Valuation Methods.

Price, Cost and Income (PCI) are distinct economic indicators intrinsically linked to the values they denote. These observables take center stage in the multi-criteria decision-making process that enables economic agents to convey subjective utilities of market-exchanged commodities objectively. The valuation of these commodities heavily relies on PCI-based empirical observables and their supported methodologies. This valuation measure's accuracy is critical, as it influences subsequent decisions within the market chain. However, measurement errors often arise due to inherent uncertainties in the value state, impacting economic agents' wealth, particularly when trading significant commodities such as real estate properties. This paper addresses this issue by incorporating entropy measurements into real estate valuation. This mathematical technique adjusts and integrates triadic PCI estimates, improving the final stage of appraisal systems where definitive value decisions are crucial. Employing entropy within the appraisal system can also aid market agents in devising informed production/trading strategies for optimal returns. The results from our practical demonstration indicate promising implications. The entropy's integration with PCI estimates significantly improved the value measurement's precision and reduced economic decision-making errors.

[Interaction assessment of polysaccharides and minor bioactive compounds in functional food ingredients of plant origin].

At present, the scientific based view of creation enriched, specialized and functional products based on bioactive compounds (BAC) of plant origin has been formed. Interactions between polysaccharides (hydrocolloids), macronutrients of the food system and minor BAC are a determining factor in their bioavailability and should be taken into account when developing formulations and evaluated accordingly. The objective of the research was to consider the theoretical aspects of the interaction of polysaccharides and minor BAC in functional food ingredients of plant origin, as well as to provide an overview of currently available methods for their evaluation. Material and methods. The search and analysis of publications were carried out using the eLIBRARY, PubMed, Scopus, Web of Science databases, mainly in the last 10 years. Results. The main interaction mechanisms of the polysaccharides with minor BAC were determined using the example of the components of the polyphenol complex (flavonoids), ecdysteroids. These include: adsorption, the formation of an "inclusion complex", hydrogen bonding between OH-groups. The interaction of BAC with other macromolecules can occur with their significant modification as a result of the formation of complexes and cause a decrease in biological activity. The assessment of the degree of interaction of hydrocolloids with minor BAC can be carried out using both in vitro and in vivo methods. Most of these studies are carried out in vitro, do not take into account many factors that affect the bioavailability of BAC. Thus, it can be noted that, despite significant progress in the development of functional food ingredients based on medicinal plant materials, the studies of the interactions of BAC with polysaccharides using relevant models are not currently carried out to the extent necessary. Conclusion. Based on the data presented in the review, it can be concluded that plant polysaccharides (hydrocolloids) have a significant effect on the biological activity and availability of minor BAC (polyphenols, ecdysteroids). As an optimal technique for a preliminary assessment of the degree of interaction, it is recommended to use a model that includes the main enzymatic systems, which allows you to accurately reproduce the processes occurring in the gastrointestinal tract; at the final stage, it is necessary to confirm the biological activity in vivo.

[Introduction and Reflection on Novel Medical Device Regulatory Science Tool MDDT].

In recent years, emerging technology medical devices have developed rapidly. How to more scientifically and more efficiently regulate these novel medical devices so as to improve access to advanced medical technology while ensuring safety and effectiveness is a new challenge faced by regulatory authorities, and is also the core topic of regulatory science. New tools, new standards and new methods are important means to achieve regulatory science. "Medical Device Development Tool" proposed by the U.S. FDA is a novel medical device regulatory science tool, which can help medical device developers to predict and evaluate product performance more efficiently. It is also helpful for regulatory authorities to make regulatory decisions more efficiently. This study introduces the concept, qualification process, role of MDDT in medical device regulation and MDDT examples, and makes some discussion on the device evaluation from the perspective of reliability and validity. MDDT can facilitate the developing of novel medical device.

Evaluation of methods used to verify nasogastric feeding tube placement in hospitalized infants and children - A follow-up study.

BACKGROUND: Nasogastric (NG) feeding tubes are used to deliver nutrition, hydration, and medications to hospitalized infants and children but the ongoing use of non-evidence-based practice (EBP) methods to confirm NG tube (NGT) placement has been associated with adverse patient events. METHODS: A study was undertaken to ascertain if practice changes have occurred since findings from a previous study were published by the New Opportunities for Verification of Enteral tube Location (NOVEL) project. The NOVEL project was an initiative of the American Society of Parenteral and Enteral Nutrition (ASPEN). A survey was distributed to member organizations participating in the NOVEL project. Respondents were also asked if and when a change in practice occurred in the policy for NGT placement verification, if there was variation within the institutional units and if there were barriers to practice change. FINDINGS: Respondents were primarily nurses (205/245) from 166 institutions that provided care to combined adult/pediatric/neonatal (122/166) patients. Respondents indicated a radiograph (64%) or pH measurement (24%) were best practice but in actual practice 42% use pH measurement and 23% use a radiograph to verify NGT placement. There was variability within institutions, with the Neonatal Intensive Care Unit (NICU) most often using aspiration and direct eye visualization to verify placement and the other units within the institutions using EBP method(s). DISCUSSION: Comparing these results to previous work by the NOVEL project shows an increase toward the use of EBP method(s) to verify NGT placement verification. APPLICATION TO PRACTICE: This study demonstrates variation within units at the same facility using methods unsupported by the literature, demonstrating that many centers still rely on non-EBP methods of NG placement confirmation, despite cautions issued by many major healthcare organizations.

Clinical Relevancy of Circulating Tumor Cells in Breast Cancer: Epithelial or Mesenchymal Characteristics, Single Cells or Clusters?

Metastatic breast cancer (MBC) is typically an incurable disease with high mortality rates; thus, early identification of metastatic features and disease recurrence through precise biomarkers is crucial. Circulating tumor cells (CTCs) consisting of heterogeneous subpopulations with different morphology and genetic, epigenetic, and gene expression profiles represent promising candidate biomarkers for metastatic potential. The experimentally verified role of epithelial-to-mesenchymal transition in cancer dissemination has not been clearly described in BC patients, but the stemness features of CTCs strongly contributes to metastatic potency. Single CTCs have been shown to be protected in the bloodstream against recognition by the immune system through impaired interactions with T lymphocytes and NK cells, while associations of heterotypic CTC clusters with platelets, leucocytes, neutrophils, tumor-associated macrophages, and fibroblasts improve their tumorigenic behavior. In addition to single CTC and CTC cluster characteristics, we reviewed CTC evaluation methods and clinical studies in early and metastatic BCs. The variable CTC tests were developed based on specific principles and strategies. However, CTC count and the presence of CTC clusters were shown to be most clinically relevant in existing clinical trials. Despite the known progress in CTC research and sampling of BC patients, implementation of CTCs and CTC clusters in routine diagnostic and treatment strategies still requires improvement in detection sensitivity and precise molecular characterizations, focused predominantly on the role of CTC clusters for their higher metastatic potency.

Pre-location for temporary distribution station of urban emergency materials considering priority under COVID-19: A case study of Wuhan City, China.

In order to avoid the huge hidden dangers caused by emergencies, it is particularly vital to make a reasonable pre-location and layout of emergency logistics facilities. A multi-objective pre-location model of temporary distribution station for emergency materials was built, which considered the problems of information shortage and uncertain demand after the incident with minimum time, maximum time satisfaction, minimum delivery cost and psychological trauma to the masses. The priority of candidate points was solved by comprehensive evaluation methods, the nominal demand of served points was estimated by triangular fuzzy number theory (TFN), and the location model was solved by non-dominated sorting genetic algorithm. In addition, the optimal schemes without priority and considering it were compared and analyzed, the practicability of the model is verified by concrete examples. The results show the time and cost reduction of 7.754% and 25.651%, an increment of total satisfaction value of the scheme considering location priority. Therefore, the model and algorithm provide theoretical support and practical ideas for solving the location problem, which can better complete the task of the location problem for temporary distribution stations of urban emergency materials.

An evaluation of a rapid conversion to teleSANE in response to COVID-19.

Sexual Assault Nurse Examiners (SANEs) provide expert, comprehensive medical forensic care to patients who present for services following a sexual assault. Because SANEs are not consistently available, telehealth technology is being explored as a means to provide access to this expert care (i.e., teleSANE). During the COVID-19 pandemic, teleSANE offered additional potential benefits by reducing the length of time spent and number of providers in patient exam rooms, the need for personal protective equipment that was in high demand and short supply, and provider anxiety related to providing in-person care. In the summer of 2020, the Massachusetts SANE program rapidly and temporarily converted five hospitals from in-person SANE care to teleSANE. An evaluation team interviewed 23 providers using a rapid research and evaluation methods approach to assess the temporary model and inform the future of SANE care. Evaluation findings reveal it is possible to rapidly and temporarily convert hospitals from in-person to teleSANE care in a time of broad uncertainty, and that such a change requires intensive and thoughtful planning; a shared commitment to being supportive, flexible, and responsive; and specific experience and expertise. Considerations for communities exploring how best to ensure consistent, equitable access to SANEs are discussed.

Impact of COVID-19 on a Community Health Coalition and Its Residents in Allegheny County, Pennsylvania: Insights Into Adaptation From Focus Groups and Evaluation Reports.

Live Well Allegheny: Lifting Wellness for African Americans (LWA2) is a coalition in Allegheny County, Pennsylvania, funded by the Centers for Disease Control and Prevention's (CDC) Racial and Ethnic Approaches to Community Health (REACH) initiative. LWA2 consists of partner organizations addressing chronic disease prevention in six Black communities through nutrition, physical activity, and community-clinical linkage strategies. This analysis focuses on qualitative data exploring the influence of COVID-19 on coalition functioning and communities. We conducted focus groups with residents in REACH communities and collected evaluation reports from partner organizations. Three focus groups assessed awareness of and participation in the REACH initiative, feedback, and the impact of COVID-19 when applicable. An additional focus group included questions related to flu vaccine messaging and the COVID-19 vaccine. These data sources provided insight regarding how COVID-19 affected planned tasks. Evaluation team members analyzed focus groups and collated summaries as part of a larger comprehensive evaluation. Partner organizations experienced an increase in food stamp applications, delays in opening farmers' markets, a shift to virtual preventive health programs, canceled in-person events, and programmatic interruptions that shifted long-term goals. Community resident concerns included difficulty accessing public transportation, decreased physical activity, fear of in-person interactions, and increased wait times for mental health services. Coalition members developed methods to continue functioning and sustaining program activities. Residents were able to engage differently with chronic illness prevention techniques. Reports from the ongoing analysis will be used to adapt coalition functioning.

Rock fisher behaviours and perceptions regarding drowning risk assessed by direct observation and self-report: A public awareness campaign evaluation.

ISSUE ADDRESSED: Participants engaged in rock fishing are at risk of drowning. Following coronial investigation of fatalities, a 3-year safety campaign targeting rock fishers was developed in Victoria, Australia. Key campaign messages were wearing lifejackets, not fishing alone, and checking sea and weather conditions. The reported study provides results from a campaign evaluation. METHODS: Evaluation by self-report and direct observation of safety attitudes and behaviours was undertaken pre- and during campaign. Data collections were as follows: (a) online survey of rock fishers recruited from panels, social media and rock fishing networks (n = 350) and (b) rock fisher direct observation and self-report at selected Victorian rock fishing platforms (n = 282; n = 58, respectively). RESULTS: Safety message recall was reported by 51.7% of rock fishers surveyed online though far fewer recalled campaign key messages. No effect on key safety behaviours or attitudes were detected for fishers on platforms during campaign. Never wearing a lifejacket was reported by 31.8% online, 60.3% at platforms and observed for 97.4%. From direct observation, most participants did not fish alone and checked conditions on arrival. CONCLUSION: Campaign evaluation measures showed mixed outcomes. Irrespective, most rock fishers carry high drowning risk through failure to wear lifejackets. Legal mandating of lifejackets for identified high-risk platform is being introduced for Victoria, although careful evaluation is required to detect unanticipated outcomes. Informing future campaign evaluation, complementary methods highlight likely bias in self-reporting through faulty recall or social desirability. SO WHAT?: Future campaigns require innovative or novel design, over longer duration, to capture attention and change rock fisher behaviours.

Accuracy of guided implant surgery obtained using 3D-printed surgical guides - An in vitro comparison of four evaluation methods.

AIM: To test four different measurement methods to evaluate deviations between planned and actual implant positions within a digital workflow applying 3D-printed surgical guides. MATERIALS AND METHODS: A fully digital workflow was applied to simulate the single implant insertion to replace a maxillary missing central incisor and first molar in 10 gypsum casts (n = 10). Surgical guides (n = 10 per site) were printed by digital light processing for implant bed preparation and implant insertion. Four methods were used to analyze 3D deviations between the planned (target) and achieved implant positions: Methods 1 and 2 used an automated computer program (ACP) to assess deviations between the initial planning file and a file that represented the actual implant position either by the implant bed [ACP_BED] or by the inserted implant [ACP_IMP]. For Method 3, a standard tessellation language dataset representing the actual implant position was used and equipped with reference planes. This dataset was registered with the target planning, allowing manual measurements [MAN_MEAS]. Method 4 used a reverse engineering approach based on 3D high-resolution scans [REVERSE]. RESULTS: Mean 3D deviations, including for anterior and posterior implant sites, ranged between 0.26 ± 0.11 mm [REVERSE] and 0.40 ± 0.09 mm [ACP_BED] at the implant shoulder, between 0.52 ± 0.24 mm [REVERSE] and 0.91 ± 0.24 mm [ACP_BED] at the implant apex, and between 1.68 and 2.35 degrees in angular deviation. Implant sites did not differ significantly, while some of the evaluation methods differed for shoulder and apex. CONCLUSION: [REVERSE] revealed the smallest deviations between planned and actual implant position. 3D implant deviations were comparable with findings in the literature or even lower.

Are UX Evaluation Methods Providing the Same Big Picture?

The success of a software application is related to users' willingness to keep using it. In this sense, evaluating User eXperience (UX) became an important part of the software development process. Researchers have been carrying out studies by employing various methods to evaluate the UX of software products. Some studies reported varied and even contradictory results when applying different UX evaluation methods, making it difficult for practitioners to identify which results to rely upon. However, these works did not evaluate the developers' perspectives and their impacts on the decision process. Moreover, such studies focused on one-shot evaluations, which cannot assess whether the methods provide the same big picture of the experience (i.e., deteriorating, improving, or stable). This paper presents a longitudinal study in which 68 students evaluated the UX of an online judge system by employing AttrakDiff, UEQ, and Sentence Completion methods at three moments along a semester. This study reveals contrasting results between the methods, which affected developers' decisions and interpretations. With this work, we intend to draw the HCI community's attention to the contrast between different UX evaluation methods and the impact of their outcomes in the software development process.