Detecting Algorithmic Errors and Patient Harms for AI-Enabled Medical Devices in Randomized Controlled Trials: Protocol for a Systematic Review.

BACKGROUND: Artificial intelligence (AI) medical devices have the potential to transform existing clinical workflows and ultimately improve patient outcomes. AI medical devices have shown potential for a range of clinical tasks such as diagnostics, prognostics, and therapeutic decision-making such as drug dosing. There is, however, an urgent need to ensure that these technologies remain safe for all populations. Recent literature demonstrates the need for rigorous performance error analysis to identify issues such as algorithmic encoding of spurious correlations (eg, protected characteristics) or specific failure modes that may lead to patient harm. Guidelines for reporting on studies that evaluate AI medical devices require the mention of performance error analysis; however, there is still a lack of understanding around how performance errors should be analyzed in clinical studies, and what harms authors should aim to detect and report. OBJECTIVE: This systematic review will assess the frequency and severity of AI errors and adverse events (AEs) in randomized controlled trials (RCTs) investigating AI medical devices as interventions in clinical settings. The review will also explore how performance errors are analyzed including whether the analysis includes the investigation of subgroup-level outcomes. METHODS: This systematic review will identify and select RCTs assessing AI medical devices. Search strategies will be deployed in MEDLINE (Ovid), Embase (Ovid), Cochrane CENTRAL, and clinical trial registries to identify relevant papers. RCTs identified in bibliographic databases will be cross-referenced with clinical trial registries. The primary outcomes of interest are the frequency and severity of AI errors, patient harms, and reported AEs. Quality assessment of RCTs will be based on version 2 of the Cochrane risk-of-bias tool (RoB2). Data analysis will include a comparison of error rates and patient harms between study arms, and a meta-analysis of the rates of patient harm in control versus intervention arms will be conducted if appropriate. RESULTS: The project was registered on PROSPERO in February 2023. Preliminary searches have been completed and the search strategy has been designed in consultation with an information specialist and methodologist. Title and abstract screening started in September 2023. Full-text screening is ongoing and data collection and analysis began in April 2024. CONCLUSIONS: Evaluations of AI medical devices have shown promising results; however, reporting of studies has been variable. Detection, analysis, and reporting of performance errors and patient harms is vital to robustly assess the safety of AI medical devices in RCTs. Scoping searches have illustrated that the reporting of harms is variable, often with no mention of AEs. The findings of this systematic review will identify the frequency and severity of AI performance errors and patient harms and generate insights into how errors should be analyzed to account for both overall and subgroup performance. TRIAL REGISTRATION: PROSPERO CRD42023387747; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=387747. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/51614.

Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study.

BACKGROUND: Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation. OBJECTIVE: This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England. METHODS: This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote. RESULTS: Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024. CONCLUSIONS: The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/50568.

Glyceryl Trinitrate: History, Mystery, and Alcohol Intolerance.

Glyceryl trinitrate (GTN) is one of the earliest known treatments for angina with a fascinating history that bridges three centuries. However, despite its central role in the nitric oxide (NO) story as a NO-donating compound, establishing the precise mechanism of how GTN exerts its medicinal benefit has proven to be far more difficult. This review brings together the explosive and vasodilatory nature of this three-carbon molecule while providing an update on the likely in vivo pathways through which GTN, and the rest of the organic nitrate family, release NO, nitrite, or a combination of both, while also trying to explain nitrate tolerance. Over the last 20 years the alcohol detoxification enzyme, aldehyde dehydrogenase (ALDH), has undoubtedly emerged as the front runner to explaining GTN's bioactivation. This is best illustrated by reduced GTN efficacy in subjects carrying the single point mutation (Glu504Lys) in ALDH, which is also responsible for alcohol intolerance, as characterized by flushing. While these findings are significant for anyone following the GTN story, they appear particularly relevant for healthcare professionals, and especially so, if administering GTN to patients as an emergency treatment. In short, although the GTN puzzle has not been fully solved, clinical study data continue to cement the importance of ALDH, as uncovered in 2002, as a key GTN activator.

Medical care in Australian residential aged care: Perspectives of residents, family, nurses and general practitioners.

The importance of residential aged care facility (RACF)'s medical care is growing, driven by world-wide demographic trends in ageing populations. Despite this, there is a paucity of research into this care delivery from the perspective of those most involved. This study aimed to identify the enablers of and barriers to satisfactory RACF medical care by focusing on the general practitioner (GP) visit in the experience of residents, their family, registered nurses (RNs) and GPs. A multi-site case study was conducted at four purposively chosen RACFs in rural and regional New South Wales, Australia. Data derived from semi-structured interviews with 35 randomly selected aforementioned stakeholders and conducted in 2017 were evaluated using thematic, specifically framework analysis. The study's first key finding was related to the care team and to care recipients. It was evident that the quality of the RN-GP interprofessional collaboration was important for satisfactory care delivery. However, the care team was observed to additionally include RACF care staff and family members. Families were also in need of care. The study's second key finding was related to continuity of care. The interpersonal continuity of care provided by the existing GP continuing a new resident's care was beneficial. Informational continuity of care was found to be important but often disrupted by patient's information being initially unavailable, then fragmented and stored in different places. Medication management systems when accessed were poorly organised, time consuming and complex. This research suggests two useful new paradigms for residential aged care. The first is a re-envisaging of the resident care team to include the RN, GP, family and care staff, and those needing care to include residents and family. Secondly, care teams informed by interpersonal and informational continuity of care, and satisfactory resident care appears inextricably and positively linked.

Continence technologies whitepaper: Informing new engineering science research.

Advances in healthcare technology for continence have historically been limited compared to other areas of medicine, reflecting the complexities of the condition and social stigma which act as a barrier to participation. This whitepaper has been developed to inspire and direct the engineering science community towards research opportunities that exist for continence technologies that address unmet needs in diagnosis, treatment and long-term management. Our aim is to pinpoint key challenges and highlight related research opportunities for novel technological advances. To do so, we draw on experience and expertise from academics, clinicians, patients and patient groups linked to continence healthcare. This is presented in four areas of consideration: the clinical pathway, patient perspective, research challenges and effective innovation. In each we introduce seminal research, background information and demonstrative case-studies, before discussing their relevance to engineering science researchers who are interested in approaching this overlooked but vital area of healthcare.

Bottom-Up Evolution of Vesicles from Disks to High-Genus Polymersomes.

Polymersomes are vesicles formed by the self-assembly of amphiphilic copolymers in water. They represent one of the most promising alternatives of natural vesicles as they add new possibilities in the amphiphiles' molecular engineering of aqueous compartments. Here we report the design of polymersomes using a bottom-up approach wherein self-assembly of amphiphilic copolymers poly(2-(methacryloyloxy) ethyl phosphorylcholine)-poly(2-(diisopropylamino) ethyl methacrylate) (PMPC-PDPA) into membranes is tuned using pH and temperature. We report evolution from disk micelles, to vesicles, to high-genus vesicles (vesicles with many holes), where each passage is controlled by pH switch or temperature. We show that the process can be rationalized, adapting membrane physics theories to disclose scaling principles that allow the estimation of minimal radius of vesiculation as well as chain entanglement and coupling. This approach allows us to generate nanoscale vesicles with genus from 0 to 70, which have been very elusive and difficult to control so far.

The Meisenheimer Complex as a Paradigm in Drug Discovery: Reversible Covalent Inhibition through C67 of the ATP Binding Site of PLK1.

The polo kinase family are important oncology targets that act in regulating entry into and progression through mitosis. Structure-guided discovery of a new class of inhibitors of Polo-like kinase 1 (PLK1) catalytic activity that interact with Cys67 of the ATP binding site is described. Compounds containing the benzothiazole N-oxide scaffold not only bind covalently to this residue, but are reversible inhibitors through the formation of Meisenheimer complexes. This mechanism of kinase inhibition results in compounds that can target PLK1 with high selectivity, while avoiding issues with irreversible covalent binding and interaction with other thiol-containing molecules in the cell. Due to renewed interest in covalent drugs and the plethora of potential drug targets, these represent prototypes for the design of kinase inhibitory compounds that achieve high specificity through covalent interaction and yet still bind reversibly to the ATP cleft, a strategy that could be applied to avoid issues with conventional covalent binders.

Australian general practitioner attitudes to residential aged care facility visiting.

Demographic trends suggest that the sustainability of the general practitioner (GP) Residential Aged Care Facility (RACF) workforce, worldwide and in Australia, is under threat, compromising the ongoing care of chronically ill RACF residents. It is therefore important to ascertain current GP attitudes towards this work, to better understand and hypothesise means of reversing this trend. To this end, during 2014 the views of 26 GPs and GP Registrars working in rural and regional New South Wales, Australia, were captured during focus group discussions and one-on-one interviews. Analysis of the qualitative date revealed that GP attitudes towards RACF visiting fell into five key themes: pleasure, duty, remuneration and logistics, hesitation, and frustration. The data also revealed that the overriding emotion GPs felt about RACF visitation was frustration with the avoidable delays and inefficiencies associated with the work. Despite the pleasure GPs derived from their work in RACFs and their sense of obligation to be involved, their hesitation and frustration was compounded by the work's perceived poor remuneration. This research suggests that the barriers to GP participation in RACF visiting were managerial rather than attitudinal, and that a strategic focus upon improving administrative and logistical support is needed.

New 4-aryl-1,3,2-oxathiazolylium-5-olates: Chemical synthesis and photochemical stability of a novel series of S-nitrosothiols.

S-nitrosothiols (RSNOs) remain one of the most popular classes of NO-donating compounds due to their ability to release nitric oxide (NO) under non-enzymatic means whilst producing an inert disulphide by-product. However, alligning these compounds to the different biological fields of NO research has proved to be problematic due to the inherent instability of such compounds under a variety of conditions including heat, light and the presence of copper ions. 1,3,2-Oxathiazolylium-5-olates (OZOs) represent an interesting subclass of S-nitrosothiols that lock the -SNO moiety into a five membered heterocyclic ring in an attempt to improve the compound's overall stability. The synthesis of a novel series of halogen-containing OZOs was comprehensively studied resulting in a seven-step route and overall yields ranging between 21 and 37%. The photochemical stability of these compounds was assessed to determine if S-nitrosothiols locked within these mesoionic ring systems can offer greater stability and thereby release NO in a more controllable fashion than their non-cyclic counterparts.

A circle of silence: The attitudes of patients older than 65 years of age to ceasing long-term sleeping tablets.

BACKGROUND: Nocturnal benzodiazepines have a significant negative health impact on the elderly, yet they continue to be used. OBJECTIVE: The aim of this study was to assess elderly patients' use and knowledge of nocturnal benzodiazepines, and their attitudes to cessation. METHODS: Semi-structured telephone interviews were conducted with elderly patients (n = 17) from four general practices in Australia. RESULTS: Our study found that the initiation of benzodiazepine use was often at a time of stress for the patient. Long-term use was not in-tended, and patients conveyed poor awareness of the side effects and addictive potential of benzodiazepines. Patients' perceived attitudes of their general practitioner (GP) to prescribing benzodiazepines and lack of awareness of alternative therapies were key to continuation. Confounding factors such as pain often contributed to sleep disturbance. Many patients expressed a willingness to cease nocturnal benzodiazepine use. DISCUSSION: These data assist in raising GPs' awareness of patients' attitudes to cessation of nocturnal benzodiazepine use. More time spent with patients presenting for repeat prescriptions, explaining side effects, discussing alternative options and investigating reasons for not sleeping could reduce benzodiazepine use among the elderly.

Simulated drug discovery process to conduct a synoptic assessment of pharmacy students.

OBJECTIVE. To implement and assess a task-based learning exercise that prompts pharmacy students to integrate their understanding of different disciplines. DESIGN. Master of pharmacy (MPharm degree) students were provided with simulated information from several preclinical science and from clinical trials and asked to synthesize this into a marketing authorization application for a new drug. Students made a link to pharmacy practice by creating an advice leaflet for pharmacists. ASSESSMENT. Students' ability to integrate information from different disciplines was evaluated by oral examination. In 2 successive academic years, 96% and 82% of students demonstrated an integrated understanding of their proposed new drug. Students indicated in a survey that their understanding of the links between different subjects improved. CONCLUSION. Simulated drug discovery provides a learning environment that emphasizes the connectivity of the preclinical sciences with each other and the practice of pharmacy.

Cross-sectional survey of older patients' views regarding multidisciplinary care for chronic conditions in general practice.

The ageing population and increasing prevalence of chronic illness have contributed to the need for significant primary care reform, including increased use of multidisciplinary care and task substitution. This cross-sectional study explores conditions under which older patients would accept having health professionals other than their general practitioner (GP) involved in their care for chronic disease management (CDM). Ten practices were randomly sampled from a contiguous major city and inner regional area. Questionnaires were distributed to consecutive patients aged 60 years and over in each practice. Agency theory was used to inform analyses. Statistical analysis was undertaken using Wald's test, growth modelling and linear regression, controlling for the clustered design. The response rate was 53% (n=272). Most respondents (79%) had at least one chronic health condition. Respondents were more comfortable with GP than with practice nurse management in the CDM scenario (Wald's test=105.49, P<0.001). Comfort with practice nurse CDM was positively associated with increased contact with their GP at the time of the visit (ß=0.41, P<0.001), negatively associated with the number of the respondent's chronic conditions (ß=-0.13, P=0.030) and not associated with the frequency of other health professional visits. Agency theory suggests that patients employ continuity of care to optimise factors important in CDM: information symmetry and goal alignment. Our findings are consistent with the theory and lend support to ensuring that interpersonal continuity of care is not lost in health care reform. Further research exploring patients' acceptance of differing systems of care is required.

Nile Blue-based nanosized pH sensors for simultaneous far-red and near-infrared live bioimaging.

Diblock copolymer vesicles are tagged with pH-responsive Nile Blue-based labels and used as a new type of pH-responsive colorimetric/fluorescent biosensor for far-red and near-infrared imaging of live cells. The diblock copolymer vesicles described herein are based on poly(2-(methacryloyloxy)ethyl phosphorylcholine-block-2-(diisopropylamino)ethyl methacrylate) [PMPC-PDPA]: the biomimetic PMPC block is known to facilitate rapid cell uptake for a wide range of cell lines, while the PDPA block constitutes the pH-responsive component that enables facile vesicle self-assembly in aqueous solution. These biocompatible vesicles can be utilized to detect interstitial hypoxic/acidic regions in a tumor model via a pH-dependent colorimetric shift. In addition, they are also useful for selective intracellular staining of lysosomes and early endosomes via subtle changes in fluorescence emission. Such nanoparticles combine efficient cellular uptake with a pH-responsive Nile Blue dye label to produce a highly versatile dual capability probe. This is in marked contrast to small molecule dyes, which are usually poorly uptaken by cells, frequently exhibit cytotoxicity, and are characterized by intracellular distributions invariably dictated by their hydrophilic/hydrophobic balance.

Adaptability - building an academic workforce with GPs.

The University of Wollongong Graduate School of Medicine (UWGSM) opened in 2007. This is one of a new wave of medical schools to enable the more than doubling of the number of medical students graduating in the period from 2006-2014. However, this rapid expansion has exposed a relative paucity of experienced medical academics and the regional medical schools especially have found difficulty immediately attracting a full complement of academic staff. These schools have therefore sought to recruit locally and train staff who vary widely in previous experience in teaching.

Are patients willing participants in the new wave of community-based medical education in regional and rural Australia?

OBJECTIVE: Community-based medical education is growing to meet the increased demand for quality clinical education in expanded settings, and its sustainability relies on patient participation. This study investigated patients' views on being used as an educational resource for teaching medical students. DESIGN: Questionnaire-based survey. SETTING AND PARTICIPANTS: Patients attending six rural and 11 regional general practices in New South Wales over 18 teaching sessions in November 2008, who consented to student involvement in their consultation. MAIN OUTCOME MEASURES: Patient perceptions, expectations and acceptance of medical student involvement in consultations, assessed by surveys before and after their consultations. RESULTS: 118 of 122 patients consented to medical student involvement; of these, 117 (99%) completed a survey before the consultation, and 100 (85%) after the consultation. Patients were overwhelmingly positive about their doctor and practice being involved in student teaching and felt they themselves played an important role. Pre-consultation, patients expressed reluctance to allow students to conduct some or all aspects of the consultation independently. However, after the consultation, they reported they would have accepted higher levels of involvement than actually occurred. CONCLUSIONS: Patients in regional and rural settings were willing partners in developing skills of junior medical students, who had greater involvement in patient consultations than previously reported for urban students. Our study extends the findings from urban general practice that patients are underutilised partners in community-based medical training. The support of patients from regional and rural settings could facilitate the expansion of primary care-based medical education in these areas of workforce need.

Synthesis and chemical characterisation of target identification reagents based on an inhibitor of human cell invasion by the parasite Toxoplasma gondii.

The use of phenotype-based screens as an approach for identifying novel small molecule tools is reliant on successful protein target identification strategies. Here we report on the synthesis and chemical characterisation of a novel reagent for protein target identification based on a small molecule inhibitor of human cell invasion by the parasite Toxoplasma gondii. A detailed (1)H NMR study and biological testing confirmed that incorporation of an amino-containing functional group into the aryl ring of this inhibitor was possible without loss of biological activity. Interesting chemical reactivity differences were identified resulting from incorporation of the new substituent. The amine functionality was then used to prepare a biotinylated reagent that is central to our current protein target identification studies with this inhibitor.

Probing structural effects on replication efficiency through comparative analyses of families of potential self-replicators.

A formidable synthetic apparatus for the creation of nanoscale molecular structures and supramolecular assemblies through molecular structures can potentially be created from systems that are capable of parallel automultiplication (self-replication). In order to achieve this goal, a detailed understanding of the relationship between molecular structure and replication efficiency is necessary. Diastereoisomeric templates that are capable of specific and simultaneous autocatalysis have been synthesised. A systematic experimental and theoretical evaluation of their behaviour and that of structurally-related systems reveals the key determinants that dictate the emergence of self-replicative function and defines the structural space within which this behaviour is observed.