Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Systematic review and cumulative analysis of clinical properties of BRAF V600E mutations in PLNTY histological samples.

PURPOSE: Polymorphous low-grade neuroepithelial tumors of the young (PLNTY) represent a rare pediatric-type tumor that most commonly presents as medically refractory epilepsy. PLNTY has only recently been recognized as a distinct clinical entity, having been first described in 2016 and added to the World Health Organization classification of CNS tumors in 2021. Molecular studies have determined that PLNTY is uniformly driven by aberrant MAPK pathway activation, with most tumors carrying either a BRAF V600E mutation or activating FGFR2 or FGFR3 fusion protein. Although it is known that these driver mutations are mutually exclusive, little is known about differences in clinical presentation or treatment outcomes between PLNTY cases driven by these distinct mutations. METHODS: We performed a systematic review and cumulative analysis of PLNTY cases to assess whether or not PLNTY tumors carrying the BRAF V600E mutation exhibit different clinical behaviors. By searching the literature for all cases of PLNTY wherein BRAF V600E status was characterized, we compiled a dataset of 62 unique patient instances. Using a logistic regression-based approach, we assessed a primary outcome of what factors of a clinical presentation were associated with BRAF V600E mutations and a secondary outcome of what factors predicted total seizure freedom post-surgical resection. RESULTS: PLNTY cases carrying BRAF V600E mutations in the literature were strongly positively associated with adult patients (p = 0.0055, OR = 6.556; 95% Conf. Int. = 1.737-24.742). BRAF V600E status was also positively associated with tumor involvement of the temporal lobe (p = 0.0046, OR = 11.036; 95% Conf. Int. = 2.100-58.006). Male sex was also positively associated with BRAF V600E status, but the result did not quite achieve statistical significance (p = 0.0731). BRAF V600E status was not found to be associated with post-operative seizure freedom. CONCLUSIONS: These findings indicate that BRAF V600E-positive PLNTY exhibit characteristic clinical presentations but are not necessarily different in treatment responsiveness. Non-BRAF V600E tumors are more commonly associated with young patients.

Advanced approaches in Pediatric Epilepsy surgery.

While recent technological advancements are reshaping the landscape of surgical epilepsy management, the established techniques of resective and disconnective surgeries guided by electrographic monitoring remain the workhorse interventions for the management of refractory seizures and have the highest likelihood of achieving complete seizure resolution. Here we discuss examples of recent developments in surgical approaches and techniques for resective and disconnective surgeries with discussion of their indications and potential advantages.

Burr Hole Hemispherotomy-Modification of Trans-Sylvian Peri-Insular Technique: 2-Dimensional Operative Video.

Trans-sylvian peri-insular hemispherotomy represents a functional hemispherectomy with minimal brain removal used for treatment of refractory hemispheric epilepsy.1 Exposure for this procedure is achieved by craniotomy. Refinement in the hemispherotomy technique, including trends toward minimizing cortical resection, has contributed to a substantial drop in complication rates.2 We present a refinement of this technique, allowing for complete hemispheric disconnection through a single burr hole. In this instance, this technique was applied in the case of a 4-year-old girl who presented with medically refractory epilepsy, which had developed on the first day of life due to a perinatal incomplete left middle cerebral artery stroke. Postoperatively, the patient experienced no worsening of her preexisting right-sided hemiparesis and remains seizure-free 18 months postoperatively, now off medication. While the trans-sylvian peri-insular hemispherotomy represents an established surgical technique, this is the first report of this procedure performed in a minimally invasive fashion through a single burr hole. Beyond the minimal incision and small aperture in the skull, seldom appreciated nuances of hemispheric disconnection are described and demonstrated, including amygdala disconnection, hippocampal tail disconnection directly into splenium disconnection, concomitant intermediate disconnection and callosotomy, and frontobasal disconnection landmarks. Consent was obtained from the patient's parents for the surgical procedure, use of outcome videos, and for publication of this video and associated materials. The participants and patient's parents consented to publication of their images and that of the patient.

Diving into the Digital Landscape: Assessing the Quality of Online Information on Neonatal Jaundice for Parents.

BACKGROUND: Hyperbilirubinemia is a common condition in newborns. While mild cases of jaundice are common and typically resolve spontaneously, severe hyperbilirubinemia can lead to serious neurologic complications if left untreated. With the constant adaptation of guidelines, clinical management has significantly improved, and treatment has become routine for pediatricians. However, for parents of affected children, managing the condition is not routine. In today's digital age, parents often seek additional information by accessing a wide range of medical resources on the internet. While this can be empowering, it also presents challenges, as the quality and accuracy of online medical information can vary widely. Therefore, we analyzed the current quality of information on jaundice found on the internet by parents. METHODS: A simulated internet search (using the Google search engine) was conducted from a layperson's perspective using German ("Neugeborenes Gelbsucht", "Baby Gelbsucht") and English ("jaundice newborn", "jaundice baby") search terms. Subsequently, the quality of the search results was assessed by two independent neonatologists based on the DISCERN Plus Score, HONcode certification, and the JAMA criteria. RESULTS: Websites targeting non-medical laypersons exhibited significant variability in quality. Notably, the content of English websites was superior to that of websites in the German language. The majority of English sites were predominantly institutional, whereas most German sites were commercially oriented. CONCLUSIONS: Although information on jaundice is readily accessible online for non-medical individuals, there were notable differences in quality based on language and significant variability in the quality of information warranting attention from healthcare professionals. Furthermore, German websites providing information on jaundice were often hosted by commercial organizations. We propose that pediatric societies engage in developing and maintaining organization-based medical information to improve online resources for parents.

Predictors of urine toxicology and other biologic specimen missingness in randomized trials of substance use disorders.

BACKGROUND: High levels of missing outcome data for biologically confirmed substance use (BCSU) threaten the validity of substance use disorder (SUD) clinical trials. Underlying attributes of clinical trials could explain BCSU missingness and identify targets for improved trial design. METHODS: We reviewed 21 clinical trials funded by the NIDA National Drug Abuse Treatment Clinical Trials Network (CTN) and published from 2005 to 2018 that examined pharmacologic and psychosocial interventions for SUD. We used configurational analysis-a Boolean algebra approach that identifies an attribute or combination of attributes predictive of an outcome-to identify trial design features and participant characteristics associated with high levels of BCSU missingness. Associations were identified by configuration complexity, consistency, coverage, and robustness. We limited results using a consistency threshold of 0.75 and summarized model fit using the product of consistency and coverage. RESULTS: For trial design features, the final solution consisted of two pathways: psychosocial treatment as a trial intervention OR larger trial arm size (complexity=2, consistency=0.79, coverage=0.93, robustness score=0.71). For participant characteristics, the final solution consisted of two pathways: interventions targeting individuals with poly- or nonspecific substance use OR younger age (complexity=2, consistency=0.75, coverage=0.86, robustness score=1.00). CONCLUSIONS: Psychosocial treatments, larger trial arm size, interventions targeting individuals with poly- or nonspecific substance use, and younger age among trial participants were predictive of missing BCSU data in SUD clinical trials. Interventions to mitigate missing data that focus on these attributes may reduce threats to validity and improve utility of SUD clinical trials.