Depth-Specific Hypoxic Responses to Spreading Depolarizations in Gyrencephalic Swine Cortex Unveiled by Photoacoustic Imaging.

Spreading depolarizations (SDs) are a marker of brain injury and have a causative effect on ischemic lesion progression. The hemodynamic responses elicited by SDs are contingent upon the metabolic integrity of the affected tissue, with vasoconstrictive reactions leading to pronounced hypoxia often indicating poor outcomes. The stratification of hemodynamic responses within different cortical layers remains poorly characterized. This pilot study sought to elucidate the depth-specific hemodynamic changes in response to SDs within the gray matter of the gyrencephalic swine brain. Employing a potassium chloride-induced SD model, we utilized multispectral photoacoustic imaging (PAI) to estimate regional cerebral oxygen saturation (rcSO2%) changes consequent to potassium chloride-induced SDs. Regions of interest were demarcated at three cortical depths covering up to 4 mm. Electrocorticography (ECoG) strips were placed to validate the presence of SDs. Through PAI, we detected 12 distinct rcSO2% responses, which corresponded with SDs detected in ECoG. Notably, a higher frequency of hypoxic responses was observed in the deeper cortical layers compared to superficial layers, where hyperoxic and mixed responses predominated (p < 0.001). This data provides novel insights into the differential oxygenation patterns across cortical layers in response to SDs, underlining the complexity of cerebral hemodynamics post-injury.

Test-time augmentation with synthetic data addresses distribution shifts in spectral imaging.

PURPOSE: Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically. METHODS: Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation. RESULTS: The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times. CONCLUSION: Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging.

Correction: Comparison of Four Active SARS-CoV-2 Surveillance Strategies in Representative Population Sample Points: Two-Factor Factorial Randomized Controlled Trial.

[This corrects the article DOI: 10.2196/44204.].

Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study.

BACKGROUND: With Surgomics, we aim for personalized prediction of the patient's surgical outcome using machine-learning (ML) on multimodal intraoperative data to extract surgomic features as surgical process characteristics. As high-quality annotations by medical experts are crucial, but still a bottleneck, we prospectively investigate active learning (AL) to reduce annotation effort and present automatic recognition of surgomic features. METHODS: To establish a process for development of surgomic features, ten video-based features related to bleeding, as highly relevant intraoperative complication, were chosen. They comprise the amount of blood and smoke in the surgical field, six instruments, and two anatomic structures. Annotation of selected frames from robot-assisted minimally invasive esophagectomies was performed by at least three independent medical experts. To test whether AL reduces annotation effort, we performed a prospective annotation study comparing AL with equidistant sampling (EQS) for frame selection. Multiple Bayesian ResNet18 architectures were trained on a multicentric dataset, consisting of 22 videos from two centers. RESULTS: In total, 14,004 frames were tag annotated. A mean F1-score of 0.75 ± 0.16 was achieved for all features. The highest F1-score was achieved for the instruments (mean 0.80 ± 0.17). This result is also reflected in the inter-rater-agreement (1-rater-kappa > 0.82). Compared to EQS, AL showed better recognition results for the instruments with a significant difference in the McNemar test comparing correctness of predictions. Moreover, in contrast to EQS, AL selected more frames of the four less common instruments (1512 vs. 607 frames) and achieved higher F1-scores for common instruments while requiring less training frames. CONCLUSION: We presented ten surgomic features relevant for bleeding events in esophageal surgery automatically extracted from surgical video using ML. AL showed the potential to reduce annotation effort while keeping ML performance high for selected features. The source code and the trained models are published open source.

Efficient Photoacoustic Image Synthesis with Deep Learning.

Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5×108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5×106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.

CholecTriplet2022: Show me a tool and tell me the triplet - An endoscopic vision challenge for surgical action triplet detection.

Formalizing surgical activities as triplets of the used instruments, actions performed, and target anatomies is becoming a gold standard approach for surgical activity modeling. The benefit is that this formalization helps to obtain a more detailed understanding of tool-tissue interaction which can be used to develop better Artificial Intelligence assistance for image-guided surgery. Earlier efforts and the CholecTriplet challenge introduced in 2021 have put together techniques aimed at recognizing these triplets from surgical footage. Estimating also the spatial locations of the triplets would offer a more precise intraoperative context-aware decision support for computer-assisted intervention. This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection. It includes weakly-supervised bounding box localization of every visible surgical instrument (or tool), as the key actors, and the modeling of each tool-activity in the form of triplet. The paper describes a baseline method and 10 new deep learning algorithms presented at the challenge to solve the task. It also provides thorough methodological comparisons of the methods, an in-depth analysis of the obtained results across multiple metrics, visual and procedural challenges; their significance, and useful insights for future research directions and applications in surgery.

HeiPorSPECTRAL - the Heidelberg Porcine HyperSPECTRAL Imaging Dataset of 20 Physiological Organs.

Hyperspectral Imaging (HSI) is a relatively new medical imaging modality that exploits an area of diagnostic potential formerly untouched. Although exploratory translational and clinical studies exist, no surgical HSI datasets are openly accessible to the general scientific community. To address this bottleneck, this publication releases HeiPorSPECTRAL ( https://www.heiporspectral.org ; https://doi.org/10.5281/zenodo.7737674 ), the first annotated high-quality standardized surgical HSI dataset. It comprises 5,758 spectral images acquired with the TIVITA® Tissue and annotated with 20 physiological porcine organs from 8 pigs per organ distributed over a total number of 11 pigs. Each HSI image features a resolution of 480 × 640 pixels acquired over the 500-1000 nm wavelength range. The acquisition protocol has been designed such that the variability of organ spectra as a function of several parameters including the camera angle and the individual can be assessed. A comprehensive technical validation confirmed both the quality of the raw data and the annotations. We envision potential reuse within this dataset, but also its reuse as baseline data for future research questions outside this dataset. Measurement(s) Spectral Reflectance Technology Type(s) Hyperspectral Imaging Sample Characteristic - Organism Sus scrofa.

Comparison of Four Active SARS-CoV-2 Surveillance Strategies in Representative Population Sample Points: Two-Factor Factorial Randomized Controlled Trial.

BACKGROUND: The COVID-19 pandemic is characterized by rapid increases in infection burden owing to the emergence of new variants with higher transmissibility and immune escape. To date, monitoring the COVID-19 pandemic has mainly relied on passive surveillance, yielding biased epidemiological measures owing to the disproportionate number of undetected asymptomatic cases. Active surveillance could provide accurate estimates of the true prevalence to forecast the evolution of the pandemic, enabling evidence-based decision-making. OBJECTIVE: This study compared 4 different approaches of active SARS-CoV-2 surveillance focusing on feasibility and epidemiological outcomes. METHODS: A 2-factor factorial randomized controlled trial was conducted in 2020 in a German district with 700,000 inhabitants. The epidemiological outcome comprised SARS-CoV-2 prevalence and its precision. The 4 study arms combined 2 factors: individuals versus households and direct testing versus testing conditioned on symptom prescreening. Individuals aged ≥7 years were eligible. Altogether, 27,908 addresses from 51 municipalities were randomly allocated to the arms and 15 consecutive recruitment weekdays. Data collection and logistics were highly digitized, and a website in 5 languages enabled low-barrier registration and tracking of results. Gargle sample collection kits were sent by post. Participants collected a gargle sample at home and mailed it to the laboratory. Samples were analyzed with reverse transcription loop-mediated isothermal amplification (RT-LAMP); positive and weak results were confirmed with real-time reverse transcription-polymerase chain reaction (RT-PCR). RESULTS: Recruitment was conducted between November 18 and December 11, 2020. The response rates in the 4 arms varied between 34.31% (2340/6821) and 41.17% (2043/4962). The prescreening classified 16.61% (1207/7266) of the patients as COVID-19 symptomatic. Altogether, 4232 persons without prescreening and 7623 participating in the prescreening provided 5351 gargle samples, of which 5319 (99.4%) could be analyzed. This yielded 17 confirmed SARS-CoV-2 infections and a combined prevalence of 0.36% (95% CI 0.14%-0.59%) in the arms without prescreening and 0.05% (95% CI 0.00%-0.108%) in the arms with prescreening (initial contacts only). Specifically, we found a prevalence of 0.31% (95% CI 0.06%-0.58%) for individuals and 0.35% (95% CI 0.09%-0.61%) for households, and lower estimates with prescreening (0.07%, 95% CI 0.0%-0.15% for individuals and 0.02%, 95% CI 0.0%-0.06% for households). Asymptomatic infections occurred in 27% (3/11) of the positive cases with symptom data. The 2 arms without prescreening performed the best regarding effectiveness and accuracy. CONCLUSIONS: This study showed that postal mailing of gargle sample kits and returning home-based self-collected liquid gargle samples followed by high-sensitivity RT-LAMP analysis is a feasible way to conduct active SARS-CoV-2 population surveillance without burdening routine diagnostic testing. Efforts to improve participation rates and integration into the public health system may increase the potential to monitor the course of the pandemic. TRIAL REGISTRATION: Deutsches Register Klinischer Studien (DRKS) DRKS00023271; https://tinyurl.com/3xenz68a. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1186/s13063-021-05619-5.

CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII.

Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.

Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark.

PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

Engaging European society at the forefront of cancer research and care: How discussions at the 5th Gago Conference on European Science policy led to the Heidelberg Manifesto

European cancer research stakeholders met in October 2022 in Heidelberg, Germany, at the 5th Gago conference on European Cancer Policy, to discuss the current cancer research and cancer care policy landscape in Europe. Meeting participants highlighted gaps in the existing European programmes focusing on cancer research, including Europe's Beating Cancer Plan (EBCP), the Mission on Cancer (MoC), Understanding Cancer (UNCAN.eu), and the joint action CRANE, and put forward the next priorities, in the form of the Heidelberg Manifesto for cancer research. This meeting report presents all discussions that shed light on how infrastructures can be effectively shaped for translational, prevention, clinical and outcomes cancer research, with a focus on implementation and sustainability and while engaging patients and the public. In addition, we summarize recommendations on how to introduce frameworks for the digitalization of European cancer research. Finally, we discuss what structures, commitment, and resources are needed to establish a collaborative cancer research environment in Europe to achieve the scale required for innovation.

Beyond rankings: Learning (more) from algorithm validation.

Challenges have become the state-of-the-art approach to benchmark image analysis algorithms in a comparative manner. While the validation on identical data sets was a great step forward, results analysis is often restricted to pure ranking tables, leaving relevant questions unanswered. Specifically, little effort has been put into the systematic investigation on what characterizes images in which state-of-the-art algorithms fail. To address this gap in the literature, we (1) present a statistical framework for learning from challenges and (2) instantiate it for the specific task of instrument instance segmentation in laparoscopic videos. Our framework relies on the semantic meta data annotation of images, which serves as foundation for a General Linear Mixed Models (GLMM) analysis. Based on 51,542 meta data annotations performed on 2,728 images, we applied our approach to the results of the Robust Medical Instrument Segmentation Challenge (ROBUST-MIS) challenge 2019 and revealed underexposure, motion and occlusion of instruments as well as the presence of smoke or other objects in the background as major sources of algorithm failure. Our subsequent method development, tailored to the specific remaining issues, yielded a deep learning model with state-of-the-art overall performance and specific strengths in the processing of images in which previous methods tended to fail. Due to the objectivity and generic applicability of our approach, it could become a valuable tool for validation in the field of medical image analysis and beyond.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery.

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Mild hypothermia reduces spreading depolarizations and infarct size in a swine model.

Spreading depolarizations (SDs) have been linked to infarct volume expansion following ischemic stroke. Therapeutic hypothermia provides a neuroprotective effect after ischemic stroke. This study aimed to evaluate the effect of hypothermia on the propagation of SDs and infarct volume in an ischemic swine model. Through left orbital exenteration, middle cerebral arteries were surgically occluded (MCAo) in 16 swine. Extensive craniotomy and durotomy were performed. Six hypothermic and five normothermic animals were included in the analysis. An intracranial temperature probe was placed right frontal subdural. One hour after ischemic onset, mild hypothermia was induced and eighteen hours of electrocorticographic (ECoG) and intrinsic optical signal (IOS) recordings were acquired. Postmortem, 4 mm-thick slices were stained with 2,3,5-triphenyltetrazolium chloride to estimate the infarct volume. Compared to normothermia (36.4 ± 0.4°C), hypothermia (32.3 ± 0.2°C) significantly reduced the frequency and expansion of SDs (ECoG: 3.5 ± 2.1, 73.2 ± 5.2% vs. 1.0 ± 0.7, 41.9 ± 21.8%; IOS 3.9 ± 0.4, 87.6 ± 12.0% vs. 1.4 ± 0.7, 67.7 ± 8.3%, respectively). Further, infarct volume among hypothermic animals (23.2 ± 1.8% vs. 32.4 ± 2.5%) was significantly reduced. Therapeutic hypothermia reduces infarct volume and the frequency and expansion of SDs following cerebral ischemia.

ICG-augmented hyperspectral imaging for visualization of intestinal perfusion compared to conventional ICG fluorescence imaging: an experimental study.

BACKGROUND: Small bowel malperfusion (SBM) can cause high morbidity and severe surgical consequences. However, there is no standardized objective measuring tool for the quantification of SBM. Indocyanine green (ICG) imaging can be used for visualization, but lacks standardization and objectivity. Hyperspectral imaging (HSI) as a newly emerging technology in medicine might present advantages over conventional ICG fluorescence or in combination with it. METHODS: HSI baseline data from physiological small bowel, avascular small bowel and small bowel after intravenous application of ICG was recorded in a total number of 54 in-vivo pig models. Visualizations of avascular small bowel after mesotomy were compared between HSI only (1), ICG-augmented HSI (IA-HSI) (2), clinical evaluation through the eyes of the surgeon (3) and conventional ICG imaging (4). The primary research focus was the localization of resection borders as suggested by each of the four methods. Distances between these borders were measured and histological samples were obtained from the regions in between in order to quantify necrotic changes 6 h after mesotomy for every region. RESULTS: StO2 images (1) were capable of visualizing areas of physiological perfusion and areas of clearly impaired perfusion. However, exact borders where physiological perfusion started to decrease could not be clearly identified. Instead, IA-HSI (2) suggested a sharp-resection line where StO2 values started to decrease. Clinical evaluation (3) suggested a resection line 23 mm (±7 mm) and conventional ICG imaging (4) even suggested a resection line 53 mm (±13 mm) closer towards the malperfused region. Histopathological evaluation of the region that was sufficiently perfused only according to conventional ICG (R3) already revealed a significant increase in pre-necrotic changes in 27% (±9%) of surface area. Therefore, conventional ICG seems less sensitive than IA-HSI with regards to detection of insufficient tissue perfusion. CONCLUSIONS: In this experimental animal study, IA-HSI (2) was superior for the visualization of segmental SBM compared to conventional HSI imaging (1), clinical evaluation (3) or conventional ICG imaging (4) regarding histopathological safety. ICG application caused visual artifacts in the StO2 values of the HSI camera as values significantly increase. This is caused by optical properties of systemic ICG and does not resemble a true increase in oxygenation levels. However, this empirical finding can be used to visualize segmental SBM utilizing ICG as contrast agent in an approach for IA-HSI. Clinical applicability and relevance will have to be explored in clinical trials. LEVEL OF EVIDENCE: Not applicable. Translational animal science. Original article.

Spatial and temporal frequency band changes during infarct induction, infarct progression, and spreading depolarizations in the gyrencephalic brain.

Aim: To describe the spatial and temporal electrocorticographic (ECoG) changes after middle cerebral artery occlusion (MCAo), including those caused by spreading depolarization (SD) in the pig brain. Methods: The left middle cerebral arteries (MCAs) were clipped in six pigs. The clipping procedure lasted between 8 and 12 min, achieving a permanent occlusion (MCAo). Five-contact ECoG stripes were placed bilaterally over the frontoparietal cortices corresponding to the irrigation territory of the MCA and anterior cerebral artery (ACA). ECoG recordings were performed around 24 h: 1 h before and 23 h after the MCAo, and SDs were quantified. Five-minute ECoG signal segments were sampled before, 5 min, and 4, 8, and 12 h after cerebral artery occlusion and before, during, and after the negative direct current shift of the SDs. The power spectrum of the signals was decomposed into delta, theta, alpha, beta, and gamma bands. Descriptive statistics, Wilcoxon matched-pairs signed-rank tests, and Friedman tests were performed. Results: Electrodes close to the MCAo showed instant decay in all frequency bands and SD onset during the first 5 h. Electrodes far from the MCAo exhibited immediate loss of fast frequencies and progressive decline of slow frequencies with an increased SD incidence between 6 and 14 h. After 8 h, the ACA electrode reported a secondary reduction of all frequency bands except gamma and high SD incidence within 12-17 h. During the SD, all electrodes showed a decline in all frequency bands. After SD passage, frequency band recovery was impaired only in MCA electrodes. Conclusion: ECoG can identify infarct progression and secondary brain injury. Severe disturbances in all the frequency bands are generated in the cortices where the SDs are passing by.

Photoacoustic image synthesis with generative adversarial networks.

Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).

Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data.

BACKGROUND: Personalized medicine requires the integration and analysis of vast amounts of patient data to realize individualized care. With Surgomics, we aim to facilitate personalized therapy recommendations in surgery by integration of intraoperative surgical data and their analysis with machine learning methods to leverage the potential of this data in analogy to Radiomics and Genomics. METHODS: We defined Surgomics as the entirety of surgomic features that are process characteristics of a surgical procedure automatically derived from multimodal intraoperative data to quantify processes in the operating room. In a multidisciplinary team we discussed potential data sources like endoscopic videos, vital sign monitoring, medical devices and instruments and respective surgomic features. Subsequently, an online questionnaire was sent to experts from surgery and (computer) science at multiple centers for rating the features' clinical relevance and technical feasibility. RESULTS: In total, 52 surgomic features were identified and assigned to eight feature categories. Based on the expert survey (n = 66 participants) the feature category with the highest clinical relevance as rated by surgeons was "surgical skill and quality of performance" for morbidity and mortality (9.0 ± 1.3 on a numerical rating scale from 1 to 10) as well as for long-term (oncological) outcome (8.2 ± 1.8). The feature category with the highest feasibility to be automatically extracted as rated by (computer) scientists was "Instrument" (8.5 ± 1.7). Among the surgomic features ranked as most relevant in their respective category were "intraoperative adverse events", "action performed with instruments", "vital sign monitoring", and "difficulty of surgery". CONCLUSION: Surgomics is a promising concept for the analysis of intraoperative data. Surgomics may be used together with preoperative features from clinical data and Radiomics to predict postoperative morbidity, mortality and long-term outcome, as well as to provide tailored feedback for surgeons.