Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Competition for cysteine acylation by C16:0 and C18:0 derived lipids is a global phenomenon in the proteome.

S-acylation is a reversible posttranslational protein modification consisting of attachment of a fatty acid to a cysteine via a thioester bond. Research over the last few years has shown that a variety of different fatty acids, such as palmitic acid (C16:0), stearate (C18:0), or oleate (C18:1), are used in cells to S-acylate proteins. We recently showed that GNAI proteins can be acylated on a single residue, Cys3, with either C16:0 or C18:1, and that the relative proportion of acylation with these fatty acids depends on the level of the respective fatty acid in the cell's environment. This has functional consequences for GNAI proteins, with the identity of the acylating fatty acid affecting the subcellular localization of GNAIs. Unclear is whether this competitive acylation is specific to GNAI proteins or a more general phenomenon in the proteome. We perform here a proteome screen to identify proteins acylated with different fatty acids. We identify 218 proteins acylated with C16:0 and 308 proteins acylated with C18-lipids, thereby uncovering novel targets of acylation. We find that most proteins that can be acylated by C16:0 can also be acylated with C18-fatty acids. For proteins with more than one acylation site, we find that this competitive acylation occurs on each individual cysteine residue. This raises the possibility that the function of many different proteins can be regulated by the lipid environment via differential S-acylation.

Artificial intelligence-based, volumetric assessment of the bone marrow metabolic activity in [18F]FDG PET/CT predicts survival in multiple myeloma.

PURPOSE: Multiple myeloma (MM) is a highly heterogeneous disease with wide variations in patient outcome. [18F]FDG PET/CT can provide prognostic information in MM, but it is hampered by issues regarding standardization of scan interpretation. Our group has recently demonstrated the feasibility of automated, volumetric assessment of bone marrow (BM) metabolic activity on PET/CT using a novel artificial intelligence (AI)-based tool. Accordingly, the aim of the current study is to investigate the prognostic role of whole-body calculations of BM metabolism in patients with newly diagnosed MM using this AI tool. MATERIALS AND METHODS: Forty-four, previously untreated MM patients underwent whole-body [18F]FDG PET/CT. Automated PET/CT image segmentation and volumetric quantification of BM metabolism were based on an initial CT-based segmentation of the skeleton, its transfer to the standardized uptake value (SUV) PET images, subsequent application of different SUV thresholds, and refinement of the resulting regions using postprocessing. In the present analysis, ten different uptake thresholds (AI approaches), based on reference organs or absolute SUV values, were applied for definition of pathological tracer uptake and subsequent calculation of the whole-body metabolic tumor volume (MTV) and total lesion glycolysis (TLG). Correlation analysis was performed between the automated PET values and histopathological results of the BM as well as patients' progression-free survival (PFS) and overall survival (OS). Receiver operating characteristic (ROC) curve analysis was used to investigate the discrimination performance of MTV and TLG for prediction of 2-year PFS. The prognostic performance of the new Italian Myeloma criteria for PET Use (IMPeTUs) was also investigated. RESULTS: Median follow-up [95% CI] of the patient cohort was 110 months [105-123 months]. AI-based BM segmentation and calculation of MTV and TLG were feasible in all patients. A significant, positive, moderate correlation was observed between the automated quantitative whole-body PET/CT parameters, MTV and TLG, and BM plasma cell infiltration for all ten [18F]FDG uptake thresholds. With regard to PFS, univariable analysis for both MTV and TLG predicted patient outcome reasonably well for all AI approaches. Adjusting for cytogenetic abnormalities and BM plasma cell infiltration rate, multivariable analysis also showed prognostic significance for high MTV, which defined pathological [18F]FDG uptake in the BM via the liver. In terms of OS, univariable and multivariable analysis showed that whole-body MTV, again mainly using liver uptake as reference, was significantly associated with shorter survival. In line with these findings, ROC curve analysis showed that MTV and TLG, assessed using liver-based cut-offs, could predict 2-year PFS rates. The application of IMPeTUs showed that the number of focal hypermetabolic BM lesions and extramedullary disease had an adverse effect on PFS. CONCLUSIONS: The AI-based, whole-body calculations of BM metabolism via the parameters MTV and TLG not only correlate with the degree of BM plasma cell infiltration, but also predict patient survival in MM. In particular, the parameter MTV, using the liver uptake as reference for BM segmentation, provides solid prognostic information for disease progression. In addition to highlighting the prognostic significance of automated, global volumetric estimation of metabolic tumor burden, these data open up new perspectives towards solving the complex problem of interpreting PET scans in MM with a simple, fast, and robust method that is not affected by operator-dependent interventions.

The German sickle cell disease registry reveals a surprising risk of acute splenic sequestration and an increased transfusion requirement in patients with compound heterozygous sickle cell disease HbS/ß-thalassaemia and no or low HbA expression.

Patients with sickle cell disease (SCD) in Germany exhibit a substantial genetic diversity in the ß-globin genotype. Data collected by the national German SCD registry reflect this diversity and allowed us to analyze the phenotypes associated with different SCD genotypes. Our study focused on 90 patients with HbS/ß-thalassaemia (HbS/ß-thal) and compared these to patients with HbSS and HbSC. Patients with HbS/ß-thal were classified into three groups: HbS/ß0-thal (no HbA), HbS/ß+-thal (HbA < 14%), and HbS/ß++-thal (HbA≥14%). In comparison to HbSS, patients with HbS/ß++-thal had higher Hb-levels, lower hemolytic activity and rarely required red blood cell transfusions. HbS/ß0-thal and HbS/ß+-thal closely resembled each other and are jointly referred to as HbS/ß0/+-thal. Compared to HbSS, patients with HbS/ß0/+-thal experienced a similar frequency of vasoocclusive crises and degree of hemolysis. However, the frequency of red blood cell transfusions (0.6 vs. 0.39/year, p = .0049) and splenic sequestration crises (42.4 vs. 15.5% of patients, p = 3.799e-05) was higher in HbS/ß0/+-thal than in HbSS, but close to zero in HbS/ß++-thal. In conclusion, the level of HbA expression determines the phenotype of HbS/ß+-thal. HbS/ß-thal expressing no or little HbA is hematologically similar to HbSS, but causes a previously unknown high risk of splenic sequestration.

Simulating and reporting frequentist operating characteristics of clinical trials that borrow external information: Towards a fair comparison in case of one-arm and hybrid control two-arm trials.

Borrowing information from historical or external data to inform inference in a current trial is an expanding field in the era of precision medicine, where trials are often performed in small patient cohorts for practical or ethical reasons. Even though methods proposed for borrowing from external data are mainly based on Bayesian approaches that incorporate external information into the prior for the current analysis, frequentist operating characteristics of the analysis strategy are often of interest. In particular, type I error rate and power at a prespecified point alternative are the focus. We propose a procedure to investigate and report the frequentist operating characteristics in this context. The approach evaluates type I error rate of the test with borrowing from external data and calibrates the test without borrowing to this type I error rate. On this basis, a fair comparison of power between the test with and without borrowing is achieved. We show that no power gains are possible in one-sided one-arm and two-arm hybrid control trials with normal endpoint, a finding proven in general before. We prove that in one-arm fixed-borrowing situations, unconditional power (i.e., when external data is random) is reduced. The Empirical Bayes power prior approach that dynamically borrows information according to the similarity of current and external data avoids the exorbitant type I error inflation occurring with fixed borrowing. In the hybrid control two-arm trial we observe power reductions as compared to the test calibrated to borrowing that increase when considering unconditional power.

Robust incorporation of historical information with known type I error rate inflation.

Bayesian clinical trials can benefit from available historical information through the specification of informative prior distributions. Concerns are however often raised about the potential for prior-data conflict and the impact of Bayes test decisions on frequentist operating characteristics, with particular attention being assigned to inflation of type I error (TIE) rates. This motivates the development of principled borrowing mechanisms, that strike a balance between frequentist and Bayesian decisions. Ideally, the trust assigned to historical information defines the degree of robustness to prior-data conflict one is willing to sacrifice. However, such relationship is often not directly available when explicitly considering inflation of TIE rates. We build on available literature relating frequentist and Bayesian test decisions, and investigate a rationale for inflation of TIE rate which explicitly and linearly relates the amount of borrowing and the amount of TIE rate inflation in one-arm studies. A novel dynamic borrowing mechanism tailored to hypothesis testing is additionally proposed. We show that, while dynamic borrowing prevents the possibility to obtain a simple closed-form TIE rate computation, an explicit upper bound can still be enforced. Connections with the robust mixture prior approach, particularly in relation to the choice of the mixture weight and robust component, are made. Simulations are performed to show the properties of the approach for normal and binomial outcomes, and an exemplary application is demonstrated in a case study.

A decision-theoretic approach to Bayesian clinical trial design and evaluation of robustness to prior-data conflict.

Bayesian clinical trials allow taking advantage of relevant external information through the elicitation of prior distributions, which influence Bayesian posterior parameter estimates and test decisions. However, incorporation of historical information can have harmful consequences on the trial's frequentist (conditional) operating characteristics in case of inconsistency between prior information and the newly collected data. A compromise between meaningful incorporation of historical information and strict control of frequentist error rates is therefore often sought. Our aim is thus to review and investigate the rationale and consequences of different approaches to relaxing strict frequentist control of error rates from a Bayesian decision-theoretic viewpoint. In particular, we define an integrated risk which incorporates losses arising from testing, estimation, and sampling. A weighted combination of the integrated risk addends arising from testing and estimation allows moving smoothly between these two targets. Furthermore, we explore different possible elicitations of the test error costs, leading to test decisions based either on posterior probabilities, or solely on Bayes factors. Sensitivity analyses are performed following the convention which makes a distinction between the prior of the data-generating process, and the analysis prior adopted to fit the data. Simulation in the case of normal and binomial outcomes and an application to a one-arm proof-of-concept trial, exemplify how such analysis can be conducted to explore sensitivity of the integrated risk, the operating characteristics, and the optimal sample size, to prior-data conflict. Robust analysis prior specifications, which gradually discount potentially conflicting prior information, are also included for comparison. Guidance with respect to cost elicitation, particularly in the context of a Phase II proof-of-concept trial, is provided.

The prognostic value of [18F]FDG PET/CT based response monitoring in metastatic melanoma patients undergoing immunotherapy: comparison of different metabolic criteria.

PURPOSE: To investigate the prognostic value of [18F]FDG PET/CT as part of response monitoring in metastatic melanoma patients treated with immune checkpoint inhibitors (ICIs). METHODS: Sixty-seven patients underwent [18F]FDG PET/CT before start of treatment (baseline PET/CT), after two cycles (interim PET/CT) and after four cycles of ICIs administration (late PET/CT). Metabolic response evaluation was based on the conventional EORTC and PERCIST criteria, as well as the newly introduced, immunotherapy-modified PERCIMT, imPERCIST5 and iPERCIST criteria. Metabolic response to immunotherapy was classified according to four response groups (complete metabolic response [CMR], partial metabolic response [PMR], stable metabolic disease [SMD], progressive metabolic disease [PMD]), and further dichotomized by response rate (responders = [CMR] + [PMR] vs. non-responders = [PMD] + [SMD]), and disease control rate (disease control = [CMR] + [PMR] + [SMD] vs. [PMD]). The spleen-to-liver SUV ratios (SLRmean, SLRmax) and bone marrow-to-liver SUV ratios (BLRmean, BLRmax) were also calculated. The results of PET/CT were correlated with patients' overall survival (OS). RESULTS: Median patient follow up [95% CI] was 61.5 months [45.3 - 66.7 months]. On interim PET/CT, the application of the novel PERCIMT demonstrated significantly longer survival for metabolic responders, while the rest criteria revealed no significant survival differences between the different response groups. Respectively on late PET/CT, both a trend for longer OS and significantly longer OS were observed in patients responding to ICIs with metabolic response and disease control after application of various criteria, both conventional and immunotherapy-modified. Moreover, patients with lower SLRmean values demonstrated significantly longer OS. CONCLUSION: In patients with metastatic melanoma PET/CT-based response assessment after four ICIs cycles is significantly associated with OS after application of different metabolic criteria. The prognostic performance of the modality is also high after the first two ICIs cycles, especially with employment of novel criteria. In addition, investigation of spleen glucose metabolism may provide further prognostic information.

Can physiologic colonic [18F]FDG uptake in PET/CT imaging predict response to immunotherapy in metastatic melanoma?

AIM: The development of biomarkers that can reliably and early predict response to immune checkpoint inhibitors (ICIs) is crucial in melanoma. In recent years, the gut microbiome has emerged as an important regulator of immunotherapy response, which may, moreover, serve as a surrogate marker and prognosticator in oncological patients under immunotherapy. Aim of the present study is to investigate if physiologic colonic [18F]FDG uptake in PET/CT before start of ICIs correlates with clinical outcome of metastatic melanoma patients. The relation between [18F]FDG uptake in lymphoid cell-rich organs and long-term patient outcome is also assessed. METHODOLOGY: One hundred nineteen stage IV melanoma patients scheduled for immunotherapy with ipilimumab, applied either as monotherapy or in combination with nivolumab, underwent baseline [18F]FDG PET/CT. PET/CT data analysis consisted of standardized uptake value (SUV), metabolic tumor volume (MTV), and total lesion glycolysis (TLG) calculations in the colon as well as measurements of the colon-to-liver SUV ratios (CLRmean, CLRmax). Visual grading of colon uptake based on a four-point scale was also performed. Moreover, the spleen-to-liver SUV ratios (SLRmean, SLRmax) and the bone marrow-to-liver SUV ratios (BLRmean, BLRmax) were calculated. We also measured serum lipopolysaccharide (LPS) levels as a marker for bacterial translocation and surrogate for mucosal defense homeostasis. The results were correlated with patients' best clinical response, progression-free survival (PFS), and overall survival (OS) as well as clinical signs of colitis. RESULTS: Median follow-up [95%CI] from the beginning of immunotherapy was 64.6 months [61.0-68.6 months]. Best response to treatment was progressive disease (PD) for 60 patients, stable disease (SD) for 37 patients, partial response (PR) for 18 patients, and complete response (CR) for 4 patients. Kaplan-Meier curves demonstrated a trend for longer PFS and OS in patients with lower colonic SUV and CLR values; however, no statistical significance for these parameters as prognostic factors was demonstrated. On the other hand, patients showing disease control as best response to treatment (SD, PR, CR) had significantly lower colonic MTV and TLG than those showing PD. With regard to lymphoid cell-rich organs, significantly lower baseline SLRmax and BLRmax were observed in patients responding with disease control than progression to treatment. Furthermore, patients with lower SLRmax and BLRmax values had a significantly longer OS when dichotomized at their median. In multivariate analysis, PET parameters that were found to significantly adversely correlate with patient survival were colonic MTV for PFS, colonic TLG for PFS, and BLRmax for PFS and OS. CONCLUSIONS: Physiologic colonic [18F]FDG uptake in PET/CT, as assessed by means of SUV, before start of ipilimumab-based treatment does not seem to independently predict patient survival of metastatic melanoma. On the other hand, volumetric PET parameters, such as MTV and TLG, derived from the normal gut may identify patients showing disease control to immunotherapy and significantly correlate with PFS. Moreover, the investigation of glucose metabolism in the spleen and the bone marrow may offer prognostic information.

Application of the long axial field-of-view PET/CT with low-dose [18F]FDG in melanoma.

AIM: The recent introduction of long axial field-of-view (LAFOV) PET/CT scanners has yielded very promising results regarding image quality and sensitivity in oncological patients. We, herein, aim to determine an appropriate acquisition time range for the new long axial field of view Biograph Vision Quadra PET/CT (Siemens Healthcare) using low dose [18F]FDG activity in a group of melanoma patients. METHODOLOGY: Forty-nine melanoma patients were enrolled in the study. All patients underwent total body PET/CT from the top of the head through the feet in two bed positions (field-of-view 106 cm) after i.v. injection of 2.0 MBq/kg [18F]FDG. The PET images of the first bed position (head to upper thigh; PET-10) were reconstructed and further split into 8-min (PET-8), 6-min (PET-6), 5-min (PET-5), 4-min (PET-4), and 2-min (PET-2) duration groups. Comparisons were performed between the different reconstructed scan times with regard to the visual evaluation of the PET/CT scans using the PET-10 images as reference and by calculating the 95%-CI for the differences between different time acquisitions. Moreover, objective evaluation of PET/CT image quality was performed based on SUV calculations of tumor lesions and background, leading to calculation of liver signal-to-noise ratio (SNR), and tumor-to-background ratio (TBR). RESULTS: A total of 60 scans were evaluated. Concerning visual analysis, 49/60 (81.7%) PET-10 scans were pathological, while the respective frequencies were 49/60 (81.7%) for PET-8 (95%-CI: - 0.0602-0.0602), 49/60 (81.7%) for PET-6 (95%-CI: - 0.0602-0.0602), 48/60 (80%) for PET-5 (95%-CI: - 0.0445-0.0886), 46/60 (76.7%) for PET-4 (95%-CI: - 0.0132-0.1370), and 45/60 (75%) for PET-2 (95%-CI: 0.0025-0.1593). In 18 PET-10 scans, the extent of metastatic involvement was very large, rendering the accurate calculation of [18F]FDG-avid tumor lesions very complicated. In the remaining 42 PET-10 scans, for which the exact calculation of tumor lesions was feasible, a total of 119 tumor lesions were counted, and the respective lesion detection rates for shorter acquisitions were as follows: 97.5% (116/119) for PET-8 (95%-CI: 0-1), 95.0% (113/119) for PET-6 (95%-CI: 0-1), 89.9% (107/119) for PET-5 (95%-CI: 0-2), 83.2% (99/119) for PET-4 (95%-CI: 1-2), and 73.9% (88/119) for PET-2 (95%-CI: 2-4). With regard to objective image quality evaluations, as a general trend, the reduction of acquisition time was associated with a decrease of liver SNR and a decrease of TBR, although in lesion-based analysis the change in TBR and tumor SUVmean values was non-significant up to 6 and 5 min acquisitions, respectively. CONCLUSIONS: In melanoma, low-dose LAFOV PET/CT imaging is feasible and can reduce the total scan time from head to upper thigh up to 5 min providing comparable diagnostic data to standard lengths of acquisition. This may have significant implications for the diagnostic work-up of patients with melanoma, given the need for true whole-body imaging in this type of cancer.

Application of an artificial intelligence-based tool in [18F]FDG PET/CT for the assessment of bone marrow involvement in multiple myeloma.

PURPOSE: [18F]FDG PET/CT is an imaging modality of high performance in multiple myeloma (MM). Nevertheless, the inter-observer reproducibility in PET/CT scan interpretation may be hampered by the different patterns of bone marrow (BM) infiltration in the disease. Although many approaches have been recently developed to address the issue of standardization, none can yet be considered a standard method in the interpretation of PET/CT. We herein aim to validate a novel three-dimensional deep learning-based tool on PET/CT images for automated assessment of the intensity of BM metabolism in MM patients. MATERIALS AND METHODS: Whole-body [18F]FDG PET/CT scans of 35 consecutive, previously untreated MM patients were studied. All patients were investigated in the context of an open-label, multicenter, randomized, active-controlled, phase 3 trial (GMMG-HD7). Qualitative (visual) analysis classified the PET/CT scans into three groups based on the presence and number of focal [18F]FDG-avid lesions as well as the degree of diffuse [18F]FDG uptake in the BM. The proposed automated method for BM metabolism assessment is based on an initial CT-based segmentation of the skeleton, its transfer to the SUV PET images, the subsequent application of different SUV thresholds, and refinement of the resulting regions using postprocessing. In the present analysis, six different SUV thresholds (Approaches 1-6) were applied for the definition of pathological tracer uptake in the skeleton [Approach 1: liver SUVmedian × 1.1 (axial skeleton), gluteal muscles SUVmedian × 4 (extremities). Approach 2: liver SUVmedian × 1.5 (axial skeleton), gluteal muscles SUVmedian × 4 (extremities). Approach 3: liver SUVmedian × 2 (axial skeleton), gluteal muscles SUVmedian × 4 (extremities). Approach 4: ≥ 2.5. Approach 5: ≥ 2.5 (axial skeleton), ≥ 2.0 (extremities). Approach 6: SUVmax liver]. Using the resulting masks, subsequent calculations of the whole-body metabolic tumor volume (MTV) and total lesion glycolysis (TLG) in each patient were performed. A correlation analysis was performed between the automated PET values and the results of the visual PET/CT analysis as well as the histopathological, cytogenetical, and clinical data of the patients. RESULTS: BM segmentation and calculation of MTV and TLG after the application of the deep learning tool were feasible in all patients. A significant positive correlation (p < 0.05) was observed between the results of the visual analysis of the PET/CT scans for the three patient groups and the MTV and TLG values after the employment of all six [18F]FDG uptake thresholds. In addition, there were significant differences between the three patient groups with regard to their MTV and TLG values for all applied thresholds of pathological tracer uptake. Furthermore, we could demonstrate a significant, moderate, positive correlation of BM plasma cell infiltration and plasma levels of ß2-microglobulin with the automated quantitative PET/CT parameters MTV and TLG after utilization of Approaches 1, 2, 4, and 5. CONCLUSIONS: The automated, volumetric, whole-body PET/CT assessment of the BM metabolic activity in MM is feasible with the herein applied method and correlates with clinically relevant parameters in the disease. This methodology offers a potentially reliable tool in the direction of optimization and standardization of PET/CT interpretation in MM. Based on the present promising findings, the deep learning-based approach will be further evaluated in future prospective studies with larger patient cohorts.

Performance of phase-I dose finding designs with and without a run-in intra-patient dose escalation stage.

Dose-finding designs for phase-I trials aim to determine the recommended phase-II dose (RP2D) for further phase-II drug development. If the trial includes patients for whom several lines of standard therapy failed or if the toxicity of the investigated agent does not necessarily increase with dose, optimal dose-finding designs should limit the frequency of treatment with suboptimal doses. We propose a two-stage design strategy with a run-in intra-patient dose escalation part followed by a more traditional dose-finding design. We conduct simulation studies to compare the 3 + 3 design, the Bayesian Optimal Interval Design (BOIN) and the Continual Reassessment Method (CRM) with and without intra-patient dose escalation. The endpoints are accuracy, sample size, safety, and therapeutic efficiency. For scenarios where the correct RP2D is the highest dose, inclusion of an intra-patient dose escalation stage generally increases accuracy and therapeutic efficiency. However, for scenarios where the correct RP2D is below the highest dose, intra-patient dose escalation designs lead to increased risk of overdosing and an overestimation of RP2D. The magnitude of the change in operating characteristics after including an intra-patient stage is largest for the 3 + 3 design, decreases for the BOIN and is smallest for the CRM.

Comparison Of Observation-Based And Model-Based Identification Of Alert Concentrations From Concentration-Expression Data.

MOTIVATION: An important goal of concentration-response studies in toxicology is to determine an 'alert' concentration where a critical level of the response variable is exceeded. In a classical observation-based approach, only measured concentrations are considered as potential alert concentrations. Alternatively, a parametric curve is fitted to the data that describes the relationship between concentration and response. For a prespecified effect level, both an absolute estimate of the alert concentration and an estimate of the lowest concentration where the effect level is exceeded significantly are of interest. RESULTS: In a simulation study for gene expression data, we compared the observation-based and the model-based approach for both absolute and significant exceedance of the prespecified effect level. Results show that, compared to the observation-based approach, the model-based approach overestimates the true alert concentration less often and more frequently leads to a valid estimate, especially for genes with large variance. AVAILABILITY AND IMPLEMENTATION: The code used for the simulation studies is available via the GitHub repository: https://github.com/FKappenberg/Paper-IdentificationAlertConcentrations. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Identification of Interleukin1ß as an Amplifier of Interferon alpha-induced Antiviral Responses.

The induction of an interferon-mediated response is the first line of defense against pathogens such as viruses. Yet, the dynamics and extent of interferon alpha (IFNα)-induced antiviral genes vary remarkably and comprise three expression clusters: early, intermediate and late. By mathematical modeling based on time-resolved quantitative data, we identified mRNA stability as well as a negative regulatory loop as key mechanisms endogenously controlling the expression dynamics of IFNα-induced antiviral genes in hepatocytes. Guided by the mathematical model, we uncovered that this regulatory loop is mediated by the transcription factor IRF2 and showed that knock-down of IRF2 results in enhanced expression of early, intermediate and late IFNα-induced antiviral genes. Co-stimulation experiments with different pro-inflammatory cytokines revealed that this amplified expression dynamics of the early, intermediate and late IFNα-induced antiviral genes can also be achieved by co-application of IFNα and interleukin1 beta (IL1ß). Consistently, we found that IL1ß enhances IFNα-mediated repression of viral replication. Conversely, we observed that in IL1ß receptor knock-out mice replication of viruses sensitive to IFNα is increased. Thus, IL1ß is capable to potentiate IFNα-induced antiviral responses and could be exploited to improve antiviral therapies.

Genetic modifiers of fetal hemoglobin affect the course of sickle cell disease in patients treated with hydroxyurea.

The course of sickle cell disease (SCD) is modified by polymorphisms boosting fetal hemoglobin (HbF) synthesis. However, it has remained an open question how these polymorphisms affect patients who are treated with the HbF-inducing drug hydroxyurea/ hydroxycarbamide. The German SCD registry offers the opportunity to answer this question, because >90% of patients are treated according to national guidelines recommending the use of hydroxyurea in all patients above 2 years of age. We analyzed the modifying effect of HbF-related genetic polymorphisms in 417 patients with homozygous SCD >2 years old who received hydroxyurea. HbF levels were correlated with higher total hemoglobin levels, lower rates of hemolysis, a lower frequency of painful crises and of red blood cell transfusions. The minor alleles of the polymorphisms in the γ-globin promoter (rs7482144), BCL11A (rs1427407) and HMIP (rs66650371) were strongly associated with increased HbF levels. However, these associations did not translate into lower frequencies of vaso-occlusive events which did not differ between patients either carrying or not carrying the HMIP and BCL11A polymorphisms. Patients on hydroxyurea carrying the γ-globin promoter polymorphism demonstrated substantially higher hemoglobin levels (P<10-4) but also higher frequencies of painful crises and hospitalizations (P<0.01) when compared to patients without this polymorphism. Taken together, these data indicate that the γ-globin, HMIP and BCL11A polymorphisms correlate with increased HbF in SCD patients on hydroxyurea. While HbF is negatively correlated with the frequency of painful crises and hospitalizations, this was not observed for the presence of known HbF-boosting alleles.

On the feasibility of pediatric dose-finding trials in small samples with information from a preceding trial in adults.

We consider the case of pediatric dose-finding trials with extremely limited sample size. The operating characteristics of the standard design, the Continual Reassessment Method (CRM), are only well described for sample sizes of about 20 patients or more. In this simulation study, we assume the situation of a pediatric trial with only 10 patients and a preceding dose-finding trial in adults. Based on the adult data, we reduce the set of pediatric doses and formulate (partially) informative prior distributions for the pediatric trial. Our simulations show that such small pediatric dose-finding trials with robustified priors may provide sufficient operating characteristics.

A comparative assessment of intra- and inter- observer repeatability of three widely used software packages for the quantification of defect size in stress myocardial perfusion scintigraphy.

OBJECTIVE: To assess the intra- and inter-observer repeatability of popular software packages for the quantitative determination of abnormality size in stress myocardial perfusion scintigraphy. SUBJECTS AND METHODS: A total of 182 tomographic stress myocardial perfusion scans were processed in duplicate by an experienced and trainee observer to assess SSSext (summed stress score multiplied by 100/68) and total defect extent (TDE), as % of the left ventricle, with 4 dimension-myocardial (4DM), emory cardiac toolbox (ECTb) and quantitative perfusion SPECT (QPS) packages. The Bland-Altman (B-A) analysis and Lin's concordance correlation coefficient (CCC) were used to assess agreement. RESULTS: In SSSext's intra-observer repeatability, CCC showed substantial agreement for 4DM and QPS, and moderate agreement for ECTb for both observers. In inter-observer repeatability, CCC revealed substantial agreement for 4DM and QPS, and poor agreement for ECTb. Regarding TDE, CCC showed substantial intra-observer repeatability for both operators using all packages, while the inter-observer repeatability was substantial for 4DM and QPS, and moderate for ECTb.In SSSext's intra-observer repeatability for 4DM, ECTb and QPS, the B-A analysis provided (mean±1.96SD of paired measurements) 0.0±4.3, 0.2±7.8, -0.6±7.6 for the experienced physician and 0.2±5.9, 0.0±7.5, -0.5±5.4 for the trainee, respectively; in inter-observer repeatability it provided 0.2±5.4, 0.1±9.6, 0.2±8.1, respectively. Regarding TDE, the B-A values for intra-observer repeatability were 0.1±5.2, 0.1±7.9, 0.1±2.8 for the experienced reader and 0.3±6.6, -0.1±6.4, -0.1±2.4 for the trainee, respectively; in inter-observer agreement the B-A provided 0.6±6.4, -0.2±10.3, -0.1±4.3, respectively. CONCLUSION: Considerable differences in intra- and inter- observer agreement were noted for the quantitative determination of defect size using widely employed software packages, suggesting limitations in the clinical use of these measurements. Quantitative perfusion SPECT appears preferable, but with no significant advantage over 4DM. There were no significant differences between the observers.

Interim [18F]FDG PET/CT can predict response to anti-PD-1 treatment in metastatic melanoma.

PURPOSE: In an attempt to identify biomarkers that can reliably predict long-term outcomes to immunotherapy in metastatic melanoma, we investigated the prognostic role of [18F]FDG PET/CT, performed at baseline and early during the course of anti-PD-1 treatment. METHODS: Twenty-five patients with stage IV melanoma, scheduled for treatment with PD-1 inhibitors, were enrolled in the study (pembrolizumab, n = 8 patients; nivolumab, n = 4 patients; nivolumab/ipilimumab, 13 patients). [18F]FDG PET/CT was performed before the start of treatment (baseline PET/CT) and after the initial two cycles of PD-1 blockade administration (interim PET/CT). Seventeen patients underwent also a third PET/CT scan after administration of four cycles of treatment. Evaluation of patients' response by means of PET/CT was performed after application of the European Organization for Research and Treatment of Cancer (EORTC) 1999 criteria and the PET Response Evaluation Criteria for IMmunoTherapy (PERCIMT). Response to treatment was classified into 4 categories: complete metabolic response (CMR), partial metabolic response (PMR), stable metabolic disease (SMD), and progressive metabolic disease (PMD). Patients were further grouped into two groups: those demonstrating metabolic benefit (MB), including patients with SMD, PMR, and CMR, and those demonstrating no MB (no-MB), including patients with PMD. Moreover, patterns of [18F]FDG uptake suggestive of radiologic immune-related adverse events (irAEs) were documented. Progression-free survival (PFS) was measured from the date of interim PET/CT until disease progression or death from any cause. RESULTS: Median follow-up from interim PET/CT was 24.2 months (19.3-41.7 months). According to the EORTC criteria, 14 patients showed MB (1 CMR, 6 PMR, and 7 SMD), while 11 patients showed no-MB (PMD). Respectively, the application of the PERCIMT criteria revealed that 19 patients had MB (1 CMR, 6 PMR, and 12 SMD), and 6 of them had no-MB (PMD). With regard to PFS, no significant difference was observed between patients with MB and no-MB on interim PET/CT according to the EORTC criteria (p = 0.088). In contrary, according to the PERCIMT criteria, patients demonstrating MB had a significantly longer PFS than those showing no-MB (p = 0.045). The emergence of radiologic irAEs (n = 11 patients) was not associated with a significant survival benefit. Regarding the sub-cohort undergoing also a third PET/CT, 14/17 patients (82%) showed concordant responses and 3/17 (18%) had a mismatch of response assessment between interim and late PET/CT. CONCLUSION: PET/CT-based response of metastatic melanoma to PD-1 blockade after application of the recently proposed PERCIMT criteria is significantly correlated with PFS. This highlights the potential ability of [18F]FDG PET/CT for early stratification of response to anti-PD-1 agents, a finding with possible significant clinical and financial implications. Further studies including larger numbers of patients are necessary to validate these results.

Equilibrium radionuclide angiography: Intra- and inter-observer repeatability and reproducibility in the assessment of cardiac systolic and diastolic function.

BACKGROUND: This study aimed to assess intra- and inter-observer agreement in assessing the systolic and diastolic function with equilibrium radionuclide angiography (ERNA). MATERIALS AND METHODS: Thirty-two adults underwent baseline and repeat ERNA. An experienced and a trainee operator analyzed the data by assigning regions of interest manually, fully automatically, and semi-automatically. The Bland-Altman statistic (mean ± 1.96 standard deviations of the differences) was used to assess the repeatability (two different assessments of a single acquisition) and reproducibility (assessments of two different acquisitions). RESULTS: Using the semi-automated technique the intraobserver repeatability and reproducibility of left ventricular ejection fraction for the experienced physician were - 0.1 ± 3.7 and 0.0 ± 3.8 and for the trainee 2.2 ± 10.6 and 1.9 ± 8.4, respectively. The inter-observer repeatability and reproducibility were - 1.8 ± 6.4 and 0.4 ± 9.0, respectively. Among the parameters of diastolic function, the intraobserver repeatability and reproducibility of the peak filling rate for the experienced physician were - 0.0 ± 1.1 and - 0.1 ± 1.1 and for the trainee 0.2 ± 3.5 and 0.4 ± 3.7, respectively. The inter-observer repeatability and reproducibility were 0.3 ± 1.5 and 0.5 ± 4.0, respectively. Similar was the pattern for the other diastolic indices. In all cases the limits of agreement varied according to the quantification approach. CONCLUSION: A good repeatability but a moderate reproducibility was found in the assessment of the LVEF. Less good were the findings in the assessment of diastolic function.