Datasets for Benchmarking RNA Design Algorithms.

RNA molecules play vital roles in many biological processes, such as gene regulation or protein synthesis. The adoption of a specific secondary and tertiary structure by RNA is essential to perform these diverse functions, making RNA a popular tool in bioengineering therapeutics. The field of RNA design responds to the need to develop novel RNA molecules that possess specific functional attributes. In recent years, computational tools for predicting RNA sequences with desired folding characteristics have improved and expanded. However, there is still a lack of well-defined and standardized datasets to assess these programs. Here, we present a large dataset of internal and multibranched loops extracted from PDB-deposited RNA structures that encompass a wide spectrum of design difficulties. Furthermore, we conducted benchmarking tests of widely utilized open-source RNA design algorithms employing this dataset.

Impact of signal-to-noise ratio and contrast definition on the sensitivity assessment and benchmarking of fluorescence molecular imaging systems.

Significance: Standardization of fluorescence molecular imaging (FMI) is critical for ensuring quality control in guiding surgical procedures. To accurately evaluate system performance, two metrics, the signal-to-noise ratio (SNR) and contrast, are widely employed. However, there is currently no consensus on how these metrics can be computed. Aim: We aim to examine the impact of SNR and contrast definitions on the performance assessment of FMI systems. Approach: We quantified the SNR and contrast of six near-infrared FMI systems by imaging a multi-parametric phantom. Based on approaches commonly used in the literature, we quantified seven SNRs and four contrast values considering different background regions and/or formulas. Then, we calculated benchmarking (BM) scores and respective rank values for each system. Results: We show that the performance assessment of an FMI system changes depending on the background locations and the applied quantification method. For a single system, the different metrics can vary up to ∼ 35 dB (SNR), ∼ 8.65 a . u . (contrast), and ∼ 0.67 a . u . (BM score). Conclusions: The definition of precise guidelines for FMI performance assessment is imperative to ensure successful clinical translation of the technology. Such guidelines can also enable quality control for the already clinically approved indocyanine green-based fluorescence image-guided surgery.

Development of the Local Food Systems Policy Index (Local Food-EPI+) tool and assessment process to benchmark the implementation of local government policies for creating healthy, equitable and environmentally sustainable food systems.

OBJECTIVE: Local governments have an important role to play in creating healthy, equitable and environmentally sustainable food systems. This study aimed to develop and pilot a tool and process for local governments in Australia to benchmark their policies for creating healthy, equitable and environmentally sustainable food systems. DESIGN: The Healthy Food Environment Policy Index (Food-EPI), developed in 2013 for national governments, was tailored to develop the Local Food Systems Policy Index (Local Food-EPI+) tool for local governments. To incorporate environmental sustainability and the local government context, this process involved a literature review and collaboration with an international and domestic expert advisory committee (n 35) and local government officials. SETTING: Local governments. RESULTS: The tool consists of sixty-one indicators across ten food policy domains (weighted based on relative importance): leadership; governance; funding and resources; monitoring and intelligence; food production and supply chain; food promotion; food provision and retail in public facilities and spaces; supermarkets and food sources in the community; food waste reuse, redistribution and reduction; and support for communities. Pilot implementation of the tool in one local government demonstrated that the assessment process was feasible and likely to be helpful in guiding policy implementation. CONCLUSION: The Local Food-EPI+ tool and assessment process offer a comprehensive mechanism to assist local governments in benchmarking their actions to improve the healthiness, equity and environmental sustainability of food systems and prioritise action areas. Broad use of this tool will identify and promote leading practices, increase accountability for action and build capacity and collaborations.

The Children's National Hospital Outcomes Registry: a plea for a benchmarking tool that provides longitudinal outcomes for patients and families.

Surgical advancements in paediatric cardiovascular surgery have led to improved survival rates for those patients with the most complex CHDs leading to greater numbers of patients who are living well into adulthood. Despite this new era of long-term survival, our current reporting systems continue to focus largely on using short-term postoperative outcomes as the criteria to both rate and rank hospitals. Using such limited criteria to rate and rank hospitals may mislead the intended audiences: patients and families. The goal of this article is to describe the creation of a local benchmarking report which aims to retrospectively review long-term outcomes from our single centre. This report is updated annually and published on our cardiac surgery webpage in an effort to be as transparent as possible for our patient and family communities.

Jointly benchmarking small and structural variant calls with vcfdist.

In this work, we extend vcfdist to be the first variant call benchmarking tool to jointly evaluate phased single-nucleotide polymorphisms (SNPs), small insertions/deletions (INDELs), and structural variants (SVs) for the whole genome. First, we find that a joint evaluation of small and structural variants uniformly reduces measured errors for SNPs (- 28.9%), INDELs (- 19.3%), and SVs (- 52.4%) across three datasets. vcfdist also corrects a common flaw in phasing evaluations, reducing measured flip errors by over 50%. Lastly, we show that vcfdist is more accurate than previously published works and on par with the newest approaches while providing improved result interpretability.

Assessing different next-generation sequencing technologies for wastewater-based epidemiology.

Wastewater-based epidemiology has proven to be an important public health asset during the COVID-19 pandemic. It can provide less biassed and more cost-effective population-level monitoring of the disease burden as compared to clinical testing. An essential component of SARS-CoV-2 wastewater monitoring is next-generation sequencing, providing genomic data to identify and quantify circulating viral strains rapidly. However, the specific choice of sequencing method influences the quality and timeliness of generated data and hence its usefulness for wastewater-based pathogen surveillance. Here, we systematically benchmarked Illumina Novaseq 6000, Element Aviti, ONT R9.4.1 MinION flow cell, and ONT R9.4.1 Flongle flow cell sequencing data to facilitate the selection of sequencing technology. Using a time series of wastewater samples from influent of six wastewater treatment plants throughout Switzerland, along with spike-in experiments, we show that higher sequencing error rates of ONT Nanopore sequencing reduce the accuracy of estimates of the relative abundance of viral variants, but the overall trend is in good concordance among all technologies. We find that the sequencing runtime for ONT Nanopore flow cells can be reduced to as little as five hours without significant impact on the quality of variant estimates. Our findings suggest that SARS-CoV-2 variant tracking is readily achievable with all tested technologies, albeit with different tradeoffs in terms of cost, timeliness and accuracy.

WHO-listed authorities (WLA) framework: transparent evidence-based approach for promoting regulatory reliance towards increased access to quality-assured medical products.

Background: Increased global access to safe, effective and quality-assured medical products remains a primary goal for the full realization of the World Health Assembly Resolution WHA 67.20 on regulatory systems strengthening for medical products as well as target 3.8 of the Sustainable Development Goals (SDG). To promote the development of efficient regulatory systems, the WHO introduced the Global Benchmarking Tool (GBT) in 2016, upon which the WHO-Listed Authority (WLA) framework was later established. This study aimed to appraise the development of the WLA framework across various phases while highlighting its achievements, challenges, and areas for improvement. Methods: An exploratory study design using a qualitative approach was used to gather information from relevant documents as well as views and experiences from purposefully selected participants from diverse backgrounds. Data was collected using a combination of desk reviews and In-depth one-to-one or small group interviews employing semi-structured interview guides with open-ended questions. Data was analysed using an inductive thematic analysis approach. Results: The leading role of the WHO was noted in developing and implementing essential documents and mediating consultative processes among stakeholders. The framework was revealed to bring an evidence-based, inclusive, and transparent approach to recognizing regulatory authorities (RAs) operating at the highest standards of performance. The framework was anticipated to promote regulatory reliance among all RAs, the WHO's prequalification programme, and procurement agencies. Furthermore, remarkable progress towards WLA listing was noted among transitional WLAs including the Stringent Regulatory Authorities (SRAs). Challenges related to the availability of resources, resistance to change, and complexity were associated with the framework. Conclusion: The study provides a well-rounded view with regard to the roles of the WHO, Member States and other stakeholders in establishing and operationalizing the WLA framework. Furthermore, evaluating the performance and possible WLA designation of RAs operating at international regulatory standards underscores its high relevance in contributing to public health globally. Maintenance along with timely addressing of highlighted next steps to improve the framework particularly in creating better understanding, more communication, and coordination are highly encouraged.

Detecting Rhythmic Gene Expression in Single-cell Transcriptomics.

An autonomous, environmentally synchronizable circadian rhythm is a ubiquitous feature of life on Earth. In multicellular organisms, this rhythm is generated by a transcription-translation feedback loop present in nearly every cell that drives daily expression of thousands of genes in a tissue-dependent manner. Identifying the genes that are under circadian control can elucidate the mechanisms by which physiological processes are coordinated in multicellular organisms. Today, transcriptomic profiling at the single-cell level provides an unprecedented opportunity to understand the function of cell-level clocks. However, while many cycling detection algorithms have been developed to identify genes under circadian control in bulk transcriptomic data, it is not known how best to adapt these algorithms to single-cell RNA seq data. Here, we benchmark commonly used circadian detection methods on their reliability and efficiency when applied to single-cell RNA seq data. Our results provide guidance on adapting existing cycling detection methods to the single-cell domain and elucidate opportunities for more robust and efficient rhythm detection in single-cell data. We also propose a subsampling procedure combined with harmonic regression as an efficient strategy to detect circadian genes in the single-cell setting.

New Parameter for Benchmarking Plasmonic Gas Sensors Demonstrated with Densely Packed Au Nanoparticle Layers.

Localized surface plasmon resonance (LSPR) gas sensitivity is introduced as a new parameter to evaluate the performance of plasmonic gas sensors. A model is proposed to consider the plasmonic sensors' surface sensitivity and plasmon decay length and correlate the LSPR response, measured upon gas exchange, with an equivalent refractive index change consistent with adsorbed gas layers. To demonstrate the applicability of this new parameter, ellipsoidal gold nanoparticles (NPs) arranged in densely packed hexagonal lattices were fabricated. The main advantages of these sensors are the small and tunable interparticle gaps (18-29 nm) between nanoparticles (diameters: 72-88 nm), with their robust and scalable fabrication technology that allows the well-ordered arrangement to be maintained on a large (cm2 range) area. The LSPR response of the sensors was tested using an LSPR sensing system by switching the gas atmosphere between inorganic gases, namely He/Ar and Ar/CO2, at constant pressure and room temperature. It was shown that this newly proposed parameter can be generally used for benchmarking plasmonic gas sensors and is independent of the type and pressure of the tested gases for a sensor structure. Furthermore, it resolves the apparent disagreement when comparing the response of plasmonic sensors tested in liquids and gases.

The development of sodium reduction targets for New Zealand fast foods and a comparison with the current sodium contents of products.

Sodium intake attributed to fast food is increasing globally. This research aims to develop maximum sodium reduction targets for New Zealand (NZ) fast foods and compare them with the current sodium content of products. Sodium content and serving size data were sourced from an existing database of major NZ fast-food chains. Target development followed a step-by-step process, informed by international targets and serving sizes, and previous methods for packaged supermarket foods. Sodium reduction targets were set per 100 g and serving, using a 40% reduction in the mean sodium content or the value met by 35-45% of products. Thirty-four per cent (1797/5246) of products in the database had sodium data available for target development. Sodium reduction targets were developed for 17 fast-food categories. Per 100 g targets ranged from 158 mg for 'Other salads' to 665 mg for 'Mayonnaise and dressings'. Per serving targets ranged from 118 mg for 'Sauce' to 1270 mg for 'Burgers with cured meat'. The largest difference between the current mean sodium content and corresponding target was for 'Other salads' and 'Grilled Chicken' (both -40% per 100g) and 'Fries and potato products' (-45% per serving), and the smallest, 'Pizza with cured meat toppings' (-3% per 100 g) and 'Pies, tarts, sausage rolls and quiches' (-4% per serving). The results indicate the display of nutrition information should be mandated and there is considerable room for sodium reduction in NZ fast foods. The methods described provide a model for other countries to develop country-specific, fast-food sodium reduction targets.

A machine learning algorithm-based risk prediction score for in-hospital/30 day mortality after adult cardiac surgery.

OBJECTIVES: A study of the performance of in-hospital/30 day mortality risk prediction models using an alterantive machine learning algorithm(XGBoost) in adults undergoing cardiac surgery. METHODS: Retrospective analyses of prospectively routinely collected data on adult patients undergoing cardiac surgery in the UK(UK) from January 2012 to March 2019.Data was temporally split 70:30 into training and validation subsets. Independent mortality prediction models were created using sequential backward floating selection(SBFS) starting with 61 variables. Assessments of discrimination, calibration and clinical utility of the resultant XGBoost model with 23 variables were then conducted. RESULTS: 224,318 adults underwent cardiac surgery during the study period with a 2.76%(N = 6,100) mortality. In the testing cohort, there was good discrimination(AUC 0.846, F1 0.277) and calibration(especially in high-risk patients). Decision curve analysis showed XGBoost-23 had a greater net benefit till a threshold probability of 60%.The most important variables were the type of operation, age, creatinine clearance, urgency of the procedure and the New York Heart Association score. CONCLUSIONS: Feature-selected XGBoost showed good discrimination, calibration and clinical benefit when predicting mortality post-cardiac surgery. Prospective external validation of a XGBoost-derived model performance is warranted.

The value of simplified models of radionuclide transport for the safety assessment of nuclear waste repositories: A benchmark study.

In order to assess sites for a deep geological repository for storing high-level nuclear waste safely in Germany, various numerical models and tools will be in use. For their interaction within one workflow, their reproducibility, and reliability version-controlled open-source solutions and careful documentation of model setups, results and verifications are of special value. However, spatially fully resolved models including all relevant physical and chemical processes are neither computationally feasible for large domains nor is the data typically available to parameterize such models. Thus, simplified models are crucial for the pre-assessment of possible sites to narrow down the list of suitable candidates for which detailed site investigations and fully resolved models will be done at a later stage. Still, the accuracy of these simplified models is of importance as the pre-assessment of suitable sites will be based on them. In this study, we compare the modelling capabilities of TransPyREnd, a one-dimensional transport code based on finite differences, specifically developed for the fast estimation of radionuclide transport by the German federal company for radioactive waste disposal (BGE), with OpenGeoSys, which is a modelling platform based on finite elements in up to three spatial dimensions. Both codes are used in the site selection procedure for the German nuclear waste repository. The comparison of the model results of TransPyREnd and OpenGeoSys is augmented by comparisons with an analytical solution for a homogeneous material. For the purpose of numerical benchmarking, we consider a geological profile located in southern Germany as an example where the hypothetical repository is located in a clay-stone formation. TransPyREnd and OpenGeoSys yield overall similar results. However, both codes use different discretizations which impact is the highest for strongly sorbing compounds, while the difference gets negligible for less sorbing and more diffusive compounds as higher diffusion tends to blur the initial conditions. Overall, the OpenGeoSys model is more exact whereas the TransPyREnd model has considerable faster run times. We found in our example, that significant substance amounts are only leaving the host rock formation, if apparent diffusion is high, for which case both codes give similar results, while relative differences are considerable for strongly sorbing compounds. However, in the latter case no significant substance amount of radionuclides leaves the host-rock formation, thus deeming the differences in the model results minor for the overall safety assessments of sites.

SpeechBrain-MOABB: An open-source Python library for benchmarking deep neural networks applied to EEG signals.

Deep learning has revolutionized EEG decoding, showcasing its ability to outperform traditional machine learning models. However, unlike other fields, EEG decoding lacks comprehensive open-source libraries dedicated to neural networks. Existing tools (MOABB and braindecode) prevent the creation of robust and complete decoding pipelines, as they lack support for hyperparameter search across the entire pipeline, and are sensitive to fluctuations in results due to network random initialization. Furthermore, the absence of a standardized experimental protocol exacerbates the reproducibility crisis in the field. To address these limitations, we introduce SpeechBrain-MOABB, a novel open-source toolkit carefully designed to facilitate the development of a comprehensive EEG decoding pipeline based on deep learning. SpeechBrain-MOABB incorporates a complete experimental protocol that standardizes critical phases, such as hyperparameter search and model evaluation. It natively supports multi-step hyperparameter search for finding the optimal hyperparameters in a high-dimensional space defined by the entire pipeline, and multi-seed training and evaluation for obtaining performance estimates robust to the variability caused by random initialization. SpeechBrain-MOABB outperforms other libraries, including MOABB and braindecode, with accuracy improvements of 14.9% and 25.2% (on average), respectively. By enabling easy-to-use and easy-to-share decoding pipelines, our toolkit can be exploited by neuroscientists for decoding EEG with neural networks in a replicable and trustworthy way.

Benchmarking deep learning-based low-dose CT image denoising algorithms.

BACKGROUND: Long-lasting efforts have been made to reduce radiation dose and thus the potential radiation risk to the patient for computed tomography (CT) acquisitions without severe deterioration of image quality. To this end, various techniques have been employed over the years including iterative reconstruction methods and noise reduction algorithms. PURPOSE: Recently, deep learning-based methods for noise reduction became increasingly popular and a multitude of papers claim ever improving performance both quantitatively and qualitatively. However, the lack of a standardized benchmark setup and inconsistencies in experimental design across studies hinder the verifiability and reproducibility of reported results. METHODS: In this study, we propose a benchmark setup to overcome those flaws and improve reproducibility and verifiability of experimental results in the field. We perform a comprehensive and fair evaluation of several state-of-the-art methods using this standardized setup. RESULTS: Our evaluation reveals that most deep learning-based methods show statistically similar performance, and improvements over the past years have been marginal at best. CONCLUSIONS: This study highlights the need for a more rigorous and fair evaluation of novel deep learning-based methods for low-dose CT image denoising. Our benchmark setup is a first and important step towards this direction and can be used by future researchers to evaluate their algorithms.

[Geriatric assessment goes digital-Possible applications in the emergency department setting]. / Geriatrisches Assessment goes digital ­ Einsatzmöglichkeiten im Setting der Notaufnahme.

The geriatric assessment is a basic requirement and a key quality parameter in geriatric care. An increasing number of older patients are presenting to emergency or central admission departments and discharge units in hospitals. For this reason, and in view of the time-critical decision-making requirements in this setting, digital applications of basic geriatric assessment data are becoming increasingly more important for the high-quality follow-up care of geriatric patients.

VCF observer: a user-friendly software tool for preliminary VCF file analysis and comparison.

BACKGROUND: Advancements over the past decade in DNA sequencing technology and computing power have created the potential to revolutionize medicine. There has been a marked increase in genetic data available, allowing for the advancement of areas such as personalized medicine. A crucial type of data in this context is genetic variant data which is stored in variant call format (VCF) files. However, the rapid growth in genomics has presented challenges in analyzing and comparing VCF files. RESULTS: In response to the limitations of existing tools, this paper introduces a novel web application that provides a user-friendly solution for VCF file analyses and comparisons. The software tool enables researchers and clinicians to perform high-level analysis with ease and enhances productivity. The application's interface allows users to conveniently upload, analyze, and visualize their VCF files using simple drag-and-drop and point-and-click operations. Essential visualizations such as Venn diagrams, clustergrams, and precision-recall plots are provided to users. A key feature of the application is its support for metadata-based file grouping, accomplished through flexible data matrix uploads, streamlining organization and analysis of user-defined categories. Additionally, the application facilitates standardized benchmarking of VCF files by integrating user-provided ground truth regions and variant lists. CONCLUSIONS: By providing a user-friendly interface and supporting essential visualizations, this software enhances the accessibility of VCF file analysis and assists researchers and clinicians in their scientific inquiries.

The Benchmarking Exercise Programme for Older People (BEPOP): Design, Results and Recommendations from The First Wave of Data Collection.

Objectives: The Benchmarking Exercise Programme for Older People (BEPOP) service improvement project seeks to determine and promote the exercise training characteristics associated with positive outcomes for resistance exercise for older people living with, or at risk of, sarcopenia or physical frailty. Methods: Mixed-methods service improvement project. Ten UK National Health Service physiotherapist-led therapy services delivering exercise interventions for older people submitted anonymized data for up to 20 consecutive patients. A multidisciplinary expert panel generated a report and recommendations with site-specific benchmarking data and feedback. In parallel, participating physiotherapy team members were interviewed to elicit feedback on BEPOP rationale, processes and perceived value. Results: Data from 188 patients were included, mean age 80 years (range 60-101). 115 (61%) received objective assessment of strength-based physical performance. Bodyweight exercises (173 [92%]) and resistance bands (49 [26%]) were the commonest exercise modalities. Exercises progressed predominantly through increased repetitions (163 [87%]) rather than increased load. 50 (30%) had no reassessment of outcomes; only 68 (41%) were signposted to follow-on exercise services. Staff interviews identified themes around knowledge, diagnosis, data collection and practice reflection. Conclusion: BEPOP was feasible to deliver and generated actionable insights for service improvement via improved diagnosis, measurement and progression of resistance exercise.

Development of a Quality Indicator Set for the Optimal Acute Management of Moderate to Severe Traumatic Brain Injury in the Australian Context.

BACKGROUND: The aim of this study was to develop a consensus-based set of indicators of high-quality acute moderate to severe traumatic brain injury (msTBI) clinical management that can be used to measure structure, process, and outcome factors that are likely to influence patient outcomes. This is the first stage of the PRECISION-TBI program, which is a prospective cohort study that aims to identify and promote optimal clinical management of msTBI in Australia. METHODS: A preliminary set of 45 quality indicators was developed based on available evidence. An advisory committee of established experts in the field refined the initial indicator set in terms of content coverage, proportional representation, contamination, and supporting evidence. The refined indicator set was then distributed to a wider Delphi panel for assessment of each indicator in terms of validity, measurement feasibility, variability, and action feasibility. Inclusion in the final indicator set was contingent on prespecified inclusion scoring. RESULTS: The indicator set was structured according to the care pathway of msTBI and included prehospital, emergency department, neurosurgical, intensive care, and rehabilitation indicators. Measurement domains included structure indicators, logistic indicators, and clinical management indicators. The Delphi panel consisted of 44 participants (84% physician, 12% nursing, and 4% primary research) with a median of 15 years of practice. Of the 47 indicators included in the second round of the Delphi, 32 indicators were approved by the Delphi group. CONCLUSIONS: This study identified a set of 32 quality indicators that can be used to structure data collection to drive quality improvement in the clinical management of msTBI. They will also be used to guide feedback to PRECISION-TBI's participating sites.

Monte Carlo Model Validation of 6MV Beam of OMID, the First Iranian Linear Accelerator.

Monte Carlo (MC) techniques are regarded as an accurate method to simulate the dose calculation in radiotherapy for many years. The present paper aims to validate the simulated model of the 6-MV beam of OMID linear accelerator (BEHYAAR Company) by EGSnrc codes system and also investigate the effects of initial electron beam parameters (energy, radial full width at half maximum, and mean angular spread) on dose distributions. For this purpose, the comparison between the calculated and measured percentage depth dose (PDD) and lateral dose profiles was done by gamma index (GI) with 1%-1 mm acceptance criteria. MC model validating was done for 3 cm × 3 cm, 5 cm × 5 cm, 8 cm × 8 cm, 10 cm × 10 cm, and 20 cm × 20 cm field sizes. To study the sensitivity of model to beam parameters, the field size was selected as 10 cm × 10 cm and 30 cm × 30 cm. All lateral dose profiles were obtained at 10 cm. Excellent agreement was achieved with a 99.2% GI passing percentage for PDD curves and at least 93.8% GI for lateral dose profiles for investigated field sizes. Our investigation confirmed that the lateral dose profile severely depends on the considered source parameters in this study. PDD only considerably depends on the initial electron beam energy. Therefore, source parameters should not be specified independently. These results indicate that the current model of OMID 6-MV Linac is well established, and the accuracy of the simulation is high enough to be used in various applications.