Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology.

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.

EMPAIA App Interface: An open and vendor-neutral interface for AI applications in pathology.

BACKGROUND AND OBJECTIVE: Artificial intelligence (AI) apps hold great potential to make pathological diagnoses more accurate and time efficient. Widespread use of AI in pathology is hampered by interface incompatibilities between pathology software. We studied the existing interfaces in order to develop the EMPAIA App Interface, an open standard for the integration of pathology AI apps. METHODS: The EMPAIA App Interface relies on widely-used web communication protocols and containerization. It consists of three parts: A standardized format to describe the semantics of an app, a mechanism to deploy and execute apps in computing environments, and a web API through which apps can exchange data with a host application. RESULTS: Five commercial AI app manufacturers successfully adapted their products to the EMPAIA App Interface and helped improve it with their feedback. Open source tools facilitate the adoption of the interface by providing reusable data access and scheduling functionality and enabling automatic validation of app compliance. CONCLUSIONS: Existing AI apps and pathology software can be adapted to the EMPAIA App Interface with little effort. It is a viable alternative to the proprietary interfaces of current software. If enough vendors join in, the EMPAIA App Interface can help to advance the use of AI in pathology.

SimRNA: a coarse-grained method for RNA folding simulations and 3D structure prediction.

RNA molecules play fundamental roles in cellular processes. Their function and interactions with other biomolecules are dependent on the ability to form complex three-dimensional (3D) structures. However, experimental determination of RNA 3D structures is laborious and challenging, and therefore, the majority of known RNAs remain structurally uncharacterized. Here, we present SimRNA: a new method for computational RNA 3D structure prediction, which uses a coarse-grained representation, relies on the Monte Carlo method for sampling the conformational space, and employs a statistical potential to approximate the energy and identify conformations that correspond to biologically relevant structures. SimRNA can fold RNA molecules using only sequence information, and, on established test sequences, it recapitulates secondary structure with high accuracy, including correct prediction of pseudoknots. For modeling of complex 3D structures, it can use additional restraints, derived from experimental or computational analyses, including information about secondary structure and/or long-range contacts. SimRNA also can be used to analyze conformational landscapes and identify potential alternative structures.

RNA-Puzzles: a CASP-like evaluation of RNA three-dimensional structure prediction.

We report the results of a first, collective, blind experiment in RNA three-dimensional (3D) structure prediction, encompassing three prediction puzzles. The goals are to assess the leading edge of RNA structure prediction techniques; compare existing methods and tools; and evaluate their relative strengths, weaknesses, and limitations in terms of sequence length and structural complexity. The results should give potential users insight into the suitability of available methods for different applications and facilitate efforts in the RNA structure prediction community in ongoing efforts to improve prediction tools. We also report the creation of an automated evaluation pipeline to facilitate the analysis of future RNA structure prediction exercises.

Wavelength-resolved measurements of fluorescence lifetime of indocyanine green.

We study fluorescence lifetime of indocyanine green (ICG) using femtosecond laser and sensitive detection based on time-correlated single-photon counting. A time-resolved multichannel spectral system is constructed and applied for determination of the fluorescence lifetime of the ICG in different solvents. Emission properties of ICG in water, milk, and 1% intralipid solution are investigated. Fluorescence of the fluorophore of different concentrations (in a range of 1.7-160 µM) dissolved in different solutions is excited by femtosecond pulses generated with the use of Ti:Sa laser tuned within the range of 740-790 nm. It is observed that fluorescence lifetime of ICG in water is 0.166 ± 0.02 ns and does not depend on excitation and emission wavelengths. We also show that for the diffusely scattering solvents (milk and intralipid), the lifetime may depend on the dye concentration (especially for large concentrations of ICG). This effect should be taken into account when analyzing changes in the mean time of arrival of fluorescence photons excited in ICG dissolved in such optically turbid media.