Interactive and automated application of virtual microscopy.

Virtual microscopy can be applied in an interactive and an automated manner. Interactive application is performed in close association to conventional microscopy. It includes image standardization suitable to the performance of an individual pathologist such as image colorization, white color balance, or individual adjusted brightness. The steering commands have to include selection of wanted magnification, easy navigation, notification, and simple measurements (distances, areas). The display of the histological image should be adjusted to the physical limits of the human eye, which are determined by a view angle of approximately 35 seconds. A more sophisticated performance should include acoustic commands that replace the corresponding visual commands. Automated virtual microscopy includes so-called microscopy assistants which can be defined similar to the developed assistants in computer based editing systems (Microsoft Word, etc.). These include an automated image standardization and correction algorithms that excludes images of poor quality (for example uni-colored or out-of-focus images), an automated selection of the most appropriate field of view, an automated selection of the best magnification, and finally proposals of the most probable diagnosis. A quality control of the final diagnosis, and feedback to the laboratory determine the proposed system. The already developed tools of such a system are described in detail, as well as the results of first trials. In order to enhance the speed of such a system, and to allow further user-independent development a distributed implementation probably based upon Grid technology seems to be appropriate. The advantages of such a system as well as the present pathology environment and its expectations will be discussed in detail.

How to measure diagnosis-associated information in virtual slides.

The distribution of diagnosis-associated information in histological slides is often spatial dependent. A reliable selection of the slide areas containing the most significant information to deriving the associated diagnosis is a major task in virtual microscopy. Three different algorithms can be used to select the appropriate fields of view: 1) Object dependent segmentation combined with graph theory; 2) time series associated texture analysis; and 3) geometrical statistics based upon geometrical primitives. These methods can be applied by sliding technique (i.e., field of view selection with fixed frames), and by cluster analysis. The implementation of these methods requires a standardization of images in terms of vignette correction and gray value distribution as well as determination of appropriate magnification (method 1 only). A principle component analysis of the color space can significantly reduce the necessary computation time. Method 3 is based upon gray value dependent segmentation followed by graph theory application using the construction of (associated) minimum spanning tree and Voronoi's neighbourhood condition. The three methods have been applied on large sets of histological images comprising different organs (colon, lung, pleura, stomach, thyroid) and different magnifications, The trials resulted in a reproducible and correct selection of fields of view in all three methods. The different algorithms can be combined to a basic technique of field of view selection, and a general theory of "image information" can be derived. The advantages and constraints of the applied methods will be discussed.

Grid computing in image analysis.

Diagnostic surgical pathology or tissue­based diagnosis still remains the most reliable and specific diagnostic medical procedure. The development of whole slide scanners permits the creation of virtual slides and to work on so-called virtual microscopes. In addition to interactive work on virtual slides approaches have been reported that introduce automated virtual microscopy, which is composed of several tools focusing on quite different tasks. These include evaluation of image quality and image standardization, analysis of potential useful thresholds for object detection and identification (segmentation), dynamic segmentation procedures, adjustable magnification to optimize feature extraction, and texture analysis including image transformation and evaluation of elementary primitives. Grid technology seems to possess all features to efficiently target and control the specific tasks of image information and detection in order to obtain a detailed and accurate diagnosis. Grid technology is based upon so-called nodes that are linked together and share certain communication rules in using open standards. Their number and functionality can vary according to the needs of a specific user at a given point in time. When implementing automated virtual microscopy with Grid technology, all of the five different Grid functions have to be taken into account, namely 1) computation services, 2) data services, 3) application services, 4) information services, and 5) knowledge services. Although all mandatory tools of automated virtual microscopy can be implemented in a closed or standardized open system, Grid technology offers a new dimension to acquire, detect, classify, and distribute medical image information, and to assure quality in tissue­based diagnosis.

Theory of sampling and its application in tissue based diagnosis.

BACKGROUND: A general theory of sampling and its application in tissue based diagnosis is presented. Sampling is defined as extraction of information from certain limited spaces and its transformation into a statement or measure that is valid for the entire (reference) space. The procedure should be reproducible in time and space, i.e. give the same results when applied under similar circumstances. Sampling includes two different aspects, the procedure of sample selection and the efficiency of its performance. The practical performance of sample selection focuses on search for localization of specific compartments within the basic space, and search for presence of specific compartments. METHODS: When a sampling procedure is applied in diagnostic processes two different procedures can be distinguished: I) the evaluation of a diagnostic significance of a certain object, which is the probability that the object can be grouped into a certain diagnosis, and II) the probability to detect these basic units. Sampling can be performed without or with external knowledge, such as size of searched objects, neighbourhood conditions, spatial distribution of objects, etc. If the sample size is much larger than the object size, the application of a translation invariant transformation results in Kriege's formula, which is widely used in search for ores. Usually, sampling is performed in a series of area (space) selections of identical size. The size can be defined in relation to the reference space or according to interspatial relationship. The first method is called random sampling, the second stratified sampling. RESULTS: Random sampling does not require knowledge about the reference space, and is used to estimate the number and size of objects. Estimated features include area (volume) fraction, numerical, boundary and surface densities. Stratified sampling requires the knowledge of objects (and their features) and evaluates spatial features in relation to the detected objects (for example grey value distribution around an object). It serves also for the definition of parameters of the probability function in so-called active segmentation. CONCLUSION: The method is useful in standardization of images derived from immunohistochemically stained slides, and implemented in the EAMUS system http://www.diagnomX.de. It can also be applied for the search of "objects possessing an amplification function", i.e. a rare event with "steering function". A formula to calculate the efficiency and potential error rate of the described sampling procedures is given.

AI (artificial intelligence) in histopathology--from image analysis to automated diagnosis.

The technological progress in digitalization of complete histological glass slides has opened a new door in tissue--based diagnosis. The presentation of microscopic images as a whole in a digital matrix is called virtual slide. A virtual slide allows calculation and related presentation of image information that otherwise can only be seen by individual human performance. The digital world permits attachments of several (if not all) fields of view and the contemporary visualization on a screen. The presentation of all microscopic magnifications is possible if the basic pixel resolution is less than 0.25 microns. To introduce digital tissue--based diagnosis into the daily routine work of a surgical pathologist requires a new setup of workflow arrangement and procedures. The quality of digitized images is sufficient for diagnostic purposes; however, the time needed for viewing virtual slides exceeds that of viewing original glass slides by far. The reason lies in a slower and more difficult sampling procedure, which is the selection of information containing fields of view. By application of artificial intelligence, tissue--based diagnosis in routine work can be managed automatically in steps as follows: 1. The individual image quality has to be measured, and corrected, if necessary. 2. A diagnostic algorithm has to be applied. An algorithm has be developed, that includes both object based (object features, structures) and pixel based (texture) measures. 3. These measures serve for diagnosis classification and feedback to order additional information, for example in virtual immunohistochemical slides. 4. The measures can serve for automated image classification and detection of relevant image information by themselves without any labeling. 5. The pathologists' duty will not be released by such a system; to the contrary, it will manage and supervise the system, i.e., just working at a "higher level". Virtual slides are already in use for teaching and continuous education in anatomy and pathology. First attempts to introduce them into routine work have been reported. Application of AI has been established by automated immunohistochemical measurement systems (EAMUS, www.diagnomX.eu). The performance of automated diagnosis has been reported for a broad variety of organs at sensitivity and specificity levels >85%). The implementation of a complete connected AI supported system is in its childhood. Application of AI in digital tissue--based diagnosis will allow the pathologists to work as supervisors and no longer as primary "water carriers". Its accurate use will give them the time needed to concentrating on difficult cases for the benefit of their patients.

How to implement grid technology in tissue-based diagnosis: diagnostic surgical pathology.

BACKGROUND: Tissue-based diagnosis or diagnostic surgical pathology is a highly accurate, sensitive and specific medical diagnostic technique that has expanded rapidly in using both molecular biology and computer technology. OBJECTIVE: The objective is to analyze the present stage and potential influence of distributed data acquisition, analysis and presentation in tissue-based diagnosis by using recently developed standardized network systems such as grids. METHODS: Interpretation of medical data is often based upon specialized examination, visual information acquisition and transfer as well as upon data collected from various sources. Efficient and accurate diagnostics require standardized data and transfer modes, which can be provided by a grid environment. The medical requirements, construction of an adequate grid environment, practical experiences in various medical disciplines and potential use in tissue-based diagnosis are described. CONCLUSIONS: Grid technology is probably a useful tool to meet the conditions of tissue-based diagnosis in the near future, and will probably play a significant role in its further development.

How to measure image quality in tissue-based diagnosis (diagnostic surgical pathology).

BACKGROUND: Automated image analysis, measurements of virtual slides, and open access electronic measurement user systems require standardized image quality assessment in tissue-based diagnosis. AIMS: To describe the theoretical background and the practical experiences in automated image quality estimation of colour images acquired from histological slides. THEORY, MATERIAL AND MEASUREMENTS: Digital images acquired from histological slides should present with textures and objects that permit automated image information analysis. The quality of digitized images can be estimated by spatial independent and local filter operations that investigate in homogenous brightness, low peak to noise ratio (full range of available grey values), maximum gradients, equalized grey value distribution, and existence of grey value thresholds. Transformation of the red-green-blue (RGB) space into the hue-saturation-intensity (HSI) space permits the detection of colour and intensity maxima/minima. The feature distance of the original image to its standardized counterpart is an appropriate measure to quantify the actual image quality. These measures have been applied to a series of H&E stained, fluorescent (DAPI, Texas Red, FITC), and immunohistochemically stained (PAP, DAB) slides. More than 5,000 slides have been measured and partly analyzed in a time series. RESULTS: Analysis of H&E stained slides revealed low shading corrections (10%) and moderate grey value standardization (10 - 20%) in the majority of cases. Immunohistochemically stained slides displayed greater shading and grey value correction. Fluorescent stained slides are often revealed to high brightness. Images requiring only low standardization corrections possess at least 5 different statistically significant thresholds, which are useful for object segmentation. Fluorescent images of good quality only possess one singular intensity maximum in contrast to good images obtained from H&E stained slides that present with 2 - 3 intensity maxima. CONCLUSION: Evaluation of image quality and creation of formally standardized images should be performed prior to automatic analysis of digital images acquired from histological slides. Spatial dependent and local filter operations as well as analysis of the RGB and HSI spaces are appropriate methods to reproduce evaluated formal image quality.

Image standards in tissue-based diagnosis (diagnostic surgical pathology).

BACKGROUND: Progress in automated image analysis, virtual microscopy, hospital information systems, and interdisciplinary data exchange require image standards to be applied in tissue-based diagnosis. AIMS: To describe the theoretical background, practical experiences and comparable solutions in other medical fields to promote image standards applicable for diagnostic pathology. THEORY AND EXPERIENCES: Images used in tissue-based diagnosis present with pathology-specific characteristics. It seems appropriate to discuss their characteristics and potential standardization in relation to the levels of hierarchy in which they appear. All levels can be divided into legal, medical, and technological properties. Standards applied to the first level include regulations or aims to be fulfilled. In legal properties, they have to regulate features of privacy, image documentation, transmission, and presentation; in medical properties, features of disease-image combination, human-diagnostics, automated information extraction, archive retrieval and access; and in technological properties features of image acquisition, display, formats, transfer speed, safety, and system dynamics. The next lower second level has to implement the prescriptions of the upper one, i.e. describe how they are implemented. Legal aspects should demand secure encryption for privacy of all patient related data, image archives that include all images used for diagnostics for a period of 10 years at minimum, accurate annotations of dates and viewing, and precise hardware and software information. Medical aspects should demand standardized patients' files such as DICOM 3 or HL 7 including history and previous examinations, information of image display hardware and software, of image resolution and fields of view, of relation between sizes of biological objects and image sizes, and of access to archives and retrieval. Technological aspects should deal with image acquisition systems (resolution, colour temperature, focus, brightness, and quality evaluation procedures), display resolution data, implemented image formats, storage, cycle frequency, backup procedures, operation system, and external system accessibility. The lowest third level describes the permitted limits and threshold in detail. At present, an applicable standard including all mentioned features does not exist to our knowledge; some aspects can be taken from radiological standards (PACS, DICOM 3); others require specific solutions or are not covered yet. CONCLUSION: The progress in virtual microscopy and application of artificial intelligence (AI) in tissue-based diagnosis demands fast preparation and implementation of an internationally acceptable standard. The described hierarchic order as well as analytic investigation in all potentially necessary aspects and details offers an appropriate tool to specifically determine standardized requirements.

Grid technology in tissue-based diagnosis: fundamentals and potential developments.

Tissue-based diagnosis still remains the most reliable and specific diagnostic medical procedure. It is involved in all technological developments in medicine and biology and incorporates tools of quite different applications. These range from molecular genetics to image acquisition and recognition algorithms (for image analysis), or from tissue culture to electronic communication services. Grid technology seems to possess all features to efficiently target specific constellations of an individual patient in order to obtain a detailed and accurate diagnosis in providing all relevant information and references. Grid technology can be briefly explained by so-called nodes that are linked together and share certain communication rules in using open standards. The number of nodes can vary as well as their functionality, depending on the needs of a specific user at a given point in time. In the beginning of grid technology, the nodes were used as supercomputers in combining and enhancing the computation power. At present, at least five different Grid functions can be distinguished, that comprise 1) computation services, 2) data services, 3) application services, 4) information services, and 5) knowledge services. The general structures and functions of a Grid are described, and their potential implementation into virtual tissue-based diagnosis is analyzed. As a result Grid technology offers a new dimension to access distributed information and knowledge and to improving the quality in tissue-based diagnosis and therefore improving the medical quality.