Large Language Models as a Rapid and Objective Tool for Pathology Report Data Extraction.

Medical institutions continuously create a substantial amount of data that is used for scientific research. One of the departments with a great amount of archived data is the pathology department. Pathology archives hold the potential to create a case series of valuable rare entities or large cohorts of common entities. The major problem in creation of these databases is data extraction which is still commonly done manually and is highly laborious and error prone. For these reasons, we offer using large language models to overcome these challenges. Ten pathology reports of selected resection specimens were retrieved from electronic archives of Koç University Hospital for the initial set. These reports were de-identified and uploaded to ChatGPT and Google Bard. Both algorithms were asked to turn the reports in a synoptic report format that is easy to export to a data editor such as Microsoft Excel or Google Sheets. Both programs created tables with Google Bard facilitating the creation of a spreadsheet from the data automatically. In conclusion, we propose the use of AI-assisted data extraction for academic research purposes, as it may enhance efficiency and precision compared to manual data entry.

Artificial intelligence assisted patient blood and urine droplet pattern analysis for non-invasive and accurate diagnosis of bladder cancer.

Bladder cancer is one of the most common cancer types in the urinary system. Yet, current bladder cancer diagnosis and follow-up techniques are time-consuming, expensive, and invasive. In the clinical practice, the gold standard for diagnosis remains invasive biopsy followed by histopathological analysis. In recent years, costly diagnostic tests involving the use of bladder cancer biomarkers have been developed, however these tests have high false-positive and false-negative rates limiting their reliability. Hence, there is an urgent need for the development of cost-effective, and non-invasive novel diagnosis methods. To address this gap, here we propose a quick, cheap, and reliable diagnostic method. Our approach relies on an artificial intelligence (AI) model to analyze droplet patterns of blood and urine samples obtained from patients and comparing them to cancer-free control subjects. The AI-assisted model in this study uses a deep neural network, a ResNet network, pre-trained on ImageNet datasets. Recognition and classification of complex patterns formed by dried urine or blood droplets under different conditions resulted in cancer diagnosis with a high specificity and sensitivity. Our approach can be systematically applied across droplets, enabling comparisons to reveal shared spatial behaviors and underlying morphological patterns. Our results support the fact that AI-based models have a great potential for non-invasive and accurate diagnosis of malignancies, including bladder cancer.

HENLE FIBER LAYER MAPPING WITH DIRECTIONAL OPTICAL COHERENCE TOMOGRAPHY.

PURPOSE: To perform a macular volumetric and topographic analysis of Henle fiber layer (HFL) from retinal scans acquired by directional optical coherence tomography. METHODS: Thirty healthy eyes of 17 subjects were imaged using the Heidelberg spectral-domain optical coherence tomography (Spectralis, Heidelberg Engineering, Heidelberg, Germany) with varied horizontal and vertical pupil entry. Manual segmentation of HFL was performed from retinal sections of horizontally and vertically tilted optical coherence tomography images acquired within macular 20 × 20° area. Total HFL volume, mean HFL thickness, and HFL coverage area within Early Treatment for Diabetic Retinopathy Study grid were calculated from mapped images. RESULTS: Henle fiber layer of 30 eyes were imaged, segmented and mapped. The mean total HFL volume was 0.74 ± 0.08 mm 3 with 0.16 ± 0.02 mm 3 , 0.18 ± 0.03 mm 3 , 0.17 ± 0.02 mm 3 , and 0.19 ± 0.03 mm 3 for superior, temporal, inferior, and nasal quadrants, respectively. The mean HFL thickness was 26.5 ± 2.9 µ m. Central 1-mm macular zone had the highest mean HFL thickness with 51.0 ± 7.6 µ m. The HFL coverage that have thickness equal or above to the mean value had a mean 10.771 ± 0.574 mm 2 of surface area. CONCLUSION: Henle fiber layer mapping is a promising tool for structural analysis of HFL. Identifying a normative data of HFL morphology will allow further studies to investigate HFL involvement in various ocular and systemic disorders.

FourierNet: Shape-Preserving Network for Henle's Fiber Layer Segmentation in Optical Coherence Tomography Images.

Henle's fiber layer (HFL), a retinal layer located in the outer retina between the outer nuclear and outer plexiform layers (ONL and OPL, respectively), is composed of uniformly linear photoreceptor axons and Müller cell processes. However, in the standard optical coherence tomography (OCT) imaging, this layer is usually included in the ONL since it is difficult to perceive HFL contours on OCT images. Due to its variable reflectivity under an imaging beam, delineating the HFL contours necessitates directional OCT, which requires additional imaging. This paper addresses this issue by introducing a shape-preserving network, FourierNet, which achieves HFL segmentation in standard OCT scans with the target performance obtained when directional OCT is available. FourierNet is a new cascaded network design that puts forward the idea of benefiting the shape prior of the HFL in the network training. This design proposes to represent the shape prior by extracting Fourier descriptors on the HFL contours and defining an additional regression task of learning these descriptors. FourierNet then formulates HFL segmentation as concurrent learning of regression and classification tasks, in which Fourier descriptors are estimated from an input image to encode the shape prior and used together with the input image to construct the HFL segmentation map. Our experiments on 1470 images of 30 OCT scans of healthy-looking macula reveal that quantifying the HFL shape with Fourier descriptors and concurrently learning them with the main segmentation task leads to significantly better results. These findings indicate the effectiveness of designing a shape-preserving network to facilitate HFL segmentation by reducing the need to perform directional OCT imaging.

AttentionBoost: Learning What to Attend for Gland Segmentation in Histopathological Images by Boosting Fully Convolutional Networks.

Fully convolutional networks (FCNs) are widely used for instance segmentation. One important challenge is to sufficiently train these networks to yield good generalizations for hard-to-learn pixels, correct prediction of which may greatly affect the success. A typical group of such hard-to-learn pixels are boundaries between instances. Many studies have developed strategies to pay more attention to learning these boundary pixels. They include designing multi-task networks with an additional task of boundary prediction and increasing the weights of boundary pixels' predictions in the loss function. Such strategies require defining what to attend beforehand and incorporating this defined attention to the learning model. However, there may exist other groups of hard-to-learn pixels and manually defining and incorporating the appropriate attention for each group may not be feasible. In order to provide an adaptable solution to learn different groups of hard-to-learn pixels, this article proposes AttentionBoost, which is a new multi-attention learning model based on adaptive boosting, for the task of gland instance segmentation in histopathological images. AttentionBoost designs a multi-stage network and introduces a new loss adjustment mechanism for an FCN to adaptively learn what to attend at each stage directly on image data without necessitating any prior definition. This mechanism modulates the attention of each stage to correct the mistakes of previous stages, by adjusting the loss weight of each pixel prediction separately with respect to how accurate the previous stages are on this pixel. Working on histopathological images of colon tissues, our experiments demonstrate that the proposed AttentionBoost model improves the results of gland segmentation compared to its counterparts.

DeepDistance: A multi-task deep regression model for cell detection in inverted microscopy images.

This paper presents a new deep regression model, which we call DeepDistance, for cell detection in images acquired with inverted microscopy. This model considers cell detection as a task of finding most probable locations that suggest cell centers in an image. It represents this main task with a regression task of learning an inner distance metric. However, different than the previously reported regression based methods, the DeepDistance model proposes to approach its learning as a multi-task regression problem where multiple tasks are learned by using shared feature representations. To this end, it defines a secondary metric, normalized outer distance, to represent a different aspect of the problem and proposes to define its learning as complementary to the main cell detection task. In order to learn these two complementary tasks more effectively, the DeepDistance model designs a fully convolutional network (FCN) with a shared encoder path and end-to-end trains this FCN to concurrently learn the tasks in parallel. For further performance improvement on the main task, this paper also presents an extended version of the DeepDistance model that includes an auxiliary classification task and learns it in parallel to the two regression tasks by also sharing feature representations with them. DeepDistance uses the inner distances estimated by these FCNs in a detection algorithm to locate individual cells in a given image. In addition to this detection algorithm, this paper also suggests a cell segmentation algorithm that employs the estimated maps to find cell boundaries. Our experiments on three different human cell lines reveal that the proposed multi-task learning models, the DeepDistance model and its extended version, successfully identify the locations of cell as well as delineate their boundaries, even for the cell line that was not used in training, and improve the results of its counterparts.

Unsupervised Feature Extraction via Deep Learning for Histopathological Classification of Colon Tissue Images.

Histopathological examination is today's gold standard for cancer diagnosis. However, this task is time consuming and prone to errors as it requires a detailed visual inspection and interpretation of a pathologist. Digital pathology aims at alleviating these problems by providing computerized methods that quantitatively analyze digitized histopathological tissue images. The performance of these methods mainly relies on the features that they use, and thus, their success strictly depends on the ability of these features by successfully quantifying the histopathology domain. With this motivation, this paper presents a new unsupervised feature extractor for effective representation and classification of histopathological tissue images. This feature extractor has three main contributions: First, it proposes to identify salient subregions in an image, based on domain-specific prior knowledge, and to quantify the image by employing only the characteristics of these subregions instead of considering the characteristics of all image locations. Second, it introduces a new deep learning-based technique that quantizes the salient subregions by extracting a set of features directly learned on image data and uses the distribution of these quantizations for image representation and classification. To this end, the proposed deep learning-based technique constructs a deep belief network of the restricted Boltzmann machines (RBMs), defines the activation values of the hidden unit nodes in the final RBM as the features, and learns the quantizations by clustering these features in an unsupervised way. Third, this extractor is the first example for successfully using the restricted Boltzmann machines in the domain of histopathological image analysis. Our experiments on microscopic colon tissue images reveal that the proposed feature extractor is effective to obtain more accurate classification results compared to its counterparts.

Object-Oriented Segmentation of Cell Nuclei in Fluorescence Microscopy Images.

Cell nucleus segmentation remains an open and challenging problem especially to segment nuclei in cell clumps. Splitting a cell clump would be straightforward if the gradients of boundary pixels in-between the nuclei were always higher than the others. However, imperfections may exist: inhomogeneities of pixel intensities in a nucleus may cause to define spurious boundaries whereas insufficient pixel intensity differences at the border of overlapping nuclei may cause to miss some true boundary pixels. In contrast, these imperfections are typically observed at the pixel-level, causing local changes in pixel values without changing the semantics on a large scale. In response to these issues, this article introduces a new nucleus segmentation method that relies on using gradient information not at the pixel level but at the object level. To this end, it proposes to decompose an image into smaller homogeneous subregions, define edge-objects at four different orientations to encode the gradient information at the object level, and devise a merging algorithm, in which the edge-objects vote for subregion pairs along their orientations and the pairs are iteratively merged if they get sufficient votes from multiple orientations. Our experiments on fluorescence microscopy images reveal that this high-level representation and the design of a merging algorithm using edge-objects (gradients at the object level) improve the segmentation results.

Cartilage Endplate Thickness Variation Measured by Ultrashort Echo-Time MRI Is Associated With Adjacent Disc Degeneration.

STUDY DESIGN: A magnetic resonance imaging study of human cadaver spines. OBJECTIVE: To investigate associations between cartilage endplate (CEP) thickness and disc degeneration. SUMMARY OF BACKGROUND DATA: Damage to the CEP is associated with spinal injury and back pain. However, CEP morphology and its association with disc degeneration have not been well characterized. METHODS: Ten lumbar motion segments with varying degrees of disc degeneration were harvested from six cadaveric spines and scanned with magnetic resonance imaging in the sagittal plane using a T2-weighted two-dimensional (2D) sequence, a three-dimensional (3D) ultrashort echo-time (UTE) imaging sequence, and a 3D T1ρ mapping sequence. CEP thicknesses were calculated from 3D UTE image data using a custom, automated algorithm, and these values were validated against histology measurements. Pfirrmann grades and T1ρ values in the disc were assessed and correlated with CEP thickness. RESULTS: The mean CEP thickness calculated from UTE images was 0.74 ±â€Š0.04 mm. Statistical comparisons between histology and UTE-derived measurements of CEP thickness showed significant agreement, with the mean difference not significantly different from zero (P = 0.32). Within-disc variation of T1ρ (standard deviation) was significantly lower for Pfirrmann grade 4 than Pfirrmann grade 3 (P < 0.05). Within-disc variation of T1ρ and adjacent CEP thickness heterogeneity (coefficient of variation) had a significant negative correlation (r = -0.65, P = 0.04). The standard deviation of T1ρand the mean CEP thickness showed a moderate positive correlation (r = 0.40, P = 0.26). CONCLUSION: This study demonstrates that quantitative measurements of CEP thickness measured from UTE magnetic resonance imaging are associated with disc degeneration. Our results suggest that variability in CEP thickness and T1ρ, rather than their mean values, may serve as valuable diagnostic markers for disc degeneration. LEVEL OF EVIDENCE: N/A.

Iterative h-minima-based marker-controlled watershed for cell nucleus segmentation.

Automated microscopy imaging systems facilitate high-throughput screening in molecular cellular biology research. The first step of these systems is cell nucleus segmentation, which has a great impact on the success of the overall system. The marker-controlled watershed is a technique commonly used by the previous studies for nucleus segmentation. These studies define their markers finding regional minima on the intensity/gradient and/or distance transform maps. They typically use the h-minima transform beforehand to suppress noise on these maps. The selection of the h value is critical; unnecessarily small values do not sufficiently suppress the noise, resulting in false and oversegmented markers, and unnecessarily large ones suppress too many pixels, causing missing and undersegmented markers. Because cell nuclei show different characteristics within an image, the same h value may not work to define correct markers for all the nuclei. To address this issue, in this work, we propose a new watershed algorithm that iteratively identifies its markers, considering a set of different h values. In each iteration, the proposed algorithm defines a set of candidates using a particular h value and selects the markers from those candidates provided that they fulfill the size requirement. Working with widefield fluorescence microscopy images, our experiments reveal that the use of multiple h values in our iterative algorithm leads to better segmentation results, compared to its counterparts. © 2016 International Society for Advancement of Cytometry.

Two-tier tissue decomposition for histopathological image representation and classification.

In digital pathology, devising effective image representations is crucial to design robust automated diagnosis systems. To this end, many studies have proposed to develop object-based representations, instead of directly using image pixels, since a histopathological image may contain a considerable amount of noise typically at the pixel-level. These previous studies mostly employ color information to define their objects, which approximately represent histological tissue components in an image, and then use the spatial distribution of these objects for image representation and classification. Thus, object definition has a direct effect on the way of representing the image, which in turn affects classification accuracies. In this paper, our aim is to design a classification system for histopathological images. Towards this end, we present a new model for effective representation of these images that will be used by the classification system. The contributions of this model are twofold. First, it introduces a new two-tier tissue decomposition method for defining a set of multityped objects in an image. Different than the previous studies, these objects are defined combining texture, shape, and size information and they may correspond to individual histological tissue components as well as local tissue subregions of different characteristics. As its second contribution, it defines a new metric, which we call dominant blob scale, to characterize the shape and size of an object with a single scalar value. Our experiments on colon tissue images reveal that this new object definition and characterization provides distinguishing representation of normal and cancerous histopathological images, which is effective to obtain more accurate classification results compared to its counterparts.

A color and shape based algorithm for segmentation of white blood cells in peripheral blood and bone marrow images.

Computer-based imaging systems are becoming important tools for quantitative assessment of peripheral blood and bone marrow samples to help experts diagnose blood disorders such as acute leukemia. These systems generally initiate a segmentation stage where white blood cells are separated from the background and other nonsalient objects. As the success of such imaging systems mainly depends on the accuracy of this stage, studies attach great importance for developing accurate segmentation algorithms. Although previous studies give promising results for segmentation of sparsely distributed normal white blood cells, only a few of them focus on segmenting touching and overlapping cell clusters, which is usually the case when leukemic cells are present. In this article, we present a new algorithm for segmentation of both normal and leukemic cells in peripheral blood and bone marrow images. In this algorithm, we propose to model color and shape characteristics of white blood cells by defining two transformations and introduce an efficient use of these transformations in a marker-controlled watershed algorithm. Particularly, these domain specific characteristics are used to identify markers and define the marking function of the watershed algorithm as well as to eliminate false white blood cells in a postprocessing step. Working on 650 white blood cells in peripheral blood and bone marrow images, our experiments reveal that the proposed algorithm improves the segmentation performance compared with its counterparts, leading to high accuracies for both sparsely distributed normal white blood cells and dense leukemic cell clusters.

Local object patterns for the representation and classification of colon tissue images.

This paper presents a new approach for the effective representation and classification of images of histopathological colon tissues stained with hematoxylin and eosin. In this approach, we propose to decompose a tissue image into its histological components and introduce a set of new texture descriptors, which we call local object patterns, on these components to model their composition within a tissue. We define these descriptors using the idea of local binary patterns, which quantify a pixel by constructing a binary string based on relative intensities of its neighbors. However, as opposed to pixel-level local binary patterns, we define our local object pattern descriptors at the component level to quantify a component. To this end, we specify neighborhoods with different locality ranges and encode spatial arrangements of the components within the specified local neighborhoods by generating strings. We then extract our texture descriptors from these strings to characterize histological components and construct the bag-of-words representation of an image from the characterized components. Working on microscopic images of colon tissues, our experiments reveal that the use of these component-level texture descriptors results in higher classification accuracies than the previous textural approaches.

Semi-automatic segmentation of subcutaneous tumours from micro-computed tomography images.

This paper outlines the first attempt to segment the boundary of preclinical subcutaneous tumours, which are frequently used in cancer research, from micro-computed tomography (microCT) image data. MicroCT images provide low tissue contrast, and the tumour-to-muscle interface is hard to determine, however faint features exist which enable the boundary to be located. These are used as the basis of our semi-automatic segmentation algorithm. Local phase feature detection is used to highlight the faint boundary features, and a level set-based active contour is used to generate smooth contours that fit the sparse boundary features. The algorithm is validated against manually drawn contours and micro-positron emission tomography (microPET) images. When compared against manual expert segmentations, it was consistently able to segment at least 70% of the tumour region (n = 39) in both easy and difficult cases, and over a broad range of tumour volumes. When compared against tumour microPET data, it was able to capture over 80% of the functional microPET volume. Based on these results, we demonstrate the feasibility of subcutaneous tumour segmentation from microCT image data without the assistance of exogenous contrast agents. Our approach is a proof-of-concept that can be used as the foundation for further research, and to facilitate this, the code is open-source and available from www.setuvo.com.

Attributed relational graphs for cell nucleus segmentation in fluorescence microscopy images.

More rapid and accurate high-throughput screening in molecular cellular biology research has become possible with the development of automated microscopy imaging, for which cell nucleus segmentation commonly constitutes the core step. Although several promising methods exist for segmenting the nuclei of monolayer isolated and less-confluent cells, it still remains an open problem to segment the nuclei of more-confluent cells, which tend to grow in overlayers. To address this problem, we propose a new model-based nucleus segmentation algorithm. This algorithm models how a human locates a nucleus by identifying the nucleus boundaries and piecing them together. In this algorithm, we define four types of primitives to represent nucleus boundaries at different orientations and construct an attributed relational graph on the primitives to represent their spatial relations. Then, we reduce the nucleus identification problem to finding predefined structural patterns in the constructed graph and also use the primitives in region growing to delineate the nucleus borders. Working with fluorescence microscopy images, our experiments demonstrate that the proposed algorithm identifies nuclei better than previous nucleus segmentation algorithms.

A hybrid classification model for digital pathology using structural and statistical pattern recognition.

Cancer causes deviations in the distribution of cells, leading to changes in biological structures that they form. Correct localization and characterization of these structures are crucial for accurate cancer diagnosis and grading. In this paper, we introduce an effective hybrid model that employs both structural and statistical pattern recognition techniques to locate and characterize the biological structures in a tissue image for tissue quantification. To this end, this hybrid model defines an attributed graph for a tissue image and a set of query graphs as a reference to the normal biological structure. It then locates key regions that are most similar to a normal biological structure by searching the query graphs over the entire tissue graph. Unlike conventional approaches, this hybrid model quantifies the located key regions with two different types of features extracted using structural and statistical techniques. The first type includes embedding of graph edit distances to the query graphs whereas the second one comprises textural features of the key regions. Working with colon tissue images, our experiments demonstrate that the proposed hybrid model leads to higher classification accuracies, compared against the conventional approaches that use only statistical techniques for tissue quantification.

Smart markers for watershed-based cell segmentation.

Automated cell imaging systems facilitate fast and reliable analysis of biological events at the cellular level. In these systems, the first step is usually cell segmentation that greatly affects the success of the subsequent system steps. On the other hand, similar to other image segmentation problems, cell segmentation is an ill-posed problem that typically necessitates the use of domain-specific knowledge to obtain successful segmentations even by human subjects. The approaches that can incorporate this knowledge into their segmentation algorithms have potential to greatly improve segmentation results. In this work, we propose a new approach for the effective segmentation of live cells from phase contrast microscopy. This approach introduces a new set of "smart markers" for a marker-controlled watershed algorithm, for which the identification of its markers is critical. The proposed approach relies on using domain-specific knowledge, in the form of visual characteristics of the cells, to define the markers. We evaluate our approach on a total of 1,954 cells. The experimental results demonstrate that this approach, which uses the proposed definition of smart markers, is quite effective in identifying better markers compared to its counterparts. This will, in turn, be effective in improving the segmentation performance of a marker-controlled watershed algorithm.

Multilevel segmentation of histopathological images using cooccurrence of tissue objects.

This paper presents a new approach for unsupervised segmentation of histopathological tissue images. This approach has two main contributions. First, it introduces a new set of high-level texture features to represent the prior knowledge of spatial organization of the tissue components. These texture features are defined on the tissue components, which are approximately represented by tissue objects, and quantify the frequency of two component types being cooccurred in a particular spatial relationship. As they are defined on components, rather than on image pixels, these object cooccurrence features are expected to be less vulnerable to noise and variations that are typically observed at the pixel level of tissue images. Second, it proposes to obtain multiple segmentations by multilevel partitioning of a graph constructed on the tissue objects and combine them by an ensemble function. This multilevel graph partitioning algorithm introduces randomization in graph construction and refinements in its multilevel scheme to increase diversity of individual segmentations, and thus, improve the final result. The experiments on 200 colon tissue images reveal that the proposed approach--the object cooccurrence features together with the multilevel segmentation algorithm--is effective to obtain high-quality results. The experiments also show that it improves the segmentation results compared to the previous approaches.

A resampling-based Markovian model for automated colon cancer diagnosis.

In recent years, there has been a great effort in the research of implementing automated diagnostic systems for tissue images. One major challenge in this implementation is to design systems that are robust to image variations. In order to meet this challenge, it is important to learn the systems on a large number of labeled images from a different range of variation. However, acquiring labeled images is quite difficult in this domain, and hence, the labeled training data are typically very limited. Although the issue of having limited labeled data is acknowledged by many researchers, it has rarely been considered in the system design. This paper successfully addresses this issue, introducing a new resampling framework to simulate variations in tissue images. This framework generates multiple sequences from an image for its representation and models them using a Markov process. Working with colon tissue images, our experiments show that this framework increases the generalization capacity of a learner by increasing the size and variation of the training data and improves the classification performance of a given image by combining the decisions obtained on its sequences.

Graph run-length matrices for histopathological image segmentation.

The histopathological examination of tissue specimens is essential for cancer diagnosis and grading. However, this examination is subject to a considerable amount of observer variability as it mainly relies on visual interpretation of pathologists. To alleviate this problem, it is very important to develop computational quantitative tools, for which image segmentation constitutes the core step. In this paper, we introduce an effective and robust algorithm for the segmentation of histopathological tissue images. This algorithm incorporates the background knowledge of the tissue organization into segmentation. For this purpose, it quantifies spatial relations of cytological tissue components by constructing a graph and uses this graph to define new texture features for image segmentation. This new texture definition makes use of the idea of gray-level run-length matrices. However, it considers the runs of cytological components on a graph to form a matrix, instead of considering the runs of pixel intensities. Working with colon tissue images, our experiments demonstrate that the texture features extracted from "graph run-length matrices" lead to high segmentation accuracies, also providing a reasonable number of segmented regions. Compared with four other segmentation algorithms, the results show that the proposed algorithm is more effective in histopathological image segmentation.