Performance and Agreement When Annotating Chest X-ray Text Reports-A Preliminary Step in the Development of a Deep Learning-Based Prioritization and Detection System.

A chest X-ray report is a communicative tool and can be used as data for developing artificial intelligence-based decision support systems. For both, consistent understanding and labeling is important. Our aim was to investigate how readers would comprehend and annotate 200 chest X-ray reports. Reports written between 1 January 2015 and 11 March 2022 were selected based on search words. Annotators included three board-certified radiologists, two trained radiologists (physicians), two radiographers (radiological technicians), a non-radiological physician, and a medical student. Consensus labels by two or more of the experienced radiologists were considered "gold standard". Matthew's correlation coefficient (MCC) was calculated to assess annotation performance, and descriptive statistics were used to assess agreement between individual annotators and labels. The intermediate radiologist had the best correlation to "gold standard" (MCC 0.77). This was followed by the novice radiologist and medical student (MCC 0.71 for both), the novice radiographer (MCC 0.65), non-radiological physician (MCC 0.64), and experienced radiographer (MCC 0.57). Our findings showed that for developing an artificial intelligence-based support system, if trained radiologists are not available, annotations from non-radiological annotators with basic and general knowledge may be more aligned with radiologists compared to annotations from sub-specialized medical staff, if their sub-specialization is outside of diagnostic radiology.

Inter- and Intra-Observer Agreement When Using a Diagnostic Labeling Scheme for Annotating Findings on Chest X-rays-An Early Step in the Development of a Deep Learning-Based Decision Support System.

Consistent annotation of data is a prerequisite for the successful training and testing of artificial intelligence-based decision support systems in radiology. This can be obtained by standardizing terminology when annotating diagnostic images. The purpose of this study was to evaluate the annotation consistency among radiologists when using a novel diagnostic labeling scheme for chest X-rays. Six radiologists with experience ranging from one to sixteen years, annotated a set of 100 fully anonymized chest X-rays. The blinded radiologists annotated on two separate occasions. Statistical analyses were done using Randolph's kappa and PABAK, and the proportions of specific agreements were calculated. Fair-to-excellent agreement was found for all labels among the annotators (Randolph's Kappa, 0.40-0.99). The PABAK ranged from 0.12 to 1 for the two-reader inter-rater agreement and 0.26 to 1 for the intra-rater agreement. Descriptive and broad labels achieved the highest proportion of positive agreement in both the inter- and intra-reader analyses. Annotating findings with specific, interpretive labels were found to be difficult for less experienced radiologists. Annotating images with descriptive labels may increase agreement between radiologists with different experience levels compared to annotation with interpretive labels.

The Added Effect of Artificial Intelligence on Physicians' Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review.

Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.

Altered evoked low-frequency connectivity from SI to ACC following nerve injury in rats.

Objective. Despite decades of research on central processing of pain, there are still several unanswered questions, in particular regarding the brain regions that may contribute to this alerting sensation. Since it is generally accepted that more than one cortical area is responsible for pain processing, there is an increasing focus on the interaction between areas known to be involved.Approach. In this study, we aimed to investigate the bidirectional information flow from the primary somatosensory cortex (SI) to the anterior cingulate cortex (ACC) in an animal model of neuropathic pain.19 rats (nine controls and ten intervention) had an intracortical electrode implanted with six pins in SI and six pins in ACC, and a cuff stimulation electrode around the sciatic nerve. The intervention rats were subjected to the spared nerve injury (SNI) after baseline recordings. Electrical stimulation at three intensities of both noxious and non-noxious stimulation was used to record electrically evoked cortical potentials. To investigate information flow, two connectivity measures were used: phase lag index (PLI) and granger prediction (GP). The rats were anesthetized during the entire study.Main results. Immediately after the intervention (<5 min after intervention), the high frequency (γandγ+) PLI was significantly decreased compared to controls. In the last recording cycle (3-4 h after intervention), the GP increased consistently in the intervention group. Peripheral nerve injury, as a model of neuropathic pain, resulted in an immediate decrease in information flow between SI and ACC, possibly due to decreased sensory input from the injured nerve. Hours after injury, the connectivity between SI and ACC increased, likely indicating hypersensitivity of this pathway.Significance. We have shown that both a directed and non-directed connectivity between SI and ACC approach can be used to show the acute changes resulting from the SNI model.

Modulation of SI and ACC response to noxious and non-noxious electrical stimuli after the spared nerve injury model of neuropathic pain.

BACKGROUND: The current knowledge on the role of SI and ACC in acute pain processing and how these contribute to the development of chronic pain is limited. Our objective was to investigate differences in and modulation of intracortical responses from SI and ACC in response to different intensities of peripheral presumed noxious and non-noxious stimuli in the acute time frame of a peripheral nerve injury in rats. METHODS: We applied non-noxious and noxious electrical stimulation pulses through a cuff electrode placed around the sciatic nerve and measured the cortical responses (six electrodes in each cortical area) before and after the spared nerve injury model. RESULTS: We found that the peak response correlated with the stimulation intensity and that SI and ACC differed in both amplitude and latency of cortical response. The cortical response to both noxious and non-noxious stimulation showed a trend towards faster processing of non-noxious stimuli in ACC and increased cortical processing of non-noxious stimuli in SI after SNI. CONCLUSIONS: We found different responses in SI and ACC to different intensity electrical stimulations based on two features and changes in these features following peripheral nerve injury. We believe that these features may be able to assist to track cortical changes during the chronification of pain in future animal studies. SIGNIFICANCE: This study showed distinct cortical processing of noxious and non-noxious peripheral stimuli in SI and ACC. The processing latency in ACC and accumulated spiking activity in SI appeared to be modulated by peripheral nerve injury, which elaborated on the function of these two areas in the processing of nociception.

Nerve Injury Decreases Hyperacute Resting-State Connectivity Between the Anterior Cingulate and Primary Somatosensory Cortex in Anesthetized Rats.

A better understanding of neural pain processing and of the development of pain over time, is critical to identify objective measures of pain and to evaluate the effect of pain alleviation therapies. One issue is, that the brain areas known to be related to pain processing are not exclusively responding to painful stimuli, and the neuronal activity is also influenced by other brain areas. Functional connectivity reflects synchrony or covariation of activation between groups of neurons. Previous studies found changes in connectivity days or weeks after pain induction. However, less in known on the temporal development of pain. Our objective was therefore to investigate the interaction between the anterior cingulate cortex (ACC) and primary somatosensory cortex (SI) in the hyperacute (minute) and sustained (hours) response in an animal model of neuropathic pain. Intra-cortical local field potentials (LFP) were recorded in 18 rats. In 10 rats the spared nerve injury model was used as an intervention. The intra-cortical activity was recorded before, immediately after, and three hours after the intervention. The interaction was quantified as the calculated correlation and coherence. The results from the intervention group showed a decrease in correlation between ACC and SI activity, which was most pronounced in the hyperacute phase but a longer time frame may be required for plastic changes to occur. This indicated that both SI and ACC are involved in hyperacute pain processing.