Assessing expert reliability in determining intracranial EEG channel quality and introducing the automated bad channel detection algorithm.

Objective.To evaluate the inter- and intra-rater reliability for the identification of bad channels among neurologists, EEG Technologists, and naïve research personnel, and to compare their performance with the automated bad channel detection (ABCD) algorithm for detecting bad channels.Approach.Six Neurologists, ten EEG Technologists, and six naïve research personnel (22 raters in total) were asked to rate 1440 real intracranial EEG channels as good or bad. Intra- and interrater kappa statistics were calculated for each group. We then compared each group to the ABCD algorithm which uses spectral and temporal domain features to classify channels as good or bad.Main results.Analysis of channel ratings from our participants revealed variable intra-rater reliability within each group, with no significant differences across groups. Inter-rater reliability was moderate among neurologists and EEG Technologists but minimal among naïve participants. Neurologists demonstrated a slightly higher consistency in ratings than EEG Technologists. Both groups occasionally misclassified flat channels as good, and participants generally focused on low-frequency content for their assessments. The ABCD algorithm, in contrast, relied more on high-frequency content. A logistic regression model showed a linear relationship between the algorithm's ratings and user responses for predominantly good channels, but less so for channels rated as bad. Sensitivity and specificity analyses further highlighted differences in rating patterns among the groups, with neurologists showing higher sensitivity and naïve personnel higher specificity.Significance.Our study reveals the bias in human assessments of intracranial electroencephalography (iEEG) data quality and the tendency of even experienced professionals to overlook certain bad channels, highlighting the need for standardized, unbiased methods. The ABCD algorithm, outperforming human raters, suggests the potential of automated solutions for more reliable iEEG interpretation and seizure characterization, offering a reliable approach free from human biases.

Fundamental Stability Skills: Reliability Analysis Using the Alfamov Assessment Tool.

Fundamental movement skills (FMS), considered as building blocks of movement, have received growing interest due to their significant impact on both present and future health. FMS are categorized into locomotor, object control and stability skills. While there has been extensive research on assessing the proficiency and reliability of locomotor and object control skills, stability skills have received comparatively less attention. For this reason, this study aimed to assess the test-retest, intrarater and interrater reliability of five stability skills included in the Alfamov app. The performance of eighty-four healthy primary school children (60.8% girls), aged 6 to 12 years (mean ± standard deviation of 8.7 ± 1.8 years), in five stability skills was evaluated and scored by four raters, including two experts and two novices. The Alfamov tool, integrating various process-oriented tests, was used for the assessment. Reliability analyses were conducted through the computation of the intraclass correlation coefficient (ICC) along with the corresponding 95% confidence intervals. Good-to-excellent intrarater reliability, excellent interrater reliability and moderate-to-good reliability in the test-retest were achieved. The results proved that Alfamov is a robust test for evaluating stability skills and can be suitable for use by different professionals with less experience in assessing children's motor competence.

Reliability of Markerless Motion Capture Systems for Assessing Movement Screenings.

Background: Movement screenings are commonly used to detect unfavorable movement patterns. Markerless motion capture systems have been developed to track 3-dimensional motion. Purpose: To determine the reliability of movement screenings assessed using a markerless motion capture system when comparing the results of multiple systems and multiple collection periods. Study Design: Descriptive laboratory study. Methods: The inter- and intrarater reliability of a commercially available markerless motion capture system were investigated in 21 recreationally active participants aged between 18 and 22 years. A total of 39 kinematic variables arising from 10 fundamental upper and lower body movements typical of a screening procedure in sports performance were considered. The data were statistically analyzed in terms of relative error via the intraclass correlation coefficient (ICC) and absolute error via the residual standard error (RSE). Results: Both inter- and intrarater reliability ICCs were at least moderate across all variables (ICC, >0.50), with most movements and corresponding variables having excellent reliability (ICC, >0.90). Although maximum knee valgus angles were the kinematic variables with the lowest interrater reliability (ICCs, 0.59-0.82) and moderate relative agreement, there was agreement in absolute terms with an RSE of <1.3°. Conclusion: Findings indicated that markerless motion capture provides reliable measurements of joint position during a movement screen, which allows for a more objective evaluation of the direction and subsequent success of interventions. However, practitioners should consider relative and absolute agreements when applying information provided by these systems. Clinical Relevance: Markerless motion capture systems may assist clinicians by reliably assessing movement screenings using different systems over different collection periods.

Review of sample size determination methods for the intraclass correlation coefficient in the one-way analysis of variance model.

Reliability of measurement instruments providing quantitative outcomes is usually assessed by an intraclass correlation coefficient. When participants are repeatedly measured by a single rater or device, or, are each rated by a different group of raters, the intraclass correlation coefficient is based on a one-way analysis of variance model. When planning a reliability study, it is essential to determine the number of participants and measurements per participant (i.e. number of raters or number of repeated measurements). Three different sample size determination approaches under the one-way analysis of variance model were identified in the literature, all based on a confidence interval for the intraclass correlation coefficient. Although eight different confidence interval methods can be identified, Wald confidence interval with Fisher's large sample variance approximation remains most commonly used despite its well-known poor statistical properties. Therefore, a first objective of this work is comparing the statistical properties of all identified confidence interval methods-including those overlooked in previous studies. A second objective is developing a general procedure to determine the sample size using all approaches since a closed-form formula is not always available. This procedure is implemented in an R Shiny app. Finally, we provide advice for choosing an appropriate sample size determination method when planning a reliability study.

Ultrasound measurement of subsynovial connective tissue thickness in the carpal tunnel: An intrarater/interrater reliability and agreement study.

OBJECTIVES: Non-inflammatory thickening of the subsynovial connective tissue (SSCT) in the carpal tunnel is commonly found in subjects with carpal tunnel syndrome (CTS), and quantification may shed light on CTS pathogenesis. To date, information on the reliability of ultrasound quantification of SSCT is scarce. Therefore, we investigated intrarater and interrater reliability/agreement for ultrasound quantification of SSCT thickness in subjects with and without CTS, and predictors for tissue thickness. MATERIAL AND METHODS: Two investigators quantified SSCT thickness and thickness ratio on ultrasound in 16 healthy subjects (age, 24-65 years; 16 left/14 right wrists) and 17 subjects with CTS (age, 37-83 years; 14 left/14 right wrists). Intra- and inter-rater reliability/agreement were assessed on intraclass correlation coefficients, standard error of measurement and minimal detectable change. A mixed-effects model was used to evaluate potential predictors for SSCT thickness. RESULTS: Intra- and inter-rater reliability analysis showed good to excellent intraclass correlation coefficients in both groups, ranging from 0.772 to 0.965. The maximum percentage standard error of measurement was 8%. The maximum minimal detectable change was 14% within raters, and 20% between raters. Both intra- and inter-rater reliability values for thickness ratio were poor. Presence of CTS (ß = 0.180; p = 0.015) correlated positively with SSCT thickness. CONCLUSIONS: Ultrasound is a reliable method for quantification of SSCT thickness, but not for thickness ratio. Presence of CTS correlates positively with SSCT thickness.

Validity, Reliability, and Efficiency of a Standard Goniometer, Medical Inclinometer, and Builder's Inclinometer.

Background: Joint range of motion (ROM) is an important assessment to aid diagnostic and clinical decision-making for persons with a wide variety of neuromusculoskeletal conditions. The current clinical standard for assessing ROM is the standard goniometer (SG). Purpose: The purpose of this study was to investigate the validity, reliability and time required to assess ROM using a standard goniometer (SG), medical inclinometer (MI), and builder's digital inclinometer (BI). Study Design: Cross-sectional study. Methods: Fifty participants with no current shoulder, elbow, or forearm pain limiting movement were assessed by a single tester. The tester measured three repetitions of passive forearm and shoulder rotation with an SG, MI, and BI. Device order was randomized. Time to complete assessment with each device was measured. Results: BI and MI were significantly faster than the SG (p < 0.001) for all motions. Inclinometer measurements were more reliable (average ICC = 0.933 for MI and 0.919 for BI) than SG measurements (average ICC = 0.822). There was good correlation between MI and BI and mean differences between devices was less than 2°. Correlations between the SG and the inclinometers ranged from poor to fair and mean differences between devices was 4°. Conclusion: The BI and MI were reliable for measuring forearm and shoulder rotation. The poor correlation between the SG and inclinometers indicates that clinicians should utilize the same device for testing. Because time can be a barrier to clinician assessment, the greater efficiency and reliability of inclinometers warrants consideration as the new measurement standard. Standard patient and inclinometer positioning is recommended to enhance reliability. Level of Evidence: 2©The Author(s).

Objective evaluation of tongue diagnosis ability using a tongue diagnosis e-learning/e-assessment system based on a standardized tongue image database.

Background: In Kampo medicine, tongue examination is used to diagnose the pathological condition "Sho," but an objective evaluation method for its diagnostic ability has not been established. We constructed a tongue diagnosis electronic learning and evaluation system based on a standardized tongue image database. Purpose: This study aims to verify the practicality of this assessment system by evaluating the tongue diagnosis ability of Kampo specialists (KSs), medical professionals, and students. Methods: In the first study, we analyzed the answer data of 15 KSs in an 80-question tongue diagnosis test that assesses eight aspects of tongue findings and evaluated the (i) test score, (ii) test difficulty and discrimination index, (iii) diagnostic consistency, and (iv) diagnostic match rate between KSs. In the second study, we administered a 20-question common Kampo test and analyzed the answer data of 107 medical professionals and 56 students that assessed the tongue color discrimination ability and evaluated the (v) correct answer rate, (vi) test difficulty, and (vii) factors related to the correct answer rate. Result: In the first study, the average test score was 62.2 ± 10.7 points. Twenty-eight questions were difficult (correct answer rate, <50%), 34 were moderate (50%-85%), and 18 were easy (≥85%). Regarding intrarater reliability, the average diagnostic match rate of five KSs involved in database construction was 0.66 ± 0.08, and as for interrater reliability, the diagnostic match rate between the 15 KSs was 0.52 (95% confidence interval, 0.38-0.65) for Gwet's agreement coefficient 1, and the degree of the match rate was moderate. In the second study, the difficulty level of questions was moderate, with a correct rate of 81.3% for medical professionals and 82.1% for students. The discrimination index was good for medical professionals (0.35) and poor for students (0.06). Among medical professionals, the correct answer group of this question had a significantly higher total score on the Kampo common test than the incorrect answer group (85.3 ± 8.4 points vs. 75.8 ± 11.8 points, p < 0.01). Conclusion: This system can objectively evaluate tongue diagnosis ability and has high practicality. Utilizing this system can be expected to contribute to improving learners' tongue diagnosis ability and standardization of tongue diagnosis.

Multilesion Segmentations in Patients with Intracerebral Hemorrhage: Reliability of ICH, IVH and PHE Masks.

Background and Purpose: Fully automated methods for segmentation and volume quantification of intraparenchymal hemorrhage (ICH), intraventricular hemorrhage extension (IVH), and perihematomal edema (PHE) are gaining increasing interest. Yet, reliabilities demonstrate considerable variances amongst each other. Our aim was therefore to evaluate both the intra- and interrater reliability of ICH, IVH and PHE on ground-truth segmentation masks. Methods: Patients with primary spontaneous ICH were retrospectively included from a German tertiary stroke center (Charité Berlin; January 2016−June 2020). Baseline and follow-up non-contrast Computed Tomography (NCCT) scans were analyzed for ICH, IVH, and PHE volume quantification by two radiology residents. Raters were blinded to all demographic and outcome data. Inter- and intrarater agreements were determined by calculating the Intraclass Correlation Coefficient (ICC) for a randomly selected set of patients with ICH, IVH, and PHE. Results: 100 out of 670 patients were included in the analysis. Interrater agreements ranged from an ICC of 0.998 for ICH (95% CI [0.993; 0.997]), to an ICC of 0.979 for IVH (95% CI [0.984; 0.993]), and an ICC of 0.886 for PHE (95% CI [0.760; 0.938]), all p-values < 0.001. Intrarater agreements ranged from an ICC of 0.997 for ICH (95% CI [0.996; 0.998]), to an ICC of 0.995 for IVH (95% CI [0.992; 0.996]), and an ICC of 0.980 for PHE (95% CI [0.971; 0.987]), all p-values < 0.001. Conclusion Manual segmentations of ICH, IVH, and PHE demonstrate good-to-excellent inter- and intrarater reliabilities, with the highest agreement for ICH and IVH and lowest for PHE. Therefore, the degree of variances reported in fully automated quantification methods might be related amongst others to variances in ground-truth masks.

Reliability and validity testing of the medicines related - consultation assessment tool for assessing pharmacists' consultations.

BACKGROUND: Demonstrating a person-centred approach in a consultation is a key component of delivering high-quality healthcare. To support development of such an approach requires training underpinned by valid assessment tools. Given the lack of a suitable pharmacy-specific tool, a new global consultation skills assessment tool: the medicines related-consultation assessment tool (MR-CAT) was designed and tested. AIM: This study aimed to test the validity and reliability of the MR-CAT using psychometric methods. METHOD: Psychometric testing involved analysis of participants' (n = 13) assessment of fifteen pre-recorded simulated consultations using the MR-CAT. Analysis included discriminant validity testing, intrarater and interrater reliability testing for each of the five sections of the MR-CAT and for the overall global assessment of the consultation. Analysis also included internal consistency testing for the whole tool. RESULTS: Internal consistency for the overall global assessment of the consultation was good (Cronbach's alpha = 0.97). The MR-CAT discriminated well for the overall global assessment of the consultation (p < 0.001). Moderate to high intrarater reliability was observed for the overall global assessment of the consultation and for all five sections of the MR-CAT (rho = 0.64-0.84) in the test-retest analysis. Moderate to good interrater reliability (Kendall's W = 0.68-0.90) was observed for the overall global assessment of the consultation and for all five sections of the MR-CAT. CONCLUSION: The MR-CAT is a valid and reliable tool for assessing person-centred pharmacist's consultations. Moreover, its unique design means that the MR-CAT can be used in both formative and summative assessment.

Reliability of the Modified Star Excursion Balance Test in individuals with Down Syndrome.

Neuromotor problems such as hypotonia, incoordination, impaired sensory-motor integration lead to significant delays in motor skills and balance development in individuals with Down Syndrome (DS). Balance control is essential for performing many motor skills independently and safely. Standardised testing of balance control can contribute significantly to the rehabilitation of individuals with DS. The purpose of this study was to determine intrarater and interrater reliability of the Modified Star Excursion Balance Test (SEBT) for individuals with DS. Thirteen individuals with DS were recruited in this study. Intraclass correlation coefficients (ICC [3,1]) with 95% confidence intervals, standard error of measurement (SEM), the smallest detectable difference (SDD) and the Spearman rank correlation coefficient were calculated. In all directions of the Modified SEBT, no statistically significant difference was found between two raters' first and second measurements (p > 0.05). Interrater reliability for all reach directions of the Modified SEBT was high, with ICC ranging from 0.990 to 0.998.95% confidence intervals, SEM and SDD ranged from 0.924 to 0.999, 0.180-2.434 and 3.270-6.747, respectively. The Modified SEBT are reliable for evaluating dynamic balance in individuals with DS aged between 6 and 24 years.

Reliability and preliminary reference values for the Total Faulty Breathing Scale (TFBS): A cross-sectional study.

Background and Aims: The evaluation of breathing function is crucial in the clinical examination of the respiratory system. The Total Faulty Breathing Scale (TFBS) could be used in clinical settings to quantify the measurement of breathing dysfunction. Reliability data for the TFBS are available for males, but there is a requirement to determine reliability for females and to develop reference values. The aim of this study, therefore, was to determine the reliability in females and to establish the preliminary reference values for the TFBS. Methods: Twenty-three healthy female participants for reliability and 44 (7 male and 37 female) participants for preliminary reference values participated in this cross-sectional study. For both aspects of the study, participants were instructed to breathe at their own pace with no specific instruction. Then each participant was observed carrying out normal breathing for a period of 10 breaths and deep breathing for a period of 10 breaths while being assessed with the TFBS. Results: Intrarater and interrater reliability of the TFBS showed a kappa value of 0.769 and 0.751, respectively, indicating substantial agreement for female participants. The preliminary reference categories for TFBS were reported to be normal for 20 (45.5%) participants and mild faulty breathing for the remaining 24 (54.4%) participants. Conclusions: The findings of this study suggested that TFBS was reliable to measure breathing function among female participants, and the reference categories may be helpful in the identification of normal and faulty breathing.

Impedance of Extracellular Fluid, Volume, and Local Tissue Water Can Be Reliably Measured in People With Lower Limb Lymphedema.

OBJECTIVE: Lower limb lymphedema (LLL) is a chronic condition. To be able to evaluate changes of LLL over time and effects of interventions, reliable measurement methods are important. Currently, there is limited knowledge of the reliability of commonly used measurement methods in LLL. The study objective was to evaluate the test-retest (intrarater) reliability of impedance of extracellular fluid, volume, and local tissue water measurements in people with unilateral or bilateral LLL and measurement errors both for a group of people and for a single individual. METHODS: Forty-two people with mild to moderate unilateral or bilateral, primary or secondary LLL were measured twice, 2 weeks apart. Impedance of extracellular fluid was measured by bioimpedance spectroscopy and calculated as arm-to-leg ratio, volume with circumference measurements every 4 cm, and local tissue water with tissue dielectric constant at 14 points. Test-retest reliability was evaluated using the intraclass correlation coefficient [ICC(2,1)], changes in the mean, SE of measurement in relative terms (SEM%), and the smallest real difference in relative terms (SRD%). RESULTS: For the impedance ratio, the reliability was high [ICC(2,1) = 0.79-0.90] and the measurement errors were acceptable (SEM% = 5.0%-5.2%; SRD% = 14.0%-14.4%). For volume, the reliability was high (ICC = 0.99) and the measurement errors were low (SEM% = 1.1%-1.7%; SRD% = 3.1%-4.6%). For the tissue dielectric constant, the reliability was fair to excellent [ICC(2,1) = 0.68-0.96] and the measurement errors were acceptable (SEM% = 4.2%-9.7%; SRD% = 11.7%-26.8%). CONCLUSIONS: Measurements of impedance of extracellular fluid, volume, and local tissue water are reliable in people with mild to moderate LLL. The measurement errors were acceptable in all 3 methods indicating that real, clinical changes in lymphedema can be measured both for a group of people and a single individual. IMPACT: The results from this test-retest reliability study can help clinicians and researchers to interpret if real clinical changes in lymphedema occur over time or after an intervention in people with mild to moderate LLL.

Relative Validity and Reliability of Isometric Lower Extremity Strength Assessment in Older Adults by Using a Handheld Dynamometer.

BACKGROUND: Handheld dynamometry (HHD) is a practical alternative to traditional testing of lower extremity strength. However, its reliability and validity across different populations and settings are not clear. HYPOTHESIS: We hypothesize that HHD is a valid and reliable device to assess lower extremity strength in a population of older adults. STUDY DESIGN: Cross-sectional/cohort. LEVEL OF EVIDENCE: Level 3. METHODS: This study included 258 older adults (≥65 years). Isometric knee extension and flexion force were measured by 1 examiner, using an HHD (n = 222), including 3 repetitions to calculate within-day intrarater reliability. These measurements were repeated by the examiner in a subgroup (n = 23) to analyze intrarater reliability over a test-retest period of on average 8 weeks. In addition, HHD force measures were performed by a second examiner (n = 29) to analyze interrater reliability. In another subgroup (n = 77), isometric knee extension and flexion torque were measured by 1 examiner using both the HHD and Biodex System 4 to assess relative validity. RESULTS: HHD and Biodex measurements were highly correlated and showed excellent concurrent validity. HHD systematically overestimated torque as compared with Biodex by 8 N·m on average. Same-day intrarater intraclass correlation coefficients (ICCs) ranged from 0.97 to 0.98. Interrater reliability ICCs ranged from 0.83 to 0.95. CONCLUSION: HHD represents a reliable and valid alternative to Biodex to rank individuals on leg strength, or to assess within-person changes in leg strength over time, because of the high validity and reliability. The HHD is less suited for absolute strength assessment because of significant systematic overestimations. CLINICAL RELEVANCE: Clinicians are encouraged to use HHD to rank older adults on leg strength, or to assess within-person changes in leg strength over time, but not to compare readings with cut-offs or normative values.

Injuries to the anterolateral ligament are observed more frequently compared to lesions to the deep iliotibial tract (Kaplan fibers) in anterior cruciate ligamant deficient knees using magnetic resonance imaging.

PURPOSE: To determine the accuracy of detection, injury rate and inter- and intrarater reproducibility in visualizing lesions to the anterolateral ligament (ALL) and the deep portion of the iliotibial tract (dITT) in anterior cruciate ligament (ACL) deficient knees. METHODS: Ninety-one consecutive patients, out of those 25 children (age 14.3 ± 3.5 years), with diagnosed ACL tears were included. Two musculoskeletal radiologists retrospectively reviewed MRI data focusing on accuracy of detection and potential injuries to the ALL or dITT. Lesion were diagnosed in case of discontinued fibers in combination with intra- or peri-ligamentous edema and graded as intact, partial or complete tears. Cohen's Kappa and 95% confidence intervals (95% CI) were determined for inter- and intrarater reliability measures. RESULTS: The ALL and dITT were visible in 52 (78.8%) and 56 (84.8%) of adult-and 25 (100%) and 19 (76.0%) of pediatric patients, respectively. The ALL was injured in 45 (58.5%; partial: 36.4%, compleate: 22.1%) patients. Partial and comleate tears, where visualized in 21 (40.4%) and 16 (30.8%) adult- and seven (28.0%) and one (4%) peditric patients. A total of 16 (21.3%; partial: 13.3%, compleate: 8.0%) dITT injuries were identified. Partal and complete lesions were seen in seven (12.5%) and five (8.9%) adult- and three (15.8%) and one (5.3%) pediatric patients. Combined injuries were visualized in nine (12.7%) patients. Inter-observer (0.91-0.95) and intra-observer (0.93-0.95) reproducibility was high. CONCLUSION: In ACL injured knees, tears of the ALL are observed more frequently compared to lesions to the deep iliotibial tract. Combined injuries of both structures are rare. Clinically, the preoperative visualization of potentially injured structures of the anterolateral knee is crucial and is important for a more personalized preoperative planning and tailored anatomical reconstruction. The clinical implication of injuries to the anterolateral complex of the knee needs further investigation. LEVEL OF EVIDENCE: II.

Intrarater Reliability of Digital Thermography in Detecting Pin Site Infection: A Proof of Concept Study.

AIM AND OBJECTIVE: The purpose of this study was to explore the capability and Intrarater reliability of thermography in detecting pin site infection. MATERIALS AND METHODS: This is an explorative proof of concept study. Clinical assessment of pin sites was performed by one examiner with the Modified Gordon Pin Infection Classification from grade 0 to 6. Thermography of the pin sites was performed with a FLIR C3 camera. The analysis of the thermographic images was done in the software FLIR Tools. The maximum skin temperature around the pin site and the maximum temperature for the whole thermographic picture were measured. An Intrarater agreement was established and test-retests were performed with different camera angles. RESULTS: Thirteen (four females, nine males) patients (age 9-72 years) were included. Indications for frames: Fracture (n=4), two deformity correction, one lengthening and six bone transport. Days from surgery to thermography ranged from 27 to 385 days. Overall, 231 pin sites were included. Eleven pin sites were diagnosed with early signs of infection: five grade 1, five grade 2 and one grade 3. Mean pin site temperature for each patient was calculated, varied between patients from 29.0°C to 35.4°C (mean 33.9°C). With 34°C as cut-off value for infection, sensitivity was 73%; specificity, 67%; positive predictive value, 10%; and negative predictive value, 98%. Intrarater agreement for thermography was ICC 0.85 (0.77-0.92). The temperature measured was influenced by the camera positioning in relation to the pin site with a variance of 0.2. CONCLUSIONS: Measurements of pin site temperature using the hand-held FLIR C3 infrared camera was a reliable method and the temperature was related to infection grading. CLINICAL SIGNIFICANCE: This study demonstrated that digital thermography with a hand-held camera might be used for monitoring the pin sites after operations to detect early infection. HOW TO CITE THIS ARTICLE: Rahbek O, Husum HC, Fridberg M, et al. Intrarater Reliability of Digital Thermography in Detecting Pin Site Infection: A Proof of Concept Study. Strategies Trauma Limb Reconstr 2021;16(1):1-7.

Inter- and Intrarater Agreement of Spot Sign and Noncontrast CT Markers for Early Intracerebral Hemorrhage Expansion.

BACKGROUND: The aim of this study was to assess the inter- and intrarater reliability of noncontrast CT (NCCT) markers [Black Hole Sign (BH), Blend Sign (BS), Island Sign (IS), and Hypodensities (HD)] and Spot Sign (SS) on CTA in patients with spontaneous intracerebral hemorrhage (ICH). METHODS: Patients with spontaneous ICH at three German tertiary stroke centers were retrospectively included. Each CT scan was rated for four NCCT markers and SS on CTA by two radiology residents. Raters were blind to all demographic and outcome data. Inter- and intrarater agreement was determined by Cohen's kappa (κ) coefficient and percentage of agreement. RESULTS: Interrater agreement was excellent in 473 included patients, ranging from 96% to 99%. Interrater κ ranged from 0.85 (95% CI [0.78-0.91]) to 0.97 (95% CI [0.94-0.99]) for NCCT markers and 0.93 (95% CI [0.88-0.98]) for SS, all p-values < 0.001. Intrarrater agreement ranged from 96% to 100%, with κ ranging from 0.85 (95% CI [0.78-0.91]) to 1.00 (95% CI [0.10-0.85]) for NCCT markers and 0.96 (95% CI [0.92-1.00]) for SS, all p-values < 0.001. CONCLUSIONS: NCCT imaging findings and SS on CTA have good-to-excellent inter- and intrarater reliabilities, with the highest agreement for BH and SS.

Relevance of SPECT-CT in Complex Cases of Foot and Ankle Surgery: A Comparison With MRI.

BACKGROUND: Finding the right diagnoses in patients with complex foot and ankle disorders can be challenging. Single-photon emission computed tomography and computed tomography (SPECT-CT) has shown to be feasible in foot and ankle surgery. The aim of this study was to evaluate the reliability and accuracy of SPECT-CT and thereby its impact on final treatment decision compared with magnetic resonance imaging (MRI). METHODS: A retrospective study was performed on 49 patients treated at our institution. Experienced foot and ankle surgeons independently, and blinded, analyzed clinical data and radiographs together with MRI, SPECT-CT, or a combination of both. Based on the determined final treatment decision Cohen's kappa values were calculated to illustrate interrater and intrarater reliability. RESULTS: The kappa values for interrater reliability were higher for SPECT-CT at .68 and MRI + SPECT-CT at .71 compared to 0.38 for MRI alone (P < .05). The kappa values for intrarater reliability of MRI + SPECT-CT were higher at .75 compared with SPECT-CT alone at .67 (P < .05) and MRI at .35 (P < .01). CONCLUSION: We found a higher interrater and intrarater reliability for SPECT-CT compared with MRI alone for diagnosing complex foot and ankle pathologies. SPECT-CT has a high impact on final treatment decision. The main indications are bony pathologies with diagnostic uncertainty especially in closely adjacent structures as the joints of the midfoot, occult coalitio, stress fractures, verification or exclusion of nonfusion, periprosthetic disorders after total ankle replacement and osteochondral lesion in cases of combined pathologies.Levels of Evidence: Level IV: Retrospective study.

Prospective validation study of an epilepsy seizure risk system for outpatient evaluation.

OBJECTIVE: We conducted clinical testing of an automated Bayesian machine learning algorithm (Epilepsy Seizure Assessment Tool [EpiSAT]) for outpatient seizure risk assessment using seizure counting data, and validated performance against specialized epilepsy clinician experts. METHODS: We conducted a prospective longitudinal study of EpiSAT performance against 24 specialized clinician experts at three tertiary referral epilepsy centers in the United States. Accuracy, interrater reliability, and intra-rater reliability of EpiSAT for correctly identifying changes in seizure risk (improvements, worsening, or no change) were evaluated using 120 seizures from four synthetic seizure diaries (seizure risk known) and 120 seizures from four real seizure diaries (seizure risk unknown). The proportion of observed agreement between EpiSAT and clinicians was evaluated to assess compatibility of EpiSAT with clinical decision patterns by epilepsy experts. RESULTS: EpiSAT exhibited substantial observed agreement (75.4%) with clinicians for assessing seizure risk. The mean accuracy of epilepsy providers for correctly assessing seizure risk was 74.7%. EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries, corresponding to a significant improvement of 17.4% (P = .002). Clinicians exhibited low-to-moderate interrater reliability for seizure risk assessment (Krippendorff's α = 0.46) with good intrarater reliability across a 4- to 12-week evaluation period (Scott's π = 0.89). SIGNIFICANCE: These results validate the ability of EpiSAT to yield objective clinical recommendations on seizure risk which follow decision patterns similar to those from specialized epilepsy providers, but with improved accuracy and reproducibility. This algorithm may serve as a useful clinical decision support system for quantitative analysis of clinical seizure frequency in clinical epilepsy practice.

Intrarater and Interrater Reliability of the Soong Classification for Distal Radius Volar Locking Plate Placement.

Background: The purpose of this study was to analyze the intrarater and interrater reliability of the Soong classification for volar locking plate placement on a randomly selected, consecutive series of radiographs. Our hypothesis was that the classification would be reliable. Methods: Six physicians of differing levels of training (orthopedic surgery intern to fellowship-trained upper extremity staff) were asked to review 40 radiographs in a random order on 2 separate occasions, 4 weeks apart. All observers graded each image (0, 1, or 2) based on the corresponding Soong grade. A weighted κ was used to determine the intrarater agreement. The interrater agreement was determined using an intraclass coefficient: Results: The intrarater reliability using a weighted κ ranged from 0.229 (95% confidence interval [CI]: 0.048-0.411) to 0.946 (95% CI: 0.840-1.051). The interrater intraclass coefficient for Randomization 1 was 0.944 (95% 0.912-0.967) and Randomization 2 was 0.877 (95% CI: 0.797-0.930). Conclusion: The Soong classification is a reliable tool, both interrater and intrarater, for assessing distal radius volar locking plate placement. The classification system remained reliable despite a randomly selected, consecutive series of images and physician observers of varying levels of training.