Reliability and agreement in intrapartum fetal heart rate monitoring interpretation: A systematic review.

INTRODUCTION: Fetal heart rate (FHR) monitoring is routine in intrapartum care worldwide and one of the most common obstetrical procedures. Intrapartum FHR monitoring helps assess fetal wellbeing and interpretation of the FHR help form decisions for clinical management and intervention. It relies on the observers' subjective assessments, with variation in interpretations leading to variations in intrapartum care. The purpose of this systematic review was to summarize and evaluate extant inter- and intrarater reliability research on the human interpretation of intrapartum FHR monitoring. MATERIAL AND METHODS: We searched for the terms "fetal heart rate monitoring," "interpretation agreement" and related concepts on Embase, Medline, Maternity and Infant Care Database and CINAHL. The last search was made on January 31, 2022. The protocol for the study was prospectively registered in PROSPERO (CRD42021260937). Studies that assess inter- and intrarater reliability and agreement of health professionals' intrapartum FHR monitoring were included and studies including other assessment of fetal wellbeing excluded. We extracted data in reviewer pairs using quality appraisal tool for studies of diagnostic reliability (QAREL) forms. The data retrieved from the studies are presented as narrative synthesis and in additional tables. RESULTS: Forty-nine articles concerning continuous FHR monitoring were included in the study. For interrater reliability and agreement, in total 577 raters assessed 6315 CTG tracings. There was considerable heterogeneity in quality and measures across the included articles. We found higher reliability and agreement for the basic FHR features than for overall classification and higher agreement for intrarater reliability and agreement than for their interrater counterparts. CONCLUSIONS: There is great variation in reliability and agreement measures for continuous intrapartum FHR monitoring, implying that intrapartum CTG should be used with caution for clinical decision making given its questionable reliability. We found few high-quality studies and noted methodological concerns in the studies. We recommend a more standardized approach to future reliability studies on FHR monitoring.

Affective Symptoms and Oropharyngeal Dysphagia in Head-and-Neck Cancer Patients: A Systematic Review.

Oropharyngeal dysphagia (OD) is a high impact morbidity in head-and-neck cancer (HNC) patients. A wide variety of instruments are developed to screen for affective symptoms and OD. The current paper aims to systematically review and appraise the literature to obtain insight into the prevalence, strength, and causal direction of the relationship between affective symptoms and OD in HNC patients. This review was conducted in accordance with the PRISMA statement. A systematic search of the literature was performed using PubMed, PsycINFO, Cochrane, and Embase. All available publications reporting on the relationship between affective conditions and swallowing function in HNC patients were included. Conference papers, tutorials, reviews, and studies with less than 5 patients were excluded. Fifteen studies met the inclusion criteria. The level of evidence and methodological quality were assessed using the ABC-rating scale and QualSyst critical appraisal tool. Eleven studies reported a positive relationship between affective symptoms and OD. The findings of this paper highlight the importance of affective symptom screening in dysphagic HNC patients as clinically relevant affective symptoms and OD seems to be prevalent and coincident in this population. Considering the impact of affective symptoms and OD on patients' daily life, early detection and an integrated interdisciplinary approach are recommended. However, due to the heterogeneity of study designs, outcomes, and outcome measures, the generalization of study results is limited.

Chasing consistency: On the measurement error in self-reported affect in experiments.

How feelings change over time is a central topic in emotion research. To study these affective fluctuations, researchers often ask participants to repeatedly indicate how they feel on a self-report rating scale. Despite widespread recognition that this kind of data is subject to measurement error, the extent of this error remains an open question. Complementing many daily-life studies, this study aimed to investigate this question in an experimental setting. In such a setting, multiple trials follow each other at a fast pace, forcing experimenters to use a limited number of questions to measure affect during each trial. A total of 1398 participants completed a probabilistic reward task in which they were unknowingly presented with the same string of outcomes multiple times throughout the study. This allowed us to assess the test-retest consistency of their affective responses to the rating scales under investigation. We then compared these consistencies across different types of rating scales in hopes of finding out whether a given type of scale led to a greater consistency of affective measurements. Overall, we found moderate to good consistency of the affective measurements. Surprisingly, however, we found no differences in consistency across rating scales, which suggests that the specific rating scale that is used does not influence the measurement consistency.

Punch Trackers: Correct Recognition Depends on Punch Type and Training Experience.

To determine the ability of different punch trackers (PT) (Corner (CPT), Everlast (EPT), and Hykso (HPT)) to recognize specific punch types (lead and rear straight punches, lead and rear hooks, and lead and rear uppercuts) thrown by trained (TR, n = 10) and untrained punchers (UNTR, n = 11), subjects performed different punch combinations, and PT data were compared to data from video recordings to determine how well each PT recognized the punches that were actually thrown. Descriptive statistics and multilevel modelling were used to analyze the data. The CPT, EPT and HPT detected punches more accurately in TR than UNTR, evidenced by a lower percentage error in TR (p = 0.007). The CPT, EPT, and HPT detected straight punches better than uppercuts and hooks, with a lower percentage error for straight punches (p < 0.001). The recognition of punches with CPT and HPT depended on punch order, with earlier punches in a sequence recognized better. The same may or may not have occurred with EPT, but EPT does not allow for data to be exported, meaning the order of individual punches could not be analyzed. The CPT and HPT both seem to be viable options for tracking punch count and punch type in TR and UNTR.

Medically Unexplained Oropharyngeal Dysphagia at the University Hospital ENT Outpatient Clinic for Dysphagia: A Cross-Sectional Cohort Study.

Medically unexplained oropharyngeal dysphagia (MUNOD) is a rare condition. It presents without demonstrable abnormalities in the anatomy of the upper aero-digestive tract and/or swallowing physiology. This study investigates whether MUNOD is related to affective or other psychiatric conditions. The study included patients with dysphagic complaints who had no detectible structural or physiological abnormalities upon swallowing examination. Patients with any underlying disease or disorder that could explain the oropharyngeal dysphagia were excluded. All patients underwent a standardized examination protocol, with FEES examination, the Hospital Anxiety and Depression Scale (HADS), and the Dysphagia Severity Scale (DSS). Two blinded judges scored five different FEES variables. None of the 14 patients included in this study showed any structural or physiological abnormalities during FEES examination. However, the majority did show abnormal piecemeal deglutition, which could be a symptom of MUNOD. Six patients (42.8%) had clinically relevant symptoms of anxiety and/or depression. The DSS scores did not differ significantly between patients with and without affective symptoms. Affective symptoms are common in patients with MUNOD, and their psychiatric conditions could possibly be related to their swallowing problems.

Modeling agreement on categorical scales in the presence of random scorers.

Kappa coefficients are often used to assess agreement between two fixed scorers on categorical scales. Cohen's version is popular for nominal scales and the weighted version for ordinal scales. In the present paper, similar agreement coefficients are defined for random scorers. A partial-Bayesian methodology is then developed to directly relate these agreement coefficients to predictors through a multilevel model. Statistical properties of the proposed approach are studied using simulations. Finally, the approach is applied to gynecological and medical imaging data.

Comparing dependent kappa coefficients obtained on multilevel data.

Reliability and agreement are two notions of paramount importance in medical and behavioral sciences. They provide information about the quality of the measurements. When the scale is categorical, reliability and agreement can be quantified through different kappa coefficients. The present paper provides two simple alternatives to more advanced modeling techniques, which are not always adequate in case of a very limited number of subjects, when comparing several dependent kappa coefficients obtained on multilevel data. This situation frequently arises in medical sciences, where multilevel data are common. Dependent kappa coefficients can result from the assessment of the same individuals at various occasions or when each member of a group is compared to an expert, for example. The method is based on simple matrix calculations and is available in the R package "multiagree". Moreover, the statistical properties of the proposed method are studied using simulations. Although this paper focuses on kappa coefficients, the method easily extends to other statistical measures.

Diagnostic accuracy for apical and chronic periodontitis biomarkers in gingival crevicular fluid: an exploratory study.

AIM: The aim of this study was to assess the levels and diagnostic accuracy of a set of potential biomarkers of periodontal tissue metabolism in gingival crevicular fluid (GCF) from patients with chronic periodontitis (CP) and asymptomatic apical periodontitis ( AAP). MATERIALS AND METHODS: Thirty one GCF samples from 11 CP patients, 44 GCF samples from 38 AAP patients and 31 GCF samples from 13 healthy volunteers were obtained (N = 106). Matrix metalloproteinases (MMPs) -2 and -9 were determined by zymography; levels of MMP-8 by ELISA and IFMA and MPO by ELISA. IL-1, IL-6, TNFα, DKK-1, Osteonectin, Periostin, TRAP-5 and OPG were determined by a multiplex quantitative panel. Statistical analysis was performed using linear mixed-effects models. RESULTS: The MMP-9 and MMP-8 were higher in CP, followed by AAP, versus healthy individuals (p < 0.05). ProMMP-2, MPO, IL-1, IL-6, PTN, TRAP-5 and OPG were significantly higher in CP when compared with AAP and healthy patients (p < 0.05). The highest diagnostic accuracies were observed for ProMMP-2, ProMMP-9, MMP-8 and TRAP-5 (AUC > 0.97) in CP, and for the active form of MMP-9 and MMP-8 (AUC > 0.90) in AAP. CONCLUSION: Gingival crevicular fluid composition is modified by CP and AAP. MMP-9 and MMP-8 show diagnostic potential for CP and AAP, whereas MMP-2 and TRAP-5 are useful only for CP.

Observers' Agreement on Measurements in Fiberoptic Endoscopic Evaluation of Swallowing.

This study analyzed the effect that dysphagia etiology, different observers, and bolus consistency might have on the level of agreement for measurements in FEES images reached by independent versus consensus panel rating. Sixty patients were included and divided into two groups according to dysphagia etiology: neurological or head and neck oncological. All patients underwent standardized FEES examination using thin and thick liquid consistencies. Two observers scored the same exams, first independently and then in a consensus panel. Four ordinal FEES variables were analyzed. Statistical analysis was performed using a linear weighted kappa coefficient and Bayesian multilevel model. Intra- and interobserver agreement on FEES measurements ranged from 0.76 to 0.93 and from 0.61 to 0.88, respectively. Dysphagia etiology did not influence observers' agreement level. However, bolus consistency resulted in decreased interobserver agreement for all measured FEES variables during thin liquid swallows. When rating on the consensus panel, the observers deviated considerably from the scores they had previously given on the independent rating task. Observer agreement on measurements in FEES exams was influenced by bolus consistency, not by dysphagia etiology. Therefore, observer agreement on FEES measurements should be analyzed by taking bolus consistency into account, as it might affect the interpretation of the outcome. Identifying factors that might influence agreement levels could lead to better understanding of the rating process and assist in developing a more precise measurement scale that would ensure higher levels of observer agreement for measurements in FEES exams.

Statistical inference for agreement between multiple raters on a binary scale.

Agreement studies often involve more than two raters or repeated measurements. In the presence of two raters, the proportion of agreement and of positive agreement are simple and popular agreement measures for binary scales. These measures were generalized to agreement studies involving more than two raters with statistical inference procedures proposed on an empirical basis. We present two alternatives. The first is a Wald confidence interval using standard errors obtained by the delta method. The second involves Bayesian statistical inference not requiring any specific Bayesian software. These new procedures show better statistical behaviour than the confidence intervals initially proposed. In addition, we provide analytical formulas to determine the minimum number of persons needed for a given number of raters when planning an agreement study. All methods are implemented in the R package simpleagree and the Shiny app simpleagree.

Review of sample size determination methods for the intraclass correlation coefficient in the one-way analysis of variance model.

Reliability of measurement instruments providing quantitative outcomes is usually assessed by an intraclass correlation coefficient. When participants are repeatedly measured by a single rater or device, or, are each rated by a different group of raters, the intraclass correlation coefficient is based on a one-way analysis of variance model. When planning a reliability study, it is essential to determine the number of participants and measurements per participant (i.e. number of raters or number of repeated measurements). Three different sample size determination approaches under the one-way analysis of variance model were identified in the literature, all based on a confidence interval for the intraclass correlation coefficient. Although eight different confidence interval methods can be identified, Wald confidence interval with Fisher's large sample variance approximation remains most commonly used despite its well-known poor statistical properties. Therefore, a first objective of this work is comparing the statistical properties of all identified confidence interval methods-including those overlooked in previous studies. A second objective is developing a general procedure to determine the sample size using all approaches since a closed-form formula is not always available. This procedure is implemented in an R Shiny app. Finally, we provide advice for choosing an appropriate sample size determination method when planning a reliability study.

Prevalence and clinical characteristics of remission during treatment in generalized anxiety.

OBJECTIVE: Although the remission criteria for generalized anxiety are well defined, there is not much data available on the point prevalence of remission. The Measuring Impact of Remission in Anxiety Disorders in Belgium (MIRABEL) study is a naturalistic study designed to document the point prevalence of remission in patients treated for general anxiety and potential factors affecting this prevalence. METHODS: The study population consisted of 618 adult outpatients being treated for generalized anxiety. The sample is defined by the key symptoms of generalized anxiety disorder rather than by fulfilling the exact DSM-IV-TR diagnostic criteria. Remission was defined as a Hamilton Anxiety Scale (HAM-A) score of less than or equal to 7. To reduce the interrater reliability, the HAM-A was assessed by the attending physicians who had no specific training. Factors investigated as possibly related to remission included sociodemographic, disease and treatment characteristics. RESULTS: The point prevalence of remission in the study population was estimated at 13.3%. Remission prevalence varied with occupational status and severity of the current anxiety episode. Remission prevalence was lower in the presence of comorbidity and was proportional to the number of comorbid symptoms. Remitters took fewer medications but were treated longer. Remission prevalence was higher in patients who were taking antidepressants, but was lower in patients who were taking sedatives. CONCLUSIONS: These findings underline the poor prognosis of generalized anxiety.

Hierarchical modeling of agreement.

Kappa-like agreement indexes are often used to assess the agreement among examiners on a categorical scale. They have the particularity of correcting the level of agreement for the effect of chance. In the present paper, we first define two agreement indexes belonging to this family in a hierarchical context. In particular, we consider the cases of a random and fixed set of examiners. Then, we develop a method to evaluate the influence of factors on these indexes. Agreement indexes are directly related to a set of covariates through a hierarchical model. We obtain the posterior distribution of the model parameters in a Bayesian framework. We apply the proposed approach on dental data and compare it with the generalized estimating equations approach.

Impact of the biopsy forceps size on histological analysis and performances of the histological scoring systems.

To improve the reliability of the quantitative scorings of the synovial biopsies, we evaluate whether diameter of arthroscopic forceps influences histological quality of synovial tissue and/or histological scores and we compare the intra- and inter-observer performances of the main histological scoring systems. Synovial biopsies were retrieved in the same part of the joint using 1, 2 and 4 mm diameters grasping forceps. After standard staining and immunohistochemistry with anti-CD68 antibody, slides were scored blindly by 2 independent experienced operators for tissue quality and with Krenn score, de Bois-Tak score and CD68 semi-quantitative score. Four samples did not pass quality control. No difference other than a higher number of vessels in the 4 mm versus 2 mm forceps (p = 0.01) was found among the 3 groups. CD68 score was significantly higher in the 2 versus 4 mm forceps (p = 0.009). So we concluded that only vessels quantification and CD68 semi-quantitative score seemed affected by the forceps size. The intra-reader agreement was variable across observers and features: 0.78 (0.66-0.87) for the Krenn scoring system, 0.89 (0.78-0.97) for the de Bois-Tak score and 0.93 (0.81-1.00) for the CD68 score. Interobserver reliabilities of Krenn score, de Bois-Tak score and CD68 scores were satisfactory: 0.95 (0.92-0.99) for Krenn, 0.98 (0.96-0.99) for de Bois-Tak and 0.80 (0.71-0.89) for CD68.

Objective physical activity assessment in patients with chronic organ failure: a validation study of a new single-unit activity monitor.

OBJECTIVE: To validate a new activity monitor (CAM) in patients with chronic organ failure during 1 hour of unconstrained activity assessment. DESIGN: A validation study in which participants wore a CAM (placed on the thigh) for 1 hour while they were videotaped. Participants were instructed to continue their normal daily routine at the rehabilitation center. SETTING: CIRO+, A Centre of Expertise for Chronic Organ Failure. PARTICIPANTS: Chronic organ failure patients (N=10) with a large range of functional exercise capacity. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: Agreement in time spent on activities and postures according to video and CAM. RESULTS: Cohen kappa coefficients for transient events resulted in an almost perfect agreement (0.85 ± 0.08) between the CAM and video. CONCLUSIONS: The CAM is a promising single-sensor unobtrusive tool for providing accurate data on the type and duration of daily activities in the home environment of patients with chronic organ failure.

Effect of an audiovisual message for tetanus booster vaccination broadcast in the waiting room.

BACKGROUND: General practitioners (GPs) often lack time and resources to invest in health education; audiovisual messages broadcast in the waiting room may be a useful educational tool. This work was designed to assess the effect of a message inviting patients to ask for a tetanus booster vaccination. METHODS: A quasi experimental study was conducted in a Belgian medical practice consisting of 6 GPs and 4 waiting rooms (total: 20,000 contacts/year). A tetanus booster vaccination audiovisual message was continuously broadcast for 6 months in 2 randomly selected waiting rooms (intervention group--3 GPs) while the other 2 waiting rooms remained unequipped (control group--3 GPs). At the end of the 6-month period, the number of vaccine adult-doses delivered by local pharmacies in response to GPs' prescriptions was recorded. As a reference, the same data were also collected retrospectively for the general practice during the same 6-month period of the previous year. RESULTS: During the 6-month reference period where no audiovisual message was broadcast in the 4 waiting rooms, the number of prescriptions presented for tetanus vaccines was respectively 52 (0.44%) in the intervention group and 33 (0.38%) in the control group (p = 0.50). By contrast, during the 6-month study period, the number of prescriptions differed between the two groups (p < 0.0001), rising significantly to 91 (0.79%) in the intervention group (p = 0.0005) while remaining constant in the control group (0.38% vs 0.39%; p = 0.90). CONCLUSIONS: Broadcasting an audiovisual health education message in the GPs' waiting room was associated with a significant increase in the number of adult tetanus booster vaccination prescriptions delivered by local pharmacies.

Voice and Vocal Fold Condition Following Short-Term General Anesthesia: A Prospective Study.

BACKGROUND: Dysphonia, with or without laryngeal changes, has been reported as a complication following prolonged intubation. In contrast, it is unknown if laryngeal changes also occur following short-term airway instrumentation. The objectives of this study were to determine the prevalence of laryngeal changes in patients undergoing short-term routine general anesthesia using an endotracheal tube (ETT) or supraglottic airway (SGA), and to identify predictors to these changes. METHODS: Standardized voice assessments were performed preoperatively, postoperatively, and at follow-up on adults undergoing general anesthesia for an elective procedure of less than three hours requiring an ETT or a SGA. The standardized voice assessment protocol comprised a rigid videolaryngostroboscopy, the Voice Handicap Index (VHI), and acoustic voice analysis. The effects of demographic and anesthetic characteristics and type of airway instrumentation on the videolaryngostroboscopic variables were studied using multilevel logistic regression. Multilevel linear regression was used to reveal preoperative versus postoperative changes in VHI and acoustic voice scores. RESULTS: Overall, the prevalence of postoperative laryngeal changes was low. Significant postoperative laryngeal changes were found for the variables right-sided vocal fold redness in the ETT group (P = 0.048) and right-sided vocal fold blood vessels in both groups (ETT versus SGA). However, after adjustment for all demographic and anesthetic characteristics in the regression model, the effect of the type of airway instrumentation (ETT versus SGA) on the variable right-sided vocal fold redness was no longer significant. CONCLUSIONS: ETT and SGA short-term airway instrumentation are vocal fold function sparing techniques with negligible laryngeal changes.

The use of spectrograms improves the classification of wheezes and crackles in an educational setting.

Chest auscultation is a widely used method in the diagnosis of lung diseases. However, the interpretation of lung sounds is a subjective task and disagreements arise. New technological developments like the use of visSual representation of sounds through spectrograms could improve the agreement when classifying lung sounds, but this is not yet known. In this study, we tested if the use of spectrograms improves the agreement when classifying wheezes and crackles. To do this, we asked twenty-three medical students at UiT the Arctic University of Norway to classify 30 lung sounds recordings for the presence of wheezes and crackles. The sample contained 15 normal recordings and 15 with wheezes or crackles. The students classified the recordings in a random order twice. First sound only, then sound with spectrograms. We calculated kappa values for the agreement between each student and the expert classification with and without display of spectrograms and tested for significant improvement between these two coefficients. We also calculated Fleiss kappa for the 23 observers with and without the spectrogram. In an individual analysis comparing each student to an expert annotated reference standard we found that 13 out of 23 students had a positive change in kappa when classifying wheezes with the help of spectrograms. When classifying crackles 16 out of 23 showed improvement when spectrograms were used. In a group analysis we observed that Fleiss kappa values were k = 0.51 and k = 0.56 (p = 0.63) for classifying wheezes without and with spectrograms. For crackles, these values were k = 0.22 and k = 0.40 (p = <0.01) in the same order. Thus, we conclude that the use of spectrograms had a positive impact on the inter-rater agreement and the agreement with experts. We observed a higher improvement in the classification of crackles compared to wheezes.

Affective symptoms and swallow-specific quality of life in total laryngectomy patients.

BACKGROUND: The aim of this study is to determine the prevalence of clinically relevant affective symptoms and level of swallow-specific quality of life (QoL) in dysphagic patients with total laryngectomy (TL) and to explore the relationship between affective symptoms and swallow-specific QoL. METHODS: Thirty-five TL patients completed the Hospital Anxiety and Depression Scale (HADS) and the MD Anderson Dysphagia Inventory (MDADI). Student's t test and linear regression were used. RESULTS: Eight (23%) patients showed clinically relevant symptoms of anxiety, 8 (23%) of depression, and 11 (31%) showed either one. These groups had significantly lower mean MDADI scores. One-point increase in HADS-anxiety or HADS-depression subscale score corresponds with a decrease of 2.7 or 3.0 points, on average, respectively, of the MDADI total score. CONCLUSIONS: Clinically relevant affective symptoms were present in approximately one-third of the TL patients. These preliminary results show that increased affective symptom scores correlate with a decreased swallow-specific QoL.