Multicenter Interrater Reliability in the Endoscopic Assessment of Velopharyngeal Function Using a Video Instruction Tool.

ABSTRACT

OBJECTIVE: Assess interrater agreement of endoscopic assessment of velopharyngeal (VP) function before and after viewing the video instruction tool (VIT). We hypothesized improvement in interrater agreement using the Golding-Kushner scale (GKS) after viewing the VIT. STUDY DESIGN: Prospective study. SETTING: Multi-institutional. METHODS: Sixteen fellowship-trained pediatric otolaryngologists who treat velopharyngeal insufficiency (VPI) rated 50 video segments using the GKS before and after watching the VIT. Raters assessed gap size percentage and lateral pharyngeal wall (LPW), soft palate (SP), and posterior pharyngeal wall (PPW) movement. Intraclass correlation coefficient was estimated for these continuous measures. Raters also indicated the presence of a palatal notch, Passavant's ridge, and aberrant pulsations (categorical variables). Fleiss κ coefficient was used for categorical variables. Wilcoxon signed-rank test was performed on the difference between the pre/post individual video ratings. RESULTS: Reliability improved for all continuous variables after watching the instructional video. The improvement was significant for PPW (0.22-0.30, P < .001), SP (left 0.63-0.68, P < .001 and right 0.64-0.68, P = .001), and LPW (left 0.49-0.54, P = .01 and right 0.49-0.54, P = .09) but not significant for gap size (0.65-0.69, P = .36). Among categorical variables, agreement on Passavant's ridge significantly improved (0.30-0.36, P = .03). CONCLUSION: Exposure to a video instruction tool improves interrater agreement of endoscopic assessment of VP function. Significant improvement was observed in our primary end points, specifically posterior pharyngeal wall movement, soft palate movement, and lateral pharyngeal wall movement. There was less impact of the VIT on the interrater agreement of the categorical variables, palatal notch, Passavant's ridge, and aberrant pulsations.