Reliability of video recordings to evaluate quality of anesthesia recovery in dogs.

OBJECTIVE: To assess the reliability of using video recordings to evaluate anesthesia recovery in dogs. STUDY DESIGN: Prospective study. ANIMALS: A total of 30 dogs undergoing surgery. METHODS: Recovery monitoring and video recording lasted from extubation until 1 hour later. Scoring was done in real time at the end of the hour by a graduate student using three systems: a simple descriptive scale, visual analog scale and numeric rating scale. Videos were distributed to three American College of Veterinary Anesthesia and Analgesia board-certified anesthesiologist raters as well as the original rater to score. These videos were revisited 4 months later, and the recoveries were scored again. To assess reliability, Cohen's and Fleiss' kappa values evaluated the agreement between sessions. Wilcoxon signed-rank tests were run comparing each observer's two sessions. Recoveries were classified as 'good' or 'bad' according to the cut-off values for each system. RESULTS: Correlation values among raters ranged from 0.50 to 0.82 and from 0.26 to 0.60 in the first and second session, respectively. Bland-Altman plots revealed biases between 0.133 and 1.633 points for each of the three scoring systems. Cohen's kappa had agreement ranging from 0.29 to 0.79 during the first viewing and from 0.17 to 0.44 during the second. Fleiss' kappa values were 0.06, 0.16, 0.22 and 0.26 for various data combinations. Considering overall recovery, Fleiss' kappa showed agreement ranging from 0.54 to 0.71 and from 0.13 to 0.49 for the first and second session, respectively. Of the 12 Wilcoxon tests run, seven found significantly different scores between the two scoring sessions. The recoveries given an overall good or bad were the same on both occasions that they were reviewed. CONCLUSIONS AND CLINICAL RELEVANCE: The use of a video to evaluate recovery in dogs should be used with caution. Individual raters' agreement for specific scores was poor, but evaluating recovery overall had perfect agreement.