Interobserver Variability in the International Study Group for Pancreas Surgery (ISGPS)-Defined Complications After Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study.

ABSTRACT

OBJECTIVE: To determine the interobserver variability for complications of pancreatoduodenectomy as defined by the International Study Group for Pancreatic Surgery (ISGPS) and others. SUMMARY BACKGROUND DATA Good interobserver variability for the definitions of surgical complications is of major importance in comparing surgical outcomes between and within centers. However, data on interobserver variability for pancreatoduodenectomy-specific complications are lacking. METHODS: International cross-sectional multicenter study including 52 raters from 13 high-volume pancreatic centers in 8 countries on 3 continents. Per center, 4 experienced raters scored 30 randomly selected patients after pancreatoduodenectomy. In addition, all raters scored six standardized case vignettes. This variability and the 'within centers' variability were calculated for twofold scoring (no complication/grade A vs grade B/C) and threefold scoring (no complication/grade A vs grade B vs grade C) of postoperative pancreatic fistula (POPF), post-pancreatoduodenectomy hemorrhage (PPH), chyle leak (CL), bile leak (BL), and delayed gastric emptying (DGE). Interobserver variability is presented with Gwet's AC-1 measure for agreement. RESULTS: Overall, 390 patients after pancreatoduodenectomy were included. The overall agreement rate for the standardized cases vignettes for twofold scoring was 68% (95%-CI 55%-81%, AC1 score moderate agreement) and for threefold scoring 55% (49%-62%, AC1 score fair agreement). The mean 'within centers' agreement for twofold scoring was 84% (80%-87%, AC1 score; substantial agreement). CONCLUSION: The interobserver variability for the ISGPS defined complications of pancreatoduodenectomy was too high even though the 'within centers' agreement was acceptable. Since these findings will decrease the quality and validity of clinical studies, ISGPS has started efforts aimed at reducing the interobserver variability.