Crowdsourced Assessment of Surgical Skill Proficiency in Cataract Surgery.

OBJECTIVE: To test whether crowdsourced lay raters can accurately assess cataract surgical skills. DESIGN: Two-armed study: independent cross-sectional and longitudinal cohorts. SETTING: Washington University Department of Ophthalmology. PARTICIPANTS AND METHODS: Sixteen cataract surgeons with varying experience levels submitted cataract surgery videos to be graded by 5 experts and 300+ crowdworkers masked to surgeon experience. Cross-sectional study: 50 videos from surgeons ranging from first-year resident to attending physician, pooled by years of training. Longitudinal study: 28 videos obtained at regular intervals as residents progressed through 180 cases. Surgical skill was graded using the modified Objective Structured Assessment of Technical Skill (mOSATS). Main outcome measures were overall technical performance, reliability indices, and correlation between expert and crowd mean scores. RESULTS: Experts demonstrated high interrater reliability and accurately predicted training level, establishing construct validity for the modified OSATS. Crowd scores were correlated with (r = 0.865, p < 0.0001) but consistently higher than expert scores for first, second, and third-year residents (p < 0.0001, paired t-test). Longer surgery duration negatively correlated with training level (r = -0.855, p < 0.0001) and expert score (r = -0.927, p < 0.0001). The longitudinal dataset reproduced cross-sectional study findings for crowd and expert comparisons. A regression equation transforming crowd score plus video length into expert score was derived from the cross-sectional dataset (r2 = 0.92) and demonstrated excellent predictive modeling when applied to the independent longitudinal dataset (r2 = 0.80). A group of student raters who had edited the cataract videos also graded them, producing scores that more closely approximated experts than the crowd. CONCLUSIONS: Crowdsourced rankings correlated with expert scores, but were not equivalent; crowd scores overestimated technical competency, especially for novice surgeons. A novel approach of adjusting crowd scores with surgery duration generated a more accurate predictive model for surgical skill. More studies are needed before crowdsourcing can be reliably used for assessing surgical proficiency.

Genomic and Phenotypic Characterization of a Broad Panel of Patient-Derived Xenografts Reflects the Diversity of Glioblastoma.

PURPOSE: Glioblastoma is the most frequent and lethal primary brain tumor. Development of novel therapies relies on the availability of relevant preclinical models. We have established a panel of 96 glioblastoma patient-derived xenografts (PDX) and undertaken its genomic and phenotypic characterization. EXPERIMENTAL DESIGN: PDXs were established from glioblastoma, IDH-wildtype (n = 93), glioblastoma, IDH-mutant (n = 2), diffuse midline glioma, H3 K27M-mutant (n = 1), and both primary (n = 60) and recurrent (n = 34) tumors. Tumor growth rates, histopathology, and treatment response were characterized. Integrated molecular profiling was performed by whole-exome sequencing (WES, n = 83), RNA-sequencing (n = 68), and genome-wide methylation profiling (n = 76). WES data from 24 patient tumors was compared with derivative models. RESULTS: PDXs recapitulate many key phenotypic and molecular features of patient tumors. Orthotopic PDXs show characteristic tumor morphology and invasion patterns, but largely lack microvascular proliferation and necrosis. PDXs capture common and rare molecular drivers, including alterations of TERT, EGFR, PTEN, TP53, BRAF, and IDH1, most at frequencies comparable with human glioblastoma. However, PDGFRA amplification was absent. RNA-sequencing and genome-wide methylation profiling demonstrated broad representation of glioblastoma molecular subtypes. MGMT promoter methylation correlated with increased survival in response to temozolomide. WES of 24 matched patient tumors showed preservation of most genetic driver alterations, including EGFR amplification. However, in four patient-PDX pairs, driver alterations were gained or lost on engraftment, consistent with clonal selection. CONCLUSIONS: Our PDX panel captures the molecular heterogeneity of glioblastoma and recapitulates many salient genetic and phenotypic features. All models and genomic data are openly available to investigators.