Professional Practice Evaluation for Pathologists: The Development, Life, and Death of the Evalumetrics Program.

CONTEXT: - In 2008, the Joint Commission (JC) implemented a standard mandating formal monitoring of physician professional performance as part of the process of granting and maintaining practice privileges. OBJECTIVE: - To create a pathology-specific management tool to aid pathologists in constructing a professional practice-monitoring program, thereby meeting the JC mandate. DESIGN: - A total of 105 College of American Pathologists (CAP)-defined metrics were created. Metrics were based on the job descriptions of pathologists' duties in the laboratory, and metric development was aided by experience from the Q-Probes and Q-Tracks programs. The program was offered in a Web-based format, allowing secure data entry, customization of metrics, and central data collection for future benchmarking. RESULTS: - The program was live for 3 years, with 347 pathologists subscribed from 61 practices (median, 4 per institution; range, 1-35). Subscribers used 93 of the CAP-defined metrics and created 109 custom metrics. The median number of CAP-defined metrics used per pathologist was 5 (range, 1-43), and the median custom-defined metrics per pathologist was 2 (range, 1-5). Most frequently, 1 to 3 metrics were monitored (42.7%), with 20% each following 4 to 6 metrics, 5 to 9 metrics, or greater than 10 metrics. Anatomic pathology metrics were used more commonly than clinical pathology metrics. Owing to low registration, the program was discontinued in 2016. CONCLUSIONS: - Through careful vetting of metrics it was possible to develop a pathologist-specific management tool to address the JC mandate. While this initial product failed, valuable metrics were developed and implementation knowledge was gained that may be used to address new regulatory requirements for emerging value-based payment systems.

Prevalence of Traditional and Reverse-Algorithm Syphilis Screening in Laboratory Practice: A Survey of Participants in the College of American Pathologists Syphilis Serology Proficiency Testing Program.

CONTEXT: -Syphilis serology screening in laboratory practice is evolving. Traditionally, the syphilis screening algorithm begins with a nontreponemal immunoassay, which is manually performed by a laboratory technologist. In contrast, the reverse algorithm begins with a treponemal immunoassay, which can be automated. The Centers for Disease Control and Prevention has recognized both approaches, but little is known about the current state of laboratory practice, which could impact test utilization and interpretation. OBJECTIVE: -To assess the current state of laboratory practice for syphilis serologic screening. DESIGN: -In August 2015, a voluntary questionnaire was sent to the 2360 laboratories that subscribe to the College of American Pathologists syphilis serology proficiency survey. RESULTS: -Of the laboratories surveyed, 98% (2316 of 2360) returned the questionnaire, and about 83% (1911 of 2316) responded to at least some questions. Twenty-eight percent (378 of 1364) reported revision of their syphilis screening algorithm within the past 2 years, and 9% (170 of 1905) of laboratories anticipated changing their screening algorithm in the coming year. Sixty-three percent (1205 of 1911) reported using the traditional algorithm, 16% (304 of 1911) reported using the reverse algorithm, and 2.5% (47 of 1911) reported using both algorithms, whereas 9% (169 of 1911) reported not performing a reflex confirmation test. Of those performing the reverse algorithm, 74% (282 of 380) implemented a new testing platform when introducing the new algorithm. CONCLUSION: -The majority of laboratories still perform the traditional algorithm, but a significant minority have implemented the reverse-screening algorithm. Although the nontreponemal immunologic response typically wanes after cure and becomes undetectable, treponemal immunoassays typically remain positive for life, and it is important for laboratorians and clinicians to consider these assay differences when implementing, using, and interpreting serologic syphilis screening algorithms.

Assessing Clinical Laboratory Quality: A College of American Pathologists Q-Probes Study of Prothrombin Time INR Structures, Processes, and Outcomes in 98 Laboratories.

CONTEXT: The anticoagulant warfarin has been identified as the second most frequent drug responsible for serious, disabling, and fatal adverse drug events in the United States, and its effect on blood coagulation is monitored by the laboratory test called international normalized ratio (INR). OBJECTIVE: To determine the presence of INR policies and procedures, INR practices, and completeness and timeliness of reporting critical INR results in participants' clinical laboratories. DESIGN: Participants reviewed their INR policies and procedure requirements, identified their practices by using a questionnaire, and studied completeness of documentation and timeliness of reporting critical value INR results for outpatients and emergency department patients. RESULTS: In 98 participating institutions, the 5 required policies and procedures were in place in 93% to 99% of clinical laboratories. Fifteen options for the allowable variations among duplicate results from different analyzers, 12 different timeliness goals for reporting critical values, and 18 unique critical value limits were used by participants. All required documentation elements were present in 94.8% of 192 reviewed INR validation reports. Critical value INR results were reported within the time frame established by the laboratory for 93.4% of 2604 results, but 1.0% of results were not reported. Although the median laboratories successfully communicated all critical results within their established time frames and had all the required validation elements based in their 2 most recent INR calculations, those participants at the lowest 10th percentile were successful in 80.0% and 85.7% of these requirements, respectively. CONCLUSIONS: Significant opportunities exist for adherence to INR procedural requirements and for practice patterns and timeliness goals for INR critical results' reporting.

Twenty-five years of accomplishments of the College of American Pathologists Q-probes program for clinical pathology.

CONTEXT: During the past 25 years, the College of American Pathologists' (CAP) Q-Probes program has been available as a subscription program to teach laboratorians how to improve the quality of clinical laboratory services. OBJECTIVE: To determine the accomplishments of the CAP Q-Probes program. DESIGN: We reviewed Q-Probes participant information, study data and conclusions, author information, and program accomplishments. RESULTS: During this time 117 Q-Probes clinical pathology studies were conducted by 54 authors and coauthors, 42,899 laboratories enrolled from 24 countries, 98 peer-reviewed publications occurred and were cited more than 1600 times, and the studies were featured 59 times in CAP Today. The most frequent studies (19) focused on turnaround times for results or products at specific locations (emergency department, operating room, inpatients, outpatients), specific diseases (acute myocardial infarction, urinary tract), availability for specific events such as morning rounds or surgery, a specific result (positive blood cultures), and a method on how to use data for improvement (stat test outliers). Percentile ranking of study participants with better performance provided benchmarks for each study with attributes statistically defined that influenced improved performance. Other programs, such as an ongoing quality improvement program (Q-Tracks), a laboratory competency assessment program, a pathologist certification program, and an ongoing physician practice evaluation program (Evalumetrics), have been developed from Q-Probes studies. CONCLUSIONS: The CAP's Q-Probes program has made significant contributions to the medical literature and has developed a worldwide reputation for improving the quality of clinical pathology services worldwide.

Fifteen years' experience of a College of American Pathologists program for continuous monitoring and improvement.

CONTEXT: The Q-Tracks program, created in 1999, is a quality monitoring subscription service offered by the College of American Pathologists. OBJECTIVE: To establish benchmarks in quality metrics, monitor changes in performance over time, and identify practice characteristics associated with better performance. DESIGN: The Q-Tracks program provides ongoing study of multiple metrics offered in most laboratory disciplines. The design enables measuring the effects of process changes and comparisons with other participating laboratories. Each laboratory Q-Tracks monitor has a primary quality indicator and additional secondary indicators. RESULTS: To date, 19 Q-Tracks monitors have been offered, with 12 currently active monitors. Q-Tracks are primarily conducted in hospital-based laboratories in the United States, Canada, and 21 other countries. Common to most Q-Tracks monitors is a demonstration of performance improvement by subscribers with long-term participation. This finding was seen in preanalytic, turnaround time, and postanalytic measures. Q-Tracks monitors contribute to the overall demonstration and improvement of laboratory and hospital quality because they address core quality measures for the College of American Pathologists Laboratory Accreditation Program and multiple Joint Commission National Patient Safety Goals. CONCLUSIONS: The Q-Tracks program has established multiple benchmarks in most disciplines of the laboratory and has demonstrated significant performance improvement in benchmarks and individual laboratories over time.

Q-Probes studies in anatomic pathology: quality improvement through targeted benchmarking.

CONTEXT: The Q-Probes program is a peer-comparison quality assurance service offered by the College of American Pathologists that was created in 1989. OBJECTIVE: To establish national benchmarks around a specific quality metric at a specific point in time in anatomic pathology (AP). DESIGN: Q-Probes are based on a voluntary subscription for an individual study. Hospital-based laboratories in the United States, Canada, and 16 other countries have participated. Approximately one-third of all Q-Probes studies address AP metrics. Each Q-Probes study has a primary quality indicator and additional minor indicators. RESULTS: There have been 52 AP Q-Probes studies addressing process-, outcome-, and structure-related quality assurance issues. These Q-Probes studies often represented the first standardized national benchmark for specific metrics in the disciplines of cytopathology, surgical pathology, and autopsy pathology, and as such have been cited more than 1700 times in peer-reviewed literature. The AP Q-Probes studies that have been repeated over time demonstrate improvement in laboratory performance across an international spectrum. CONCLUSIONS: The Q-Probes program has produced important national benchmarks in AP, addressing preanalytic, analytic, and postanalytic factors in the disciplines of cytopathology, surgical pathology, and autopsy pathology. Q-Probes study data have been published, cited, and used in the creation of laboratory accreditation standards and other national guidelines.