Credentialing, Certification, and Peer Review Essentials for the Neurosurgeon.

Credentialing and certification are essential processes during hiring to ensure that the physician is competent and possesses the qualifications and skill sets claimed. Peer review ensures the continuing evolution of these skills to meet a standard of care. We have provided an overview and discussion of these processes in the United States. Credentialing is the process by which a physician is determined to be competent and able to practice, used to ensure that medical staff meets specific standards, and to grant operative privileges at an institution. Certification is a standardized affirmation of a physician's competence on a nationwide basis. Although not legally required to practice in the United States, many institutions emphasize certification for full privileges on an ongoing basis at a hospital. In the United States, peer review of adverse events is a mandatory prerequisite for accreditation. The initial lack of standardization led to the development of the Health Care Quality Improvement Act, which protects those involved in the peer review process from litigation, and the National Provider Databank, which was established as a national database to track misconduct. A focus on quality improvement in the peer review process can lead to improved performance and patient outcomes. A thorough understanding of the processes of credentialing, certification, and peer review in the United States will benefit neurosurgeons by allowing them to know what institutions are looking for as well and their rights and responsibilities in any given situation. It could also be useful to compare these policies and practices in the United States to those in other countries.

Association between leniency of anesthesiologists when evaluating certified registered nurse anesthetists and when evaluating didactic lectures.

Daily evaluations of certified registered nurse anesthetists' (CRNAs') work habits by anesthesiologists should be adjusted for rater leniency. The current study tested the hypothesis that there is a pairwise association by rater between leniencies of evaluations of CRNAs' daily work habits and of didactic lectures. The historical cohorts were anesthesiologists' evaluations over 53 months of CRNAs' daily work habits and 65 months of didactic lectures by visiting professors and faculty. The binary endpoints were the Likert scale scores for all 6 and 10 items, respectively, equaling the maximums of 5 for all items, or not. Mixed effects logistic regression estimated the odds of each ratee performing above or below average adjusted for rater leniency. Bivariate errors in variables least squares linear regression estimated the association between the leniency of the anesthesiologists' evaluations of work habits and didactic lectures. There were 29/107 (27%) raters who were more severe in their evaluations of CRNAs' work habits than other anesthesiologists (two-sided P < 0.01); 34/107 (32%) raters were more lenient. When evaluating lectures, 3/81 (4%) raters were more severe and 8/81 (10%) more lenient. Among the 67 anesthesiologists rating both, leniency (or severity) for work habits was not associated with that for lectures (P = 0.90, unitless slope between logits 0.02, 95% confidence interval -0.34 to 0.30). Rater leniency is of large magnitude when making daily clinical evaluations, even when using a valid and psychometrically reliable instrument. Rater leniency was context dependent, not solely a reflection of raters' personality or rating style.

100% peer review in radiation oncology: is it feasible?

PURPOSE: Peer review has been proposed as a strategy to ensure patient safety and plan quality in radiation oncology. Despite its potential benefits, barriers commonly exist to its optimal implementation in daily clinical routine. Our purpose is to analyze peer-review process at our institution. METHODS AND MATERIALS: Based on our group peer-review process, we quantified the rate of plan changes, time and resources needed for this process. Prospectively, data on cases presented at our institutional peer-review conference attended by physicians, resident physicians and physicists were collected. Items such as time to present per case, type of patient (adult or pediatric), treatment intent, dose, aimed technique, disease location and receipt of previous radiation were gathered. Cases were then analyzed to determine the rate of major change, minor change and plan rejection after presentation as well as the median time per session. RESULTS: Over a period of 4 weeks, 148 cases were reviewed. Median of attendants was six physicians, three in-training-physicians and one physicist. Median time per session was 38 (4-72) minutes. 59.5% of cases presented in 1-4 min, 32.4% in 5-9 min and 8.1% in ≥ 10 min. 79.1% of cases were accepted without changes, 11.5% with minor changes, 6% with major changes and 3.4% were rejected with indication of new presentation. Most frequent reason of change was contouring corrections (53.8%) followed by dose or fractionation (26.9%). CONCLUSION: Everyday group consensus peer review is an efficient manner to recollect clinical and technical data of cases presented to ensure quality radiation care before initiation of treatment as well as ensuring department quality in a feedback team environment. This model is feasible within the normal operation of every radiation oncology Department.

Realizing One's Own Subjectivity: Assessors' Perceptions of the Influence of Training on Their Conduct of Workplace-Based Assessments.

PURPOSE: Assessor training is essential for defensible assessments of physician performance, yet research on the effectiveness of training programs for promoting assessor consistency has produced mixed results. This study explored assessors' perceptions of the influence of training and assessment tools on their conduct of workplace-based assessments of physicians. METHOD: In 2017, the authors used a constructivist grounded theory approach to interview 13 physician assessors about their perceptions of the effects of training and tool development on their conduct of assessments. RESULTS: Participants reported that training led them to realize that there is a potential for variability in assessors' judgments, prompting them to change their scoring and feedback behaviors to enhance consistency. However, many participants noted they had not substantially changed their numerical scoring. Nonetheless, most thought training would lead to increased standardization and consistency among assessors, highlighting a "standardization paradox" in which participants perceived a programmatic shift toward standardization but minimal changes in their own ratings. An "engagement effect" was also found in which participants involved in both tool development and training cited more substantial learnings than participants involved only in training. CONCLUSIONS: Findings suggest that training may help assessors recognize their own subjectivity when judging performance, which may prompt behaviors that support rigorous and consistent scoring but may not lead to perceptible changes in assessors' numeric ratings. Results also suggest that participating in tool development may help assessors align their judgments with the scoring criteria. Overall, results support the continued study of assessor training programs as a means of enhancing assessor consistency.

The Congenital Heart Technical Skill Study: Rationale and Design.

BACKGROUND: We report the rationale and design for a peer-evaluation protocol of attending congenital heart surgeon technical skill using direct video observation. METHODS: All surgeons contributing data to The Society of Thoracic Surgeons-Congenital Heart Surgery Database (STS-CHSD) are invited to submit videos of themselves operating, to rate peers, or both. Surgeons may submit Norwood procedures, complete atrioventricular canal repairs, and/or arterial switch operations. A HIPPA-compliant website allows secure transmission/evaluation. Videos are anonymously rated using a modified Objective Structured Assessment of Technical Skills score. Ratings are linked to five years of contemporaneous outcome data from the STS-CHSD and surgeon questionnaires. The primary outcome is a composite for major morbidity/mortality. RESULTS: Two hundred seventy-six surgeons from 113 centers are eligible for participation: 83 (30%) surgeons from 53 (45%) centers have agreed to participate, with recruitment ongoing. These surgeons vary considerably in years of experience and outcomes. Participants, both early and late in their careers, describe the process as "very rewarding" and "less time consuming than anticipated." An initial subset of 10 videos demonstrated excellent interrater reliability (interclass correlation = 0.85). CONCLUSIONS: This study proposes to evaluate the technical skills of attending pediatric cardiothoracic surgeons by video observation and peer-review. It is notable that over a quarter of congenital heart surgeons, across a range of experiences, from almost half of United States centers have already agreed to participate. This study also creates a mechanism for peer feedback; we hypothesize that feedback could yield broad and meaningful quality improvement.

A Prospective Analysis of Radiation Oncologist Compliance With Early Peer Review Recommendations.

PURPOSE: We conducted a prospective observational cohort study of physician compliance with daily early pretreatment planning peer review recommendations and quantified factors associated with compliance. METHODS AND MATERIALS: All patient cases in our department are presented at 2 peer review conferences: (1) "early" preplanning, occurring daily for patients who have undergone simulation review, and (2) "late" (chart rounds), occurring weekly for patients who have started treatment. Peer review recommendations were prospectively recorded during early review, and compliance with recommendations was determined at chart rounds. Recommendations were assigned magnitude scores (minor, moderate, or major). We analyzed the association of patient, physician, and recommendation characteristics and compliance (scored as a binary variable) with early peer review recommendations, using logistic regression with a mixed effects model. RESULTS: From February 2017 to May 2018, 1271 patient cases underwent early peer review, and 326 (26%) received peer-based recommendations. Of 356 recommendations, 37% were minor, 36% were moderate, and 27% were major. Overall compliance was 59% (95% confidence interval, 54%-64%). On univariate analysis, compliance decreased as the recommendation magnitude increased (minor, 65%; moderate, 60%; major, 47%; P = .019; odds ratio, 0.71 per increase in magnitude). Compliance also differed among different treating physicians (range, 38%-73%, χ2 test, P = .003) but was not associated with other physician characteristics. Disease group and treatment technique were not associated with compliance. On multivariable analysis, increasing recommendation magnitude remained significantly associated with decreased compliance (multivariate P = .042; odds ratio, 0.74). CONCLUSIONS: Daily early peer review resulted in a substantial proportion of recommended changes. Compliance with early peer review recommendations was fair but varied among physicians. Compliance declined with increasing recommendation magnitude, suggesting that physicians may be reluctant to adopt major changes. These results highlight the potential importance of peer review timing.

Toxicologic Pathology Forum*: Opinion on Considerations for the Use of Whole Slide Images in GLP Pathology Peer Review.

Whole slide imaging (WSI) technology has advanced to a point where it has replaced the glass slide as the primary means of pathology evaluation within many areas of medical pathology. The deployment of WSI in the field of toxicologic pathology has been delayed by a lack of clarity around the degree of validation required for its use on Good Laboratory Practice (GLP) studies. The current opinion piece attempts to provide a high-level overview of WSI technology to include basic methodology, advantages and disadvantages over a conventional microscope, validation status of WSI scanners, and perceived concerns over regulatory acceptance for the use of WSI for (GLP) peer review in the field of toxicologic pathology. Observations are based on the extensive use by AstraZeneca of WSI for the peer review of non-GLP studies conducted at Charles River facilities and represent the experiences of the authors. Note: This is an opinion article submitted to the Toxicologic Pathology Forum. It represents the views of the author(s). It does not constitute an official position of the Society of Toxicologic Pathology, British Society of Toxicological Pathology, or European Society of Toxicologic Pathology, and the views expressed might not reflect the best practices recommended by these Societies. This article should not be construed to represent the policies, positions, or opinions of their respective organizations, employers, or regulatory agencies.

Radiation therapy quality assurance in head and neck radiotherapy - Moving forward.

Head and Neck Cancer (HNC) radiation oncologists (ROs) enjoy the immense pleasure of curing patients, working within a large multidisciplinary team to effectively deliver curative intent treatment whilst also aiming to minimise late treatment toxicity. Secondary analyses of large-scale HNC clinical trials have shown the critical impact of the quality of radiotherapy plans, where protocol non-compliant plans have yielded inferior survival rates approximating 20%. The peer review process in routine day-to-day HNC practice shows that even in major academic centers a significant proportion of RT plans may require changes to the radiotherapy planning volume. Optimising the therapeutic ratio in HNC has been dramatically facilitated by intensity modulated radiotherapy (IMRT), but that technology has also increased the complexity of HNC radiotherapy treatment and high-volume centers with experienced clinicians may be best placed to deliver this most accurately. International consensus guidelines to standardise or benchmark best practice with respect to the RT-QA process in HNC are needed. The aim of this paper is to highlight the importance of the RT-QA process in the HNC treatment process and to make some recommendations for its inclusion in both clinical trials and routine clinical practice.

How do physicians behave when they participate in audit and feedback activities in a group with their peers?

BACKGROUND: Audit and feedback interventions may be strengthened using social interaction. With this in mind, the Calgary office of the Alberta Physician Learning Program developed a process for audit and group feedback for physician groups. As a part of a larger project to develop a practical approach to the design and implementation of audit and group feedback projects, we explored patterns of physician behavior during facilitated audit and group feedback sessions. METHODS: Six audit and group feedback sessions were recorded, transcribed, and analyzed thematically to derive a conceptual model of physicians' behaviors during audit and group feedback sessions. RESULTS: A predictable cycle of behaviors emerged from audit and group feedback sessions. This cycle would repeat with discussion of each new data element: reacting to the data, questioning and understanding the data, justifying and contextualizing, sharing and reflecting on the data and relevant guidelines, and planning for change. "Change cues" that emerged within groups reliably pivoted the discussion towards action planning. CONCLUSIONS: In audit and group feedback sessions, physicians display a predictable series of behaviors as they move towards commitment to change. Establishing the meaning and credibility of the data is a necessary precursor to reflection. Group reflection leads to "change cues" triggered by group members, which stimulate action planning.

Random Versus Nonrandom Peer Review: A Case for More Meaningful Peer Review.

OBJECTIVE: Random peer review programs are not optimized to discover cases with diagnostic error and thus have inherent limitations with respect to educational and quality improvement value. Nonrandom peer review offers an alternative approach in which diagnostic error cases are targeted for collection during routine clinical practice. The objective of this study was to compare error cases identified through random and nonrandom peer review approaches at an academic center. METHODS: During the 1-year study period, the number of discrepancy cases and score of discrepancy were determined from each approach. RESULTS: The nonrandom peer review process collected 190 cases, of which 60 were scored as 2 (minor discrepancy), 94 as 3 (significant discrepancy), and 36 as 4 (major discrepancy). In the random peer review process, 1,690 cases were reviewed, of which 1,646 were scored as 1 (no discrepancy), 44 were scored as 2 (minor discrepancy), and none were scored as 3 or 4. Several teaching lessons and quality improvement measures were developed as a result of analysis of error cases collected through the nonrandom peer review process. CONCLUSIONS: Our experience supports the implementation of nonrandom peer review as a replacement to random peer review, with nonrandom peer review serving as a more effective method for collecting diagnostic error cases with educational and quality improvement value.

Peer-assisted debriefing of multisource feedback: an exploratory qualitative study.

BACKGROUND: The Manitoba Physician Achievement Review (MPAR) is a 360-degree feedback assessment that physicians undergo every 7 years to retain licensure. Deliberate reflection on feedback has been demonstrated to encourage practice change. The MPAR Reflection Exercise (RE), a peer-assisted debriefing tool, was developed whereby the physician selects a peer with whom to review and reflect on feedback, committing to change. This qualitative study explores how physicians who had undergone the MPAR used the RE, what areas of change are identified and committed to, and what they perceived as the role of reflection in the MPAR process. METHODS: The MPAR RE was piloted out to a cohort of MPAR-reviewed physicians. Thematic analysis was conducted on completed exercises (n = 61). Semi-structured interviews were conducted with individuals (n = 6) who completed the MPAR RE until saturation was reached. RESULTS: Physicians reviewed feedback with a range of peers, including colleagues, staff, and spouses. Many physicians were surprised by feedback, both positive and negative, but interviewees found the RE useful in processing feedback. Areas where physicians committed to change were diverse, covering all CanMEDS roles. Most physicians identified themselves as being successful in implementing change, though time, habit, and structures were cited as barriers. CONCLUSIONS: Peer-assisted debriefing can assist reflection of multisource feedback. It is easy to implement, is not resource-intensive, and feedback implies that it is effective at promoting change. Participants, with the aid of peers, identified areas for change, developed approaches for change, and largely thought themselves successful at implementing changes. Areas of change included all seven CanMEDS roles.

The Impact of a State Medicaid Peer-Review Authorization Program on Pediatric Use of Antipsychotic Medications.

OBJECTIVE: This cross-sectional study assessed the impact of a peer-review program on the prevalence of pediatric antipsychotic use among Medicaid-insured youths in a Mid-Atlantic state. METHODS: Medicaid claims (2010-2014) were assessed among continuously enrolled youths in the 12 months before and after implementation of peer review. The study identified children ages zero to four preimplementation (N=118,815) and postimplementation (N=121,431), ages five to nine preimplementation (N=98,681) and postimplementation (N=107,872), and ages 10 to 17 preimplementation (N=154,696) and postimplementation (N=161,370). (Age ranges are inclusive of the final number). In each age group, multivariable logistic regression models with generalized estimating equations assessed the change in annual prevalence of antipsychotic use pre- to postimplementation. Use of other leading psychotropic classes and antipsychotic prescribing by medical specialty were also examined. RESULTS: The annual pre- to postimplementation prevalence of antipsychotic use decreased significantly, from .07% to .03% (adjusted odds ratio [AOR]=.41) among children ages zero to four, from 1.57% to .86% (AOR=.54) among those ages five to nine, and from 3.28% to 2.40% (AOR=.72) among those ages 10 to 17. With the exception of alpha-agonist use, which increased postimplementation (AOR=1.30) among those ages zero to four, no clinically significant pre-post change was noted in other leading psychotropic classes among children ages zero to four and 10 to 17. By contrast, postimplementation use of other psychotropic medications decreased among those ages five to nine (AOR=.73). CONCLUSIONS: A state Medicaid peer-review program resulted in decreased antipsychotic use across all age groups, particularly among children younger than ten. No notable substitution of other psychotropic classes for antipsychotics was observed.

Validity of using a work habits scale for the daily evaluation of nurse anesthetists' clinical performance while controlling for the leniencies of the rating anesthesiologists.

STUDY OBJECTIVE: Anesthesiologists can provide psychometrically reliable daily evaluations of certified registered nurse anesthetist (CRNA) work habits for purposes of the mandatory ongoing professional practice evaluation (OPPE). Our goal was to evaluate the validity of assessing CRNA work habits. DESIGN: Observational study. SETTING: Large teaching hospital. MEASUREMENTS: N=77 anesthesiologists evaluated work habits of N=67 CRNAs. The non-technical attribute of work habits was measured on a 6-item scale (e.g., 1="Only assumed responsibility when forced to, and failed to follow through consistently" versus 5="Consistently identified tasks and completed them efficiently and thoroughly"). One year of scores were used to assess validity. Each daily evaluation could also be accompanied by a written comment. Content analysis of comments was performed using two years of data. Statistical analyses were performed using mixed effects logistic regression, treating each anesthesiologist as a fixed effect to compensate for the leniency of their ratings. MAIN RESULTS: The N=77 anesthesiologists' response rate was 97.3%, obtained at a mean 2.93days after the request. The internal consistency of the scale was large: Cronbach's alpha 0.952. Controlling the false discovery rate at 5.0%, among the 67 CRNAs, 8 were significantly below average (each P≤0.0048) and 6 were above average (each P≤0.0018). During the 6months after CRNAs knew that their work habits scores would be used for OPPE, there were significant increases in the scores compared with the preceding 6months (odds ratio 1.93, P<0.0001). Greater CRNA's qualitative annual evaluation scores made by the chief CRNA, without knowledge of the work habit scores or comments, were associated with greater odds of the CRNA's leniency-adjusted work habit scores equaling 5.00 (odds ratio 1.53, P=0.0004). Comments of negative sentiment made by the anesthesiologists were associated with greater odds of the leniency-adjusted work habit scores being <5.00 (odds ratio 54.5, P<0.0001). Even though the anesthesiologists were already providing information about work habits using the work habits scale, approximately half the comments of negative sentiment included the theme of work habits (92/153, 60.1%). CONCLUSIONS: Reporting OPPE metrics are mandatory for the maintenance of clinical privileges of anesthesia practitioners in the USA. Basing such peer review on work habits can be quantitative, psychometrically reliable, and valid.

Identifying preventable trauma death: does autopsy serve a role in the peer review process?

BACKGROUND: Missing life-threatening injuries is a persistent concern in any trauma program. Autopsy is a tool routinely utilized to determine an otherwise occult cause of death in many fields of medicine. It has been adopted as a required component of the trauma peer review (PR) process by both the American College of Surgeons and the Pennsylvania Trauma Foundation. We hypothesized that autopsy would not identify preventable deaths for augmentation of the PR process. MATERIALS AND METHODS: A retrospective chart review using our institutional trauma registry of all trauma deaths between January 2012 and December 2015 was performed. Per the protocol of our level 1 center, all trauma deaths are referred to the medical examiner (ME) and reviewed as part of the trauma PR process. All autopsy results are evaluated with relation to injury severity score (ISS), trauma injury severity score (TRISS), nature of death, and injuries added by autopsy. ME reports are reviewed by the trauma medical director and referred back to the trauma PR committee if warranted. Trauma injury severity score methodology determines the probability of survival (Ps) given injuries identified. A patient with Ps of ≥0.5 is expected to survive their injuries. Cohorts were created based on when in the hospitalization death occurred: <24 h, or immediate death; 24 to 48 h, or early death; and death >48 h, or late death. A comparison was conducted between the ISS and Ps calculated during trauma workup and on autopsy using chi-square and Fischer's exact tests. RESULTS: A total of 173 patient deaths were referred to the ME with 123 responses received. Average length of stay was 2.61 d. Twenty-six patients had autopsy declined by the ME, 25 received an external examination only, and 72 received a full autopsy. Autopsy identified one case that was reconsidered in PR (P = 0.603) and added diagnoses, but not injuries, to one patient in the early death group (P = 1) and two in the late death group (P = 0.4921). No preventable cause of death was uncovered, and educational use was minimal. Autopsy did identify injuries in seven cases that were initially not consistent with expected mortality, but postmortem Ps was consistent with expected mortality (P = 0.254). Mean ISS was 34.48, and mean Ps was 0.275 among all patients. The most commonly identified injuries added by autopsy were rib injuries, lung injuries, and intracranial hemorrhage. CONCLUSIONS: Autopsy does not identify causes of preventable in an otherwise highly functioning trauma program and may be a poor use of institutional resources. In fact, it adds few diagnoses when death occurs after a full trauma assessment has had time to take place. Autopsy may be of use to identify protocol failure in maturing trauma programs, to give answers to grieving families and in select situations where death was unanticipated even after a full evaluation took place.

Evidence-based Peer Review for Radiation Therapy - Updated Review of the Literature with a Focus on Tumour Subsite and Treatment Modality.

Technological advances in radiation therapy permit steep dose gradients from the target to spare normal tissue, but increase the risk of geographic miss. Suboptimal target delineation adversely affects clinical outcomes. Prospective peer review is a method for quality assurance of oncologists' radiotherapy plans. Published surveys suggest it is widely implemented. However, it may not be feasible to review every case before commencement of radiation therapy in all departments. The rate of plan changes following peer review of cases without a specific subsite or modality is typically around 10%. Stereotactic body radiation therapy, head and neck, gynaecological, gastrointestinal, haematological and lung cases are associated with higher rates of change of around 25%. These cases could thus be prioritised for peer review. Other factors may limit peer review efficacy including organisational culture, time constraints and the physical environment in which sessions are held. Recommendations for peer review endorsed by the American Society for Radiation Oncology were made available in 2013, but a number of relevant studies have been published since. Here we review and update the literature, and provide an updated suggestion for the implementation of peer review to serve as an adjunct to published guidelines. This may help practitioners evaluate their current processes and maximise the utility and effectiveness of peer review sessions.

Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians' Professional Performance.

INTRODUCTION: Multisource feedback (MSF) instruments are used to and must feasibly provide reliable and valid data on physicians' performance from multiple perspectives. The "INviting Co-workers to Evaluate Physicians Tool" (INCEPT) is a multisource feedback instrument used to evaluate physicians' professional performance as perceived by peers, residents, and coworkers. In this study, we report on the validity, reliability, and feasibility of the INCEPT. METHODS: The performance of 218 physicians was assessed by 597 peers, 344 residents, and 822 coworkers. Using explorative and confirmatory factor analyses, multilevel regression analyses between narrative and numerical feedback, item-total correlations, interscale correlations, Cronbach's α and generalizability analyses, the psychometric qualities, and feasibility of the INCEPT were investigated. RESULTS: For all respondent groups, three factors were identified, although constructed slightly different: "professional attitude," "patient-centeredness," and "organization and (self)-management." Internal consistency was high for all constructs (Cronbach's α ≥ 0.84 and item-total correlations ≥ 0.52). Confirmatory factor analyses indicated acceptable to good fit. Further validity evidence was given by the associations between narrative and numerical feedback. For reliable total INCEPT scores, three peer, two resident and three coworker evaluations were needed; for subscale scores, evaluations of three peers, three residents and three to four coworkers were sufficient. DISCUSSION: The INCEPT instrument provides physicians performance feedback in a valid and reliable way. The number of evaluations to establish reliable scores is achievable in a regular clinical department. When interpreting feedback, physicians should consider that respondent groups' perceptions differ as indicated by the different item clustering per performance factor.

Novel peer review method for improving controlled substance prescribing in primary care.

OBJECTIVE: Determine if peer feedback through a chart review tool (CRT) can impact opioid prescribing for patients with chronic noncancer pain in an outpatient family medicine clinic at an urban, safety-net teaching hospital and to assess providing practices. DESIGN: A quality improvement (QI) project, comparing preopioid and postopioid prescribing practices. SETTING: Outpatient family medicine clinic at urban, safety-net teaching hospital. PATIENTS, PARTICIPANTS: A convenience sample of 16 family medicine physicians. INTERVENTIONS: A CRT was developed to allow physicians to give peer feedback to one another about their opioid prescribing practices as part of a 1-year QI project. We assessed the deidentified data gathered from the CRT. MAIN OUTCOME MEASURE(S): Primary study outcome measures were the amount of opioids prescribed at the end of the QI project compared to the time of initial chart review. We also describe overall prescribing practices. RESULTS: Ninety-nine patient charts from 14 different physicians were reviewed over 1 year. Sixty percent of patients had at least one violation of the clinic's controlled substance prescribing policy in the prior 6 months, and half of the violations were due to missed appointments with specialists to help manage pain. The mean dose of opioids decreased 2.6 mg morphine equivalent dose (MED)/day from time of chart review until the end of project, compared to a 6.9 mg MED/day increase that occurred from 12 months prior to chart review to the time of chart review (p=0.01). Fourteen patients (16 percent) of patients prescribed opioids were taken-off of opioids after the chart review. CONCLUSIONS: Use of a CRT in an urban primary care clinic provided helpful insight on prescribing practices and has promise to improve quality of opioid prescribing.