Decision models for distinguishing between clinically insignificant and significant tumors in prostate cancer biopsies: an application of Bayes' Theorem to reduce costs and improve outcomes.

Prostate cancer is the second leading cause of death from cancer, behind lung cancer, for men in the U. S, with nearly 30,000 deaths per year. A key problem is the difficulty in distinguishing, after biopsy, between significant cancers that should be treated immediately and clinically insignificant tumors that should be monitored by active surveillance. Prostate cancer has been over-treated; a recent European randomized screening trial shows overtreatment rates of 40%. Overtreatment of insignificant tumors reduces quality of life, while delayed treatment of significant cancers increases the incidence of metastatic disease and death. We develop a decision analysis approach based on simulation and probability modeling. For a given prostate volume and number of biopsy needles, our rule is to treat if total length of cancer in needle cores exceeds c, the cutoff value, with active surveillance otherwise, provided pathology is favorable. We determine the optimal cutoff value, c*. There are two misclassification costs: treating a minimal tumor and not treating a small or medium tumor (large tumors were never misclassified in our simulations). Bayes' Theorem is used to predict the probabilities of minimal, small, medium, and large cancers given the total length of cancer found in biopsy cores. A 20 needle biopsy in conjunction with our new decision analysis approach significantly reduces the expected loss associated with a patient in our target population about to undergo a biopsy. Longer needles reduce expected loss. Increasing the number of biopsy cores from the current norm of 10-12 to about 20, in conjunction with our new decision model, should substantially improve the ability to distinguish minimal from significant prostate cancer by minimizing the expected loss from over-treating minimal tumors and delaying treatment of significant cancers.

Importance of operating room case scheduling on analyses of observed reductions in surgical site infections from the purchase and installation of capital equipment in operating rooms.

BACKGROUND: We review the impact of the consequences of operating room (OR) management decision making on power analyses for observational studies of surgical site infections (SSIs) among patients receiving care in ORs with interventions versus without interventions involving physical changes to ORs. Examples include ventilation systems, bactericidal lighting, and physical alterations to ORs. METHODS: We performed a narrative review of operating room management and surgical site infection articles. We used 10-years of operating room data to estimate parameters for use in statistical power analyses. RESULTS: Creating pivot tables or monthly control charts of SSI per case by OR and comparing among ORs with or without intervention is not recommended. This approach has low power to detect a difference in SSI rates among the ORs with or without the intervention. The reason is that appropriate OR case scheduling decision making causes risk factors for SSI to differ among ORs, even when stratifying by surgical specialty. Such risk factors include case duration, urgency, and American Society of Anesthesiologists' Physical Status. Instead, analyze SSI controlling for the OR, where the patient had surgery, and matching patients using these variables is preferable. With α = 0.05, 600 cases per OR, 5 intervention ORs, and 5 or 1 control patients for each intervention patient, reasonable power (≅94% or 78%, respectively) can be achieved to detect reductions (3.6% to 2.4%) in the incidence of SSI between ORs with or without the intervention. CONCLUSIONS: By using this matched cohort design, the effect of the purchase and installation of capital equipment in ORs on SSI can be evaluated meaningfully.

Operating Room Anesthesia Subspecialization Is Not Associated With Significantly Greater Quality of Supervision of Anesthesia Residents and Nurse Anesthetists.

BACKGROUND: Supervision of anesthesia residents and nurse anesthetists is a major responsibility of faculty anesthesiologists. The quality of their supervision can be assessed quantitatively by the anesthesia residents and nurse anesthetists. Supervision scores are an independent measure of the contribution of the anesthesiologist to patient care. We evaluated the association between quality of supervision and level of specialization of anesthesiologists. METHODS: We used two 6-month periods, one with no feedback to anesthesiologists of the residents' and nurse anesthetists' evaluations, and the other with feedback. Supervision scores provided by residents and nurse anesthetists were considered separately. Sample sizes among the 4 combinations ranged from n = 51 to n = 62 University of Iowa faculty. For each supervising anesthesiologist and 6-month period, we calculated the proportion of anesthetic cases attributable to each anesthesia Current Procedural Terminology code. The sum of the square of the proportions, a measurement of diversity, is known as the Herfindahl index. The inverse of this index represents the effective number of common procedures. The diversity (degree of specialization) of each faculty anesthesiologist was measured attributing each case to: (1) the anesthesiologist who supervised for the longest total period of time, (2) the anesthesiologist who started the case, or (3) the anesthesiologist who started the case, limited to cases started during "regular hours" (defined as nonholiday Monday to Friday, 07:00 AM to 02:59 PM). Inferential analysis was performed using bivariate-weighted least-squares regression. RESULTS: The point estimates of all 12 slopes were in the direction of greater specialization of practice of the evaluated faculty anesthesiologist being associated with significantly lower supervision scores. Among supervision scores provided by nurse anesthetists, the association was statistically significant for the third of the 6-month periods under the first and second ways of attributing the cases (uncorrected P < .0001). However, the slopes of the relationships were all small (eg, 0.109 ± 0.025 [SE] units on the 4-point supervision scale for a change of 10 common procedures). Among supervision scores provided by anesthesia residents, the association was statistically significant during the first period for all 3 ways of attributing the case (uncorrected P < .0001). However, again, the slopes were small (eg, 0.127 ± 0.027 units for a change of 10 common procedures). CONCLUSIONS: Greater clinical specialization of faculty anesthesiologists was not associated with meaningful improvements in quality of clinical supervision.

Diversity and Similarity of Anesthesia Procedures in the United States During and Among Regular Work Hours, Evenings, and Weekends.

BACKGROUND: Anesthesiologists providing care during off hours (ie, weekends or holidays, or cases started during the evening or late afternoon) are more likely to care for patients at greater risk of sustaining major adverse events than when they work during regular hours (eg, Monday through Friday, from 7:00 AM to 2:59 PM). We consider the logical inconsistency of using subspecialty teams during regular hours but not during weekends or evenings. METHODS: We analyzed data from the Anesthesia Quality Institute's National Anesthesia Clinical Outcomes Registry (NACOR). Among the hospitals in the United States, we estimated the average number of common types of anesthesia procedures (ie, diversity measured as inverse of Herfindahl index), and the average difference in the number of common procedures between 2 off-hours periods (regular hours versus weekends, and regular hours versus evenings). We also used NACOR data to estimate the average similarity in the distributions of procedures between regular hours and weekends and between regular hours and evenings in US facilities. Results are reported as mean ± standard error of the mean among 399 facilities nationwide with weekend cases. RESULTS: The distributions of common procedures were moderately similar (ie, not large, <.8) between regular hours and evenings (similarity index .59 ± .01) and between regular hours and weekends (similarity index, .55 ± .02). For most facilities, the number of common procedures differed by <5 procedures between regular hours and evenings (74.4% of facilities, P < .0001) and between regular hours and weekends (64.7% of facilities, P < .0001). The average number of common procedures was 13.59 ± .12 for regular hours, 13.12 ± .13 for evenings, and 9.43 ± .13 for weekends. The pairwise differences by facility were .13 ± .07 procedures (P = .090) between regular hours and evenings and 3.37 ± .12 procedures (P < .0001) between regular hours and weekends. In contrast, the differences were -5.18 ± .12 and 7.59 ± .13, respectively, when calculated using nationally pooled data. This was because the numbers of common procedures were 32.23 ± .05, 37.41 ± .11, and 24.64 ± .12 for regular hours, evenings, and weekends, respectively (ie, >2x the number of common procedures calculated by facility). CONCLUSIONS: The numbers of procedures commonly performed at most facilities are fewer in number than those that are commonly performed nationally. Thus, decisions on anesthesia specialization should be based on quantitative analysis of local data rather than national recommendations using pooled data. By facility, the number of different procedures that take place during regular hours and off hours (diversity) is essentially the same, but there is only moderate similarity in the procedures performed. Thus, at many facilities, anesthesiologists who work principally within a single specialty during regular work hours will likely not have substantial contemporary experience with many procedures performed during off hours.

Bernoulli Cumulative Sum (CUSUM) control charts for monitoring of anesthesiologists' performance in supervising anesthesia residents and nurse anesthetists.

We describe our experiences in using Bernoulli cumulative sum (CUSUM) control charts for monitoring clinician performance. The supervision provided by each anesthesiologist is evaluated daily by the Certified Registered Nurse Anesthetists (CRNAs) and/or anesthesia residents with whom they work. Each of 9 items is evaluated (1 = never, 2 = rarely, 3 = frequently, 4 = always). The score is the mean of the 9 responses. Choosing thresholds for low scores is straightforward, <2.0 for CRNAs and <3.0 for residents. Bernoulli CUSUM detection of low scores was within 50 ± 14 (median ± quartile deviation) days rather than 182 days without use of CUSUM. The true positive detection of anesthesiologists with incidences of low scores greater than the chosen "out-of-control" rate was 14 of 14. The false-positive detection rate was 0 of 29. This CUSUM performance exceeded that of Shewhart individual control charts, for which the smallest threshold sufficiently large to detect 14 of 14 true positives had false-positive detection of 16 of 29 anesthesiologists. The Bernoulli CUSUM assumes that scores are known right away, which is untrue. However, CUSUM performance was insensitive to this assumption. The Bernoulli CUSUM assumes statistical independence of scores, which also is untrue. For example, when an evaluation of an anesthesiologist 1 day by a CRNA had a low score, there was an increased chance that another CRNA working in a different operating room on the same day would also give that same anesthesiologist a low score (P < 0.0001). This correlation among scores does affect the Bernoulli CUSUM, such that detection is more likely. This is an advantage for our continual process improvement application since it flags individuals for further evaluation by managers while maintaining confidentiality of raters.

Influence of provider type (nurse anesthetist or resident physician), staff assignments, and other covariates on daily evaluations of anesthesiologists' quality of supervision.

BACKGROUND: At many U.S. healthcare facilities, supervision of anesthesiology residents and/or Certified Registered Nurse Anesthetists (CRNAs) is a major daily responsibility of anesthesiologists. Our department implemented a daily process by which the supervision provided by each anesthesiologist working in operating rooms was evaluated by the anesthesiology resident(s) and CRNA(s) with whom they worked the previous day. METHODS: Requests for evaluation were sent daily via e-mail to each resident and CRNA after working in an operating room. Supervision scores were analyzed after 6 months, and aligned with the cases' American Society of Anesthesiologists Relative Value Guide units. RESULTS: (1) Mean monthly evaluation completion rates exceeded 85% (residents P = 0.0001, CRNAs P = 0.0005). (2) Pairwise by anesthesiologist, residents and CRNAs mean supervision scores were correlated (P < 0.0001), but residents assigned greater scores than did CRNAs (P < 0.0001). The pairwise differences between residents and CRNAs were heterogeneous among anesthesiologists (P < 0.0001). (3) Anesthesiologist supervision scores provided by residents were: (a) greater when a resident had more units of work that day with the rated anesthesiologist (P < 0.0001), and (b) less when the anesthesiologist had more units of work that same day with other providers (P < 0.0001). However, the relationships were unimportantly small, Kendall τb = +0.083 ± 0.014 (SE) and τb = -0.057 ± 0.014, respectively. The correlations were even less among the CRNAs, τb = -0.029 ± 0.013 and τb = -0.004 ± 0.012, respectively. (4) There also was unimportantly small association between a resident's or CRNA's mean score for an anesthesiologist and the number of days worked together (τb = -0.069 ± 0.023 and τb = +0.038 ± 0.020, respectively). CONCLUSIONS: Although the attributes that residents and CRNA perceive as constituting "supervision" significantly share commonalities, supervision scores should be analyzed separately for residents and CRNAs. Although mean supervision scores differ markedly among anesthesiologists, supervision scores are influenced negligibly by staff assignments (e.g., how busy the anesthesiologist is with other operating rooms).

Estimating surgical case durations and making comparisons among facilities: identifying facilities with lower anesthesia professional fees.

Consumer-driven health care relies on transparency in cost estimates for surgery, including anesthesia professional fees. Using systematic narrative review, we show that providing anesthesia costs requires that each facility (anesthesia group) estimate statistics, reasonably the mean and the 90% upper prediction limit of case durations by procedure. The prediction limits need to be calculated, for many procedures, using Bayesian methods based on the log-normal distribution. Insurers and/or governments lack scheduled durations and procedures and cannot practically infer these estimates because of the large heterogeneities among facilities in the means and coefficients of variation of durations. Consequently, the insurance industry cannot provide the cost information accurately from public and private databases. Instead, the role of insurers and/or governments can be to identify facilities with significantly briefer durations (costs to the patient) than average. Such comparisons of durations among facilities should be performed with correction for the effects of the multiple comparisons. Our review also has direct implications to the potentially more important issue of how to study the association between anesthetic durations and patient morbidity and mortality. When pooling duration data among facilities, both the large heterogeneity in the means and coefficients of variation of durations among facilities need to be considered (e.g., using "multilevel" or "hierarchical" models).

Monitoring changes in individual surgeon's workloads using anesthesia data.

PURPOSE: We investigated whether changes in the number of cases performed by surgeons can be used as an appropriate surrogate for anesthesia departments' billed units. METHODS: We used both number of cases performed and the American Society of Anesthesiologists' Relative Value Guide™ (ASA RVG) units to assess all operating room anesthetics of an anesthesia group for two sets of 13 four-week periods. The units correspond to Canadian basic units and time units. RESULTS: Although the number of ASA RVG units is an economically important variable that quantifies perioperative workload, the number of cases is a suitable surrogate for ASA RVG units when used to monitor individual surgeons. The pooled mean Pearson correlation coefficient between the two variables was r = 0.95, with 95% confidence interval 0.94 to 0.96. In addition, there were essentially none to very weak pairwise correlations among surgeons. CONCLUSIONS: Informal hospital analyses of relative changes in a surgeon's caseload over one year using anesthesia workload data or anesthesia billing data will generally give equivalent results. The principal importance of our findings is that they can be used by anesthesiologists, specifically department heads, in their role as part of operating room committees. Such committees institute plans to revise the caseload of one or a few surgeons, and they then evaluate the results of those plans. The findings of this study are applicable to all anesthesia groups and may be especially valuable to the heads of anesthesiology departments who do not have the data to repeat our analyses.

Risk factors for serious injury in Finnish agriculture.

BACKGROUND: Previous studies indicate 20% of injuries represent 80% of injury costs in agriculture. To help prevent the most costly injuries, we aimed to identify characteristics and risk factors associated with serious injuries. METHODS: We analyzed insurance records of 93,550 self-employed Finnish farmers. We ranked injury causes by claim cost and used multiple logistic regressions to identify risk factors for (any) injury and serious injury (injuries exceeding claim costs of 2000 euros). RESULTS: A total of 5,507 compensated injuries occurred in 2002 (rate 5.9/100 person-years), and 1,167 or 21% of them (rate 1.25/100 person-years) were serious. The causes/sources resulting in highest average claim costs were motor vehicles; stairs, scaffoldings, and ladders; trailers and wagons; floors, walkways, and steps; other structures and obstacles; augers, mills, and grain handling equipment; horses; combines and harvesting equipment; tractor steps; and uneven and slippery terrain. Older age, male gender, higher income level, greater field size, residing on the farm, Finnish language (vs. Swedish), occupational health service (OHS) membership, and animal production were risk factors for injury. The risk factors for serious injury were similar; however, the effects of age, income level, and the raising of horses were more prominent. Language, residence, ownership status, and OHS membership were not risk factors for serious injury. CONCLUSIONS: Cost-effective prevention efforts should address the following risk factors: older age, male gender, larger income and operation size, livestock production (particularly dairy, swine, and horses), motor vehicle incidents, falls from elevation, and slips, trips and falls.

Identification of systematic underestimation (bias) of case durations during case scheduling would not markedly reduce overutilized operating room time.

STUDY OBJECTIVE: If a case has a high probability of taking longer than scheduled, then increasing the case's scheduled duration could reduce over-utilized operating room (OR) time. We studied surgeons' and schedulers' case scheduling behavior to evaluate whether such a strategy would be useful. DESIGN: Observational study. SETTING: University hospital. MEASUREMENTS: The probability of each of 66,561 cases taking longer than scheduled was estimated with an accuracy to within 1-2%. MAIN RESULTS: Overall underestimation by surgeons and schedulers was 22 minutes for each 8 hours of used operating room (OR) time. If a 90% or 95% chance of taking longer than scheduled were required to conclude that a case's duration was deliberately underestimated, and if such cases' scheduled durations were changed, overall underestimation would be reduced by only 0.2 or 0.9 minutes per 8 hours of used OR time because only 0.1% or 0.6% of used OR time met that criterion. In contrast, underestimation would be reduced by 20 minutes if the cases identified were those with only a 50% to 60% chance of taking longer than scheduled because they accounted for more than 40% of OR time. Persistent underestimation of cases' durations was caused not by poor decisions for a few outlier cases, but instead by slight underestimation for many cases. Surgeons' and schedulers' behavior that fit cases into staffed (allocated) OR time was to underestimate slightly the duration of many cases. CONCLUSION: The impact of inaccurate, scheduled case duration on staffing costs and unpredictable work hours can be reduced by allocating appropriate total hours of OR time (ie, staffing) for the cases that will get done, regardless of the inaccuracy of the scheduled durations of those cases.

Effects of premium discount on workers' compensation claims in agriculture in Finland.

BACKGROUND: The objective of this study was to measure changes in injury claim rates after a premium discount program was implemented in the Finnish farmers' workers' compensation insurance. We focused on measures that could indicate whether the changes occurred in the true underlying injury rate, or only in claims reporting. METHODS: Monthly injury claim rates were constructed at seven disability duration levels from January 1990 to December 2003. We conducted interrupted time series analyses to measure changes in the injury claim rates after the premium discount was implemented on July 1, 1997. Three additional policy change indicators were included in the analyses. RESULTS: The overall injury claim rate decreased 10.2%. Decreases occurred at four severity levels (measured by compensated disability days): 0 days (16.3%), 1-6 days (14.1%), 7-13 days (19.5%), and 14-29 days (8.4%). No changes were observed at higher severity levels. Minor injuries had a seasonal pattern with higher rates in summer months while severe injuries did not have a seasonal pattern. CONCLUSIONS: The premium discount decreased the overall claim rate. Decreases were observed in all categories up to 29 disability days. This pattern suggests that under-reporting contributes to the decrease but may not be the only factor. The value of the premium discount is lower than the value of a lost-time claim, so there was no financial reason to under-report lost-time injuries. Under-reporting would be expected to be greatest in the 0 day category, but that was not the case. These observations suggest that in addition to under-reporting, the premium discount may also have some preventive effect.

Quantifying effect of a hospital's caseload for a surgical specialty on that of another hospital using multi-attribute market segments.

Inpatient and outpatient data were used to create market segments consisting of hierarchical combinations of surgical procedure, then type of payer, and then location of patients' residences. The competitive effect of one hospital's caseload for a given surgical specialty on the caseload of another hospital was determined from the numbers of patients in each segment. Earlier methods for estimating surgical competition that ignored market segments over-estimated the competitive effects of one hospital on another. Thus, results differed from those obtained previously for all types of hospital admissions. When actual market segments with homogeneous groups of patients are used, competitive effects of hospitals in the same market area are far less than expected.

Estimating the incidence of prolonged turnover times and delays by time of day.

BACKGROUND: Prolonged turnover times cause frustration and can thereby reduce professional satisfaction and the workload surgeons bring to a hospital. METHODS: The authors analyzed 1 yr of operating room information system data from two academic, tertiary hospitals and Monte-Carlo simulations of a 15-operating room hospital surgical suite. RESULTS: Confidence interval widths for the mean turnover times at the hospitals were negligible when compared with the variation in sample mean turnover times among 31 hospitals. The authors developed a statistical method to estimate the proportion of all turnovers that were prolonged (> 15 min beyond mean) and that occurred during specified hours of the day. Confidence intervals for the proportions corrected for the effect of multiple comparisons. Statistical assumptions were satisfied at the two studied hospitals. The confidence intervals achieved family-wise type I error rates accurate to within 0.5% when applied to between five and nineteen 4-week periods of data. The diurnal pattern in the proportions of all turnovers that were prolonged provided different, more managerially relevant information than the time course throughout the day in the percentage of turnovers at each hour that were prolonged. CONCLUSIONS: Benchmarking sample mean turnover times among hospitals, without the use of confidence intervals, can be valid and useful. The authors successfully developed and validated a statistical method to estimate the percentage of turnover times at a surgical suite that are prolonged and occur at specified times of the day. Managers can target their quality improvement efforts on times of the day with the largest percentages of prolonged turnovers.

Tactical decision making for selective expansion of operating room resources incorporating financial criteria and uncertainty in subspecialties' future workloads.

We considered the allocation of operating room (OR) time at facilities where the strategic decision had been made to increase the number of ORs. Allocation occurs in two stages: a long-term tactical stage followed by short-term operational stage. Tactical decisions, approximately 1 yr in advance, determine what specialized equipment and expertise will be needed. Tactical decisions are based on estimates of future OR workload for each subspecialty or surgeon. We show that groups of surgeons can be excluded from consideration at this tactical stage (e.g., surgeons who need intensive care beds or those with below average contribution margins per OR hour). Lower and upper limits are estimated for the future demand of OR time by the remaining surgeons. Thus, initial OR allocations can be accomplished with only partial information on future OR workload. Once the new ORs open, operational decision-making based on OR efficiency is used to fill the OR time and adjust staffing. Surgeons who were not allocated additional time at the tactical stage are provided increased OR time through operational adjustments based on their actual workload. In a case study from a tertiary hospital, future demand estimates were needed for only 15% of surgeons, illustrating the practicality of these methods for use in tactical OR allocation decisions.

Managing risk and expected financial return from selective expansion of operating room capacity: mean-variance analysis of a hospital's portfolio of surgeons.

UNLABELLED: Surgeons using the same amount of operating room (OR) time differ in their achieved hospital contribution margins (revenue minus variable costs) by >1000%. Thus, to improve the financial return from perioperative facilities, OR strategic decisions should selectively focus additional OR capacity and capital purchasing on a few surgeons or subspecialties. These decisions use estimates of each surgeon's and/or subspecialty's contribution margin per OR hour. The estimates are subject to uncertainty (e.g., from outliers). We account for the uncertainties by using mean-variance portfolio analysis (i.e., quadratic programming). This method characterizes the problem of selectively expanding OR capacity based on the expected financial return and risk of different portfolios of surgeons. The assessment reveals whether the choices, of which surgeons have their OR capacity expanded, are sensitive to the uncertainties in the surgeons' contribution margins per OR hour. Thus, mean-variance analysis reduces the chance of making strategic decisions based on spurious information. We also assess the financial benefit of using mean-variance portfolio analysis when the planned expansion of OR capacity is well diversified over at least several surgeons or subspecialties. Our results show that, in such circumstances, there may be little benefit from further changing the portfolio to reduce its financial risk. IMPLICATIONS: Surgeon and subspecialty specific hospital financial data are uncertain, a fact that should be taken into account when making decisions about expanding operating room capacity. We show that mean-variance portfolio analysis can incorporate this uncertainty, thereby guiding operating room management decision-making and reducing the chance of a strategic decision being made based on spurious information.