Learning curve for completely thoracoscopic anatomic sublobar resection.

BACKGROUND: Minimally invasive anatomic sublobar resection is increasingly being considered as an alternative to lobectomy in selected cases. However, this remains a technically challenging procedure and only 5 studies evaluating learning curves have been published to date. The aim of this study was to evaluate a single surgeon's learning curve for completely thoracoscopic anatomic sublobar resection. METHODS: A retrospective review was conducted of all thoracoscopic anatomic sublobar resections by one surgeon proficient in VATS lobectomy between January 2015 and January 2020. The primary outcome was operative time. Secondary outcomes were perioperative complications, duration of chest tube drainage and length of stay. RESULTS: There were 67 thoracoscopic anatomic sublobar resections performed in 66 patients. A Time-series plot and Cumulative Sum analysis of operative times showed a drop off after case 32, suggesting achievement of competency. After case 32, mean operative times were decreased (128.59±32.42 min. vs. 153.63±40.16 min, P=0.013) and there was a trend toward decreased blood loss (124.26±76.0 vs. 175.0±141.99 mL, P=0.073). A percentage 13.6% of patients had postoperative complications other than air leak and 88,9% of these were Clavien-Dindo class 1-2; postoperative complications were evenly distributed before and after case 32. Cumlulative Sum curves for the duration of chest tube drainage and length of stay did not show any significant change during the study period. CONCLUSIONS: This study suggests that for a surgeon proficient in VATS lobectomy, competency in completely thoracoscopic anatomic sublobar resection can be achieved after 32 cases and can be accomplished in a way that does not compromise perioperative outcomes.

An overview and critique of the use of cumulative sum methods with surgical learning curve data.

Cumulative sum (CUSUM) plots and methods have wide-ranging applications in healthcare. We review and discuss some issues related to the analysis of surgical learning curve (LC) data with a focus on three types of CUSUM statistical approaches. The underlying assumptions, benefits, and weaknesses of each approach are given. Our primary conclusion is that two types of CUSUM methods are useful in providing visual aids, but are subject to overinterpretation due to the lack of well-defined decision rules and performance metrics. The third type is based on plotting the CUSUM of the differences between observations and their average value. We show that this commonly applied retrospective method is frequently interpreted incorrectly and is thus unhelpful in the LC application. Curve-fitting methods are more suitable for meeting many of the goals associated with the study of surgical LCs.

The effect of temporal aggregation level in social network monitoring.

Social networks have become ubiquitous in modern society, which makes social network monitoring a research area of significant practical importance. Social network data consist of social interactions between pairs of individuals that are temporally aggregated over a certain interval of time, and the level of such temporal aggregation can have substantial impact on social network monitoring. There have been several studies on the effect of temporal aggregation in the process monitoring literature, but no studies on the effect of temporal aggregation in social network monitoring. We use the degree corrected stochastic block model (DCSBM) to simulate social networks and network anomalies and analyze these networks in the context of both count and binary network data. In conjunction with this model, we use the Priebe scan method as the monitoring method. We demonstrate that temporal aggregation at high levels leads to a considerable decrease in the ability to detect an anomaly within a specified time period. Moreover, converting social network communication data from counts to binary indicators can result in a significant loss of information, hindering detection performance. Aggregation at an appropriate level with count data, however, can amplify the anomalous signal generated by network anomalies and improve detection performance. Our results provide both insights on the practical effects of temporal aggregation and a framework for the study of other combinations of network models, surveillance methods, and types of anomalies.

Can long-term historical data from electronic medical records improve surveillance for epidemics of acute respiratory infections? A systematic evaluation.

BACKGROUND: As the deployment of electronic medical records (EMR) expands, so is the availability of long-term datasets that could serve to enhance public health surveillance. We hypothesized that EMR-based surveillance systems that incorporate seasonality and other long-term trends would discover outbreaks of acute respiratory infections (ARI) sooner than systems that only consider the recent past. METHODS: We simulated surveillance systems aimed at discovering modeled influenza outbreaks injected into backgrounds of patients with ARI. Backgrounds of daily case counts were either synthesized or obtained by applying one of three previously validated ARI case-detection algorithms to authentic EMR entries. From the time of outbreak injection, detection statistics were applied daily on paired background+injection and background-only time series. The relationship between the detection delay (the time from injection to the first alarm uniquely found in the background+injection data) and the false-alarm rate (FAR) was determined by systematically varying the statistical alarm threshold. We compared this relationship for outbreak detection methods that utilized either 7 days (early aberrancy reporting system (EARS)) or 2-4 years of past data (seasonal autoregressive integrated moving average (SARIMA) time series modeling). RESULTS: In otherwise identical surveillance systems, SARIMA detected epidemics sooner than EARS at any FAR below 10%. The algorithms used to detect single ARI cases impacted both the feasibility and marginal benefits of SARIMA modeling. Under plausible real-world conditions, SARIMA could reduce detection delay by 5-16 days. It also was more sensitive at detecting the summer wave of the 2009 influenza pandemic. CONCLUSION: Time series modeling of long-term historical EMR data can reduce the time it takes to discover epidemics of ARI. Realistic surveillance simulations may prove invaluable to optimize system design and tuning.

Improved implementation of the risk-adjusted Bernoulli CUSUM chart to monitor surgical outcome quality.

METHODOLOGY ISSUE: The traditional implementation of the risk-adjusted Bernoulli cumulative sum (CUSUM) chart for monitoring surgical outcome quality requires waiting a pre-specified period of time after surgery before incorporating patient outcome information. PROPOSED SOLUTION: We propose a simple but powerful implementation of the risk-adjusted Bernoulli CUSUM chart that incorporates outcome information as soon as it is available, rather than waiting a pre-specified period of time after surgery. EVALUATION: A simulation study is presented that compares the performance of the traditional implementation of the risk-adjusted Bernoulli CUSUM chart to our improved implementation. We show that incorporating patient outcome information as soon as it is available leads to quicker detection of process deterioration. ADVICE TO PRACTITIONERS: Deterioration of surgical performance could be detected much sooner using our proposed implementation, which could lead to the earlier identification of problems.

Dynamic probability control limits for risk-adjusted CUSUM charts based on multiresponses.

For a patient who has survived a surgery, there could be several levels of recovery. Thus, it is reasonable to consider more than two outcomes when monitoring surgical outcome quality. The risk-adjusted cumulative sum (CUSUM) chart based on multiresponses has been developed for monitoring a surgical process with three or more outcomes. However, there is a significant effect of varying risk distributions on the in-control performance of the chart when constant control limits are applied. To overcome this disadvantage, we apply the dynamic probability control limits to the risk-adjusted CUSUM charts for multiresponses. The simulation results demonstrate that the in-control performance of the charts with dynamic probability control limits can be controlled for different patient populations because these limits are determined for each specific sequence of patients. Thus, the use of dynamic probability control limits for risk-adjusted CUSUM charts based on multiresponses allows each chart to be designed for the corresponding patient sequence of a surgeon or a hospital and therefore does not require estimating or monitoring the patients' risk distribution. Copyright © 2017 John Wiley & Sons, Ltd.

Debate: what is the best method to monitor surgical performance?

BACKGROUND: There is considerable recent interest in the monitoring of individual surgeon or hospital surgical outcomes. If one aggregates data over time and assesses performance with a funnel plot, then the detection of any process deterioration or improvement could be delayed. The variable life adjusted display (VLAD) is widely used for monitoring on a case-by-case basis, but we show that use of the risk-adjusted Bernoulli cumulative sum (RA-CUSUM) chart leads to much better performance. DISCUSSION: We use simulation to illustrate that the RA-CUSUM chart has better performance than the VLAD in detecting changes in the rates of adverse events. We recommend the RA-CUSUM approach over the VLAD approach for monitoring surgical performance. If the VLAD is used, we recommend running the RA-CUSUM chart in the background to generate signals that the process performance has changed.

Dynamic probability control limits for risk-adjusted Bernoulli CUSUM charts.

The risk-adjusted Bernoulli cumulative sum (CUSUM) chart developed by Steiner et al. (2000) is an increasingly popular tool for monitoring clinical and surgical performance. In practice, however, the use of a fixed control limit for the chart leads to a quite variable in-control average run length performance for patient populations with different risk score distributions. To overcome this problem, we determine simulation-based dynamic probability control limits (DPCLs) patient-by-patient for the risk-adjusted Bernoulli CUSUM charts. By maintaining the probability of a false alarm at a constant level conditional on no false alarm for previous observations, our risk-adjusted CUSUM charts with DPCLs have consistent in-control performance at the desired level with approximately geometrically distributed run lengths. Our simulation results demonstrate that our method does not rely on any information or assumptions about the patients' risk distributions. The use of DPCLs for risk-adjusted Bernoulli CUSUM charts allows each chart to be designed for the corresponding particular sequence of patients for a surgeon or hospital.

The impact of varying patient populations on the in-control performance of the risk-adjusted CUSUM chart.

OBJECTIVE: This research is designed to examine the impact of varying patient population distributions on the in-control performance of the risk-adjusted Bernoulli CUSUM chart. DESIGN: The in-control performance of the chart is compared based on sampling the Parsonnet scores with replacement from five realistic subsets of a given distribution. SETTINGS: Five patient mixes with different Parsonnet score distributions are created from a real patient population. MAIN OUTCOME MEASURES: The outcome measures for this research are the in-control average run lengths (ARLs) given varying patient populations. RESULTS: Our simulation results show that the in-control ARLs of the risk-adjusted Bernoulli CUSUM chart with fixed control limits and a given risk-adjustment equation vary significantly for different patient population distributions, and the in-control ARLs decrease as the mean of the Parsonnet scores increases. CONCLUSIONS: The simulation results imply that the control limits should vary based on the particular patient population of interest in order to control the in-control performance of the risk-adjusted Bernoulli CUSUM method.

Control charts for accident frequency: a motivation for real-time occupational safety monitoring.

This paper quantitatively motivates the need for active monitoring of occupational safety incident data through the use of cumulative sum (CUSUM) control charts. The frequency of incidents within a subset of historical accident data is analysed. The performance of Poisson CUSUM and exponential CUSUM (time-between-events) charts is compared in an illustrative example to show that shorter periods of aggregation and time-between-events monitoring lead to more timely indications of increased accident frequency. An extension showing the anticipated performance of these charts with real-time data is given. Various adjustments to the monitoring system are also simulated to show that quick implementation of hazard controls can significantly impact safety performance. Decreases in the frequency of safety incidents as a result of implemented hazard controls can also be monitored.

Comparison of the early aberration reporting system (EARS) W2 methods to an adaptive threshold method.

In this paper we examine some of the methodologies implemented by the Centers for Disease Control and Prevention's (CDC) BioSense program. The program uses data from hospitals and public health departments to detect outbreaks using the Early Aberration Reporting System (EARS). The EARS method W2 allows one to monitor syndrome counts (W2count) from each source and the proportion of counts of a particular syndrome relative to the total number of visits (W2rate). We investigate the performance of these methods, which are designed using an empiric recurrence interval (RI), with simulated parametric data. Counts from the Poisson and negative binomial distributions are generated, and used to examine W2 properties. An adaptive threshold monitoring method is introduced based on fitting sample data to the above distributions, then converting the current value to a Z-score through a p-value. We compare the thresholds required to obtain given values of the RI for different sets of parameter values. We then simulate 1-week outbreaks in our data and calculate the proportion of times these methods correctly signal an outbreak using Shewhart and exponentially weighted moving average (EWMA) charts. Our results indicate that the adaptive threshold method gives more consistent statistical performance across different parameter sets and amounts of baseline historical data used for computing the statistics. For the sensitivity analysis, the EWMA chart is superior to its Shewhart counterpart in nearly all cases and the adaptive threshold methods tend to outperform the W2 methods. Copyright © 2011 John Wiley & Sons, Ltd.

Spatiotemporal surveillance methods in the presence of spatial correlation.

Health surveillance involves collecting public health data on chronic and infectious diseases to detect changes in disease incidence rates in order to improve public health. Timely detection of disease clusters is essential in prospective public health surveillance. Most existing health surveillance research is based on the assumption that observations from different regions are independent. This paper proposes a set of multivariate surveillance schemes generalized from well-known detection methods in multivariate statistical process control based on likelihood ratio tests. We use Monte Carlo simulations to compare these methods for health surveillance in the presence of spatial correlations. By taking advantage of correlations among regions,the proposed schemes are able to perform better than existing surveillance methods and provide faster and more accurate detection of outbreaks. An example of breast cancer in New Hampshire is presented to demonstrate the application of these methods when observations are spatially correlated counts.

Risk-adjusted monitoring of survival times.

We consider the monitoring of surgical outcomes, where each patient has a different risk of post-operative mortality due to risk factors that exist prior to the surgery. We propose a risk-adjusted (RA) survival time CUSUM chart (RAST CUSUM) for monitoring a continuous, time-to-event variable that may be right-censored. Risk adjustment is accomplished using accelerated failure time regression models. We compare the average run length performance of the RAST CUSUM chart with the RA Bernoulli CUSUM chart using data from cardiac surgeries to motivate the details of the comparison. The comparisons show that the RAST CUSUM chart is more efficient at detecting a sudden increase in the odds of mortality than the RA Bernoulli CUSUM chart, especially when the fraction of censored observations is relatively low or when a small increase in the odds of mortality occurs. We also discuss the impact of the amount of training data used to estimate chart parameters as well as the implementation of the RAST CUSUM chart during prospective monitoring.

Detecting a rate increase using a Bernoulli scan statistic.

Scan statistics are used in public health applications to detect increases in rates or clusters of disease indicated by an unusually large number of events. Most of the work has been for the retrospective case, in which a single set of historical data is to be analyzed. A modification of this retrospective scan statistic has been recommended for use when incidences of an event are recorded as they occur over time (prospectively) to determine whether the underlying incidence rate has increased, preferably as soon as possible after such an increase. In this paper, we investigate the properties of the scan statistic when used in prospective surveillance of the incidence rate under the assumption of independent Bernoulli observations. We show how to evaluate the expected number of Bernoulli observations needed to generate a signal that the incidence rate has increased. We compare the performance of the prospective scan statistic method with that obtained using the Bernoulli-based cumulative sum (CUSUM) technique. We show that the latter tends to be more effective in detecting sustained increases in the rate, but the scan method may be preferred in some applications due to its simplicity and can be used with relatively little loss of efficiency.

A comparison of surveillance methods for small incidence rates.

A number of methods have been proposed for detecting an increase in the incidence rate of a rare health event, such as a congenital malformation. Among these are the sets method, two modifications of the sets method, and the CUSUM method based on the Poisson distribution. We consider the situation where data are observed as a sequence of Bernoulli trials and propose the Bernoulli CUSUM chart as a desirable method for the surveillance of rare health events. We compared the performance of the sets method and its modifications with that of the Bernoulli CUSUM chart under a wide variety of circumstances. Chart design parameters were chosen to satisfy a minimax criteria. We used the steady-state average run length to measure chart performance instead of the average run length (ARL), which was used in nearly all previous comparisons involving the sets method or its modifications. Except in a very few instances, we found that the Bernoulli CUSUM chart has better steady-state ARL performance than the sets method and its modifications for the extensive number of cases considered. Thus, we recommend the use of the Bernoulli CUSUM chart to monitor small incidence rates and provide practical advice for its implementation.

Use of the local Knox statistic for the prospective monitoring of disease occurrences in space and time.

The detection of clusters of events occurring close together both temporally and spatially is important in finding outbreaks of disease within a geographic region. The Knox statistic is often used in epidemiology to test for space-time clustering retrospectively. For quicker detection of epidemics, prospective methods should be used in which observed events in space and time are assessed as they are recorded. The cumulative sum (CUSUM) surveillance method for monitoring the local Knox statistic tests for space-time clustering each time there is an incoming observation. We consider the design of this control chart by determining the in-control average run length (ARL) performance of the CUSUM chart for different space and time closeness thresholds as well as for different control limit values. We also explain the effect of population density and region shape on the in-control ARL and discuss other distributional issues that should be considered when implementing this method.