Forecasting emergency department crowding: an external, multicenter evaluation.

STUDY OBJECTIVE: We apply a previously described tool to forecast emergency department (ED) crowding at multiple institutions and assess its generalizability for predicting the near-future waiting count, occupancy level, and boarding count. METHODS: The ForecastED tool was validated with historical data from 5 institutions external to the development site. A sliding-window design separated the data for parameter estimation and forecast validation. Observations were sampled at consecutive 10-minute intervals during 12 months (n=52,560) at 4 sites and 10 months (n=44,064) at the fifth. Three outcome measures-the waiting count, occupancy level, and boarding count-were forecast 2, 4, 6, and 8 hours beyond each observation, and forecasts were compared with observed data at corresponding times. The reliability and calibration were measured following previously described methods. After linear calibration, the forecasting accuracy was measured with the median absolute error. RESULTS: The tool was successfully used for 5 different sites. Its forecasts were more reliable, better calibrated, and more accurate at 2 hours than at 8 hours. The reliability and calibration of the tool were similar between the original development site and external sites; the boarding count was an exception, which was less reliable at 4 of 5 sites. Some variability in accuracy existed among institutions; when forecasting 4 hours into the future, the median absolute error of the waiting count ranged between 0.6 and 3.1 patients, the median absolute error of the occupancy level ranged between 9.0% and 14.5% of beds, and the median absolute error of the boarding count ranged between 0.9 and 2.8 patients. CONCLUSION: The ForecastED tool generated potentially useful forecasts of input and throughput measures of ED crowding at 5 external sites, without modifying the underlying assumptions. Noting the limitation that this was not a real-time validation, ongoing research will focus on integrating the tool with ED information systems.

Forecasting emergency department crowding: a prospective, real-time evaluation.

OBJECTIVE: Emergency department crowding threatens quality and access to health care, and a method of accurately forecasting near-future crowding should enable novel ways to alleviate the problem. The authors sought to implement and validate the previously developed ForecastED discrete event simulation for real-time forecasting of emergency department crowding. DESIGN AND MEASUREMENTS: The authors conducted a prospective observational study during a three-month period (5/1/07-8/1/07) in the adult emergency department of a tertiary care medical center. The authors connected the forecasting tool to existing information systems to obtain real-time forecasts of operational data, updated every 10 minutes. The outcome measures included the emergency department waiting count, waiting time, occupancy level, length of stay, boarding count, boarding time, and ambulance diversion; each forecast 2, 4, 6, and 8 hours into the future. RESULTS: The authors obtained crowding forecasts at 13,239 10-minute intervals, out of 13,248 possible (99.9%). The R(2) values for predicting operational data 8 hours into the future, with 95% confidence intervals, were 0.27 (0.26, 0.29) for waiting count, 0.11 (0.10, 0.12) for waiting time, 0.57 (0.55, 0.58) for occupancy level, 0.69 (0.68, 0.70) for length of stay, 0.61 (0.59, 0.62) for boarding count, and 0.53 (0.51, 0.54) for boarding time. The area under the receiver operating characteristic curve for predicting ambulance diversion 8 hours into the future, with 95% confidence intervals, was 0.85 (0.84, 0.86). CONCLUSIONS: The ForecastED tool provides accurate forecasts of several input, throughput, and output measures of crowding up to 8 hours into the future. The real-time deployment of the system should be feasible at other emergency departments that have six patient-level variables available through information systems.

Forecasting emergency department crowding: a discrete event simulation.

STUDY OBJECTIVE: To develop a discrete event simulation of emergency department (ED) patient flow for the purpose of forecasting near-future operating conditions and to validate the forecasts with several measures of ED crowding. METHODS: We developed a discrete event simulation of patient flow with evidence from the literature. Development was purely theoretical, whereas validation involved patient data from an academic ED. The model inputs and outputs, respectively, are 6-variable descriptions of every present and future patient in the ED. We validated the model by using a sliding-window design, ensuring separation of fitting and validation data in time series. We sampled consecutive 10-minute observations during 2006 (n=52,560). The outcome measures--all forecast 2, 4, 6, and 8 hours into the future from each observation--were the waiting count, waiting time, occupancy level, length of stay, boarding count, boarding time, and ambulance diversion. Forecasting performance was assessed with Pearson's correlation, residual summary statistics, and area under the receiver operating characteristic curve. RESULTS: The correlations between crowding forecasts and actual outcomes started high and decreased gradually up to 8 hours into the future (lowest Pearson's r for waiting count=0.56; waiting time=0.49; occupancy level=0.78; length of stay=0.86; boarding count=0.79; boarding time=0.80). The residual means were unbiased for all outcomes except the boarding time. The discriminatory power for ambulance diversion remained consistently high up to 8 hours into the future (lowest area under the receiver operating characteristic curve=0.86). CONCLUSION: By modeling patient flow, rather than operational summary variables, our simulation forecasts several measures of near-future ED crowding, with various degrees of good performance.