Prediction of multiclass surgical outcomes in glaucoma using multimodal deep learning based on free-text operative notes and structured EHR data.

OBJECTIVE: Surgical outcome prediction is challenging but necessary for postoperative management. Current machine learning models utilize pre- and post-op data, excluding intraoperative information in surgical notes. Current models also usually predict binary outcomes even when surgeries have multiple outcomes that require different postoperative management. This study addresses these gaps by incorporating intraoperative information into multimodal models for multiclass glaucoma surgery outcome prediction. MATERIALS AND METHODS: We developed and evaluated multimodal deep learning models for multiclass glaucoma trabeculectomy surgery outcomes using both structured EHR data and free-text operative notes. We compare those to baseline models that use structured EHR data exclusively, or neural network models that leverage only operative notes. RESULTS: The multimodal neural network had the highest performance with a macro AUROC of 0.750 and F1 score of 0.583. It outperformed the baseline machine learning model with structured EHR data alone (macro AUROC of 0.712 and F1 score of 0.486). Additionally, the multimodal model achieved the highest recall (0.692) for hypotony surgical failure, while the surgical success group had the highest precision (0.884) and F1 score (0.775). DISCUSSION: This study shows that operative notes are an important source of predictive information. The multimodal predictive model combining perioperative notes and structured pre- and post-op EHR data outperformed other models. Multiclass surgical outcome prediction can provide valuable insights for clinical decision-making. CONCLUSIONS: Our results show the potential of deep learning models to enhance clinical decision-making for postoperative management. They can be applied to other specialties to improve surgical outcome predictions.

Pediatric Ophthalmology Provider and Staff Attitudes and Patient Satisfaction in Telehealth Implementation During COVID-19.

Purpose:Describe a comprehensive overview of a telehealth implementation process that highlights attitudes and satisfaction scores toward telehealth from patients, providers, and staff in an academic pediatric ophthalmology practice during the early months of the coronavirus disease 2019 (COVID-19) pandemic.Methods:The electronic medical record data for telehealth and in-person visits, as well as a patient experience survey in pediatric ophthalmology were retrospectively reviewed for March 1 to July 31, 2020 and March 1 to July 31, 2019. Patient experience survey results were retrospectively reviewed. All current providers and staff were invited to participate in an anonymous and voluntary survey focused on attitudes at the time of telehealth implementation.Results:During March 1 to July 31, 2020, there was significant increase in telehealth visits (n = 1,006) compared with the same period in 2019 (n = 22). Evaluation and management (E & M) codes (n = 527) were the most commonly used billing codes, and strabismus, nystagmus, and irregular eye movement (n = 496) were the most common telehealth primary diagnoses. The telehealth attitudes survey showed more positive responses from providers than staff. The patient experience survey showed more favorable scores for telehealth visits compared with clinic visits. However, only about 50% of the respondents were satisfied with the technology in terms of ease and quality of connection during their telehealth visits.Conclusions:Telehealth was a satisfactory alternative to clinic visits in our academic pediatric ophthalmology practice during the early phase of the COVID-19 pandemic. Providers and staff had largely positive attitudes toward telehealth; however, future efforts should include strategies to increase staff buy in. Patients had high satisfaction scores with telehealth visits despite connection challenges.

Applications of Artificial Intelligence to Electronic Health Record Data in Ophthalmology.

Widespread adoption of electronic health records (EHRs) has resulted in the collection of massive amounts of clinical data. In ophthalmology in particular, the volume range of data captured in EHR systems has been growing rapidly. Yet making effective secondary use of this EHR data for improving patient care and facilitating clinical decision-making has remained challenging due to the complexity and heterogeneity of these data. Artificial intelligence (AI) techniques present a promising way to analyze these multimodal data sets. While AI techniques have been extensively applied to imaging data, there are a limited number of studies employing AI techniques with clinical data from the EHR. The objective of this review is to provide an overview of different AI methods applied to EHR data in the field of ophthalmology. This literature review highlights that the secondary use of EHR data has focused on glaucoma, diabetic retinopathy, age-related macular degeneration, and cataracts with the use of AI techniques. These techniques have been used to improve ocular disease diagnosis, risk assessment, and progression prediction. Techniques such as supervised machine learning, deep learning, and natural language processing were most commonly used in the articles reviewed.

Promoting Quality Face-to-Face Communication during Ophthalmology Encounters in the Electronic Health Record Era.

OBJECTIVE: To evaluate informatics-enabled quality improvement (QI) strategies for promoting time spent on face-to-face communication between ophthalmologists and patients. METHODS: This prospective study involved deploying QI strategies during implementation of an enterprise-wide vendor electronic health record (EHR) in an outpatient academic ophthalmology department. Strategies included developing single sign-on capabilities, activating mobile- and tablet-based applications, EHR personalization training, creating novel workflows for team-based orders, and promoting problem-based charting to reduce documentation burden. Timing data were collected during 648 outpatient encounters. Outcomes included total time spent by the attending ophthalmologist on the patient, time spent on documentation, time spent on examination, and time spent talking with the patient. Metrics related to documentation efficiency, use of personalization features, use of team-based orders, and note length were also measured from the EHR efficiency portal and compared with averages for ophthalmologists nationwide using the same EHR. RESULTS: Time spent on exclusive face-to-face communication with patients initially decreased with EHR implementation (2.9 to 2.3 minutes, p = 0.005) but returned to the paper baseline by 6 months (2.8 minutes, p = 0.99). Observed participants outperformed national averages of ophthalmologists using the same vendor system on documentation time per appointment, number of customized note templates, number of customized order lists, utilization of team-based orders, note length, and time spent after-hours on EHR use. CONCLUSION: Informatics-enabled QI interventions can promote patient-centeredness and face-to-face communication in high-volume outpatient ophthalmology encounters. By employing an array of interventions, time spent exclusively talking with the patient returned to levels equivalent to paper charts by 6 months after EHR implementation. This was achieved without requiring EHR redesign, use of scribes, or excessive after-hours work. Documentation efficiency can be achieved using interventions promoting personalization and team-based workflows. Given their efficacy in preserving face-to-face physician-patient interactions, these strategies may help alleviate risk of physician burnout.

Application of Machine Learning to Predict Patient No-Shows in an Academic Pediatric Ophthalmology Clinic.

Patient "no-shows" are missed appointments resulting in clinical inefficiencies, revenue loss, and discontinuity of care. Using secondary electronic health record (EHR) data, we used machine learning to predict patient no-shows in follow-up and new patient visits in pediatric ophthalmology and to evaluate features for importance. The best model, XGBoost, had an area under the receiver operating characteristics curve (AUC) score of 0.90 for predicting no-shows in follow-up visits. The key findings from this study are: (1) secondary use of EHR data can be used to build datasets for predictive modeling and successfully predict patient no-shows in pediatric ophthalmology, (2) models predicting no-shows for follow-up visits are more accurate than those for new patient visits, and (3) the performance of predictive models is more robust in predicting no-shows compared to individual important features. We hope these models will be used for more effective interventions to mitigate the impact ofpatient no-shows.

Electronic Health Records in Ophthalmology: Source and Method of Documentation.

PURPOSE: This study analyzed and quantified the sources of electronic health record (EHR) text documentation in ophthalmology progress notes. DESIGN: EHR documentation review and analysis. METHODS: Setting: a single academic ophthalmology department. STUDY POPULATION: a cohort study conducted between November 1, 2016, and December 31, 2018, using secondary EHR data and a follow-up manual review of a random samples. The cohort study included 123,274 progress notes documented by 42 attending providers. These notes were for patients with the 5 most common primary International Statistical Classification of Diseases and Related Health Problems, version 10, parent codes for each provider. For the manual review, 120 notes from 8 providers were randomly sampled. Main outcome measurements were characters or number of words in each note categorized by attribution source, author type, and time of creation. RESULTS: Imported text entries made up the majority of text in new and return patients, 2,978 characters (77%) and 3,612 characters (91%). Support staff members authored substantial portions of notes; 3,024 characters (68%) of new patient notes, 3,953 characters (83%) of return patient notes. Finally, providers completed large amounts of documentation after clinical visits: 135 words (35%) of new patient notes, 102 words (27%) of return patient notes. CONCLUSIONS: EHR documentation consists largely of imported text, is often authored by support staff, and is often written after the end of a visit. These findings raise questions about documentation accuracy and utility and may have implications for quality of care and patient-provider relationships.

Time Requirements of Paper-Based Clinical Workflows and After-Hours Documentation in a Multispecialty Academic Ophthalmology Practice.

PURPOSE: To assess time requirements for patient encounters and estimate after-hours demands of paper-based clinical workflows in ophthalmology. DESIGN: Time-and-motion study with a structured survey. METHODS: This study was conducted in a single academic ophthalmology department. A convenience sample consisted of 7 attending ophthalmologists from 6 subspecialties observed during 414 patient encounters for the time-motion analysis and 12 attending ophthalmologists for the survey. Outcome measurements consisted of total time spent by attending ophthalmologists per patient and time spent on documentation, examination, and talking with patients. The survey assessed time requirements of documentation-related activities performed outside of scheduled clinic hours. RESULTS: Among the 7 attending ophthalmologists observed (6 men and 1 woman), mean ± SD age 43.9 ± 7.1 years, during encounters with 414 patients (57.8 ± 24.6 years of age), total time spent per patient was 8.1 ± 4.8 minutes, with 2.8 ± 1.4 minutes (38%) for documentation, 1.2 ± 0.9 minutes (17%) for examination, and 3.3 ± 3.1 minutes (37%) for talking with patients. New patient evaluations required significantly more time than routine follow-up visits and postoperative visits. Higher clinical volumes were associated with less time per patient. Survey results indicated that paper-based documentation was associated with minimal after-hours work on weeknights and weekends. CONCLUSIONS: Paper-based documentation takes up a substantial portion of the total time spent for patient care in outpatient ophthalmology clinics but is associated with minimal after-hours work. Understanding paper-based clinical workflows may help inform targeted strategies for improving electronic health record use in ophthalmology.

Changes in Electronic Health Record Use Time and Documentation over the Course of a Decade.

PURPOSE: With the current wide adoption of electronic health records (EHRs) by ophthalmologists, there are widespread concerns about the amount of time spent using the EHR. The goal of this study was to examine how the amount of time spent using EHRs as well as related documentation behaviors changed 1 decade after EHR adoption. DESIGN: Single-center cohort study. PARTICIPANTS: Six hundred eighty-five thousand three hundred sixty-one office visits with 70 ophthalmology providers. METHODS: We calculated time spent using the EHR associated with each individual office visit using EHR audit logs and determined chart closure times and progress note length from secondary EHR data. We tracked and modeled how these metrics changed from 2006 to 2016 with linear mixed models. MAIN OUTCOME MEASURES: Minutes spent using the EHR associated with an office visit, chart closure time in hours from the office visit check-in time, and progress note length in characters. RESULTS: Median EHR time per office visit in 2006 was 4.2 minutes (interquartile range [IQR], 3.5 minutes), and increased to 6.4 minutes (IQR, 4.5 minutes) in 2016. Median chart closure time was 2.8 hours (IQR, 21.3 hours) in 2006 and decreased to 2.3 hours (IQR, 18.5 hours) in 2016. In 2006, median note length was 1530 characters (IQR, 1435 characters) and increased to 3838 characters (IQR, 2668.3 characters) in 2016. Linear mixed models found EHR time per office visit was 31.9±0.2% (P < 0.001) greater from 2014 through 2016 than from 2006 through 2010, chart closure time was 6.7±0.3 hours (P < 0.001) shorter from 2014 through 2016 versus 2006 through 2010, and note length was 1807.4±6.5 characters (P < 0.001) longer from 2014 through 2016 versus 2006 through 2010. CONCLUSIONS: After 1 decade of use, providers spend more time using the EHR for an office visit, generate longer notes, and close the chart faster. These changes are likely to represent increased time and documentation pressure for providers. Electronic health record redesign and new documentation regulations may help to address these issues.

Predicting Wait Times in Pediatric Ophthalmology Outpatient Clinic Using Machine Learning.

Patient perceptions of wait time during outpatient office visits can affect patient satisfaction. Providing accurate information about wait times could improve patients' satisfaction by reducing uncertainty. However, these are rarely known about efficient ways to predict wait time in the clinic. Supervised machine learning algorithms is a powerful tool for predictive modeling with large and complicated data sets. In this study, we tested machine learning models to predict wait times based on secondary EHR data in pediatric ophthalmology outpatient clinic. We compared several machine-learning algorithms, including random forest, elastic net, gradient boosting machine, support vector machine, and multiple linear regressions to find the most accurate model for prediction. The importance of the predictors was also identified via machine learning models. In the future, these models have the potential to combine with real-time EHR data to provide real time accurate estimates of patient wait time outpatient clinics.

Data-Driven Scheduling for Improving Patient Efficiency in Ophthalmology Clinics.

PURPOSE: To improve clinic efficiency through development of an ophthalmology scheduling template developed using simulation models and electronic health record (EHR) data. DESIGN: We created a computer simulation model of 1 pediatric ophthalmologist's clinic using EHR timestamp data, which was used to develop a scheduling template based on appointment length (short, medium, or long). We assessed its impact on clinic efficiency after implementation in the practices of 5 different pediatric ophthalmologists. PARTICIPANTS: We observed and timed patient appointments in person (n = 120) and collected EHR timestamps for 2 years of appointments (n = 650). We calculated efficiency measures for 172 clinic sessions before implementation vs. 119 clinic sessions after implementation. METHODS: We validated clinic workflow timings calculated from EHR timestamps and the simulation models based on them with observed timings. From simulation tests, we developed a new scheduling template and evaluated it with efficiency metrics before vs. after implementation. MAIN OUTCOME MEASURES: Measurements of clinical efficiency (mean clinic volume, patient wait time, examination time, and clinic length). RESULTS: Mean physician examination time calculated from EHR timestamps was 13.8±8.2 minutes and was not statistically different from mean physician examination time from in-person observation (13.3±7.3 minutes; P = 0.7), suggesting that EHR timestamps are accurate. Mean patient wait time for the simulation model (31.2±10.9 minutes) was not statistically different from the observed mean patient wait times (32.6±25.3 minutes; P = 0.9), suggesting that simulation models are accurate. After implementation of the new scheduling template, all 5 pediatric ophthalmologists showed statistically significant improvements in clinic volume (mean increase of 1-3 patients/session; P ≤ 0.05 for 2 providers; P ≤ 0.008 for 3 providers), whereas 4 of 5 had improvements in mean patient wait time (average improvements of 3-4 minutes/patient; statistically significant for 2 providers, P ≤ 0.008). All of the ophthalmologists' examination times remained the same before and after implementation. CONCLUSIONS: Simulation models based on big data from EHRs can test clinic changes before real-life implementation. A scheduling template using predicted appointment length improves clinic efficiency and may generalize to other clinics. Electronic health records have potential to become tools for supporting clinic operations improvement.

Association of the Presence of Trainees With Outpatient Appointment Times in an Ophthalmology Clinic.

Importance: Physicians face pressure to improve clinical efficiency, particularly with electronic health record (EHR) adoption and gradual shifts toward value-based reimbursement models. These pressures are especially pronounced in academic medical centers, where delivery of care must be balanced with medical education. However, the association of the presence of trainees with clinical efficiency in outpatient ophthalmology clinics is not known. Objective: To quantify the association of the presence of trainees (residents and fellows) and efficiency in an outpatient ophthalmology clinic. Design, Setting, and Participants: This single-center cohort study was conducted from January 1 through December 31, 2014, at an academic department of ophthalmology. Participants included 49 448 patient appointments with 33 attending physicians and 40 trainees. Exposures: Presence vs absence of trainees in an appointment or clinic session, as determined by review of the EHR audit log. Main Outcomes and Measures: Patient appointment time, as determined by time stamps in the EHR clinical data warehouse. Linear mixed models were developed to analyze variability among clinicians and patients. Results: Among the 33 study physicians (13 women [39%] and 20 men [61%]; median age, 44 years [interquartile range, 39-53 years]), appointments with trainees were significantly longer than appointments in clinic sessions without trainees (mean [SD], 105.0 [55.7] vs 80.3 [45.4] minutes; P < .001). The presence of a trainee in a clinic session was associated with longer mean appointment time, even in appointments for which the trainee was not present (mean [SD], 87.2 [49.2] vs 80.3 [45.4] minutes; P < .001). Among 33 study physicians, 3 (9%) had shorter mean appointment times when a trainee was present, 1 (3%) had no change, and 29 (88%) had longer mean appointment times when a trainee was present. Linear mixed models showed the presence of a resident was associated with a lengthening of appointment time of 17.0 minutes (95% CI, 15.6-18.5 minutes; P < .001), and the presence of a fellow was associated with a lengthening of appointment time of 13.5 minutes (95% CI, 12.3-14.8 minutes; P < .001). Conclusions and Relevance: Presence of trainees was associated with longer appointment times, even for patients not seen by a trainee. Although numerous limitations to this study design might affect the interpretation of the findings, these results highlight a potential challenge of maintaining clinical efficiency in academic medical centers and raise questions about physician reimbursement models.

Clinical Documentation in Electronic Health Record Systems: Analysis of Similarity in Progress Notes from Consecutive Outpatient Ophthalmology Encounters.

Content importing technology enables duplication of large amounts of clinical text in electronic health record (EHR) progress notes. It can be difficult to find key sections such as Assessment and Plan in the resulting note. To quantify the extent of text length and duplication, we analyzed average ophthalmology note length and calculated novelty of each major note section (Subjective, Objective, Assessment, Plan, Other). We performed a retrospective chart review of consecutive note pairs and found that the average encounter note was 1182 ± 374 words long and less than a quarter of words changed between visits. The Plan note section had the highest percentage of change, and both the Assessment and Plan sections comprised a small fraction of the full note. Analysis of progress notes by section and unique content helps describe physician documentation activity and inform best practices and EHR design recommendations.

Secondary Use of Electronic Health Record Data for Prediction of Outpatient Visit Length in Ophthalmology Clinics.

Electronic health record systems have dramatically transformed the process of medical care, but one challenge has been increased time requirements for physicians. In this study, we address this challenge by developing and validating analytic models for predicting patient encounter length based on secondary EHR data. Key findings from this study are: (1) Secondary use of EHR data may be captured to predict provider interaction time with patients; (2) Modeling results using secondary data may provide more accurate predictions of provider interaction time than an expert provide; (3) These findings suggest that secondary use of EHR data may be used to develop effective customized scheduling methods to improve clinical efficiency. In the future, this has the potential to contribute toward methods for improved clinical scheduling and efficiency.

Secondary use of electronic health record data for clinical workflow analysis.

Objective: Outpatient clinics lack guidance for tackling modern efficiency and productivity demands. Workflow studies require large amounts of timing data that are prohibitively expensive to collect through observation or tracking devices. Electronic health records (EHRs) contain a vast amount of timing data - timestamps collected during regular use - that can be mapped to workflow steps. This study validates using EHR timestamp data to predict outpatient ophthalmology clinic workflow timings at Oregon Health and Science University and demonstrates their usefulness in 3 different studies. Materials and Methods: Four outpatient ophthalmology clinics were observed to determine their workflows and to time each workflow step. EHR timestamps were mapped to the workflow steps and validated against the observed timings. Results: The EHR timestamp analysis produced times that were within 3 min of the observed times for >80% of the appointments. EHR use patterns affected the accuracy of using EHR timestamps to predict workflow times. Discussion: EHR timestamps provided a reasonable approximation of workflow and can be used for workflow studies. They can be used to create simulation models, analyze EHR use, and quantify the impact of trainees on workflow. Conclusion: The secondary use of EHR timestamp data is a valuable resource for clinical workflow studies. Sample timestamp data files and algorithms for processing them are provided and can be used as a template for more studies in other clinical specialties and settings.

Clinical Documentation in Electronic Health Record Systems: Analysis of Patient Record Review During Outpatient Ophthalmology Visits.

Busy clinicians struggle with productivity and usability in electronic health record systems (EHRs). While previous studies have investigated documentation practices and strategies in the inpatient setting, outpatient documentation and review practices by clinicians using EHRs are relatively unknown. In this study, we look at clinicians' patterns of note review in the EHR during outpatient follow-up office visits in ophthalmology. Key findings from this study are that the number and percentage of notes reviewed is very low, there is variation between providers, specialties, and users, and staff access more notes than physicians. These findings suggest that the vast majority of content in the EHR is not being used by clinicians; improved EHR designs would better present this data and support the information needs of outpatient clinicians.

Time Requirements for Electronic Health Record Use in an Academic Ophthalmology Center.

Importance: Electronic health record (EHR) systems have transformed the practice of medicine. However, physicians have raised concerns that EHR time requirements have negatively affected their productivity. Meanwhile, evolving approaches toward physician reimbursement will require additional documentation to measure quality and cost of care. To date, little quantitative analysis has rigorously studied these topics. Objective: To examine ophthalmologist time requirements for EHR use. Design, Setting, and Participants: A single-center cohort study was conducted between September 1, 2013, and December 31, 2016, among 27 stable departmental ophthalmologists (defined as attending ophthalmologists who worked at the study institution for ≥6 months before and after the study period). Ophthalmologists who did not have a standard clinical practice or who did not use the EHR were excluded. Exposures: Time stamps from the medical record and EHR audit log were analyzed to measure the length of time required by ophthalmologists for EHR use. Ophthalmologists underwent manual time-motion observation to measure the length of time spent directly with patients on the following 3 activities: EHR use, conversation, and examination. Main Outcomes and Measures: The study outcomes were time spent by ophthalmologists directly with patients on EHR use, conversation, and examination as well as total time required by ophthalmologists for EHR use. Results: Among the 27 ophthalmologists in this study (10 women and 17 men; mean [SD] age, 47.3 [10.7] years [median, 44; range, 34-73 years]) the mean (SD) total ophthalmologist examination time was 11.2 (6.3) minutes per patient, of which 3.0 (1.8) minutes (27% of the examination time) were spent on EHR use, 4.7 (4.2) minutes (42%) on conversation, and 3.5 (2.3) minutes (31%) on examination. Mean (SD) total ophthalmologist time spent using the EHR was 10.8 (5.0) minutes per encounter (range, 5.8-28.6 minutes). The typical ophthalmologist spent 3.7 hours using the EHR for a full day of clinic: 2.1 hours during examinations and 1.6 hours outside the clinic session. Linear mixed effects models showed a positive association between EHR use and billing level and a negative association between EHR use per encounter and clinic volume. Each additional encounter per clinic was associated with a decrease of 1.7 minutes (95% CI, -4.3 to 1.0) of EHR use time per encounter for ophthalmologists with high mean billing levels (adjusted R2 = 0.42; P = .01). Conclusions and Relevance: Ophthalmologists have limited time with patients during office visits, and EHR use requires a substantial portion of that time. There is variability in EHR use patterns among ophthalmologists.

Quantifying the Impact of Trainee Providers on Outpatient Clinic Workflow using Secondary EHR Data.

Providers today face productivity challenges including increased patient loads, increased clerical burdens from new government regulations and workflow impacts of electronic health records (EHR). Given these factors, methods to study and improve clinical workflow continue to grow in importance. Despite the ubiquitous presence of trainees in academic outpatient clinics, little is known about the impact of trainees on academic workflow. The purpose of this study is to demonstrate that secondary EHR data can be used to quantify that impact, with potentially important results for clinic efficiency and provider reimbursement models. Key findings from this study are that (1) Secondary EHR data can be used to reflect in clinic trainee activity, (2) presence of trainees, particularly in high-volume clinic sessions, is associated with longer session lengths, and (3) The timing of trainee appointments within clinic sessions impacts the session length.

Evaluating and Improving an Outpatient Clinic Scheduling Template Using Secondary Electronic Health Record Data.

Improving the efficiency of outpatient clinics is challenging in the face of increased patient loads, decreased reimbursements and potential negative productivity impacts of using electronic health records (EHR). We modeled outpatient ophthalmology clinic workflow using discrete event simulation for testing new scheduling templates that decrease patient wait time and improve clinic efficiency. Despite challenges in implementing the new scheduling templates in one outpatient clinic, the new templates improved patient wait time and clinic session length when they were followed. Analyzing EHR data about these schedules and their adherence to the template provides insight into new policies that can better balance the competing priorities of filling the schedules, meeting patient demand and minimizing wait time.

Clinic Workflow Simulations using Secondary EHR Data.

Clinicians today face increased patient loads, decreased reimbursements and potential negative productivity impacts of using electronic health records (EHR), but have little guidance on how to improve clinic efficiency. Discrete event simulation models are powerful tools for evaluating clinical workflow and improving efficiency, particularly when they are built from secondary EHR timing data. The purpose of this study is to demonstrate that these simulation models can be used for resource allocation decision making as well as for evaluating novel scheduling strategies in outpatient ophthalmology clinics. Key findings from this study are that: 1) secondary use of EHR timestamp data in simulation models represents clinic workflow, 2) simulations provide insight into the best allocation of resources in a clinic, 3) simulations provide critical information for schedule creation and decision making by clinic managers, and 4) simulation models built from EHR data are potentially generalizable.

Computerized patient-reported symptom assessment in radiotherapy: a pilot randomized, controlled trial.

PURPOSE: Computer-based, patient-reported symptom survey tools have been described for patients undergoing chemotherapy. We hypothesized that patients undergoing radiotherapy might also benefit, so we developed a computer application to acquire symptom ratings from patients and generate summaries for use at point of care office visits and conducted a randomized, controlled pilot trial to test its feasibility. METHODS: Subjects were randomized prior to beginning radiotherapy. Both control and intervention group subjects completed the computerized symptom assessment, but only for the intervention group were printed symptom summaries made available before each weekly office visit. Metrics compared included the Global Distress Index (GDI), concordance of patient-reported symptoms and symptoms discussed by the physician and numbers of new and/or adjusted symptom management medications prescribed. RESULTS: One hundred twelve patients completed the study: 54 in the control and 58 in the intervention arms. There were no differences in GDI over time between the control and intervention groups. In the intervention group, more patient-reported symptoms were actually discussed in radiotherapy office visits: 46/202 vs. 19/230. A sensitivity analysis to account for within-subjects correlation yielded 23.2 vs. 10.3 % (p = 0.03). Medications were started or adjusted at 15.4 % (43/280) of control visits compared to 20.4 % (65/319) of intervention visits (p = 0.07). CONCLUSIONS: This computer application is easy to use and makes extensive patient-reported outcome data available at the point of care. Although no differences were seen in symptom trajectory, patients who had printed symptom summaries had improved communication during office visits and a trend towards a more active symptom management during radiotherapy.