Identifying Nonpatient Authors of Patient Portal Secure Messages in Oncology: A Proof-of-Concept Demonstration of Natural Language Processing Methods.

PURPOSE: Patient portal secure messages are not always authored by the patient account holder. Understanding who authored the message is particularly important in an oncology setting where symptom reporting is crucial to patient treatment. Natural language processing has the potential to detect messages not authored by the patient automatically. METHODS: Patient portal secure messages from the Memorial Sloan Kettering Cancer Center were retrieved and manually annotated as a predicted unregistered proxy (ie, not written by the patient) or a presumed patient. After randomly splitting the annotated messages into training and test sets in a 70:30 ratio, a bag-of-words approach was used to extract features and then a Least Absolute Shrinkage and Selection Operator (LASSO) model was trained and used for classification. RESULTS: Portal secure messages (n = 2,000) were randomly selected from unique patient accounts and manually annotated. We excluded 335 messages from the data set as the annotators could not determine if they were written by a patient or proxy. Using the remaining 1,665 messages, a LASSO model was developed that achieved an area under the curve of 0.932 and an area under the precision recall curve of 0.748. The sensitivity and specificity related to classifying true-positive cases (predicted unregistered proxy-authored messages) and true negatives (presumed patient-authored messages) were 0.681 and 0.960, respectively. CONCLUSION: Our work demonstrates the feasibility of using unstructured, heterogenous patient portal secure messages to determine portal secure message authorship. Identifying patient authorship in real time can improve patient portal account security and can be used to improve the quality of the information extracted from the patient portal, such as patient-reported outcomes.

Managing Critical Patient-Reported Outcome Measures in Oncology Settings: System Development and Retrospective Study.

BACKGROUND: Remote monitoring programs based on the collection of patient-reported outcome (PRO) data are being increasingly adopted in oncology practices. Although PROs are a great source of patient data, the management of critical PRO data is not discussed in detail in the literature. OBJECTIVE: This first-of-its-kind study aimed to design, describe, and evaluate a closed-loop alerting and communication system focused on managing PRO-related alerts in cancer care. METHODS: We designed and developed a novel solution using an agile software development methodology by incrementally building new capabilities. We evaluated these new features using participatory design and the Fit between Individuals, Task, and Technology framework. RESULTS: A total of 8 questionnaires were implemented using alerting features, resulting in an alert rate of 7.82% (36,838/470,841) with 13.28% (10,965/82,544) of the patients triggering at least one alert. Alerts were reviewed by 501 staff members spanning across 191 care teams. All the alerts were reviewed with a median response time of 1 hour (SD 185 hours) during standard business hours. The most severe (red) alerts were documented 56.83% (2592/4561) of the time, whereas unlabeled alerts were documented 27.68% (1298/4689) of the time, signaling clinician concordance with the alert thresholds. CONCLUSIONS: A PRO-based alert and communication system has some initial benefits in reviewing clinically meaningful PRO data in a reasonable amount of time. We have discussed key system design considerations, workflow integration, and the mitigation of potential impact on the burden of care teams. The introduction of a PRO-based alert and communication system provides a reliable mechanism for care teams to review and respond to patient symptoms quickly. The system was standardized across many different oncology settings, demonstrating system flexibility. Future studies should focus on formally evaluating system usability through qualitative methods.

Ethics Consultation Requests After Implementation of an Electronic Health Record Order.

PURPOSE: As the role of clinical ethics consultation in health care advances, there are calls to standardize the process of consultation. The Ethics Committee at Memorial Sloan Kettering Cancer Center (MSK) hypothesized that the process of requesting an ethics consultation could be improved by instituting an electronic health record (EHR) order for consultation requests. This report summarizes the impact of adopting an EHR order for ethics consultation requests at MSK. METHODS: This retrospective review of all clinical ethics consultations requested at a tertiary cancer center from May 2017 to February 2020 spans 17 months before and after implementation of an electronic order for consultation requests. Summary statistics are presented using Pearson chi-square analyses with a significance level of 0.05. RESULTS: There was a significant increase in the total number of consultation requests placed after implementation of the EHR order (n = 165, 0.08% of total patients) compared with before (n = 108, 0.05% of total patients; P = .007). The number of consults requested by providers from inpatient (P = .02) and outpatient (P = .04) settings significantly increased. The proportion of consults placed by medical versus nonmedical providers remained unchanged (P = .32). CONCLUSION: In this large single-institution retrospective study, implementation of an EHR order for ethics consultation requests was associated with a significant increase in the number of consultation requests. Implementation of an electronic order may decrease barriers to ethics consultation in diverse practice settings. Further longitudinal, multicenter studies are needed to assess strategies to improve access to clinical ethics consultation for oncology patients.

Adoption of Patient-Generated Health Data in Oncology: A Report From the NCCN EHR Oncology Advisory Group.

BACKGROUND: Collecting, monitoring, and responding to patient-generated health data (PGHD) are associated with improved quality of life and patient satisfaction, and possibly with improved patient survival in oncology. However, the current state of adoption, types of PGHD collected, and degree of integration into electronic health records (EHRs) is unknown. METHODS: The NCCN EHR Oncology Advisory Group formed a Patient-Reported Outcomes (PRO) Workgroup to perform an assessment and provide recommendations for cancer centers, researchers, and EHR vendors to advance the collection and use of PGHD in oncology. The issues were evaluated via a survey of NCCN Member Institutions. Questions were designed to assess the current state of PGHD collection, including how, what, and where PGHD are collected. Additionally, detailed questions about governance and data integration into EHRs were asked. RESULTS: Of 28 Member Institutions surveyed, 23 responded. The collection and use of PGHD is widespread among NCCN Members Institutions (96%). Most centers (90%) embed at least some PGHD into the EHR, although challenges remain, as evidenced by 88% of respondents reporting the use of instruments not integrated. Forty-seven percent of respondents are leveraging PGHD for process automation and adherence to best evidence. Content type and integration touchpoints vary among the members, as well as governance maturity. CONCLUSIONS: The reported variability regarding PGHD suggests that it may not yet have reached its full potential for oncology care delivery. As the adoption of PGHD in oncology continues to expand, opportunities exist to enhance their utility. Among the recommendations for cancer centers is establishment of a governance process that includes patients. Researchers should consider determining which PGHD instruments confer the highest value. It is recommended that EHR vendors collaborate with cancer centers to develop solutions for the collection, interpretation, visualization, and use of PGHD.

Preferences in Oncology History Documentation Styles Among Clinical Practitioners.

PURPOSE: Clinical notes function as the de facto handoff between providers and assume great importance during unplanned medical encounters. An organized and thorough oncology history is essential in care coordination. We sought to understand reader preferences for oncology history organization by comparing between chronologic and narrative formats. METHODS: A convenience sample of 562 clinicians from 19 National Comprehensive Cancer Network Member Institutions responded to a survey comparing two formats of oncology histories, narrative and chronologic, for the same patient. Both histories were consensus-derived real-world examples. Each history was evaluated using semantic differential attributes (thorough, useful, organized, comprehensible, and succinct). Respondents choose a preference between the two styles for history gathering and as the basis of a new note. Open-ended responses were also solicited. RESULTS: Respondents preferred the chronologic over the narrative history to prepare for a visit with an unknown patient (66% preference) and as a basis for their own note preparation (77% preference) (P < .01). The chronologic summary was preferred in four of the five measured attributes (useful, organized, comprehensible, and succinct); the narrative summary was favored for thoroughness (P < .01). Open-ended responses reflected the attribute scoring and noted the utility of content describing social determinants of health in the narrative history. CONCLUSION: Respondents of this convenience sample preferred a chronologic oncology history to a concise narrative history. Further studies are needed to determine the optimal structure and content of chronologic documentation for oncology patients and the provider effort to use this format.

A Randomized Controlled Trial Evaluating Electronic Outpatient Symptom Monitoring After Ambulatory Cancer Surgery.

OBJECTIVE: We implemented routine daily electronic monitoring of patient-reported outcomes (PROs) for 10 days after discharge after ambulatory cancer surgery, with alerts to clinical staff for worrying symptoms. We sought to determine whether enhancing this monitoring by adding immediate automated normative feedback to patients regarding expected symptoms would further improve the patient experience. SUMMARY OF BACKGROUND DATA: PRO monitoring reduces symptom severity in cancer patients. In ambulatory cancer surgery, it reduces potentially avoidable urgent care center (UCC) visits, defined as those UCC visits without readmission. METHODS: Patients undergoing ambulatory cancer surgery (n = 2624) were randomized to receive standard PRO monitoring or enhanced feedback. The primary study outcome was UCC visits without readmission within 30 days; secondary outcomes included patient anxiety and nursing utilization. RESULTS: There was no significant difference in the risk of a potentially avoidable UCC visit [1.0% higher in enhanced feedback, 95% confidence interval (CI) -0.2-3.1%; P = 0.12]. There were similarly no significant differences in UCC visits with readmission or readmission overall (P = 0.4 for both). Patients randomized to enhanced feedback demonstrated a quicker reduction in anxiety (P < 0.001) and required 14% (95% CI 8-19%; P < 0.001) and 10% (95% CI 5-16%, P < 0.001) fewer nursing calls over 10 and 30 days postoperatively. CONCLUSIONS: Providing patients with feedback about symptom severity during recovery from ambulatory cancer surgery reduces anxiety and nursing workload without affecting UCC visits or readmissions. These results support wider incorporation of normative feedback in systems for routine PRO monitoring.

Interfaces for collecting data from patients: 10 golden rules.

Memorial Sloan Kettering Cancer Center has more than a decade's experience creating online interfaces for obtaining data from patients as part of routine clinical care. We have developed a set of "golden rules" for design of these interfaces. Many relate to the knowledge imbalance between professional staff (whether medical or informatics) and patients, who are often old and sick and have limited knowledge of technology. Others relate to the clinical nature of the encounter: data cannot be taken from patients as part of clinical care unless there is a plan to act on whatever information is prepared. We also note that the plethora of marketing questionnaires makes patients suspicious of surveys: patient trust is hard to gain and easy to lose. Addition of these golden rules to standard approaches to interface design will maximize our ability to obtain data from patients and thus improve communication between patients and clinicians.

Ambulatory cancer care electronic symptom self-reporting (ACCESS) for surgical patients: a randomised controlled trial protocol.

INTRODUCTION: An increasing proportion of cancer surgeries are ambulatory procedures requiring a stay of 1 day or less in the hospital. Providing patients and their caregivers with ongoing, real-time support after discharge aids delivery of high-quality postoperative care in this new healthcare environment. Despite abundant evidence that patient self-reporting of symptoms improves quality of care, the most effective way to monitor and manage this self-reported information is not known. METHODS AND ANALYSIS: This is a two-armed randomised, controlled trial evaluating two approaches to the management of patient-reported data: (1) team monitoring, symptom monitoring by the clinical team, with nursing outreach if symptoms exceed normal limits, and (2) enhanced feedback, real-time feedback to patients about expected symptom severity, with patient-activated care as needed.Patients with breast, gynaecologic, urologic, and head and neck cancer undergoing ambulatory cancer surgery (n=2750) complete an electronic survey for up to 30 days after surgery that includes items from a validated instrument developed by the National Cancer Institute, the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Information provided to patients in the Enhanced Feedback group is procedure-specific and based on updated PRO-CTCAE data from previous patients. Qualitative interviews are also performed. The primary study outcomes assess unplanned emergency department visits and symptom-triggered interventions (eg, nursing calls and pain management referrals) within 30 days, and secondary outcomes assess the patient and caregiver experience (ie, patient engagement, patient anxiety and caregiver burden). ETHICS AND DISSEMINATION: This study is approved by the Institutional Review Board at Memorial Sloan Kettering Cancer Center. The relationships between the study team and stakeholders will be leveraged to disseminate study findings. Findings will be relevant in designing future coordinated care models targeting improved healthcare quality and patient experience. TRIAL REGISTRATION NUMBER: NCT03178045.

Real-World Outcomes of an Automated Physician Support System for Genome-Driven Oncology.

PURPOSE: Matching patients to investigational therapies requires new tools to support physician decision making. We designed and implemented Precision Insight Support Engine (PRECISE), an automated, just-in-time, clinical-grade informatics platform to identify and dynamically track patients on the basis of molecular and clinical criteria. Real-world use of this tool was analyzed to determine whether PRECISE facilitated enrollment to early-phase, genome-driven trials. MATERIALS AND METHODS: We analyzed patients who were enrolled in genome-driven, early-phase trials using PRECISE at Memorial Sloan Kettering Cancer Center between April 2014 and January 2018. Primary end point was the proportion of enrolled patients who were successfully identified using PRECISE before enrollment. Secondary end points included time from sequencing and PRECISE identification to enrollment. Reasons for a failure to identify genomically matched patients were also explored. RESULTS: Data were analyzed from 41 therapeutic trials led by 19 principal investigators. In total, 755 patients were accrued to these studies during the period that PRECISE was used. PRECISE successfully identified 327 patients (43%) before enrollment. Patients were diagnosed with 29 tumor types and harbored alterations in 43 oncogenes, most commonly ERBB2 (21.3%), PIK3CA (14.1%), and BRAF (8.7%). Median time from sequencing to enrollment was 163 days (interquartile range, 66 to 357 days), and from PRECISE identification to enrollment 87 days (interquartile range, 37 to 180 days). Common reasons for failing to identify patients before enrollment included accrual on the basis of molecular alterations that did not match pre-established PRECISE genomic eligibility (140 [33%] of 428) and external sequencing not available for parsing (127 [30%] of 428). CONCLUSION: PRECISE identified 43% of all patients accrued to a diverse cohort of early-phase, genome-matched studies. Purpose-built informatics platforms represent a novel and potentially effective method for matching patients to molecularly selected studies.

Informing, Reassuring, or Alarming? Balancing Patient Needs in the Development of a Postsurgical Symptom Reporting System in Cancer.

After ambulatory surgeries, patients who recover at home have multiple questions about wound healing, symptoms and medication side effects, and recovery expectations. We conducted user testing and rapid application development of a newly developed symptom reporting system that supports home-based recovery by inviting patients to self-report symptoms in the days after surgery and then receive an immediate feedback report giving context for their reported symptoms. Findings showed that some participants primarily valued reassurance, whereas others prioritized receiving alerts about potential problems. Results also showed that most patients wanted feedback framed as comparing their progress to their expected progress, not to that of other patients. The final feedback report provided patients with actionable recommendations, small graphs showing their progress, and with short "gist" text interpretations. The system has been implemented, and recruitment is ongoing for a large clinical trial of its effectiveness for reducing adverse events and unnecessary emergency or urgent care visits.

Automated eligibility screening and monitoring for genotype-driven precision oncology trials.

The Information Systems Department at Memorial Sloan Kettering Cancer Center developed the DARWIN Cohort Management System (DCMS). The DCMS identifies and tracks cohorts of patients based on genotypic and clinical data. It assists researchers and treating physicians in enrolling patients to genotype-matched IRB-approved clinical trials. The DCMS sends automated, actionable, and secure email notifications to users with information about eligible or enrolled patients before their upcoming appointments. The system also captures investigators input via annotations on patient eligibility and preferences on future status updates. As of August 2015, the DCMS is tracking 159,893 patients on both clinical operations and research cohorts. 134 research cohorts have been established and track 64,473 patients. 51,192 of these have had one or more genomic tests including MSK-IMPACT, comprising the pool eligible for genotype-matched studies. This paper describes the design and evolution of this Informatics solution.

An integrated billing application to streamline clinician workflow.

Between 2008 and 2010, our academic medical center transitioned to electronic provider documentation using a commercial electronic health record system. For attending physicians, one of the most frustrating aspects of this experience was the system's failure to support their existing electronic billing workflow. Because of poor system integration, it was difficult to verify the supporting documentation for each bill and impractical to track whether billable notes had corresponding charges. We developed and deployed in 2011 an integrated billing application called "iCharge" that streamlines clinicians' documentation and billing workflow, and simultaneously populates the inpatient problem list using billing diagnosis codes. Each month, over 550 physicians use iCharge to submit approximately 23,000 professional service charges for over 4,200 patients. On average, about 2.5 new problems are added to each patient's problem list. This paper describes the challenges and benefits of workflow integration across disparate applications and presents an example of innovative software development within a commercial EHR framework.

Enabling claims-based decision support through non-interruptive capture of admission diagnoses and provider billing codes.

The patient problem list, like administrative claims data, has become an important source of data for decision support, patient cohort identification, and alerting systems. A two-fold intervention to increase capture of problems on the problem list automatically - with minimal disruption to admitting and provider billing workflows - is described. For new patients with no prior data in the electronic health record, the intervention resulted in a statistically significant increase in the number of problems recorded to the problem list (3.8 vs 2.9 problems post-and pre-intervention respectively, p value 2×10(-16)). The majority of problems were recorded in the first 24 hours of admission. The proportion of patients with at least one problem coded to the problem list within the first 24 hours increased from 94% to 98% before and after intervention (chi square 344, p value 2×10(-16)). ICD9 "V codes" connoting circumstances beyond disease were captured at a higher rate post intervention than before. Deyo/Charlson comorbidities derived from problem list data were more similar to those derived from claims data after the intervention than before (Jaccard similarity 0.3 post- vs 0.21 pre-intervention, p value 2×10(-16)). A workflow-sensitive, non-interruptive means of capturing provider-entered codes early in admission can improve both the quantity and content of problems on the patient problem list.

Provider-to-provider electronic communication in the era of meaningful use: a review of the evidence.

BACKGROUND: Electronic communication between providers occurs daily in clinical practice but has not been well studied. PURPOSE: To assess the impact of provider-to-provider electronic communication tools on communication and healthcare outcomes through literature review. DATA SOURCES: Ovid MEDLINE, PubMed, Google Scholar, Cumulative Index to Nursing and Allied Health Literature, and Academic Search Premier. STUDY SELECTION: Publication in English-language peer-reviewed journals. Studies provided quantitative provider-to-provider communication data, provider satisfaction statistics, or electronic health record (EHR) communication data. DATA EXTRACTION: Literature review. DATA SYNTHESIS: Two reviewers conducted the title review to determine eligible studies from initial search results. Three reviewers independently reviewed titles, abstracts, and full text (where appropriate) against inclusion and exclusion criteria. LIMITATIONS: Small number of eligible studies; few described trial design (20%). Homogeneous provider type (physicians). English-only studies. CONCLUSIONS: Of 25 included studies, all focused on physicians; most were observational (68%). Most (60%) described electronic specialist referral tools. Although overall use has been measured, there were no studies of the effectiveness of intra-EHR messaging. Literature describing the effectiveness of provider-to-provider electronic communications is sparse and narrow in scope. Complex care, such as that envisioned for the Patient Centered Medical Home, necessitates further research.

Relationship between nursing documentation and patients' mortality.

BACKGROUND: Nurses alter their monitoring behavior as a patient's clinical condition deteriorates, often detecting and documenting subtle changes before physiological trends are apparent. It was hypothesized that a nurse's behavior of recording optional documentation (beyond what is required) reflects concern about a patient's status and that mining data from patients' electronic health records for the presence of these features could help predict patients' mortality. METHODS: Data-mining methods were used to analyze electronic nursing documentation from a 15-month period at a large, urban academic medical center. Mortality rates and the frequency of vital sign measurements (beyond required) and optional nursing comment documentation were analyzed for a random set of patients and patients who experienced a cardiac arrest during their hospitalization. Patients were stratified by age-adjusted Charlson comorbidity index. RESULTS: A total of 15,000 acute care patients and 145 cardiac arrest patients were studied. Patients who died had a mean of 0.9 to 1.5 more optional comments and 6.1 to 10 more vital signs documented within 48 hours than did patients who survived. A higher frequency of comment and vital sign documentation was also associated with a higher likelihood of cardiac arrest. Of patients who had a cardiac arrest, those with more documented comments were more likely to die. CONCLUSIONS: For the first time, nursing documentation patterns have been linked to patients' mortality. Findings were consistent with the hypothesis that some features of nursing documentation within electronic health records can be used to predict mortality. With future work, these associations could be used in real time to establish a threshold of concern indicating a risk for deterioration in a patient's condition.

CT pulmonary angiography: increasingly diagnosing less severe pulmonary emboli.

BACKGROUND: It is unknown whether the observed increase in computed tomography pulmonary angiography (CTPA) utilization has resulted in increased detection of pulmonary emboli (PEs) with a less severe disease spectrum. METHODS: Trends in utilization, diagnostic yield, and disease severity were evaluated for 4,048 consecutive initial CTPAs performed in adult patients in the emergency department of a large urban academic medical center between 1/1/2004 and 10/31/2009. Transthoracic echocardiography (TTE) findings and peak serum troponin levels were evaluated to assess for the presence of PE-associated right ventricular (RV) abnormalities (dysfunction or dilatation) and myocardial injury, respectively. Statistical analyses were performed using multivariate logistic regression. RESULTS: 268 CTPAs (6.6%) were positive for acute PE, and 3,780 (93.4%) demonstrated either no PE or chronic PE. There was a significant increase in the likelihood of undergoing CTPA per year during the study period (odds ratio [OR] 1.05, 95% confidence interval [CI] 1.04-1.07, P<0.01). There was no significant change in the likelihood of having a CTPA diagnostic of an acute PE per year (OR 1.03, 95% CI 0.95-1.11, P = 0.49). The likelihood of diagnosing a less severe PE on CTPA with no associated RV abnormalities or myocardial injury increased per year during the study period (OR 1.39, 95% CI 1.10-1.75, P = 0.01). CONCLUSIONS: CTPA utilization has risen with no corresponding change in diagnostic yield, resulting in an increase in PE detection. There is a concurrent rise in the likelihood of diagnosing a less clinically severe spectrum of PEs.