Adoption of Patient-Generated Health Data in Oncology: A Report From the NCCN EHR Oncology Advisory Group.

BACKGROUND: Collecting, monitoring, and responding to patient-generated health data (PGHD) are associated with improved quality of life and patient satisfaction, and possibly with improved patient survival in oncology. However, the current state of adoption, types of PGHD collected, and degree of integration into electronic health records (EHRs) is unknown. METHODS: The NCCN EHR Oncology Advisory Group formed a Patient-Reported Outcomes (PRO) Workgroup to perform an assessment and provide recommendations for cancer centers, researchers, and EHR vendors to advance the collection and use of PGHD in oncology. The issues were evaluated via a survey of NCCN Member Institutions. Questions were designed to assess the current state of PGHD collection, including how, what, and where PGHD are collected. Additionally, detailed questions about governance and data integration into EHRs were asked. RESULTS: Of 28 Member Institutions surveyed, 23 responded. The collection and use of PGHD is widespread among NCCN Members Institutions (96%). Most centers (90%) embed at least some PGHD into the EHR, although challenges remain, as evidenced by 88% of respondents reporting the use of instruments not integrated. Forty-seven percent of respondents are leveraging PGHD for process automation and adherence to best evidence. Content type and integration touchpoints vary among the members, as well as governance maturity. CONCLUSIONS: The reported variability regarding PGHD suggests that it may not yet have reached its full potential for oncology care delivery. As the adoption of PGHD in oncology continues to expand, opportunities exist to enhance their utility. Among the recommendations for cancer centers is establishment of a governance process that includes patients. Researchers should consider determining which PGHD instruments confer the highest value. It is recommended that EHR vendors collaborate with cancer centers to develop solutions for the collection, interpretation, visualization, and use of PGHD.

A Randomized Controlled Trial Evaluating Electronic Outpatient Symptom Monitoring After Ambulatory Cancer Surgery.

OBJECTIVE: We implemented routine daily electronic monitoring of patient-reported outcomes (PROs) for 10 days after discharge after ambulatory cancer surgery, with alerts to clinical staff for worrying symptoms. We sought to determine whether enhancing this monitoring by adding immediate automated normative feedback to patients regarding expected symptoms would further improve the patient experience. SUMMARY OF BACKGROUND DATA: PRO monitoring reduces symptom severity in cancer patients. In ambulatory cancer surgery, it reduces potentially avoidable urgent care center (UCC) visits, defined as those UCC visits without readmission. METHODS: Patients undergoing ambulatory cancer surgery (n = 2624) were randomized to receive standard PRO monitoring or enhanced feedback. The primary study outcome was UCC visits without readmission within 30 days; secondary outcomes included patient anxiety and nursing utilization. RESULTS: There was no significant difference in the risk of a potentially avoidable UCC visit [1.0% higher in enhanced feedback, 95% confidence interval (CI) -0.2-3.1%; P = 0.12]. There were similarly no significant differences in UCC visits with readmission or readmission overall (P = 0.4 for both). Patients randomized to enhanced feedback demonstrated a quicker reduction in anxiety (P < 0.001) and required 14% (95% CI 8-19%; P < 0.001) and 10% (95% CI 5-16%, P < 0.001) fewer nursing calls over 10 and 30 days postoperatively. CONCLUSIONS: Providing patients with feedback about symptom severity during recovery from ambulatory cancer surgery reduces anxiety and nursing workload without affecting UCC visits or readmissions. These results support wider incorporation of normative feedback in systems for routine PRO monitoring.

A Framework for Integrating Telehealth Equitably across the cancer care continuum.

The COVID-19 pandemic placed a spotlight on the potential to dramatically increase the use of telehealth across the cancer care continuum, but whether and how telehealth can be implemented in practice in ways that reduce, rather than exacerbate, inequities are largely unknown. To help fill this critical gap in research and practice, we developed the Framework for Integrating Telehealth Equitably (FITE), a process and evaluation model designed to help guide equitable integration of telehealth into practice. In this manuscript, we present FITE and showcase how investigators across the National Cancer Institute's Telehealth Research Centers of Excellence are applying the framework in different ways to advance digital and health equity. By highlighting multilevel determinants of digital equity that span further than access alone, FITE highlights the complex and differential ways structural determinants restrict or enable digital equity at the individual and community level. As such, achieving digital equity will require strategies designed to not only support individual behavior but also change the broader context to ensure all patients and communities have the choice, opportunity, and resources to use telehealth across the cancer care continuum.

Transforming patient-centered cancer care using telehealth: the MATCHES Center.

Modern cancer care is costly and logistically burdensome for patients and their families despite an expansion of technology and medical advances that create the opportunity for novel approaches to care. Therefore, there is a growing appreciation for the need to leverage these innovations to make cancer care more patient centered and convenient. The Memorial Sloan Kettering Making Telehealth Delivery of Cancer Care at Home Efficient and Safe Telehealth Research Center is a National Cancer Institute-designated and funded Telehealth Research Center of Excellence poised to generate the evidence necessary to inform the appropriate use of telehealth as a strategy to improve access to cancer services that are convenient for patients. The center will evaluate telehealth as a strategy to personalize cancer care delivery to ensure that it is not only safe and effective but also convenient and efficient. In this article, we outline this new center's research strategy, as well as highlight challenges that exist in further integrating telehealth into standard oncology practice based on early experiences.

A new clustering method for detecting rare senses of abbreviations in clinical notes.

Abbreviations are widely used in clinical documents and they are often ambiguous. Building a list of possible senses (also called sense inventory) for each ambiguous abbreviation is the first step to automatically identify correct meanings of abbreviations in given contexts. Clustering based methods have been used to detect senses of abbreviations from a clinical corpus [1]. However, rare senses remain challenging and existing algorithms are not good enough to detect them. In this study, we developed a new two-phase clustering algorithm called Tight Clustering for Rare Senses (TCRS) and applied it to sense generation of abbreviations in clinical text. Using manually annotated sense inventories from a set of 13 ambiguous clinical abbreviations, we evaluated and compared TCRS with the existing Expectation Maximization (EM) clustering algorithm for sense generation, at two different levels of annotation cost (10 vs. 20 instances for each abbreviation). Our results showed that the TCRS-based method could detect 85% senses on average; while the EM-based method found only 75% senses, when similar annotation effort (about 20 instances) was used. Further analysis demonstrated that the improvement by the TCRS method was mainly from additionally detected rare senses, thus indicating its usefulness for building more complete sense inventories of clinical abbreviations.

Managing Critical Patient-Reported Outcome Measures in Oncology Settings: System Development and Retrospective Study.

BACKGROUND: Remote monitoring programs based on the collection of patient-reported outcome (PRO) data are being increasingly adopted in oncology practices. Although PROs are a great source of patient data, the management of critical PRO data is not discussed in detail in the literature. OBJECTIVE: This first-of-its-kind study aimed to design, describe, and evaluate a closed-loop alerting and communication system focused on managing PRO-related alerts in cancer care. METHODS: We designed and developed a novel solution using an agile software development methodology by incrementally building new capabilities. We evaluated these new features using participatory design and the Fit between Individuals, Task, and Technology framework. RESULTS: A total of 8 questionnaires were implemented using alerting features, resulting in an alert rate of 7.82% (36,838/470,841) with 13.28% (10,965/82,544) of the patients triggering at least one alert. Alerts were reviewed by 501 staff members spanning across 191 care teams. All the alerts were reviewed with a median response time of 1 hour (SD 185 hours) during standard business hours. The most severe (red) alerts were documented 56.83% (2592/4561) of the time, whereas unlabeled alerts were documented 27.68% (1298/4689) of the time, signaling clinician concordance with the alert thresholds. CONCLUSIONS: A PRO-based alert and communication system has some initial benefits in reviewing clinically meaningful PRO data in a reasonable amount of time. We have discussed key system design considerations, workflow integration, and the mitigation of potential impact on the burden of care teams. The introduction of a PRO-based alert and communication system provides a reliable mechanism for care teams to review and respond to patient symptoms quickly. The system was standardized across many different oncology settings, demonstrating system flexibility. Future studies should focus on formally evaluating system usability through qualitative methods.

Ethics Consultation Requests After Implementation of an Electronic Health Record Order.

PURPOSE: As the role of clinical ethics consultation in health care advances, there are calls to standardize the process of consultation. The Ethics Committee at Memorial Sloan Kettering Cancer Center (MSK) hypothesized that the process of requesting an ethics consultation could be improved by instituting an electronic health record (EHR) order for consultation requests. This report summarizes the impact of adopting an EHR order for ethics consultation requests at MSK. METHODS: This retrospective review of all clinical ethics consultations requested at a tertiary cancer center from May 2017 to February 2020 spans 17 months before and after implementation of an electronic order for consultation requests. Summary statistics are presented using Pearson chi-square analyses with a significance level of 0.05. RESULTS: There was a significant increase in the total number of consultation requests placed after implementation of the EHR order (n = 165, 0.08% of total patients) compared with before (n = 108, 0.05% of total patients; P = .007). The number of consults requested by providers from inpatient (P = .02) and outpatient (P = .04) settings significantly increased. The proportion of consults placed by medical versus nonmedical providers remained unchanged (P = .32). CONCLUSION: In this large single-institution retrospective study, implementation of an EHR order for ethics consultation requests was associated with a significant increase in the number of consultation requests. Implementation of an electronic order may decrease barriers to ethics consultation in diverse practice settings. Further longitudinal, multicenter studies are needed to assess strategies to improve access to clinical ethics consultation for oncology patients.

Preferences in Oncology History Documentation Styles Among Clinical Practitioners.

PURPOSE: Clinical notes function as the de facto handoff between providers and assume great importance during unplanned medical encounters. An organized and thorough oncology history is essential in care coordination. We sought to understand reader preferences for oncology history organization by comparing between chronologic and narrative formats. METHODS: A convenience sample of 562 clinicians from 19 National Comprehensive Cancer Network Member Institutions responded to a survey comparing two formats of oncology histories, narrative and chronologic, for the same patient. Both histories were consensus-derived real-world examples. Each history was evaluated using semantic differential attributes (thorough, useful, organized, comprehensible, and succinct). Respondents choose a preference between the two styles for history gathering and as the basis of a new note. Open-ended responses were also solicited. RESULTS: Respondents preferred the chronologic over the narrative history to prepare for a visit with an unknown patient (66% preference) and as a basis for their own note preparation (77% preference) (P < .01). The chronologic summary was preferred in four of the five measured attributes (useful, organized, comprehensible, and succinct); the narrative summary was favored for thoroughness (P < .01). Open-ended responses reflected the attribute scoring and noted the utility of content describing social determinants of health in the narrative history. CONCLUSION: Respondents of this convenience sample preferred a chronologic oncology history to a concise narrative history. Further studies are needed to determine the optimal structure and content of chronologic documentation for oncology patients and the provider effort to use this format.

Identifying Nonpatient Authors of Patient Portal Secure Messages in Oncology: A Proof-of-Concept Demonstration of Natural Language Processing Methods.

PURPOSE: Patient portal secure messages are not always authored by the patient account holder. Understanding who authored the message is particularly important in an oncology setting where symptom reporting is crucial to patient treatment. Natural language processing has the potential to detect messages not authored by the patient automatically. METHODS: Patient portal secure messages from the Memorial Sloan Kettering Cancer Center were retrieved and manually annotated as a predicted unregistered proxy (ie, not written by the patient) or a presumed patient. After randomly splitting the annotated messages into training and test sets in a 70:30 ratio, a bag-of-words approach was used to extract features and then a Least Absolute Shrinkage and Selection Operator (LASSO) model was trained and used for classification. RESULTS: Portal secure messages (n = 2,000) were randomly selected from unique patient accounts and manually annotated. We excluded 335 messages from the data set as the annotators could not determine if they were written by a patient or proxy. Using the remaining 1,665 messages, a LASSO model was developed that achieved an area under the curve of 0.932 and an area under the precision recall curve of 0.748. The sensitivity and specificity related to classifying true-positive cases (predicted unregistered proxy-authored messages) and true negatives (presumed patient-authored messages) were 0.681 and 0.960, respectively. CONCLUSION: Our work demonstrates the feasibility of using unstructured, heterogenous patient portal secure messages to determine portal secure message authorship. Identifying patient authorship in real time can improve patient portal account security and can be used to improve the quality of the information extracted from the patient portal, such as patient-reported outcomes.

Content overlap in nurse and physician handoff artifacts and the potential role of electronic health records: a systematic review.

PURPOSE: The aims of this systematic review were: (1) to analyze the content overlap between nurse and physician hospital-based handoff documentation for the purpose of developing a list of interdisciplinary handoff information for use in the future development of shared and tailored computer-based handoff tools, and (2) to evaluate the utility of the Continuity of Care Document (CCD) standard as a framework for organizing hospital-based handoff information for use in electronic health records (EHRs). METHODS: We searched PubMed for studies published through July 2010 containing the indexed terms: handoff(s), hand-off, handover(s), shift-report, shift report, signout, and sign-out. Original, hospital-based studies of acute care nursing or physician handoff were included. Handoff information content was organized into lists of nursing, physician, and interdisciplinary handoff information elements. These information element lists were organized using CCD sections, with additional sections being added as needed. RESULTS: Analysis of 36 studies resulted in a total of 95 handoff information elements. Forty-six percent (44/95) of the information overlapped between the nurse and physician handoff lists. Thirty-six percent (34/95) were specific to the nursing list and 18% (17/95) were specific to the physician list. The CCD standard was useful for categorizing 80% of the terms in the lists and 12 category names were developed for the remaining 20%. CONCLUSION: Standardized interdisciplinary, nursing-specific, and physician-specific handoff information elements that are organized around the CCD standard and incorporated into EHRs in a structured narrative format may increase the consistency of data shared across all handoffs, facilitate the establishment of common ground, and increase interdisciplinary communication.

Interfaces for collecting data from patients: 10 golden rules.

Memorial Sloan Kettering Cancer Center has more than a decade's experience creating online interfaces for obtaining data from patients as part of routine clinical care. We have developed a set of "golden rules" for design of these interfaces. Many relate to the knowledge imbalance between professional staff (whether medical or informatics) and patients, who are often old and sick and have limited knowledge of technology. Others relate to the clinical nature of the encounter: data cannot be taken from patients as part of clinical care unless there is a plan to act on whatever information is prepared. We also note that the plethora of marketing questionnaires makes patients suspicious of surveys: patient trust is hard to gain and easy to lose. Addition of these golden rules to standard approaches to interface design will maximize our ability to obtain data from patients and thus improve communication between patients and clinicians.

Methods for building sense inventories of abbreviations in clinical notes.

OBJECTIVE: To develop methods for building corpus-specific sense inventories of abbreviations occurring in clinical documents. DESIGN: A corpus of internal medicine admission notes was collected and instances of each clinical abbreviation in the corpus were clustered to different sense clusters. One instance from each cluster was manually annotated to generate a final list of senses. Two clustering-based methods (Expectation Maximization--EM and Farthest First--FF) and one random sampling method for sense detection were evaluated using a set of 12 clinical abbreviations. MEASUREMENTS: The clustering-based sense detection methods were evaluated using a set of clinical abbreviations that were manually sense annotated. "Sense Completeness" and "Annotation Cost" were used to measure the performance of different methods. Clustering error rates were also reported for different clustering algorithms. RESULTS: A clustering-based semi-automated method was developed to build corpus-specific sense inventories for abbreviations in hospital admission notes. Evaluation demonstrated that this method could largely reduce manual annotation cost and increase the completeness of sense inventories when compared with a manual annotation method using random samples. CONCLUSION: The authors developed an effective clustering-based method for building corpus-specific sense inventories for abbreviations in a clinical corpus. To the best of the authors knowledge, this is the first time clustering technologies have been used to help building sense inventories of abbreviations in clinical text. The results demonstrated that the clustering-based method performed better than the manual annotation method using random samples for the task of building sense inventories of clinical abbreviations.

Iterative evaluation of the Health Level 7--Logical Observation Identifiers Names and Codes Clinical Document Ontology for representing clinical document names: a case report.

The authors summarize their experience in iteratively testing the adequacy of three versions of the Health Level Seven (HL7) Logical Observation Identifiers Names and Codes (LOINC) Clinical Document Ontology (CDO) to represent document names at Columbia University Medical Center. The percentage of documents fully represented increased from 23.4% (Version 1) to 98.5% (Version 3). The proportion of unique representations increased from 7.9% (Analysis 1) to 39.4% (Analysis 4); the proportion reflects the level of specificity in the document names as well as the completeness and level of granularity of the CDO. The authors shared the findings of each analysis with the Clinical LOINC committee and participated in the decision-making regarding changes to the CDO on the basis of those analyses and those conducted by the Department of Veterans Affairs. The authors encourage other institutions to actively engage in testing healthcare standards and participating in standards development activities to increase the likelihood that the evolving standards will meet institutional needs.

Development and evaluation of nursing user interface screens using multiple methods.

Building upon the foundation of the Structured Narrative Electronic Health Record (EHR) model, we applied theory-based (combined Technology Acceptance Model and Task-Technology Fit Model) and user-centered methods to explore nurses' perceptions of functional requirements for an electronic nursing documentation system, design user interface screens reflective of the nurses' perspectives, and assess nurses' perceptions of the usability of the prototype user interface screens. The methods resulted in user interface screens that were perceived to be easy to use, potentially useful, and well-matched to nursing documentation tasks associated with Nursing Admission Assessment, Blood Administration, and Nursing Discharge Summary. The methods applied in this research may serve as a guide for others wishing to implement user-centered processes to develop or extend EHR systems. In addition, some of the insights obtained in this study may be informative to the development of safe and efficient user interface screens for nursing document templates in EHRs.

Sociotechnical analysis of a neonatal ICU.

Sociotechnical theory has been used to inform the development of computer systems in the complex and dynamic environment of healthcare. The key components of the sociotechnical system are the workers, their practices, their mental models, their interactions, and the tools used in the work process. We conducted a sociotechnical analysis of a neonatal intensive care unit towards the development of decision support for antimicrobial prescribing. We found that the core task was to save the baby in the face of complex and often incomplete information. Organizational climate characteristics were pride in clinical and educational practice. In addition, the structure of work identified interdisciplinary teamwork with some communication breakdown and interruptive work environment. Overall, sociotechnical analysis provided a solid method to understand work environment during the decision support development process.

Frequency and types of alerts for antibiotic prescribing in a neonatal ICU.

Sepsis in neonates is a significant problem that carries with it severe morbidity and mortality. Managing antibiotics in this population is therefore an important issue. We studied clinical alerts currently in place to support antibiotic prescribing in a neonatal intensive care unit in order to ensure that appropriate information is being provided in a way that is consistent with current recommendations. Data were obtained from our alerts tracking database. Alerts were described according to triggering orders and clinician recipients. We found that alerts most commonly associated with antibiotics are providing critical information regarding lab results and patient factors necessary in preventing adverse effects of these drugs. Clinician recipients of alerts are those responsible for entering orders and the information is being provided at the point of care.

Ambulatory cancer care electronic symptom self-reporting (ACCESS) for surgical patients: a randomised controlled trial protocol.

INTRODUCTION: An increasing proportion of cancer surgeries are ambulatory procedures requiring a stay of 1 day or less in the hospital. Providing patients and their caregivers with ongoing, real-time support after discharge aids delivery of high-quality postoperative care in this new healthcare environment. Despite abundant evidence that patient self-reporting of symptoms improves quality of care, the most effective way to monitor and manage this self-reported information is not known. METHODS AND ANALYSIS: This is a two-armed randomised, controlled trial evaluating two approaches to the management of patient-reported data: (1) team monitoring, symptom monitoring by the clinical team, with nursing outreach if symptoms exceed normal limits, and (2) enhanced feedback, real-time feedback to patients about expected symptom severity, with patient-activated care as needed.Patients with breast, gynaecologic, urologic, and head and neck cancer undergoing ambulatory cancer surgery (n=2750) complete an electronic survey for up to 30 days after surgery that includes items from a validated instrument developed by the National Cancer Institute, the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Information provided to patients in the Enhanced Feedback group is procedure-specific and based on updated PRO-CTCAE data from previous patients. Qualitative interviews are also performed. The primary study outcomes assess unplanned emergency department visits and symptom-triggered interventions (eg, nursing calls and pain management referrals) within 30 days, and secondary outcomes assess the patient and caregiver experience (ie, patient engagement, patient anxiety and caregiver burden). ETHICS AND DISSEMINATION: This study is approved by the Institutional Review Board at Memorial Sloan Kettering Cancer Center. The relationships between the study team and stakeholders will be leveraged to disseminate study findings. Findings will be relevant in designing future coordinated care models targeting improved healthcare quality and patient experience. TRIAL REGISTRATION NUMBER: NCT03178045.

Real-World Outcomes of an Automated Physician Support System for Genome-Driven Oncology.

PURPOSE: Matching patients to investigational therapies requires new tools to support physician decision making. We designed and implemented Precision Insight Support Engine (PRECISE), an automated, just-in-time, clinical-grade informatics platform to identify and dynamically track patients on the basis of molecular and clinical criteria. Real-world use of this tool was analyzed to determine whether PRECISE facilitated enrollment to early-phase, genome-driven trials. MATERIALS AND METHODS: We analyzed patients who were enrolled in genome-driven, early-phase trials using PRECISE at Memorial Sloan Kettering Cancer Center between April 2014 and January 2018. Primary end point was the proportion of enrolled patients who were successfully identified using PRECISE before enrollment. Secondary end points included time from sequencing and PRECISE identification to enrollment. Reasons for a failure to identify genomically matched patients were also explored. RESULTS: Data were analyzed from 41 therapeutic trials led by 19 principal investigators. In total, 755 patients were accrued to these studies during the period that PRECISE was used. PRECISE successfully identified 327 patients (43%) before enrollment. Patients were diagnosed with 29 tumor types and harbored alterations in 43 oncogenes, most commonly ERBB2 (21.3%), PIK3CA (14.1%), and BRAF (8.7%). Median time from sequencing to enrollment was 163 days (interquartile range, 66 to 357 days), and from PRECISE identification to enrollment 87 days (interquartile range, 37 to 180 days). Common reasons for failing to identify patients before enrollment included accrual on the basis of molecular alterations that did not match pre-established PRECISE genomic eligibility (140 [33%] of 428) and external sequencing not available for parsing (127 [30%] of 428). CONCLUSION: PRECISE identified 43% of all patients accrued to a diverse cohort of early-phase, genome-matched studies. Purpose-built informatics platforms represent a novel and potentially effective method for matching patients to molecularly selected studies.

Preliminary development of the physician documentation quality instrument.

OBJECTIVES: This study sought to design and validate a reliable instrument to assess the quality of physician documentation. DESIGN: Adjectives describing clinician attitudes about high-quality clinical documentation were gathered through literature review, assessed by clinical experts, and transformed into a semantic differential scale. Using the scale, physicians and nurse practitioners scored the importance of the adjectives for describing quality in three note types: admission, progress, and discharge notes. Psychometric methods including exploratory factor analysis were applied to provide preliminary evidence for the construct validity and internal consistency reliability. RESULTS: A 22-item Physician Documentation Quality Instrument (PDQI) was developed. Exploratory factor analysis (n = 67 clinician respondents) on three note types resulted in solutions ranging from four (discharge) to six (admission and progress) factors, and explained 65.8% (discharge) to 73% (admission and progress) of the variance. Each factor solution was unique. However, four sets of items consistently factored together across all note types: (1) up-to-date and current; (2) brief, concise, succinct; (3) organized and structured; and (4) correct, comprehensible, consistent. Internal consistency reliabilities were: admission note (factor scales = 0.52-88, overall = 0.86), progress note (factor scales = 0.59-0.84, overall = 0.87), and discharge summary (factor scales = 0.76-0.85, overall = 0.88). CONCLUSION: The exploratory factor analyses and reliability analyses provide preliminary evidence for the construct validity and internal consistency reliability of the PDQI. Two novel dimensions of the construct for document quality were developed related to form (Well-formed, Compact). Additional work is needed to assess intrarater and interrater reliability of applying of the proposed instrument and to examine the reproducibility of the factors in other samples.