The burden of diagnostic error in dentistry: A study on periodontal disease misclassification.

BACKGROUND: Periodontal disease constitutes a widely prevalent category of non-communicable diseases and ranks among the top 10 causes of disability worldwide. Little however is known about diagnostic errors in dentistry. In this work, by retrospectively deploying an electronic health record (EHR)-based trigger tool, followed by gold standard manual review, we provide epidemiological estimates on the rate of diagnostic misclassification in dentistry through a periodontal use case. METHODS: An EHR-based trigger tool (a retrospective record review instrument that uses a list of triggers (or clues), i.e., data elements within the health record, to alert reviewers to the potential presence of a wrong diagnosis) was developed, tested and run against the EHR at the two participating sites to flag all cases having a potential misdiagnosis. All cases flagged as potentially misdiagnosed underwent extensive manual reviews by two calibrated domain experts. A subset of the non-flagged cases was also manually reviewed. RESULTS: A total of 2,262 patient charts met the study's inclusion criteria. Of these, the algorithm flagged 1,124 cases as potentially misclassified and 1,138 cases as potentially correctly diagnosed. When the algorithm identified a case as potentially misclassified, compared to the diagnosis assigned by the gold standard, the kappa statistic was 0.01. However, for cases the algorithm marked as potentially correctly diagnosed, the review against the gold standard showed a kappa statistic of 0.9, indicating near perfect agreement. The observed proportion of diagnostic misclassification was 32 %. There was no significant difference by clinic or provider characteristics. CONCLUSION: Our work revealed that about a third of periodontal cases are misclassified. Diagnostic errors have been reported to happen more frequently than other types of errors, and to be more preventable. Benchmarking diagnostic quality is a first step. Subsequent research endeavor will delve into comprehending the factors that contribute to diagnostic errors in dentistry and instituting measures to prevent them. CLINICAL SIGNIFICANCE: This study sheds light on the significance of diagnostic excellence in the delivery of dental care, and highlights the potential role of technology in aiding diagnostic decision-making at the point of care.

The Nature of Adverse Events in Dentistry.

OBJECTIVES: Learning from clinical data on the subject of safety with regards to patient care in dentistry is still largely in its infancy. Current evidence does not provide epidemiological estimates on adverse events (AEs) associated with dental care. The goal of the dental practice study was to quantify and describe the nature and severity of harm experienced in association with dental care, and to assess for disparities in the prevalence of AEs. METHODS: Through a multistaged sampling procedure, we conducted in-depth retrospective review of patients' dental and medical records. RESULTS: We discovered an AE proportion of 1.4% (95% CI, 1.1% to 1.8). At least two-thirds of the detected AEs were preventable. Eight percent of patients who experienced harm due to a dental treatment presented only to their physician and not to the dentist where they originally received care. CONCLUSIONS: Although most studies of AEs have focused on hospital settings, our results show that they also occur in ambulatory care settings. Extrapolating our data, annually, at least 3.3 million Americans experience harm in relation to outpatient dental care, of which over 2 million may be associated with an error. PRACTICAL IMPLICATIONS: Measurement is foundational in enabling learning and improvement. A critical first step in preventing errors and iatrogenic harm in dentistry is to understand how often these safety incidents occur, what type of incidents occur, and what the consequences are in terms of patient suffering, and cost to the healthcare system.

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities.

OBJECTIVES: This work describes the process by which the quality of electronic health care data for a public health study was determined. The objectives were to adapt, develop, and implement data quality assessments (DQAs) based on the National Institutes of Health Pragmatic Trials Collaboratory (NIHPTC) data quality framework within the three domains of completeness, accuracy, and consistency, for an investigation into oral health care disparities of a preventive care program. METHODS: Electronic health record data for eligible children in a dental accountable care organization of 30 offices, in Oregon, were extracted iteratively from January 1, 2014, through March 31, 2022. Baseline eligibility criteria included: children ages 0-18 with a baseline examination, Oregon home address, and either Medicaid or commercial dental benefits at least once between 2014 and 2108. Using the NIHPTC framework as a guide, DQAs were conducted throughout data element identification, extraction, staging, profiling, review, and documentation. RESULTS: The data set included 91,487 subjects, 11 data tables comprising 75 data variables (columns), with a total of 6,861,525 data elements. Data completeness was 97.2%, the accuracy of EHR data elements in extracts was 100%, and consistency between offices was strong; 29 of 30 offices within 2 standard deviations of the mean (s = 94%). CONCLUSIONS: The NIHPTC framework proved to be a useful approach, to identify, document, and characterize the dataset. The concepts of completeness, accuracy, and consistency were adapted by the multidisciplinary research team and the overall quality of the data are demonstrated to be of high quality.

Learning from data in dentistry: Summary of the third annual OpenWide conference.

The overarching goal of the third scientific oral health symposium was to introduce the concept of a learning health system to the dental community and to identify and discuss cutting-edge research and strategies using data for improving the quality of dental care and patient safety. Conference participants included clinically active dentists, dental researchers, quality improvement experts, informaticians, insurers, EHR vendors/developers, and members of dental professional organizations and dental service organizations. This report summarizes the main outputs of the third annual OpenWide conference held in Houston, Texas, on October 12, 2022, as an affiliated meeting of the American Dental Association (ADA) 2022 annual conference.

Improvements in appropriate placement of dental sealants after implementation of a clinical decision support system.

BACKGROUND: Dental sealants are effective for the prevention of caries in children at elevated risk levels, and increasing the proportion of children and adolescents who have dental sealants on 1 or more molars is a Healthy People 2030 objective. Electronic health record (EHR)-based clinical decision support systems (CDSSs) have the ability to improve patient care. A dental quality measure related to dental sealant placement for children at elevated risk of caries was targeted for improvement using a CDSS. METHODS: A validated dental quality measure was adapted to assess a patient's need for dental sealant placement. A CDSS was implemented to advise care team members whether a child was at elevated risk of developing caries and had sealant-eligible first or second molars. Data on dental sealant placement at examination visits during a 5-year period were analyzed, including 32 months before CDSS implementation and 28 months after CDSS implementation. RESULTS: From January 1, 2018, through December 31, 2022, the authors assessed 59,047 examination visits for children at elevated risk of developing caries and with sealant-eligible teeth. With the implementation of a CDSS and training to support the clinical care team members in September 2020, the appropriate placement of dental sealants at examination visits increased from 27% through 60% (P < .00001). CONCLUSIONS: Integration of a CDSS into the EHR as part of a quality improvement program was effective in increasing the delivery of sealants in eligible first and second molars of children aged 5 through 15 years and considered at high risk of developing caries. PRACTICAL IMPLICATIONS: An EHR-based CDSS can be implemented to improve standardization and provide timely and appropriate patient care in dental practices.

Evaluating and improving the usability of a mHealth platform to assess postoperative dental pain.

Objectives: The use of interactive mobile health (mHealth) applications to monitor patient-reported postoperative pain outcomes is an emerging area in dentistry that requires further exploration. This study aimed to evaluate and improve the usability of an existing mHealth application. Materials and methods: The usability of the application was assessed iteratively using a 3-phase approach, including a rapid cognitive walkthrough (Phase I), lab-based usability testing (Phase II), and in situ pilot testing (Phase III). The study team conducted Phase I, while providers and patients participated in Phase II and III. Results: The rapid cognitive walkthrough identified 23 potential issues that could negatively impact user experience, with the majority classified as system issues. The lab-based usability testing yielded 141 usability issues.; 43% encountered by patients and 57% by dentists. Usability problems encountered during pilot testing included undelivered messages due to mobile phone carrier and service-related issues, errors in patients' phone number data entry, and problems in provider training. Discussion: Through collaborative and iterative work with the vendor, usability issues were addressed before launching a trial to assess its efficacy. Conclusion: The usability of the mHealth application for postoperative dental pain was remarkably improved by the iterative analysis and interdisciplinary collaboration.

Identifying predictors of the tooth loss phenotype in a large periodontitis patient cohort using a machine learning approach.

OBJECTIVES: This study aimed to identify predictors associated with the tooth loss phenotype in a large periodontitis patient cohort in the university setting. METHODS: Information on periodontitis patients and nineteen factors identified at the initial visit was extracted from electronic health records. The primary outcome is tooth loss phenotype (presence or absence of tooth loss). Prediction models were built on significant factors (single or combinatory) selected by the RuleFit algorithm, and these factors were further adopted by regression models. Model performance was evaluated by Area Under the Receiver Operating Characteristic Curve (AUROC) and Area Under the Precision-Recall Curve (AUPRC). Associations between predictors and the tooth loss phenotype were also evaluated by classical statistical approaches to validate the performance of machine learning models. RESULTS: In total, 7840 patients were included. The machine learning model predicting the tooth loss phenotype achieved AUROC of 0.71 and AUPRC of 0.66. Age, periodontal diagnosis, number of missing teeth at baseline, furcation involvement, and tooth mobility were associated with the tooth loss phenotype in both machine learning and classical statistical models. CONCLUSIONS: The rule-based machine learning approach improves model explainability compared to classical statistical methods. However, the model's generalizability needs to be further validated by external datasets. CLINICAL SIGNIFICANCE: Predictors identified by the current machine learning approach using the RuleFit algorithm had clinically relevant thresholds in predicting the tooth loss phenotype in a large and diverse periodontitis patient cohort. The results of this study will assist clinicians in performing risk assessment for periodontitis at the initial visit.

Patients' Experiences of Dental Diagnostic Failures: A Qualitative Study Using Social Media.

OBJECTIVE: Despite the many advancements made in patient safety over the past decade, combating diagnostic errors (DEs) remains a crucial, yet understudied initiative toward improvement. This study sought to understand the perception of dental patients who have experienced a dental diagnostic failure (DDF) and to identify patient-centered strategies to help reduce future occurrences of DDF. METHODS: Through social media recruitment, we conducted a screening survey, initial assessment, and 67 individual patient interviews to capture the effects of misdiagnosis, missed diagnosis, or delayed diagnosis on patient lives. Audio recordings of patient interviews were transcribed, and a hybrid thematic analysis approach was used to capture details about 4 main domains of interest: the patient's DDF experience, contributing factors, impact, and strategies to mitigate future occurrences. RESULTS: Dental patients endured prolonged suffering, disease progression, unnecessary treatments, and the development of new symptoms as a result of experiencing DE. Poor provider communication, inadequate time with provider, and lack of patient self-advocacy and health literacy were among the top attributes patients believed contributed to the development of a DE. Patients suggested that improvements in provider chairside manners, more detailed patient diagnostic workups, and improving personal self-advocacy; along with enhanced reporting systems, could help mitigate future DE. CONCLUSIONS: This study demonstrates the valuable insight the patient perspective provides in understanding DEs, therefore aiding the development of strategies to help reduce the occurrences of future DDF events. Given the challenges patients expressed, there is a significant need to create an accessible reporting system that fosters constructive clinician learning.

Patient and dentist perspectives on collecting patient reported outcomes after painful dental procedures in the National Dental PBRN.

BACKGROUND: Dental Patient Reported Outcomes (PROs) relate to a dental patient's subjective experience of their oral health. How practitioners and patients value PROs influences their successful use in practice. METHODS: Semi-structured interviews were conducted with 22 practitioners and 32 patients who provided feedback on using a mobile health (mHealth) platform to collect the pain experience after dental procedures. A themes analysis was conducted to identify implementation barriers and facilitators. RESULTS: Five themes were uncovered: (1) Sense of Better Care. (2) Tailored Follow-up based on the dental procedure and patient's pain experience. (3) Effective Messaging and Alerts. (4) Usable Digital Platform. (5) Routine mHealth Integration. CONCLUSION: Frequent automated and preferably tailored follow-up messages using an mHealth platform provided a positive care experience for patients, while providers felt it saved them time and effort. Patients thought that the mHealth questionnaires were well-developed and of appropriate length. The mHealth platform itself was perceived as user-friendly by users, and most would like to continue using it. PRACTICAL IMPLICATIONS: Patients are prepared to use mobile phones to report their pain experience after dental procedures. Practitioners will be able to close the post-operative communication gap with their patients, with little interruption of their workflow.

Development and validation of a rule-based algorithm to identify periodontal diagnosis using structured electronic health record data.

AIM: To develop and validate an automated electronic health record (EHR)-based algorithm to suggest a periodontal diagnosis based on the 2017 World Workshop on the Classification of Periodontal Diseases and Conditions. MATERIALS AND METHODS: Using material published from the 2017 World Workshop, a tool was iteratively developed to suggest a periodontal diagnosis based on clinical data within the EHR. Pertinent clinical data included clinical attachment level (CAL), gingival margin to cemento-enamel junction distance, probing depth, furcation involvement (if present) and mobility. Chart reviews were conducted to confirm the algorithm's ability to accurately extract clinical data from the EHR, and then to test its ability to suggest an accurate diagnosis. Subsequently, refinements were made to address limitations of the data and specific clinical situations. Each refinement was evaluated through chart reviews by expert periodontists at the study sites. RESULTS: Three-hundred and twenty-three charts were manually reviewed, and a periodontal diagnosis (healthy, gingivitis or periodontitis including stage and grade) was made by expert periodontists for each case. After developing the initial version of the algorithm using the unmodified 2017 World Workshop criteria, accuracy was 71.8% for stage alone and 64.7% for stage and grade. Subsequently, 16 modifications to the algorithm were proposed and 14 were accepted. This refined version of the algorithm had 79.6% accuracy for stage alone and 68.8% for stage and grade together. CONCLUSIONS: Our findings suggest that a rule-based algorithm for suggesting a periodontal diagnosis using EHR data can be implemented with moderate accuracy in support of chairside clinical diagnostic decision making, especially for inexperienced clinicians. Grey-zone cases still exist, where clinical judgement will be required. Future applications of similar algorithms with improved performance will depend upon the quality (completeness/accuracy) of EHR data.

Use GPT-J Prompt Generation with RoBERTa for NER Models on Diagnosis Extraction of Periodontal Diagnosis from Electronic Dental Records.

This study explored the usability of prompt generation on named entity recognition (NER) tasks and the performance in different settings of the prompt. The prompt generation by GPT-J models was utilized to directly test the gold standard as well as to generate the seed and further fed to the RoBERTa model with the spaCy package. In the direct test, a lower ratio of negative examples with higher numbers of examples in prompt achieved the best results with a F1 score of 0.72. The performance revealed consistency, 0.92-0.97 in the F1 score, in all settings after training with the RoBERTa model. The study highlighted the importance of seed quality rather than quantity in feeding NER models. This research reports on an efficient and accurate way to mine clinical notes for periodontal diagnoses, allowing researchers to easily and quickly build a NER model with the prompt generation approach.