Comparison of mortality prediction models for road traffic accidents: an ensemble technique for imbalanced data.

BACKGROUND: Injuries caused by RTA are classified under the International Classification of Diseases-10 as 'S00-T99' and represent imbalanced samples with a mortality rate of only 1.2% among all RTA victims. To predict the characteristics of external causes of road traffic accident (RTA) injuries and mortality, we compared performances based on differences in the correction and classification techniques for imbalanced samples. METHODS: The present study extracted and utilized data spanning over a 5-year period (2013-2017) from the Korean National Hospital Discharge In-depth Injury Survey (KNHDS), a national level survey conducted by the Korea Disease Control and Prevention Agency, A total of eight variables were used in the prediction, including patient, accident, and injury/disease characteristics. As the data was imbalanced, a sample consisting of only severe injuries was constructed and compared against the total sample. Considering the characteristics of the samples, preprocessing was performed in the study. The samples were standardized first, considering that they contained many variables with different units. Among the ensemble techniques for classification, the present study utilized Random Forest, Extra-Trees, and XGBoost. Four different over- and under-sampling techniques were used to compare the performance of algorithms using "accuracy", "precision", "recall", "F1", and "MCC". RESULTS: The results showed that among the prediction techniques, XGBoost had the best performance. While the synthetic minority oversampling technique (SMOTE), a type of over-sampling, also demonstrated a certain level of performance, under-sampling was the most superior. Overall, prediction by the XGBoost model with samples using SMOTE produced the best results. CONCLUSION: This study presented the results of an empirical comparison of the validity of sampling techniques and classification algorithms that affect the accuracy of imbalanced samples by combining two techniques. The findings could be used as reference data in classification analyses of imbalanced data in the medical field.

Descriptive epidemiology of cancer of unknown primary in South Korea, 1999-2017.

BACKGROUND: Cancers of unknown primary (CUPs) are tumors found after metastasizing from unidentified primary sites; these tumors generally have unknown treatment strategies, expected treatment results, and prognosis. We assessed the epidemiological characteristics of CUPs in Korea. METHODS: We extracted records for 1999 through 2017 from the Korea Central Cancer Registry using the International Statistical Classification of Diseases and Related Health Problems (10th revision) codes for CUP as defined by the International Agency for Research on Cancer. Age-standardized rates and relative survival rates were calculated. RESULTS: The CUPs constituted 2.1 % of the total number of cancer registrations in 1999, declining to 0.7 % in 2017. The incidence rate decreased for both sexes (5.35 to 2.20 for men, 3.15 to 1.77 for women). Patients aged 80 years and older had the highest incidence rate at 40.2, and 86.3 % of CUPs occurred in those 50 years of age or older. The cases of retroperitoneum and peritoneum sites increased over time. Cases diagnosed by microscopic methods and death certification only were 62.3 % and 7.9 %, respectively. The malignant neoplasm of the retroperitoneum and peritoneum and unknown primary site had the highest and lowest survival rates, respectively. The 5-year relative survival rate increased over time from 14.2 % (1999-2002) to 27.3 % (2013-2017). CONCLUSIONS: Our analysis of data from the Korea Central Cancer Registry found decreasing rates of CUP, although with consistent disparities by patient age and sex. Advancements in diagnostic technology may be decreasing the number of CUP diagnoses. Expanding the amount of information recorded in the registry may further improve diagnostic accuracy.

Comparison of Prediction Models for Mortality Related to Injuries from Road Traffic Accidents after Correcting for Undersampling.

In this study, four models-logistic regression (LR), random forest (RF), linear support vector machine (SVM), and radial basis function (RBF)-SVM-were compared for their accuracy in determining mortality caused by road traffic injuries. They were tested using five years of national-level data from the Korea Disease Control and Prevention Agency's (KDCA) National Hospital Discharge In-Depth Survey (2013 through to 2017). Model performance was measured for accuracy, precision, recall, F1 score, and Brier score metrics using classification analysis that included characteristics of patients, accidents, injuries, and illnesses. Due to the number of variables and differing units, the rates of survival and mortality related to road traffic accidents were imbalanced, so the data was corrected and standardized before the classification models' performances were compared. Using the importance analysis, the main diagnosis, the type of injury, the site of the injury, the type of injury, the operation status, the type of accident, the role at the time of the accident, and the sex were selected as the analysis factors. The biggest contributing factor was the role in the accident, which is the driver, and the major sites of the injuries were head injuries and deep injuries. Using selected factors, comparisons of the classification performance of each model indicated RBF-SVM and RF models were superior to the others. Of the SVM models, the RBF kernel model was superior to the linear kernel model; it can be inferred that the performance of the high-dimensional transformed RBF model is superior when the dimension is complex because of the use of multiple variables. The findings suggest there are limitations to analyses involving imbalanced, multidimensional original data, such as data on road traffic mortality. Thus, analyses must be performed after imbalances are corrected.

Comparing Logistic Regression Models with Alternative Machine Learning Methods to Predict the Risk of Drug Intoxication Mortality.

(1) Medical research has shown an increasing interest in machine learning, permitting massive multivariate data analysis. Thus, we developed drug intoxication mortality prediction models, and compared machine learning models and traditional logistic regression. (2) Categorized as drug intoxication, 8,937 samples were extracted from the Korea Centers for Disease Control and Prevention (2008-2017). We trained, validated, and tested each model through data and compared their performance using three measures: Brier score, calibration slope, and calibration-in-the-large. (3) A chi-square test demonstrated that mortality risk statistically significantly differed according to severity, intent, toxic substance, age, and sex. The multilayer perceptron model (MLP) had the highest area under the curve (AUC), and lowest Brier score in training and validation phases, while the logistic regression model (LR) showed the highest AUC (0.827) and lowest Brier score (0.0307) in the testing phase. MLP also had the second-highest AUC (0.816) and second-lowest Brier score (0.003258) in the testing phase, demonstrating better performance than the decision-making tree model. (4) Given the complexity of choosing tuning parameters, LR proved competitive when using medical datasets, which require strict accuracy.

Survey of Potential Cancer Registry Items to Enhance the Usefulness of Cancer Registry Data.

BACKGROUND: Cancer registry data can help plan for cancer services and to identify where further progress is needed, in order to improve the lives of patients with cancer. This study investigated the possibility of collecting additional information and the priority of the information by examining other cancer registry items. We aimed to suggest additional data items to be collected to enhance the usefulness of cancer registry data. METHODS: We examined items that could potentially be added by comparing the cancer registration items in five foreign registries and large hospitals in Korea. Based on the foreign and domestic hospital cancer registry data, a questionnaire survey was administered to 272 cancer registry workers nationwide and 10 cancer experts to investigate the possibility of expanding the variables. The proportion and rank of each item were analyzed. RESULTS: There were similar items for demographic information and cancer diagnosis between foreign cancer registries and the Korea Central Cancer Registry (KCCR). However, the KCCR had fewer items for staging, treatment, and follow-up. There were 29 items to be collected with high priority. Items under demographic information included date of birth, race and country of birth. Items for cancer diagnosis included type of cancer, smoking history and type of pathologic test. Treatment information included the date of treatment, chemotherapy and radiation. Items under the stage and prognostic factors included TNM stage, collaborative stage, and comorbidities. Finally, items under follow-up information included survival, cancer state and recurrence information. CONCLUSIONS: Cancer registration workers and cancer experts generally agreed on the need to expand the essential items for cancer registration. The findings of this study will be useful for devising plans to expand cancer registration items.

Qualitative Improvement Methods Through Analysis of Inquiry Contents for Cancer Registration

Background: In Korea, the national cancer database was constructed after the initiation of the national cancer registration project in 1980, and the annual national cancer registration report has been published every year since 2005. Consequently, data management must begin even at the stage of data collection in order to ensure quality. Objectives: To determine the suitability of cancer registries' inquiry tools through the inquiry analysis of the Korea Central Cancer Registry (KCCR), and identify the needs to improve the quality of cancer registration. Methods: Results of 721 inquiries to the KCCR from 2000 to 2014 were analyzed by inquiry year, question type, and medical institution characteristics. Using Stata version 14.1, descriptive analysis was performed to identify general participant characteristics, and chi-square analysis was applied to investigate significant differences in distribution characteristics by factors affecting the quality of cancer registration data. Results: The number of inquiries increased in 2005­2009. During this period, there were various changes, including the addition of cancer registration items such as brain tumors and guideline updates. Of the inquirers, 65.3% worked at hospitals in metropolitan cities and 60.89% of hospitals had 601­1000 beds. Tertiary hospitals had the highest number of inquiries (64.91%), and the highest number of questions by type were 353 (48.96%) for histological codes, 92 (12.76%) for primary sites, and 76 (10.54%) for reportable. Conclusions: A cancer registration inquiry system is an effective method when not confident about codes during cancer registration, or when confronting cancer cases in which previous clinical knowledge or information on the cancer registration guidelines are insufficient.

Cancer Registration Manual Understanding by Medical Record Administrators

Background: This study examined: (a) whether medical record administrators (MRAs) properly understand and apply a cancer registration manual; (b) weaknesses in cancer registration: and (c) factors influencing the accuracy of cancer registration. Methods: MRAs dealing with cancer registration in 81 national member hospitals were asked by mail to answer 17 questions about cancer information among registration items. After calculating rates of correct and incorrect answers to each question, an analysis was conducted of whether they were related to particular factors (hospital type, hospital location, number of hospital beds, experience of cancer registration, and experience of consulting cancer registration). Results: A total of 45.7% of the participants gave approximately 90% correct answers. The rates for correct answers about general rules were under 90% for objectives of cancer registration and treatment. The rate concerning histologic type was over 90%, while few correct answers exceeded 50% to questions concerning newly revised rules for the colorectal system. For questions about initial treatment, date of diagnosis, and method of final diagnosis, as well as four questions on checking the validity of registered data, the correct answer rates were 70-90%. Regarding the features of hospitals and respondents, number of hospital beds and prior experience in cancer registration questions were found to contribute to the high rate of correct answers. Conclusion: To improve the accuracy of cancer registration, the manual needs to be supplemented in areas demonstrating low correct answer rates. Education opportunities and methods for MRAs should be diversified.

Stakeholder analysis for adopting a personal health record standard in Korea.

BACKGROUND: Interest in health information exchanges (HIEs) is increasing. Several countries have adopted core health data standards with appropriate strategies. OBJECTIVE: This study was conducted to determine the feasibility of a continuity of care record (CCR) as the standard for an electronic version of the official transfer note and the HIE in Korean healthcare. METHOD: A technical review of the CCR standard and analysis of stakeholders' views were undertaken. Transfer notes were reviewed and matched with CCR standard categories. The standard for the Korean coding system was selected. Stakeholder analysis included an online survey of members of the Korean Society of Medical Informatics, a public hearing to derive opinions of consumers, doctors, vendors, academic societies and policy makers about the policy process, and a focus group meeting with EMR vendors to determine which HIE objects were technically applicable. RESULTS: Data objects in the official transfer note form matched CCR standards. Korean Classification of Diseases, Korean Standard Terminology of Medicine, Electronic Data Interchange code (EDI code), Logical Observation Identifiers Names and Codes, and Korean drug codes (KD code) were recommended as the Korean coding standard.'Social history', 'payers', and 'encounters' were mostly marked as optional or unnecessary sections, and 'allergies', 'alerts', 'medication list', 'problems/diagnoses', 'results',and 'procedures' as mandatory. Unlike the US, 'social history' was considered optional and 'advance directives' mandatory.At the public hearing there was some objection from the Korean Medical Association to the HIE on legal grounds in termsof intellectual property and patients' personal information. Other groups showed positive or neutral responses. Focus group members divided CCR data objects into three phases based onpredicted adoption time in CCR: (i) immediate adoption; (ii) short-term adoption ('alerts', 'family history'); and (iii) long-term adoption ('results', 'advanced directives', 'functional status', 'medical equipment', 'vital signs', 'plan of care', 'social history'). CONCLUSION: There were no technical problems in generating the CCR standard document from EMRs. Matters of concern that arose from study results should be resolved with time and consultation.

Implementation plans for the Korean certified tumor registrar qualification system.

BACKGROUND: Cancer registration data is used to understand the nation's cancer burden, and to provide significant baseline data for cancer control efforts, as well as, research on cancer incidence, mortality, survival, and prevalence. A system that approves, assesses, and manages the qualification of specialists, responsible for performing cancer registration, has not been developed in Korea. This study presents ways to implement a certification system designed for the qualification of tumor registrars in Korea. MATERIALS AND METHODS: Requirements for implementing a certified tumor registrar qualification system were determined by reviewing the system for establishing qualifications in Korea and the American qualification system via the National Cancer Registrars Association (NCRA). Moreover, a survey was conducted on Korean medical records administrators, who had taken the U.S. Certified Tumor Registrar (CTR) examination, in order to review their opinions regarding these requirements. RESULTS: This study verified the feasibility of a qualification examination based on the opinions of CTR specialists by determining the following: items, and the associated ratings, of the qualifications necessary to register individuals as certified tumor registrars in a private qualification system; status of human resources required for the examination or training processes; plans regarding the organization needed for management, and operation of qualifications, examination standards, subject areas, examination methods, examination qualifications, or education and training programs. CONCLUSIONS: The implementation of a certified tumor registrar qualification system will lead to enhanced job competency for specialists and a qualitative improvement of cancer registration data. It will also reliably foster human resources that will lay the groundwork needed to establish scientific and reasonable national cancer management policies.

Improving accuracy and completeness in the collaborative staging system for stomach cancer in South Korea.

BACKGROUND: Cancer staging enables planning for the best treatments, evaluation of prognosis, and predictions for survival. The Collaborative Stage (CS) system makes it possible to significantly reduce the proportion of patients labeled at an "unknown" stage as well as discrepancies among different staging systems. This study aims to analyze the factors that influence the accuracy and validity of CS data. MATERIALS AND METHODS: Data were randomly selected (233 cases) from stomach cancer cases enrolled for CS survey at the Korea Central Cancer Registry. Two questionnaires were used to assess CS values for each case and to review the cancer registration environment for each hospital. Data were analyzed in terms of the relationships between the time spent for acquisition and registration of CS information, environments relating to cancer registration in the hospitals, and document sources of CS information for each item. RESULTS: The time for extracting and registering data was found to be shorter when the hospitals had prior experience gained from participating in a CS pilot study and when they were equipped with full-time cancer registrars. Evaluation of the CS information according to medical record sources found that the percentage of items missing for Site Specific Factor (SSF) was 30% higher than for other CS variables. Errors in CS coding were found in variables such as "CS Extension," "CS Lymph Nodes," "CS Metastasis at Diagnosis," and "SSF25 Involvement of Cardia and Distance from Esophagogastric Junction (EGJ)." CONCLUSIONS: To build CS system data that are reliable for cancer registration and clinical research, the following components are required: 1) training programs for medical records administrators; 2) supporting materials to promote active participation; and 3) format development to improve registration validity.

Analysis of questions regarding morbidity coding posted to the online coding clinic of the Korean Medical Record Association.

Accuracy and consistency in morbidity coding are important in both clinical research and practice. However,Health Information Managers (HIMs) sometimes face difficulties in assigning morbidity codes. To assist them,the Korean Medical Record Association operates an online coding clinic bulletin board, on which HIMs can post questions and receive answers. Frequency analysis and Fisher's exact testing were performed to identify differences among the types of questions posted and the characteristics of the HIMs who posted them. Through statistical analysis, it was found that HIMs working at hospitals with fewer than 500 beds and those with more than 10 years of work experience were found to post more questions than other HIMs. The study also identified the characteristics of HIMs who require more coding education and particular diagnoses for which further training is required. Our findings will assist the development of coding procedures, guidelines, education programs, and a more user-friendly database.

The national status of tuberculosis using nationwide medical records survey of patients with tuberculosis in Korea.

BACKGROUND: The epidemiology of tuberculosis (TB) has been assessed based on the data of the analysis of TB patients notified to the surveillance system in Korea. However, the national status of TB is not validated through this surveillance system. The objective is to determine the epidemiology of TB and to understand the accurate status of TB patients treated in private institutions. METHODS: Medical records of 53,579 patients who had been diagnosed with TB in 2008 were analyzed. RESULTS: Among 53,579 patients, the number of sputum smear positive cases was 15,639(29.2%) and the number of new cases was 39,191 (73.1%). The drug resistance rate of new cases was 5.3%, while the rate stood at 13.3% for TB patients with treatment history. The number of multi-drug resistant TB (MDR-TB) patients was 2,472 (4.6%), which consists of 2.9% of new cases and 9.3% of TB patients with prior treatment history. The number of extensively drug-resistant TB patients was 749 (1.4%), consisting of 1.1% of new cases and 2.2% of TB patients with prior treatment history. In terms of treatment outcomes, 66.4% of all TB patients, 70.5% of new cases, 64.4% of relapse cases, and 46.8% of MDR-TB cases were cured or completed. It was inferred that in 2008, the total number of TB patients reached 70,767, 145.6 per 100,000 people (95% confidence interval, 145.5~145.7). CONCLUSION: We conclude that the medical records review of the Health Insurance Review and Assessment Service (HIRA) data can be very effective in promoting the understanding of the current status of TB in private institutions.

A study of the difference in volume of information in chief complaint and present illness between electronic and paper medical records.

The introduction of an electronic medical record (EMR) has been rapidly accelerating in South Korea. The EMR was expected to improve quality of care, readability, availability, and the quality of data. However, the reluctance of healthcare providers to use the EMR may have caused a reduction of information recorded in EMRs. The purpose of this study was to identify whether there was any loss of information following the introduction of a narrative text-based EMR in the recording of chief complaint and present illness in inpatient medical records. Inpatient medical records of a university hospital were retrospectively evaluated for one month before and one month after the introduction of the EMR in June 2006. The volume of information for chief complaint and present illness was measured by number of words in Korean and normalised bytes. Change in volume of information was measured by two-way ANOVA and multiple regression analyses, controlling for doctors' gender, age, and grade/year of residents, patients' readmission status, reasons for admission and service department to assess any effect of the introduction of an EMR. Total numbers of paper-based medical records (PMRs) and EMRs for analysis were 1,159 and 1,122, respectively. Forty-three doctors participated in the study. Thirty-one (72%) doctors were less than 30 years of age. Number of words proved a better outcome measure (R²=22 for CC, R²=36 for PI) than normalised bytes (R²=18 for CC, R²=35 for PI) for measuring volume of information. Results showed that the volume of information in the chief complaint and present illness was not decreased after the introduction of the EMR, except when the dependent variable was measured by number of words in the present illness. The study showed that the introduction of the EMR did not reduce the volume of information documented for chief complaint and present illness in inpatient medical records. However, further studies are needed to identify how to control the probable loss of information as showed in present illness measured by number of words.

A development of conceptual framework for structured data entry in procedure.

This study was performed to provide the well-designed conceptual framework of clinical contents in procedures that enables clinicians to execute structured data entry at the point of care. The conceptual framework was developed through applying the ISO's conceptual framework for clinical contents modeling, semantic structuring, and using the Entity-Attribute-Value (EAV) model for data representation. We provided the conceptual framework in procedures, including twenty-one attributes and structured clinical data sets for operation records that are six diseases based on the conceptual framework.