Findings from the 2017 Yearbook Section on Health Information Management.

Objectives: To summarize the recent literature and research and present a selection of the best papers published in 2016 in the field of Health Information Management (HIM). Methods: A systematic review of the literature for the IMIA Yearbook HIM section was performed by the two section editors with the help of a medical librarian. We searched bibliographic databases for HIM-related papers were searched using both MeSH headings and key words in titles and abstracts. A shortlist of candidate best papers was first selected by section editors before being peer-reviewed by independent external reviewers. Results: The five papers selected as 'Best Papers' illustrate a variety of themes, include authors from a variety of countries, and were published in some of the best journals in the field. The themes of the five best papers include health information exchange, personal health records, patient engagement, data quality, and e-quality measures. Conclusions: The discipline of Health Information Management is increasingly becoming allied with the field of Biomedical Informatics in that both disciplines have interests in common. Traditional HIM areas of expertise (in the pre-electronic health record world), such as coding and privacy and security of health information, are necessary for the electronic exchange and secondary use of health information. With the changes in healthcare delivery brought by the use of electronic health records, addressing issues of information governance is essential. This synopsis discusses these key issues at the intersection of HIM and informatics, examines the potential challenges, and points the way for best practices, future research, and public policy considerations and directions.

Bridging the Gap: A Collaborative Approach to Health Information Management and Informatics Education.

BACKGROUND: Health Information Management (HIM) and Health Informatics (HI) were very separate professions when they were first formed. However, with the increasing adoption of electronic health records, the interests of the two fields have become more aligned. OBJECTIVES: To describe the evolution of a joint master's program in health informatics(HI) and health information management (HIM). METHODS: After analyzing workforce needs, and reviewing both CAHIIM accreditation requirements and existing curricular offerings in separate programs in HIM and HI, a joint program was developed. RESULTS: An HI master's program with a core curriculum for all students and tracks in Data Analytics, User Experience and Advanced Practice HIM was developed. A model for a comprehensive examination, based on the CAHIIM competencies, to be administered prior to and after the core curriculum was also developed. CONCLUSIONS: A core and track curriculum that incorporates HIM education as part of the Master of Science of Health Informatics provides a feasible roadmap for the future as HIM and HI become more closely aligned.

Use of diagnostic decision support systems in medical education.

BACKGROUND: Diagnostic decision support systems are designed to assist physicians with making diagnoses. This article illustrates some of the issues that will be faced as diagnostic decision support systems become used in medical education. OBJECTIVES: The objectives of this article are to examine 1) the skills that are needed to properly use these programs as part of the students' clinical experiences; 2) the changes that will be necessary in our curricula once these programs are more extensively utilized, including the implications of using these systems as an educational resource or simulation tool, and 3) the research issues that arise when these systems become an established part of our educational programs. METHODS: This is a critical analysis of the literature on diagnostic decision support systems and medical education. RESULTS: To optimally use diagnostic decision support programs, students will need grounding in the basic knowledge and skills that have always been necessary to become a physician, such as the ability to accurately gather and interpret clinical information from the patient. In addition, students will need specific skills in 1) selecting appropriate system vocabulary and functions, and 2) applying the diagnostic system's suggestions to their particular patient. CONCLUSIONS: When computer-based decision support systems are incorporated in medical education, they will likely lead to changes in the traditional medical curriculum. Research will be needed on how use of these programs changes the students' knowledge, problem-solving and information-seeking skills.

Comparison of measures to assess change in diagnostic performance due to a decision support system.

Little has been done to examine the relative merit of measures used to assess the impact of diagnostic decision support systems (DDSS) on physician performance. In this study, 10 different single-measures of diagnostic performance were compared empirically. The measures were of three types: rank-order, all-or-none, and appropriateness. The responsiveness (RESP) of each measure was estimated under two repeated-measures experimental conditions. RESP is the degree to which a measure could detect differences between conditions of low and high performance. The diagnostic performance of 108 physicians was compared on medical cases of varying diagnostic difficulty and with or without a high level of assistance from a DDSS. The results showed that the RESP among the measures varied nearly tenfold. The rank-order measures tended to provide the highest RESP values (maximum = 2.14) while appropriateness measures provided the lowest RESP values (maximum = 1.41). The most responsive measures were rank-orders of the correct diagnosis within the top 5 to 10 listed diagnoses.

Effects of a decision support system on physicians' diagnostic performance.

PURPOSE: This study examines how the information provided by a diagnostic decision support system for clinical cases of varying diagnostic difficulty affects physicians' diagnostic performance. METHODS: A national sample of 67 internists, 35 family physicians, and 6 other physicians used the Quick Medical Reference (QMR) diagnostic decision support system to assist them in the diagnosis of written clinical cases. Three sets of eight cases, stratified by diagnostic difficulty and the potential of QMR to produce high-quality information, were used. The effects of using QMR on three measures of physicians' diagnostic performance were analyzed using analyses of variance. RESULTS: Physicians' diagnostic performance was significantly higher (p < 0.01) on the easier cases and the cases for which QMR could provide higher-quality information. CONCLUSIONS: Physicians' diagnostic performance can be strongly influenced by the quality of information the system produces and the type of cases on which the system is used.

Influence of case and physician characteristics on perceptions of decision support systems.

OBJECTIVE: This study examines how characteristics of clinical cases and physician users relate to the users' perceptions of the usefulness of the Quick Medical Reference (QMR) and their confidence in their diagnoses when supported by the decision support system. METHODS: A national sample (N = 108) of 67 internists, 35 family physicians, and 6 other U.S. physicians used QMR to assist in the diagnosis of written clinical cases. Three sets of eight cases stratified by diagnostic difficulty and the potential of QMR to produce high-quality information were used. A 2 x 2 repeated-measures analysis of variance was used to test whether these factors were associated with perceived usefulness of QMR and physicians' diagnostic confidence after using QMR. Correlations were computed among physician characteristics, ratings of QMR usefulness, and physicians' confidence in their own diagnoses, and between usefulness or confidence and actual diagnostic performance. RESULTS: The analyses showed that QMR was perceived to be significantly more useful (P < 0.05) on difficult cases, on cases where QMR could provide high-quality information, by non-board-certified physicians, and when diagnostic confidence was lower. Diagnostic confidence was higher when comfort with using certain QMR functions was higher. The ratings of usefulness or diagnostic confidence were not consistently correlated with diagnostic performance. CONCLUSIONS: The results suggest that users' diagnostic confidence and perceptions of QMR usefulness may be associated more with their need for decision support than with their actual diagnostic performance when using the system. Evaluators may fail to find a diagnostic decision support system useful if only easy cases are tested, if correct diagnoses are not in the system's knowledge base, or when only highly trained physicians use the system.

The effects of the PDQ patient information file (PIF) on patients' knowledge, enrollment in clinical trials, and satisfaction.

BACKGROUND: This study examined the outcomes of providing a copy of the PDQ Patient Information File (PIF) to cancer patients. METHODS: Patients with cervical, endometrial, and ovarian cancers were randomized to two groups: 1) verbal communication only and 2) verbal communication plus PIF. Cancer knowledge and satisfaction with the PIF and the information received were assessed with telephone interviews. Clinical trial registries were reviewed to determine enrollment in clinical trials. RESULTS: The overall reaction to the PIF was good or excellent for 92% of the patients surveyed, but there was no significant difference between the two groups in their cancer knowledge, enrollment in clinical trials, or satisfaction with the information they received from their physicians. The majority of patients from both groups lacked basic knowledge about their disease, did not use any source of information other than their physicians and/or nurses, and were satisfied with the information they received. CONCLUSIONS: Patients appreciate receiving written cancer information, although it may not increase their cancer knowledge.

Relationships among performance scores of four diagnostic decision support systems.

OBJECTIVE: To examine the relationships among different performance scores for each of four diagnostic decision support systems (DDSSs). DESIGN: Intercorrelations among seven performance scores on a set of 105 cases for each of four DDSSs (DXplain, Iliad, Meditel, QMR) were computed. METHODS: The performance scores for each case reflected: 1) presence or absence of the case diagnosis in the DDSS knowledge base; 2) presence or absence of the correct diagnosis anywhere on the DDSS diagnosis list; 3) presence or absence of the correct diagnosis in the top ten diagnoses; 4) relevance of the DDSS diagnosis list; 5) comprehensiveness of the DDSS diagnosis list; 6) whether the DDSS suggested additional diagnoses to the experts' list; and 7) the length of the DDSS diagnosis list. RESULTS: For all DDSSs, the two Correct Diagnosis scores (top ten and total list) were significantly related: 1) to the presence of the correct diagnosis in the knowledge base; 2) to the Comprehensiveness score; and 3) to each other. There were significant differences among the four DDSSs on the magnitude and/or direction of the relationships between: 1) the two Correct Diagnosis scores; 2) the Relevance and Length scores; and 3) the Relevance and Additional Diagnoses scores. CONCLUSION: The production of a correct diagnosis for a given case is not related to the number of diagnoses suggested by the DDSS and, across different DDSSs, is not consistently related to other measures of performance. These data indicate that multiple measures are needed to fully describe the performance of a DDSS.

Performance of four computer-based diagnostic systems.

BACKGROUND: Computer-based diagnostic systems are available commercially, but there has been limited evaluation of their performance. We assessed the diagnostic capabilities of four internal medicine diagnostic systems: Dxplain, Iliad, Meditel, and QMR. METHODS: Ten expert clinicians created a set of 105 diagnostically challenging clinical case summaries involving actual patients. Clinical data were entered into each program with the vocabulary provided by the program's developer. Each of the systems produced a ranked list of possible diagnoses for each patient, as did the group of experts. We calculated scores on several performance measures for each computer program. RESULTS: No single computer program scored better than the others on all performance measures. Among all cases and all programs, the proportion of correct diagnoses ranged from 0.52 to 0.71, and the mean proportion of relevant diagnoses ranged from 0.19 to 0.37. On average, less than half the diagnoses on the experts' original list of reasonable diagnoses were suggested by any of the programs. However, each program suggested an average of approximately two additional diagnoses per case that the experts found relevant but had not originally considered. CONCLUSIONS: The results provide a profile of the strengths and limitations of these computer programs. The programs should be used by physicians who can identify and use the relevant information and ignore the irrelevant information that can be produced.

Use of the USMLE to select residents.

Many studies have examined the relationships between students' performances on the National Board of Medical Examiners (NBME) Part I and Part II examinations and their postgraduate clinical performances. Most studies have found a positive relationship between students' NBME examination scores and the ratings of residents' clinical performances and/or scores on specialty board certification examinations. Surveys of residency directors have also shown that NBME scores are used as part of the process for selecting residents, although other data and other selection criteria are considered more useful than the NBME scores. One area of continued concern is that a large body of data on the United States Medical Licensing Examination (USMLE) is not yet available. However, the predictive validity of the NBME scores supports the use of USMLE scores as part of the review process to screen potential residents. Residency directors should continue to use a variety of criteria in their final selection decisions and are encouraged to use their own program-specific data to establish and monitor particular cutoff scores for screening applicants.

Comparison of problem-based and traditional education on student performance in the obstetrics and gynecology clerkship.

OBJECTIVE: To evaluate the hypothesis that students trained in problem-based learning during basic sciences will have an advantage over traditionally trained students during clinical activities because of greater experience in patient care settings. METHODS: A retrospective study encompassing the academic years 1985-1990 compared students' clinical performance during an obstetrics and gynecology third-year clerkship based on the method of basic science education (problem-based 78, traditional 228). RESULTS: No statistical difference could be demonstrated in student performance based on the basic science educational method and controlling for the timing of rotation. CONCLUSIONS: Participation in a problem-based curriculum does not affect students' performance on standardized objective examinations within the clinical curriculum. Using current evaluation tools, there is no measurable difference in clinical evaluations between basic science education by traditional versus problem-based learning methods.

Needs assessment for diagnostic decision support systems (DDSS).

Diagnostic decision support systems are often developed without a clear idea of how well the system will meet the needs of its users. The present study was designed to assess the information needs of clinicians. A set of questions submitted to an information service by family physicians was used to determine how much need there was for diagnostic decision support, the types of support needed, and the general content areas of their questions. Results showed that less than half of the questions were related to diagnosis and that most of those were requests for general information about a given condition. In addition, the fewest diagnosis questions were for conditions that were seen frequently in ambulatory care in a survey of family practitioners.

Evaluation issues in the development of expert systems in medicine.

Evaluation of medical decision support software (MDSS)--computer programs to assist health professionals with diagnostic and/or therapeutic decisions--has not kept pace with the development of such programs. This article describes the following formative evaluation issues that must be addressed by developers of MDSS to evaluate these programs properly: (1) How can systematic feedback be obtained about an evolving program? (2) How can enough data to evaluate the program be obtained? (3) How much instruction is necessary? (4) What are the most important aspects for users to evaluate? and (5) How can the appropriate use of a developing MDSS be assured? Data from an ongoing evaluation of an existing MDSS, Quick Medical Reference, are used to illustrate the issues and to suggest recommendations for addressing them.