Interobserver Agreement Among Uveitis Experts on Uveitic Diagnoses: The Standardization of Uveitis Nomenclature Experience.

PURPOSE: To evaluate the interobserver agreement among uveitis experts on the diagnosis of the specific uveitic disease. DESIGN: Interobserver agreement analysis. METHODS: Five committees, each comprised of 9 individuals and working in parallel, reviewed cases from a preliminary database of 25 uveitic diseases, collected by disease, and voted independently online whether the case was the disease in question or not. The agreement statistic, κ, was calculated for the 36 pairwise comparisons for each disease, and a mean κ was calculated for each disease. After the independent online voting, committee consensus conference calls, using nominal group techniques, reviewed all cases not achieving supermajority agreement (>75%) on the diagnosis in the online voting to attempt to arrive at a supermajority agreement. RESULTS: A total of 5766 cases for the 25 diseases were evaluated. The overall mean κ for the entire project was 0.39, with disease-specific variation ranging from 0.23 to 0.79. After the formalized consensus conference calls to address cases that did not achieve supermajority agreement in the online voting, supermajority agreement overall was reached on approximately 99% of cases, with disease-specific variation ranging from 96% to 100%. CONCLUSIONS: Agreement among uveitis experts on diagnosis is moderate at best but can be improved by discussion among them. These data suggest the need for validated and widely used classification criteria in the field of uveitis.

A design thinking approach to effective vaccine safety communication.

The highly complex and controversial topic of vaccine safety communication warrants innovative, user-centered solutions that would start with gaining mutual respect while taking into account the needs, concerns and underlying motives of patients, parents and physicians. To this end, a non-profit collaborative project was conducted by The Vienna Vaccine Safety Initiative, an international think tank aiming to promote vaccine safety research and communication, and the School of Design Thinking in Potsdam, Germany, the first school for innovation in Europe. The revolutionary concept of the Design Thinking approach is to group students in small multi-disciplinary teams. As a result they can generate ground-breaking ideas by combining their expertise and different points of view. The team agreed to address the following design challenge question: "How might we enable physicians to encourage parents and children to prevent infectious diseases?" The current article describes, step-by step, the ideation and innovation process as well as first tangible outcomes of the project.

Standardization and simplification of vaccination records.

The majority of vaccines are administered during childhood. Vaccination records are important documents to be kept for a lifetime, but the documentation of immunization events is poorly standardized. At the point of care, paper records are often unavailable, making it impossible to obtain accurate vaccination histories. Vaccination records should include batch specifications to allow the tracking of licensed vaccines in cases of recall. The WHO have generated the International Certificate of Vaccination or Prophylaxis for the documentation of childhood and travel vaccinations as well as seasonal and booster immunizations. When moving vaccination records into the digital age, data standards and interoperability need to be considered. The ideal vaccination record should facilitate the interpretation of safety reports and promote a data continuum from pre-licensure trials to post-marketing surveillance. The current article describes which data elements are essential, and how vaccination documentation could be streamlined and simplified.

Automated quality measurement in Department of the Veterans Affairs discharge instructions for patients with congestive heart failure.

Quality measurement is an important issue for the United States Department of Veterans Affairs (VA). In this study, we piloted the use of an informatics tool, the Multithreaded Clinical Vocabulary Server (MCVS), which extracted automatically whether the VA Office of Quality and Performance measures of quality of care were met for the completion of discharge instructions for inpatients with congestive heart failure. We used a single document, the discharge instructions, from one section of the medical records for 152 patients and developed a reference standard using two independent reviewers to assess performance. When evaluated against the reference standard, MCVS achieved a sensitivity of 0.87, a specificity of 0.86, and a positive predictive value of 0.90. The automated process using the discharge instruction document worked effectively. The use of the MCVS tool for concept-based indexing resulted in mostly accurate data capture regarding quality measurement, but improvements are needed to further increase the accuracy of data extraction.

Are posttraumatic stress disorder mental health terms found in SNOMED-CT medical terminology.

The authors sought to evaluate how well the Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) controlled vocabulary represents terms commonly used clinically when documenting posttraumatic stress disorder (PTSD). A list was constructed based on the PTSD criteria in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; American Psychiatric Association, 1994), symptom assessment instruments, and publications. Although two teams mapping the terms to SNOMED-CT differed in their approach, the consensus mapping accounted for 91% of the 153 PTSD terms. They found that the words used by clinicians in describing PTSD symptoms are represented in SNOMED-CT. These results can be used to codify mental health text reports for health information technology applications such as automated chart abstraction, algorithms for identifying documentation of symptoms representing PTSD in clinical notes, and clinical decision support.

Secondary use of clinical data.

Clinicians involved in clinical care generate daily volumes of important data. This data is important for continuity of care, referrals to specialists and back to the patient's medical home. The same data can be used to generate alerts to improve the practice and to generate care activities to ensure that all appropriate care services are provided for the patient given their known medical histories using electronic quality (eQuality) monitoring. For many years we have used patient records as a data source for human abstraction of clinical research data. With the advent of electronic health record (EHR) data we can now make use of computable EHR data that can perform retrospective research studies more rapidly and lower the activation energy necessary to ask the next important question using electronic studies (eStudies). Barriers to these eStudies include: the lack of interoperable data between and among practices, the lack of computable definitions of measures, the lack of training of health professionals to use Ontology based Informatics tools that allow the execution of this type of logic, common methods need to be developed to distribute computable best practice rules to ensure rapid dissemination of evidence, better translating research into practice.

The Health Archetype Language (HAL-42): interface considerations.

In this manuscript we report an evaluation of the reliability of clinical research rules creation by multiple clinicians using the Health Archetype Language (HAL-42) and user interface. HAL-42 is a language which allows real time epidemiological inquiry using automatically derived clinical encodings with any health Ontology. This evaluation used SNOMED CT as the underlying Ontology. The inquiries were performed on a population of 17,731 patients whose 50,000 clinical records have all been fully encoded in SNOMED CT. Four subject matter experts (SMEs) were asked independently to encode and run 10 rules/studies. The inter-rater agreement was 74.8% (p=0.6526) with a Kappa statistic of 0.49217 (p=0.5722). The ten rules were divided into three easy rules, four moderate and three complex rules. There was no significant difference in the SME's agreement when representing easy and complex rules (p=0.6243). We conclude that although the usability of the HAL-42 language is usable enough to achieve reasonable inter-rater reliability, some training will be necessary to reach high levels of reliability for ad hoc queries. We also conclude that SMEs are just as competent to perform complex queries as easy queries of ontologically indexed clinical data.

BioProspecting: novel marker discovery obtained by mining the bibleome.

BioProspecting is a novel approach that enabled our team to mine data related to genetic markers from the New England Journal of Medicine (NEJM) utilizing SNOMED CT and the Human Gene Onotology (HUGO). The Biomedical Informatics Research Collaborative was able to link genes and disorders using the Multi-threaded Clinical Vocabulary Server (MCVS) and natural language processing engine, whose output creates an ontology-network using the semantic encodings of the literature that is organized by these two terminologies. We identified relationships between (genes or proteins) and (diseases or drugs) as linked by metabolic functions and identified potentially novel functional relationships between, for example, genes and diseases (e.g. Article #1 ([Gene - IL27] = > {Enzyme - Dipeptidyl Carboxypeptidase 1}) and Article #2 ({Enzyme - Dipeptidyl Carboxypeptidase 1} < = [Disorder - Type II DM]) showing a metabolic link between IL27 and Type II DM). In this manuscript we describe our method for developing the database and its content as well as its potential to assist in the discovery of novel markers and drugs.

NLP-based identification of pneumonia cases from free-text radiological reports.

Radiological reports are a rich source of clinical data which can be mined to assist with biosurveillance of emerging infectious diseases. In addition to biosurveillance, radiological reports are an important source of clinical data for health service research.Pneumonias and other radiological findings on chest x ray or chest computed tomography (CT) are one type of relevant finding to both biosurveillance and health services research. In this study we examined the ability of a Natural Language Processing system to accurately identify pneumonias and other lesions from within free text radiological reports. The system encoded the reports in the SNOMED CT Ontology and then a set of SNOMED CT based rules were created in our Health Archetype Language aimed at the identification of these radiological findings and diagnoses. The encoded rule was executed against the SNOMED CT encodings of the radiological reports. The accuracy of the reports was compared with a Clinician review of the Radiological Reports. The accuracy of the system in the identification of pneumonias was high with a Sensitivity (recall) of 100%, a specificity of 98%, and a positive predictive value (precision) of 97%. We conclude that SNOMED CT based computable rules are accurate enough for the automated biosurveillance of pneumonias from radiological reports.

Using natural language processing for identification of pneumonia cases from clinical records of patients with serologically proven influenza.

The purpose of this study was to determine whether influenza vaccination protects against pneumonia in patients who develop influenza. By parsing a data set of records of 1455 patients with serologically proven influenza using SNOMED CT we found that of the vaccinated patients 19.3% developed pneumonia and of the unvaccinated 20.7%. These data suggest that influenza vaccine does not prevent pneumonias in patients who develop influenza despite immunization with influenza vaccine.

Whole record surveillance is superior to chief complaint surveillance for predicting influenza.

Matched records of positive and negative influenza cases were parsed with a Natural Language Processor, the Multi-threaded Clinical Vocabulary Server (MCVS). Output was coded into SNOMED-CT reference terminology and compared to the SNOMED case definition of influenza. Odds ratios for each element of the influenza case definition by each section of the record were used to generate ROC curves. C-statistics showed that whole record surveillance was superior to chief complaint surveillance for predicting influenza.

Biosurveillance evaluation of SNOMED CT's terminology (BEST Trial): coverage of chief complaints.

The current United States Health Information Technology Standards Panel's interoperability specification for biosurveillance relies heavily on chief complaint data for tracking rates of cases compatible with a case definition for diseases of interest (e.g. Avian Flu). We looked at SNOMED CT to determine how well this large general medical ontology could represent data held in chief complaints. In this experiment we took 50,000 records (Comprehensive Examinations or Limited Examinations from primary care areas at the Mayo Clinic) from December 2003 through February 2005 (Influenza Season). Of these records, 36,097 had non-null Chief Complaints. We randomly selected 1,035 non-null Chief Complaints and two Board-certified internists (one Infectious Diseases specialist and one general internist) reviewed the mappings of the 1,035 chief complaints. Where the reviewers disagreed, a third internist adjudicated. SNOMED CT had a sensitivity of 98.7% for matching clinical terms found in the chief complaint section of the clinical record. The positive predictive value was 97.4%, the negative predictive value was 89.5%, the specificity was 81.0%, the positive likelihood ratio was 5.181 and the negative likelihood ratio was 0.016. We conclude that SNOMED CT and natural language parsing engines can well represent the clinical content of chief complaint fields. Future research should focus on how well the information contained in the chief complaints can be relied upon to provide the basis of a national strategy for biosurveillance. The authors recommend that efforts be made to examine the entire clinical record to determine the level of improvement in the accuracy of biosurveillance that can be achieved if we were to incorporate the entire clinical record into our biosurveillance strategy.