Identifying musculoskeletal conditions in electronic medical records: a prevalence and validation study using the Deliver Primary Healthcare Information (DELPHI) database.

BACKGROUND: Musculoskeletal (MSK) conditions are a common presentation in primary care. This study sought to determine the prevalence of MSK conditions in primary care in Ontario and to validate the extent to which health administrative date billing codes accurately represent MSK diagnoses. METHODS: De-identified electronic medical records (EMR) from the DELPHI database in southwestern Ontario, which contains 2493 patients (55.6% female, mean age 50.3 years (SD = 22.2)) and 21,964 encounters (July 1, 2006-June 30, 2010) were used for the analyses. Outcomes included: validation measures of agreement between International Classification of Diseases (ICD-9) diagnostic codes (health administrative data) and International Classification of Primary Care (ICPC) diagnoses defined as the reference standard, time to first ICD-9 code, prevalence, and healthcare utilization. RESULTS: There were 2940 true positive MSK encounters with primary care practitioners for 998 patients. Performance of the ICD-9 diagnostic codes included sensitivity = 76.5%, specificity = 95.2%, PPV = 94.6%, and NPV = 78.7%, compared to the ICPC reference standard. The majority of 998 patients were coded with both an ICPC and ICD-9 MSK code at their first or second encounter (67.4%). However, 23.5% of patients with the ICPC reference standard MSK were never coded with ICD-9. Four-year prevalence of MSK was 52.3% and varied by age (4.5% 0-17 years, 20.1% 18-44, 42.7% 45-64, and 32.7% 65+). Patients at MSK encounters had a higher number of: investigations (17.9% compared to 9.1%, p < .0001); referrals (17.6% compared to 14.3%, p < .0001); and prescriptions for opioids (17.2% compared to 5.3%, p < .0001). CONCLUSIONS: This study determined the prevalence of musculoskeletal conditions in primary care in Ontario using a reference standard definition. The study highlighted the value of using primary care ICPC codes to validate a definition for musculoskeletal conditions. Health administrative data can be used to ascertain the presence of musculoskeletal conditions; however, ICD-9 codes may underrepresent the prevalence of MSK conditions.

A basic model for assessing primary health care electronic medical record data quality.

BACKGROUND: The increased use of electronic medical records (EMRs) in Canadian primary health care practice has resulted in an expansion of the availability of EMR data. Potential users of these data need to understand their quality in relation to the uses to which they are applied. Herein, we propose a basic model for assessing primary health care EMR data quality, comprising a set of data quality measures within four domains. We describe the process of developing and testing this set of measures, share the results of applying these measures in three EMR-derived datasets, and discuss what this reveals about the measures and EMR data quality. The model is offered as a starting point from which data users can refine their own approach, based on their own needs. METHODS: Using an iterative process, measures of EMR data quality were created within four domains: comparability; completeness; correctness; and currency. We used a series of process steps to develop the measures. The measures were then operationalized, and tested within three datasets created from different EMR software products. RESULTS: A set of eleven final measures were created. We were not able to calculate results for several measures in one dataset because of the way the data were collected in that specific EMR. Overall, we found variability in the results of testing the measures (e.g. sensitivity values were highest for diabetes, and lowest for obesity), among datasets (e.g. recording of height), and by patient age and sex (e.g. recording of blood pressure, height and weight). CONCLUSIONS: This paper proposes a basic model for assessing primary health care EMR data quality. We developed and tested multiple measures of data quality, within four domains, in three different EMR-derived primary health care datasets. The results of testing these measures indicated that not all measures could be utilized in all datasets, and illustrated variability in data quality. This is one step forward in creating a standard set of measures of data quality. Nonetheless, each project has unique challenges, and therefore requires its own data quality assessment before proceeding.

Comprehensive practice: Normative definition across 3 generations of alumni from a single family practice program, 1985 to 2012.

OBJECTIVE: To determine the range of services and procedures offered by family physicians who define themselves as comprehensive practitioners and compare responses across 3 generations of alumni of a single family practice program. DESIGN: Cross-sectional survey. SETTING: Western University in London, Ont. PARTICIPANTS: All graduates of the family medicine program between 1985 and 2012. MAIN OUTCOME MEASURES: Self-reported provision of the following types of care: in-office care, in-hospital care, intrapartum obstetrics, housecalls, palliative care, after-hours care, nursing home care, minor surgery, emergency department care, sport medicine, and walk-in care. Sex, training site (urban or rural), size of community of practice, practice model, and satisfaction with practice were also reported. RESULTS: Participants practised in 7 provinces and 1 territory across Canada, but principally in Ontario. A small number were located in the United States. There was a decline in the number of services provided across 3 generations of graduates, with newer graduates providing fewer services than the older graduates. Significant decreases across the 3 groups were observed in provision of housecalls (P = .004), palliative care (P = .028), and nursing home care (P < .001). Non-significant changes were seen in provision of intrapartum obstetrics across the 3 alumni groups, with an initial decline and then increase in reported activity. Most respondents were in a family health organization or family health network practice model and those in such models reported offering significantly more services than those in family health group or salary models (P < .001). CONCLUSION: The normative definition of comprehensive care varies across 3 generations of graduates of this family medicine program, with newer physicians reporting fewer overall services and procedures than older graduates. Greater understanding of the forces (institutional, regulatory, economic, and personal) that determine the meaning of comprehensive primary care is necessary if this foundational element of family medicine is to be preserved.

Methods to Describe Referral Patterns in a Canadian Primary Care Electronic Medical Record Database: Modelling Multilevel Count Data.

BACKGROUND:   A referral from a family physician (FP) to a specialist is an inflection point in the patient journey, with potential implications for clinical outcomes and health policy. Primary care electronic medical record (EMR) databases offer opportunities to examine referral patterns. Until recently, software techniques were not available to model these kinds of multi-level count data. OBJECTIVE:  To establish methodology for determining referral rates from FPs to medical specialists using the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) EMR database. METHOD: Retrospective cohort study, mixed effects and multi-level negative binomial regression modelling with 87,258 eligible patients between 2007 and 2012. Mean referrals compared by patient sex, age, chronic conditions, FP visits, and urban/rural practice location.  Proportion of variance in referral rates attributable to the patient and practice levels. RESULTS:  On average, males had 0.26, and females 0.31 referrals in a 12-month period.  Referrals were significantly higher for females, increased with age, FP visits, and number of chronic conditions (p<.0001). Overall, 14% of the variance in referrals could be attributed to the practice level, and 86% to patient level characteristics. CONCLUSIONS:  Both patient and practice characteristics influenced referral patterns. The methodologic insights gained from this study have relevance to future studies on many research questions that utilize count data, both within primary care and broader health services research. The utility of the CPCSSN database will continue to increase in tandem with data quality improvements, providing a valuable resource to study Canadian referral patterns over time.

Psychosocial diagnoses occurring after patients present with fatigue.

OBJECTIVE: To discover the frequency of psychosocial and other diagnoses occurring at the end of a visit when patients present to their FPs with concerns about fatigue. DESIGN: Cross-sectional study of patient-FP encounters for fatigue. SETTING: Ten FP practices in southwestern Ontario. PARTICIPANTS: A total of 259 encounters involving 167 patients presenting to their FPs between March 1, 2006, and June 30, 2010, with concerns about fatigue. MAIN OUTCOME MEASURES: The frequency of psychological and social diagnoses made at the end of visits, and whether diagnoses were made by FPs at the end of the visits versus whether the code for fatigue remained. The associations between patient age, sex, fatigue presenting with other symptoms, or the presence of previous chronic conditions and the outcomes was tested. RESULTS: Psychosocial diagnoses were made 23.9% of the time. Among psychosocial diagnoses made, depressive disorder and anxiety disorder or anxiety state were diagnosed more often in women (P = .048). Slightly less than 30% of the time, the cause of patients' fatigue remained undiagnosed at the end of the encounter. A diagnosis was made more often in men. CONCLUSION: Causes of fatigue frequently remain undiagnosed; however, when there is a diagnosis, psychosocial diagnoses are common. Therefore, it would be appropriate for FPs to screen for psychosocial issues when their patients present with fatigue, unless some other diagnosis is evident. Depression and anxiety could be considered particularly among female patients with fatigue.

Comparisons of multi-morbidity in family practice--issues and biases.

BACKGROUND: As the population ages, practice and policy need to be guided by accurate estimates of chronic disease burden in primary care. OBJECTIVE: To produce a preliminary set of methodological considerations for cross-sectional and retrospective cohort studies of multi-morbidity in primary care using three studies as examples. Prevalence rate results from the three studies were re-estimated using identical age-sex groups. METHODS: We compared the methods and results of three separate studies in primary care: (i) patients in the Saguenay region of Quebec, Canada (2005); (ii) a substudy of the BEACH (Bettering the Evaluation and Care of Health) programme in Australia (2008); and (iii) the DELPHI (Deliver Primary Health Care Information) project in South-western Ontario, Canada (2009). Areas where the methods of multi-morbidity studies may differ were identified. The percentage of patients with two or more chronic conditions was compared by age-sex groups. RESULTS: Multi-morbidity prevalence varied by as much as 61%, where reported prevalence was 95% among females aged 45-64 in the Saguenay study, 46% in the BEACH substudy and 34% in the DELPHI study. Several aspects of the methods and study designs were identified as differing among the studies, including the sampling of frequent attenders, sampling period, source of data, and both the definition and count of chronic conditions. CONCLUSIONS: Understanding the differences among the methods used to produce prevalence data on multi-morbidity in primary care can help explain the varying results. Standardization of methods would allow for more valid inter-study comparisons.

A systematic review of prevalence studies on multimorbidity: toward a more uniform methodology.

PURPOSE: We sought to identify and compare studies reporting the prevalence of multimorbidity and to suggest methodologic aspects to be considered in the conduct of such studies. METHODS: We searched the literature for English- and French-language articles published between 1980 and September 2010 that described the prevalence of multimorbidity in the general population, in primary care, or both. We assessed quality of included studies with a modified version of the Strengthening the Reporting of Observational Studies in Epidemiology checklist. Results of individual prevalence studies were adjusted so that they could be compared graphically. RESULTS: The final sample included 21 articles: 8 described studies conducted in primary care, 12 in the general population, and 1 in both. All articles were of good quality. The largest differences in prevalence of multimorbidity were observed at age 75 in both primary care (with prevalence ranging from 3.5% to 98.5% across studies) and the general population (with prevalence ranging from 13.1% to 71.8% across studies). Apart from differences in geographic settings, we identified differences in recruitment method and sample size (primary care: 980-60,857 patients; general population: 1,099-316,928 individuals), data collection, and the operational definition of multimorbidity used, including the number of diagnoses considered (primary care: 5 to all; general population: 7 to all). This last aspect seemed to be the most important factor in estimating prevalence. CONCLUSIONS: Marked variation exists among studies of the prevalence of multimorbidity with respect to both methodology and findings. When undertaking such studies, investigators should carefully consider the specific diagnoses included and their number, as well as the operational definition of multimorbidity.

What are wait times to see a specialist? an analysis of 26,942 referrals in southwestern Ontario.

BACKGROUND: Reducing wait times is a key goal of Canadian health planners and policy makers. Using data from the EMRs of 23 family physicians across southwestern Ontario, we present data on wait times to see a specialist, and evaluate these data for equity. METHODS: Cross-sectional analysis of EMR database of 29,303 patients and 544,398 encounters from October 2005 to July 2010. Wait time was calculated as the difference between the specialist appointment date and the family physician's referral date. Multilevel regression analysis was used to assess equity in waiting times. RESULTS: The median wait was 53 days, with female patients having a slightly longer wait than males (55 vs. 51 days), and younger patients (median = 45 days) having the shortest wait. Differences were noted among practices, with a range of 42-63 days. The longest waits were for neurosurgery (median = 103 days) and the shortest for paediatricians (median = 28 days). There was no correlation between wait times and income, but significant associations were noted for patient age, sex and referral urgency. INTERPRETATION: This is the first study to present data on actual wait times for a broad array of specialists over a five-year period. There is variation among specialties and by practice, and further research is needed to understand reasons for these. From a policy perspective, there is equity in wait times in southwestern Ontario, as waits are not correlated with SES. Future work should model the patient-, physician- and contextual-level factors that determine specialist wait times.

Feedback and training tool to improve provision of preventive care by physicians using EMRs: a randomised control trial.

BACKGROUND: Electronic medical records (EMRs) have the potential to improve the provision of preventive care by allowing general practitioners (GPs) to track and recall eligible patients and record testing for feedback on their service provision. OBJECTIVE: This study evaluates the effect of an educational intervention and feedback tool designed to teach GPs how to use their EMRs to improve their provision of preventive care. METHODS: A randomised controlled trial comparing rates of mammography, Papanicolaou tests, faecal occult blood tests and albumin creatinine ratios one-year pre- and post-intervention was conducted. Nine primary care practices (PCPs) representing over 30 000 patients were paired by practice size and experience of GPs, and randomly allocated to intervention or control groups. Physicians at the four intervention practices received a two-hour feedback session on their current level of preventive care and training to generate eligible patient lists for preventive services from their EMR database. RESULTS: One-year post-intervention results provided no evidence of a difference. The intervention was not a significant predictor of the one-year postintervention test rates for any of the four tests. On average, the intervention practices increased postintervention test rates on all tests by 16.8%, and control practices increased by 22.3%. CONCLUSION: The non-significant results may be due to a variety of reasons, including the level of intensity of the educational intervention, the cointervention of a government programme which provided incentives to GPs meeting specific targets for preventive care testing or the level of recording of tests performed in the EMR.

Patterns of referral in a Canadian primary care electronic health record database: retrospective cross-sectional analysis.

BACKGROUND: Databases derived from primary care electronic health records (EHRs) are ideally suited to study clinical influences on referral patterns. This is the first study outside the United Kingdom to use an EHR database to describe rates of referral per patient from family physicians to specialists. OBJECTIVE: To use a primary care EHR database to describe referrals to specialist physicians; to partition variance in referral rates between the practice and patient levels. METHODS: Retrospective cross-sectional analysis of de-identified EHRs of 33 998 patients from 10 primary care practices in Ontario, Canada. The study cohort included all patients who visited their family physician 1 April 2007 to 31 March 2008 (n ≥ 24856). Specialist referrals for each patient were counted for 12 months following their index visit. Rates of referral were compared by sex, age, number of office visits, practice location and specialist type using t-tests or Pearson's correlation. Variance partitioning determined the proportion of variance in the overall referral rate accounted for by the practice and patient levels. RESULTS: In total, 7771 patients (31.3%) had one or more referrals. The overall referral rate was 455/1000 patients/year (95% CI, 444-465). Rates were higher for females, older patients and rural practices. The referral rate correlated with the number of family physician office visits. Ninety-two percent of the total variance in referral rates was attributable to the patient (vs. practice) level. CONCLUSIONS: A Canadian primary care EHR database showed similar patterns of referral to those reported from administrative databases. Most variance in referral rates is explained at the patient level.

Quality of congestive heart failure care: assessing measurement of care using electronic medical records.

OBJECTIVE: To study the feasibility of using electronic medical record (EMR) data from the Deliver Primary Healthcare Information (DELPHI) database to measure quality of care for patients with congestive heart failure (CHF) in primary care and to determine the percentage of patients with CHF receiving the recommended care. DESIGN: Items listed on the Ontario Ministry of Health and Long-Term Care Heart Failure Patient Care Flow Sheet (CHF flow sheet) were assessed and measured using EMRs of patients diagnosed with CHF between October 1, 2005, and September 30, 2008. SETTING: Ten primary health care practices in southwestern Ontario. PARTICIPANTS: Four hundred eighty-eight patients who were considered to have CHF because at least 1 of the following was indicated in their EMRs: an International Classification of Diseases billing code for CHF (category 428), an International Classification of Primary Care diagnosis code for heart failure (ie, K77), or "CHF" reported on the problem list. MAIN OUTCOME MEASURES: Number of CHF flow sheet items that were measurable using EMR data from the DELPHI database. Percentage of patients with CHF receiving required quality-of-care items since the date of diagnosis. RESULTS: The DELPHI database contained information on 60 (65.9%) of the 91 items identified using the CHF flow sheet. The recommended tests and procedures were recorded infrequently: 55.5% of patients with CHF had chest radiographs; 32.6% had electrocardiograms; 32.2% had echocardiograms; 30.5% were prescribed angiotensin-converting enzyme inhibitors; 20.9% were prescribed ß-blockers; and 15.8% were prescribed angiotensin II receptor blockers. CONCLUSION: Low frequencies of recommended care items for patients with CHF were recorded in the EMR. Physicians explained that CHF care was documented in areas of the EMR that contained patient identifiers, such as the encounter notes, and was therefore not part of the DELPHI database. Extractable information from the EMR does not provide a complete picture of the quality of care provided to patients with CHF.

The sum of the parts detracts from the intended whole: competencies and in-training assessments.

OBJECTIVES: Despite the fact that Canadian residency programmes are required to assess trainees' performance within the context of the CanMEDS Roles Framework, there has been no inquiry into the potential relationship between residents' perceptions of the framework and their in-training assessments (ITA). Using data collected during the study of ITA, we explored residents' perceptions of these competencies. METHODS: From May 2006-07, a purposive sample of 20 resident doctors from internal medicine, paediatrics, and surgery were interviewed about their ITA experiences. Data collection and analysis proceeded in an iterative fashion consistent with grounded theory. In April 2008, a summary of recurrent themes was presented during a focus group interview of another five residents to afford further elaboration and refinement of thematic findings. RESULTS: The in-training assessment report (ITAR) was perceived as a primary source of residents' information on CanMEDS. Residents' familiarity with the set of competencies appeared to be quite limited and they possessed narrow definitions of the roles. Several trainees questioned the framework's relevance and some appeared confused about the overlapping nature of the roles. Although residents viewed the central Medical Expert role as the most relevant and important competency, they incorrectly perceived it as only involving the acquisition of medical and scientific knowledge. A visual rhetorical analysis of a typical ITAR suggests that the visual features found within this assessment tool may be misrepresenting the framework and the centrality of the Medical Expert role. CONCLUSIONS: Resident doctors' knowledge of CanMEDS was found to be limited. The visual structure of the ITAR appears to be a factor in residents' apparent distortion of the CanMEDS construct from its original holistic philosophy.

Rules of engagement: residents' perceptions of the in-training evaluation process.

BACKGROUND: In-training evaluation reports (ITERs) often fall short of their goals of promoting resident learning and development. Efforts to address this problem through faculty development and assessment-instrument modification have been disappointing. The authors explored residents' experiences and perceptions of the ITER process to gain insight into why the process succeeds or fails. METHOD: Using a grounded theory approach, semistructured interviews were conducted with 20 residents. Constant comparative analysis for emergent themes was conducted. RESULTS: All residents identified aspects of "engagement" in the ITER process as the dominant influence on the success of ITERs. Both external (evaluator-driven, such as evaluator credibility) and internal (resident-driven, such as self-assessment) influences on engagement were elaborated. When engagement was lacking, residents viewed the ITER process as inauthentic. CONCLUSIONS: Engagement is a critical factor to consider when seeking to improve ITER use. Our articulation of external and internal influences on engagement provides a starting point for targeted interventions.