Developing the evidentiary basis for family medicine in the global context: The Besrour Papers: a series on the state of family medicine in the world.

OBJECTIVE: To provide an overview of the main methodologic challenges to finding definitive evidence of the positive effects of family medicine and family medicine training on a global scale. COMPOSITION OF THE COMMITTEE: In 2012, 2013, and 2014, the College of Family Physicians of Canada hosted the Besrour Conferences to reflect on its role in advancing the discipline of family medicine globally. The Besrour Papers Working Group, which was struck at the 2013 conference, was tasked with developing a series of papers to highlight the key issues, lessons learned, and outcomes emerging from the various activities of the Besrour collaboration. The working group comprised members of various academic departments of family medicine in Canada and abroad who attended the conferences. METHODS: We performed a scoping review to determine the methodologic obstacles to understanding the positive effects of family medicine globally. REPORT: The main obstacle to evaluating family medicine globally is that one of its core dimensions and assets is its local adaptability. Family medicine takes on very different roles in different health systems, making aggregation of data difficult. In many countries family medicine competes with other disciplines rather than performing a gatekeeping role. Further, most research that has been conducted thus far comes from industrialized contexts, and patient continuity and its benefits might not be achievable in the short term in developing countries when clinical demands are great. We must find frameworks to permit strengthening the evidentiary basis of the discipline across different contexts without sacrificing its beneficial adaptability. CONCLUSION: We believe that developing family medicine and its attributes is one of the keys to achieving global health. These attributes­including its comprehensiveness, adaptability, and attention to both local and patient needs­are key to advancing global health priorities, but make common evaluative frameworks for the discipline a challenge. The spread of family medicine over the past decades is indirect evidence of its utility, but we need to generate more evidence. We present some of the initial challenges to a broader and more rigorous evaluative framework.

Continuing professional development and social accountability: a review of the literature.

The idea that health professionals should be accountable to the society they serve is not a new concept and by the 1990 s, the continuing professional development (CPD) of health professionals was being seen as one way in which Canadians' level of health could be improved. The public was, and is still today, increasingly demanding a system that is more responsive to regional and community needs. As a result, there is a need for more health professional education at all stages of the education continuum - undergraduate, postgraduate, and continuing professional development - that meets the health and social needs of the populations being served. The trend is now towards 'socially accountable' health care, meaning that the broader context of CPD must also include the personal, social, and political aspects of health care and as such, involve a widening of accountability to patients, the community, managers and policymakers. CPD planning must take into account local and national priorities as well as personal learning needs. However, the definition of social accountability and the stages at which it is addressed is sometimes vague and this added to the difficulty of identifying relevant studies in the literature. Nonetheless, there were some "best practices" evident via Canadian and American studies which focused on models of socially accountable CPD, as well as examples of interdisciplinary collaboration in Canada, the United States, Australia, Great Britain, and the United Arab Emirates. However, there is a definite need for increased research and publication of such "best practice" initiatives. There is also a need for Canadian health professional schools to facilitate this process by sharing their experiences and resources if possible. An extensive literature review was conducted between January and March 2004. Due to time constraints, it was limited to articles written in the English language. The databases/sources utilized included: Medline (now known as Pubmed), CINAHL, ERIC, PsychInfo, Canadian Business & Current Affairs (CBCA) Full-text Education (now known as CBCA Education), Research and Development Resource Base in Continuing Medical Education (RDRB/CME) at the University of Toronto, EMBASE (Excerpta Medica). This literature review was one of the first activities conducted under the auspices of "Issues of Quality and Continuing Professional Development: Maintenance of Competence", a national project funded by the Primary Health Care Transition Fund, Health Canada. The purposes of this review were to identify literature which focuses on aspects of continuing professional development, social accountability, and determinants of health; "best practices" of socially accountable CPD and inter/intra-disciplinary collaboration, and the critical success factors and challenges to implementing CPD, especially CPD that meets the needs of both health professionals and the populations they serve.

Impact of clinical information-retrieval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies.

PURPOSE: This paper appraises empirical studies examining the impact of clinical information-retrieval technology on physicians and medical students. METHODS: The world literature was reviewed up to February 2004. Two reviewers independently identified studies by scrutinising 3368 and 3249 references from bibliographic databases. Additional studies were retrieved by hand searches, and by searching ISI Web of Science for citations of articles. Six hundred and five paper-based articles were assessed for relevance. Of those, 40 (6.6%) were independently appraised by two reviewers for relevance and methodological quality. These articles were quantitative, qualitative or of mixed methods, and 26 (4.3%) were retained for further analysis. For each retained article, two teams used content analysis to review extracted textual material (quantitative results and qualitative findings). RESULTS: Observational studies suggest that nearly one-third of searches using information-retrieval technology may have a positive impact on physicians. Two experimental and three laboratory studies do not reach consensus in support of a greater impact of this technology compared with other sources of information, notably printed educational material. Clinical information-retrieval technology may affect physicians, and further research is needed to examine its impact in everyday practice.

Development and content validation of the information assessment method for patients and consumers.

BACKGROUND: Online consumer health information addresses health problems, self-care, disease prevention, and health care services and is intended for the general public. Using this information, people can improve their knowledge, participation in health decision-making, and health. However, there are no comprehensive instruments to evaluate the value of health information from a consumer perspective. OBJECTIVE: We collaborated with information providers to develop and validate the Information Assessment Method for all (IAM4all) that can be used to collect feedback from information consumers (including patients), and to enable a two-way knowledge translation between information providers and consumers. METHODS: Content validation steps were followed to develop the IAM4all questionnaire. The first version was based on a theoretical framework from information science, a critical literature review and prior work. Then, 16 laypersons were interviewed on their experience with online health information and specifically their impression of the IAM4all questionnaire. Based on the summaries and interpretations of interviews, questionnaire items were revised, added, and excluded, thus creating the second version of the questionnaire. Subsequently, a panel of 12 information specialists and 8 health researchers participated in an online survey to rate each questionnaire item for relevance, clarity, representativeness, and specificity. The result of this expert panel contributed to the third, current, version of the questionnaire. RESULTS: The current version of the IAM4all questionnaire is structured by four levels of outcomes of information seeking/receiving: situational relevance, cognitive impact, information use, and health benefits. Following the interviews and the expert panel survey, 9 questionnaire items were confirmed as relevant, clear, representative, and specific. To improve readability and accessibility for users with a lower level of literacy, 19 items were reworded and all inconsistencies in using a passive or active voice have been solved. One item was removed due to redundancy. The current version of the IAM4all questionnaire contains 28 items. CONCLUSIONS: We developed and content validated the IAM4all in partnership with information providers, information specialists, researchers and representatives of information consumers. This questionnaire can be integrated within electronic knowledge resources to stimulate users' reflection (eg, their intention to use information). We claim that any organization (eg, publishers, community organizations, or patient associations), can evaluate and improve their online consumer health information from a consumers' perspective using this method.

A new impact assessment method to evaluate knowledge resources.

RATIONALE: Methods to systematically measure the impact of knowledge resources on health professionals would enhance evaluation of these resources in the real world. OBJECTIVE: To propose a new impact assessment method. BACKGROUND: We demonstrated the feasibility of combining a 4-level scale with Computerized Ecological Momentary Assessment (CEMA) for efficiently measuring the impact of a knowledge resource. METHOD: We critically reviewed the world literature regarding the impact of clinical information-retrieval technology on trainees and doctors, and retained 26 papers for qualitative content analysis. FINDINGS: Of those, 21 use a nominal scale (yes/no), none systematically measures the impact of searches for information outside of a laboratory setting, and none uses an ordinal scale. The literature supports the proposed levels of impact, and suggests a fifth level. CONCLUSION: A new impact assessment method is proposed, which combines a 5-level revised scale and CEMA.

Improving on-line skills and knowledge. A randomized trial of teaching rural physicians to use on-line medical information.

OBJECTIVE: To assess the change in frequency and methods with which a pilot group of rural physicians consulted on-line medical resources before and after an educational intervention. DESIGN: Physicians were randomly assigned to an educational intervention or control group. Self-administered questionnaires were completed before and 3 months after the intervention. SETTING: Rural practices in southwestern Ontario. PARTICIPANTS: Eighty rural (defined as a population of 15000 or less) physicians in southwestern Ontario, with a computer with Internet access. INTERVENTIONS: Individualized 3-hour training session on using the World Wide Web to research patient-related questions. MAIN OUTCOME MEASURES: Frequency of access and comfort with on-line medical information were compared after intervention with baseline data using the Wilcoxon two-sample test. RESULTS: At follow up, the intervention group showed a significant improvement over the control group in their frequency of accessing the World Wide Web to address patient-related questions (P = .009), in their comfort level in using on-line databases (P = .032), and in their frequency of accessing on-line databases (P = .044). CONCLUSION: Rural physicians' comfort and competence in using computers to address patient problems can be improved by an individualized 3-hour training session.