Impact of clinical information-retrieval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies.

PURPOSE: This paper appraises empirical studies examining the impact of clinical information-retrieval technology on physicians and medical students. METHODS: The world literature was reviewed up to February 2004. Two reviewers independently identified studies by scrutinising 3368 and 3249 references from bibliographic databases. Additional studies were retrieved by hand searches, and by searching ISI Web of Science for citations of articles. Six hundred and five paper-based articles were assessed for relevance. Of those, 40 (6.6%) were independently appraised by two reviewers for relevance and methodological quality. These articles were quantitative, qualitative or of mixed methods, and 26 (4.3%) were retained for further analysis. For each retained article, two teams used content analysis to review extracted textual material (quantitative results and qualitative findings). RESULTS: Observational studies suggest that nearly one-third of searches using information-retrieval technology may have a positive impact on physicians. Two experimental and three laboratory studies do not reach consensus in support of a greater impact of this technology compared with other sources of information, notably printed educational material. Clinical information-retrieval technology may affect physicians, and further research is needed to examine its impact in everyday practice.

A new impact assessment method to evaluate knowledge resources.

RATIONALE: Methods to systematically measure the impact of knowledge resources on health professionals would enhance evaluation of these resources in the real world. OBJECTIVE: To propose a new impact assessment method. BACKGROUND: We demonstrated the feasibility of combining a 4-level scale with Computerized Ecological Momentary Assessment (CEMA) for efficiently measuring the impact of a knowledge resource. METHOD: We critically reviewed the world literature regarding the impact of clinical information-retrieval technology on trainees and doctors, and retained 26 papers for qualitative content analysis. FINDINGS: Of those, 21 use a nominal scale (yes/no), none systematically measures the impact of searches for information outside of a laboratory setting, and none uses an ordinal scale. The literature supports the proposed levels of impact, and suggests a fifth level. CONCLUSION: A new impact assessment method is proposed, which combines a 5-level revised scale and CEMA.