User needs analysis and usability assessment of DataMed - a biomedical data discovery index.

OBJECTIVE: To present user needs and usability evaluations of DataMed, a Data Discovery Index (DDI) that allows searching for biomedical data from multiple sources. MATERIALS AND METHODS: We conducted 2 phases of user studies. Phase 1 was a user needs analysis conducted before the development of DataMed, consisting of interviews with researchers. Phase 2 involved iterative usability evaluations of DataMed prototypes. We analyzed data qualitatively to document researchers' information and user interface needs. RESULTS: Biomedical researchers' information needs in data discovery are complex, multidimensional, and shaped by their context, domain knowledge, and technical experience. User needs analyses validate the need for a DDI, while usability evaluations of DataMed show that even though aggregating metadata into a common search engine and applying traditional information retrieval tools are promising first steps, there remain challenges for DataMed due to incomplete metadata and the complexity of data discovery. DISCUSSION: Biomedical data poses distinct problems for search when compared to websites or publications. Making data available is not enough to facilitate biomedical data discovery: new retrieval techniques and user interfaces are necessary for dataset exploration. Consistent, complete, and high-quality metadata are vital to enable this process. CONCLUSION: While available data and researchers' information needs are complex and heterogeneous, a successful DDI must meet those needs and fit into the processes of biomedical researchers. Research directions include formalizing researchers' information needs, standardizing overviews of data to facilitate relevance judgments, implementing user interfaces for concept-based searching, and developing evaluation methods for open-ended discovery systems such as DDIs.

Assessment of moral reasoning skills in the orthopaedic surgery resident applicant.

BACKGROUND: Assessing orthopaedic surgery residency applicants is a complex process. One important applicant characteristic not commonly measured during the interview process is moral behavior, a key component of the guidelines of the Accreditation Council for Graduate Medical Education for professionalism competency. We sought to determine whether a relationship exists between the results of residency selection interviews, as measured by match rank order, and moral reasoning skills, as measured by the revised version of the Defining Issues Test (DIT-2). METHODS: The DIT-2, a psychological tool that has been validated in assessing moral reasoning in >30,000 subjects, was administered to orthopaedic surgery residency candidates on a voluntary basis during their interview day. Data were collected annually from four applicant classes (2009 to 2012) at one institution. Each candidate was interviewed by an orthopaedic faculty team and was ranked comparatively with the other interviewing applicants. The five-part aggregate interview method for ranking applicants for the match list included a focused interview station presenting ethical dilemmas as one part of determining rank order. RESULTS: One hundred and twenty-four fourth-year medical students interviewing for orthopaedic surgery residency were assessed with the DIT-2. The results from the DIT-2 produced a postconventional score, which correlates with various levels of moral development. Each applicant's postconventional score was then compared with the patient's overall rank order. Applicants had highly variable moral reasoning skills, with no relationship to the final rank list. CONCLUSIONS: Despite a wide range of DIT-2 scores, the interview process did not differentiate applicants based on moral reasoning skills. It remains unclear whether the DIT-2 could (or should) be used as an additional data point to help stratify orthopaedic residency applicants and to predict their performance.

Polypharmacy reduction in youth in a residential treatment center leads to positive treatment outcomes and significant cost savings.

OBJECTIVE: The purpose of this study was to assess whether polypharmacy regimens can be safely and effectively reduced for youth placed in a residential treatment center, and to assess the cost savings achieved from medication reductions. METHODS: Data were collected for 131 youth ages 11-18, who were admitted to and discharged from a residential treatment center between 2007 and 2011. Six month postdischarge data were available for 51 youth. Data include demographics, admission and discharge medications, place of discharge, and postdischarge stability level. RESULTS: Upon admission, 30 youth were not on medication, at discharge 48 were not; a 60% increase. Mean number of admission medications was 2.16 (SD=0.97) versus 1.55 (SD=0.70) upon discharge. Upon admission, one youth was on five and nine were on four medications. At end-point, only one youth was on four medications. The number of youth needing two or more medications declined by 55%, and the number of those needing three or more declined by 69%. The largest reduction was seen in the number of antipsychotics and antidepressants. Mood stabilizer and antipsychotic combinations declined by 65%. Youth with medication reduction were more likely to be discharged to a less restrictive setting than were youth without medication reduction (72.6% vs. 53.8%), p=0.03. At 6 months postdischarge, of the 51 out of 131 youth with available follow-up data, 71% were doing well. Cost analysis based on discontinued medication by class showed monthly savings of $21,365, or $256,368 yearly. The largest contributor was the reduction in the use of antipsychotics, accounting for $205,332 of the total savings. CONCLUSIONS: Our study indicates that comprehensive treatment can lead to significant reductions in polypharmacy, and positive short- and longer-term treatment outcomes. Judicial prescribing also resulted in significant cost reduction in an already costly healthcare system.

SYFSA: a framework for systematic yet flexible systems analysis.

Although technological or organizational systems that enforce systematic procedures and best practices can lead to improvements in quality, these systems must also be designed to allow users to adapt to the inherent uncertainty, complexity, and variations in healthcare. We present a framework, called Systematic Yet Flexible Systems Analysis (SYFSA) that supports the design and analysis of Systematic Yet Flexible (SYF) systems (whether organizational or technical) by formally considering the tradeoffs between systematicity and flexibility. SYFSA is based on analyzing a task using three related problem spaces: the idealized space, the natural space, and the system space. The idealized space represents the best practice-how the task is to be accomplished under ideal conditions. The natural space captures the task actions and constraints on how the task is currently done. The system space specifies how the task is done in a redesigned system, including how it may deviate from the idealized space, and how the system supports or enforces task constraints. The goal of the framework is to support the design of systems that allow graceful degradation from the idealized space to the natural space. We demonstrate the application of SYFSA for the analysis of a simplified central line insertion task. We also describe several information-theoretic measures of flexibility that can be used to compare alternative designs, and to measure how efficiently a system supports a given task, the relative cognitive workload, and learnability.

Preliminary evaluation of medication therapy management services in assisted living facilities in rural Minnesota.

OBJECTIVES: While pharmacists are providing medication therapy management (MTM) services, few programs have been assessed for outcomes, particularly those in rural areas. The objectives are to: 1) categorize the number and type of drug-therapy problems (DTPs) that were identified by consultant pharmacists in assisted living facilities, 2) assess resident and administrator satisfaction of consultant pharmacist MTM services, and 3) assess the direct costs of providing MTM services and make some preliminary inferences about the program's economic viability. DESIGN: We conducted a cross-sectional, pilot study at 14 assisted living facilities over a 12-month period. SETTING: 14 rural assisted living facilities. PATIENTS: Assisted living facility residents. INTERVENTION: MTM visits were performed by a consultant pharmacist based on the MTM model developed by the American Pharmacists Association. MAIN OUTCOME MEASURES: Patient satisfaction, clinical (DTPs), and economic costs. RESULTS: 130 residents were enrolled with a mean age of 86 years, with 77.7% female. Residents were taking an average of 13 medications (range = 1-29), including prescription and nonprescription drugs. Common medical conditions were hypertension (80.0%), hyperlipidemia (37.7%), and diabetes (20.8%). Residents had an average of 9 comorbidities (range = 2-16) and 304 DTPs (mean 2.3 per resident). Residents reported high mean satisfaction levels with the pharmacist and the services provided. Direct costs of the intervention were $20,372.96, which suggests that only modest gains in overall patient health (or a few major avoided adverse events) caused by the intervention are necessary to ensure the program's economic viability. CONCLUSION: Through the use of MTM programs to resolve medication problems, pharmacists can improve patient satisfaction with care among assisted living facility residents.

Economic impact of a pharmacy resident in an assisted living facility-based medication therapy management program.

BACKGROUND: Pharmacists now have the opportunity to be reimbursed for providing medication therapy management (MTM) services. With 5% of the elderly population living in senior housing such as assisted living facilities, MTM programs need to be evaluated in this setting. OBJECTIVE: To evaluate the economic impact of a postgraduate year 1 (PGY1) pharmacy practice resident's interventions while performing MTM in an assisted living facility. METHODS: We conducted a prospective, evaluative study at an assisted living facility over 184 days. Patients included in the study were aged 57-100 years. MTM visits were performed by the PGY1 resident, based on the American Pharmacists Association consensus definition of the model of MTM services. The pharmacy resident prospectively collected data, including interventions made and patient demographics. Drug therapy recommendations were categorized and assessed for cost savings and acceptance by a primary care provider. RESULTS: Fifty-three patients were enrolled in the study (mean age 85.3 y). Patients were taking an average of 12 medications (prescription and nonprescription). The pharmacy resident made 125 recommendations to primary care providers; 72 of those were addressed and 90.3% of addressed recommendations were accepted. The largest category to elicit drug therapy recommendations was dose appropriateness. Of the 72 addressed recommendations, 17 (23.6%) resulted in direct cost savings totaling $3774. Costs that accrued as a result of drug therapy recommendations totaled $693. The resultant net cost-benefit was $1550, with a benefit-to-cost ratio of 1.7 and a return on investment of 70%. CONCLUSIONS: Our study demonstrates the positive value that a PGY1 pharmacy resident has on an assisted living-based MTM program, with respect to a positive drug-related cost-benefit and drug therapy recommendation acceptance.

Attitudes toward medical device use errors and the prevention of adverse events.

BACKGROUND: The design of a device's user interface often contributes to the chance of a user making an error in using the device. However, there is evidence that most such errors that occur in practice are attributed solely to the user and that the primary method of error prevention is to retrain the user. Yet this attitude may decrease the quality of error reports and the use of more effective error prevention strategies. A qualitative study was conducted to assess health care employees' attitudes toward device use errors and the prevention of adverse events. METHODS: Twenty-six health care employees from three hospital systems, including 11 device users and 15 nonusers who had participated in infusion pump purchasing decisions were given a scenario describing a device use error involving an infusion pump. Several open-ended questions assessed what they felt led to the event and how they would prevent the event from reccurring. RESULTS: The top three reported types of factors leading to the adverse event, in decreasing order of frequency, were the user, pump design problems, and lack of training. The top three prevention strategies reported by the participants were retraining the user, redesigning the device, and telling the user to be careful. DISCUSSION: These results suggest that health care employees still put too much emphasis on the traditional view of blaming and retraining the user.

Risk assessment and therapeutic indices of artesunate and artelinate in Plasmodium berghei-infected and uninfected rats.

Artesunate (AS) is being developed as a potential agent for the treatment of severe and complicated malaria. A risk assessment of the therapeutic index and related hematological changes of AS and artelinate (AL) following daily intravenous injection for 3 days was conducted in Plasmodium berghei-infected and uninfected rats. The minimum doses of AS and AL for parasitemia suppression were 2.3 and 2.5 mg/kg, respectively, and the suppressive doses for half parasitemia (SD50) were 7.4 and 8.6 mg/kg, respectively. The maximum tolerated dose (MTD) for AS was 240 mg/kg with a therapeutic index of 32.6. The MTD for AL was 80 mg/kg with a therapeutic index of 9.3. Hematological changes were studied on days 1 and 8 after the final dosing. In both AS- and AL-treated rats, dose-dependent and rapidly reversible hematological changes (significant reductions in RBC, HCT, Hb, and reticulocyte levels) were seen in the peripheral blood. Bone marrow evaluation revealed a statistically significant reduction in the myeloid/erythroid ratio only at the highest dose of AS (240 mg/kg), albeit still within the normal ratio range (1.0-1.5:1.0). Looking at the respective therapeutic indices the authors have concluded that AS is much safer than AL. Both drugs induced hematological changes in rats that parallel the dose-dependent, reversible anemia and reticulocytopenia previously reported in animals and humans. However, no significant bone marrow depression was seen for either agent.

Decisions about critical events in device-related scenarios as a function of expertise.

This paper presents the perspectives of personnel involved in decision-making about devices in critical care. We use the concept of "sharp and blunt ends" of practice to describe the performance of health care professionals. The "sharp end" is physically and temporally close to the system; the "blunt end" is removed from the system in time and space and yet affects the system through indirect influence on the sharp end. In this study, the sharp end is represented by the clinicians (nurses and doctors) and the blunt end by the administrators and biomedical engineers. These subjects represent the professionals involved in the decision-making process for purchasing biomedical equipment for the hospital. They were asked to "think aloud" while evaluating three error scenarios based on real events. The responses were recorded and transcribed for analysis. The results show differences in interpretation of critical events as a function of professional expertise. The clinicians (sharp-end practitioners) focused on clinical and human aspect of errors while the biomedical engineers focused on device-related errors. The administrators focused on documentation and training. These different interpretations mean that the problems are represented differently by these groups of subjects, and these representations result in variable decisions about devices. These results are discussed within a systems approach framework to help us assess the completeness of the problem representations of the subjects, their awareness of critical events, and how these events would collectively contribute to the occurrence of error.

Institutional decision-making for medical device purchasing: evaluating patient safety.

Many medical devices that are currently on the market are sub-optimal for human use, thus contributing to medical errors. This places significant responsibility for device selection on purchasers. This paper describes a retrospective analysis of decision making processes for infusion pump selection in three large hospitals and focuses on patient safety. Through a series of detailed interviews and a study of relevant documentation we characterized the nature of the decision-making, patterns of communication, and the roles of different participants. Findings suggest that success of the process is related to effective communication among participants with different expertise and adequate device usability assessment during the process. The paper discusses how information technology could provide support to distributed institutional decision making.

A cognitive taxonomy of medical errors.

OBJECTIVE: Propose a cognitive taxonomy of medical errors at the level of individuals and their interactions with technology. DESIGN: Use cognitive theories of human error and human action to develop the theoretical foundations of the taxonomy, develop the structure of the taxonomy, populate the taxonomy with examples of medical error cases, identify cognitive mechanisms for each category of medical error under the taxonomy, and apply the taxonomy to practical problems. MEASUREMENTS: Four criteria were used to evaluate the cognitive taxonomy. The taxonomy should be able (1) to categorize major types of errors at the individual level along cognitive dimensions, (2) to associate each type of error with a specific underlying cognitive mechanism, (3) to describe how and explain why a specific error occurs, and (4) to generate intervention strategies for each type of error. RESULTS: The proposed cognitive taxonomy largely satisfies the four criteria at a theoretical and conceptual level. CONCLUSION: Theoretically, the proposed cognitive taxonomy provides a method to systematically categorize medical errors at the individual level along cognitive dimensions, leads to a better understanding of the underlying cognitive mechanisms of medical errors, and provides a framework that can guide future studies on medical errors. Practically, it provides guidelines for the development of cognitive interventions to decrease medical errors and foundation for the development of medical error reporting system that not only categorizes errors but also identifies problems and helps to generate solutions. To validate this model empirically, we will next be performing systematic experimental studies.

Institutional decision-making to select patient care devices: identifying venues to promote patient safety.

Many medical errors that involve drug infusion devices are related to classic interface problems. Although manufacturers are becoming increasingly aware of human factors design considerations, many devices that are currently on the market are still sub-optimal for human use. This places significant responsibility for device selection on institutional purchasing groups. Theories of naturalistic decision-making point to many potential strengths and pitfalls of group decision-making processes that may affect the final outcome. This paper describes a retrospective analysis of decision-making process for infusion pump selection in a large hospital and focuses on factors related to patient safety. Through a series of detailed interviews and a study of relevant documentation we characterized the nature of the decision-making, patterns of communication, and the roles of different participants. Findings show that although the process involves a number of different professional groups and committees, the information flow among them is restricted. This results in inadequate representation of critical device usability considerations in the decision-making process. While all participants view device safety as an important consideration in the selection process, administrators (who are the final decision-makers) tend to equate safety with technical accuracy and reliability, paying less attention to the role of human factors in safe device use. Findings suggest that collaborative communication technology and automated evidence-based guidelines could provide support to institutional decision-making, ensuring that the process is efficient, effective, and ultimately safe for the patients.

International expert workshop on the analysis of the economic and public health impacts of air pollution: workshop summary.

Forty-nine experts from 18 industrial and developing countries met on 6 September 2001 in Garmisch-Partenkirchen, Germany, to discuss the economic and public health impacts of air pollution, particularly with respect to assessing the public health benefits from technologies and policies that reduce greenhouse gas (GHG) emissions. Such measures would provide immediate public health benefits, such as reduced premature mortality and chronic morbidity, through improved local air quality. These mitigation strategies also allow long-term goals--for example, reducing the buildup of GHG emissions--to be achieved alongside short-term aims, such as immediate improvements in air quality, and therefore benefits to public health. The workshop aimed to foster research partnerships by improving collaboration and communication among various agencies and researchers; providing a forum for presentations by sponsoring agencies and researchers regarding research efforts and agency activities; identifying key issues, knowledge gaps, methodological shortcomings, and research needs; and recommending activities and initiatives for research, collaboration, and communication. This workshop summary briefly describes presentations made by workshop participants and the conclusions of three separate working groups: economics, benefits transfer, and policy; indoor air quality issues and susceptible populations; and development and transfer of dose-response relationships and exposure models in developing countries. Several common themes emerged from the working group sessions and subsequent discussion. Key recommendations include the need for improved communication and extended collaboration, guidance and support for researchers, advances in methods, and resource support for data collection, assessment, and research.