Automated Item Generation: impact of item variants on performance and standard setting.

BACKGROUND: Automated Item Generation (AIG) uses computer software to create multiple items from a single question model. There is currently a lack of data looking at whether item variants to a single question result in differences in student performance or human-derived standard setting. The purpose of this study was to use 50 Multiple Choice Questions (MCQs) as models to create four distinct tests which would be standard set and given to final year UK medical students, and then to compare the performance and standard setting data for each. METHODS: Pre-existing questions from the UK Medical Schools Council (MSC) Assessment Alliance item bank, created using traditional item writing techniques, were used to generate four 'isomorphic' 50-item MCQ tests using AIG software. Isomorphic questions use the same question template with minor alterations to test the same learning outcome. All UK medical schools were invited to deliver one of the four papers as an online formative assessment for their final year students. Each test was standard set using a modified Angoff method. Thematic analysis was conducted for item variants with high and low levels of variance in facility (for student performance) and average scores (for standard setting). RESULTS: Two thousand two hundred eighteen students from 12 UK medical schools participated, with each school using one of the four papers. The average facility of the four papers ranged from 0.55-0.61, and the cut score ranged from 0.58-0.61. Twenty item models had a facility difference > 0.15 and 10 item models had a difference in standard setting of > 0.1. Variation in parameters that could alter clinical reasoning strategies had the greatest impact on item facility. CONCLUSIONS: Item facility varied to a greater extent than the standard set. This difference may relate to variants causing greater disruption of clinical reasoning strategies in novice learners compared to experts, but is confounded by the possibility that the performance differences may be explained at school level and therefore warrants further study.

The influence of candidates' physical attributes on patient ratings in simulated assessments of clinical practice.

BACKGROUND: We have previously shown that clinical examiners' scoring is not negatively impacted when a candidate has a tattoo, unnatural hair colour, or a regional accent. We investigated whether these physical attributes in exam candidates impact patient scoring. METHODS: Simulated/real patients were randomly assigned to watch five videos of simulated candidate performances of a cranial nerve examination: clear fail, borderline, good, 'clear pass' without an attribute, and 'clear pass' with one of the attributes (tattoo, purple hair, accent). Participants scored domains of communication and professionalism. We compared scores for the clear pass candidates with and without attributes. RESULTS: One hundred and eighty three patients participated. The total scores for the candidates with tattoos and purple hair were higher than the candidate with no physical attribute (p < 0.001). For the candidate with a Liverpool English accent no difference was identified (p = 0.120). CONCLUSIONS: The presence of certain physical attributes (tattoos or purple hair) was associated with higher scores given by patients to candidates in a simulated physical examination station.

Age and gender differences in value orientation among American adolescents.

This study investigated the development of a caring value orientation in four age groups. The sample included 1,247 students in grades 6, 8, 10, and 12 who were asked to complete a questionnaire, containing 14 contrasting word pairs and phrases, designed to measure preference for more caring, personal values. Gender differences were found across age groups and geographic regions. These findings are discussed in terms of theories that hypothesize gender differences in worldview.

Patient care report cards: an analysis.

In the competitive health care market of the 1990s, health care institutions face a significant number of fiscal challenges that threaten their survival. As part of the survival process, institutions must demonstrate--to the public and to regulatory agencies--that they are measuring the effect of their care structures and processes. Institutions must be willing to share these measurements with the public as well as to use them internally to identify performance problems and document the impact of process improvements or other changes. The report card is a simple, easily understood method for reporting quality data and comparing varying aspects of patient care processes and outcomes. However, as measurements, report cards are not robust enough to serve as the sole source of quality-related data, and interinstitutional comparisons may be misleading. This article explores the new phenomenon known as patient care report cards and discusses their usage in quality measurement and improvement.

The relationship of adamantinomatous craniopharyngioma to ghost cell ameloblastoma of the jaws: a histopathologic and immunohistochemical study.

The objective of this investigation was to study the relationship of the ghost cell ameloblastoma (GCA), which is a form of type II calcifying odontogenic cyst (COC), to the adamantinomatous craniopharyngioma (ACP). H&E sections of 26 examples of ACP were compared to three cases of GCA and to the reported microscopic features of that tumor. Clinical records of the ACPs were studied to determine their biologic behavior compared to that of the ameloblastomas. Immunohistochemical studies of nine examples of ACP were performed for KL1 (high mol.wt cytokeratins), 5D3 (low mol.wt cytokeratins) and involucrin (characteristic of terminally differentiated keratinocytes) using the peroxidase-antiperoxidase method. The results were compared with those reported for COC and ameloblastoma. ACP and GCA exhibited similar microscopic features, including pre-ameloblasts, tissue resembling stellate reticulum, ghost cells and calcifications; both tumors grew slowly and were invasive. ACP and COC, and by interpolation GCA, exhibited similar features with all three antibodies. The ghost cells did not exhibit any immunoreactivity but the adjacent cells stained positively for involucrin. The immunological features of ACP were similar to those reported in ameloblastomas for squamous differentiation. However, because of their rarity, no ameloblastomas exhibiting keratinization, including ghost cells, have yet been studied with these antibodies. We conclude that ACP and GCA are homologous lesions.

The promise of automated wellness systems.

Do automated wellness programs hold out the promise of improving the health of individuals and populations through the establishment of baseline wellness checkpoints? Nurse case managers have a strong interest in this question. The automation of wellness programs and the integration of this information into the continuum of a care information system model offers ongoing, interactive accountability of individual health status. The crowning glory will be the ultimate integration of the individual's personal health game plan with the provider's continuum of care offering. The strength of the nursing profession lies in its traditional role of promoting health through preventive measures (public health nursing) and managing the ill patient's progress toward his/her optimal health status. Therefore, nurses and nurse case managers should embrace wellness programs and wellness data availability. Because of their experience in using and understanding wellness data, nurses are in a unique position. Not only will they be able to assimilate rapidly the use of wellness data into their own practice, they also will be able to guide other professionals in its use. The table in this articles provides examples of the impact that automated wellness program data could have at various stages of the clinical decision-making process.

How the case manager can influence the clinical information system selection process.

The selection and implementation of a clinical information system that provides meaningful patient data in a format that is usable by the clinician is a challenge facing many healthcare institutions. In the first issue of Nursing Case Management, we explored the information needs of the nurse case manager and the available and emerging computerized information systems tools to meet these specific needs. This follow-up article will focus on a typical clinical information system selection process; outline specific ways the nurse case manager can effectively influence that process; and provide examples and a checklist that will enable the nurse case manager to assist the institution in selecting and implementing the most appropriate system.

Information systems tools available to the case manager.

Regardless of the setting, the role of the nurse as a manager of care continues to grow and develop. The nurse case manager needs to have vital information readily available to facilitate critical thinking, analysis, and decision-making on the appropriateness of care provided to patients and to ensure that optimal outcomes are achieved. While there are a variety of clinical information systems available, it is evident that the complexity of the case manager's information needs presents challenges for software developers. Accordingly, nurse case managers must take an active role in defining and communicating their needs. The purpose of this article (the first of two) is to present a general overview of the information needs of the nurse case manager and the computerized information system tools available (and emerging) to meet these needs. A follow-up article in the next issue of Nursing Case Management will focus on a typical clinical information system selection process; outline specific ways the nurse case manager can effectively influence that process; and provide a checklist that will enable the nurse case manager to assist the institution in selecting and implementing the most appropriate system.

Structural evidence that human liver and placental alkaline phosphatase isoenzymes are coded by different genes.

Human liver alkaline phosphatase [ortho-phosphoric monoester phosphohydrolase (alkaline optimum), EC 3.1.3.1] was purified, and some of its physical and chemical properties were examined and compared to those of human placental alkaline phosphatase. The results indicated a different peptide structure for each, based upon HB2-terminal residue sequence, two-dimensional tryptic peptide maps, and different amino acid compositions. These data are interpreted to indicate that the enzymes are synthesized by different structural genes. Other molecular properties differentiating the two enzymes were a higher apparent molecular weight for the liver enzyme from sodium dodecyl sulfate gel electrophoresis, a higher S20,w value, different carbohydrate content, and a different isoelectric point. The immunochemical specificity of each enzyme was not affected by removal of sialic acid groups. Both enzymes are similar in that they are dimers of equal molecular weight subunits, and are probably homodimers.