Automated Item Generation: impact of item variants on performance and standard setting.

BACKGROUND: Automated Item Generation (AIG) uses computer software to create multiple items from a single question model. There is currently a lack of data looking at whether item variants to a single question result in differences in student performance or human-derived standard setting. The purpose of this study was to use 50 Multiple Choice Questions (MCQs) as models to create four distinct tests which would be standard set and given to final year UK medical students, and then to compare the performance and standard setting data for each. METHODS: Pre-existing questions from the UK Medical Schools Council (MSC) Assessment Alliance item bank, created using traditional item writing techniques, were used to generate four 'isomorphic' 50-item MCQ tests using AIG software. Isomorphic questions use the same question template with minor alterations to test the same learning outcome. All UK medical schools were invited to deliver one of the four papers as an online formative assessment for their final year students. Each test was standard set using a modified Angoff method. Thematic analysis was conducted for item variants with high and low levels of variance in facility (for student performance) and average scores (for standard setting). RESULTS: Two thousand two hundred eighteen students from 12 UK medical schools participated, with each school using one of the four papers. The average facility of the four papers ranged from 0.55-0.61, and the cut score ranged from 0.58-0.61. Twenty item models had a facility difference > 0.15 and 10 item models had a difference in standard setting of > 0.1. Variation in parameters that could alter clinical reasoning strategies had the greatest impact on item facility. CONCLUSIONS: Item facility varied to a greater extent than the standard set. This difference may relate to variants causing greater disruption of clinical reasoning strategies in novice learners compared to experts, but is confounded by the possibility that the performance differences may be explained at school level and therefore warrants further study.

The influence of candidates' physical attributes on patient ratings in simulated assessments of clinical practice.

BACKGROUND: We have previously shown that clinical examiners' scoring is not negatively impacted when a candidate has a tattoo, unnatural hair colour, or a regional accent. We investigated whether these physical attributes in exam candidates impact patient scoring. METHODS: Simulated/real patients were randomly assigned to watch five videos of simulated candidate performances of a cranial nerve examination: clear fail, borderline, good, 'clear pass' without an attribute, and 'clear pass' with one of the attributes (tattoo, purple hair, accent). Participants scored domains of communication and professionalism. We compared scores for the clear pass candidates with and without attributes. RESULTS: One hundred and eighty three patients participated. The total scores for the candidates with tattoos and purple hair were higher than the candidate with no physical attribute (p < 0.001). For the candidate with a Liverpool English accent no difference was identified (p = 0.120). CONCLUSIONS: The presence of certain physical attributes (tattoos or purple hair) was associated with higher scores given by patients to candidates in a simulated physical examination station.

The influence of candidates' physical attributes on assessors' ratings in clinical practice.

BACKGROUND: Assessments of physician competence in the work-place are common and often contribute to high-stakes assessments. Previous research suggests that assessors' judgements can be influenced by candidates' physical attributes. We investigated whether simulated candidates' scores were influenced by assessor bias based on tattoos, hair colour, and a regional accent. METHODS: We used an experimental, video-based, single-blinded, randomised, internet-based design. We created videos of simulated medical intern performances of a clinical examination at four different standards of competence. Four videos were also created of simulated candidates performing at a 'clear pass' standard, with either no stereotypical attribute (CPX), purple hair (CPH), tattoos (CPT) or a Liverpool English accent (CPA). Assessors were randomly assigned to watch five videos including the "clear pass" candidate without an attribute and one of the "clear pass" candidates with an attribute and asked to give an overall global grade for each candidate. We compared the global grades for the clear pass candidates with and without attributes. RESULTS: Ninety-eight assessors were included in the analysis. The total scores for the candidates with stereotyped attributes were not significantly lower than the candidate with no attribute. Assessors showed moderate levels of agreement between the global grades awarded for all the candidates. The global grades awarded to candidate with a stereotypical attribute were not significantly lower than for those without. CONCLUSIONS: The presence of tattoos, purple hair, or a regional accent did not systematically negatively influence the grade or score awarded by assessors to candidates in observed clinical examination scenarios.

Current issues in care and support for HIV-positive gay men in Sydney.

We explored current access to care among HIV-positive people in Australia. In 2006, 270 HIV-positive gay men from a community-based Positive Health cohort in Sydney were asked about their health (including medical and social) service needs and, subsequently, about difficulty in accessing services. We report the prevalence of specific needs, barriers and associated factors. Participants most commonly used general practitioners (64%) for HIV management and needed at least one HIV-related medical service (usually several: doctors experienced in HIV management, dentists and hospital pharmacies). Most participants were able to access them. Barriers in accessing services were related to their convenience rather than lack or quality. Cost emerged as a substantial barrier to dental care and psychological counselling (91% and 48% respectively of those in need). Need for an HIV-related social service was reported by 46% of respondents. Difficulties in accessing these related to poor services and staff attitudes. Income was associated with limited access to multiple services. In Australia, HIV-related medical service needs outweigh those for social services. Complex health services remain essential to HIV-positive people, but some services are currently not meeting their needs. To remain adequate, services need to understand and constantly adapt to the changing needs of HIV-positive people.

Molecular dynamics simulation of the cooperative adsorption of barley lipid transfer protein and cis-isocohumulone at the vacuum-water interface.

Molecular dynamic simulations have been carried out on systems containing a mixture of barley lipid transfer protein (LTP) and cis-isocohumulone (a hop derived iso-alpha-acid) in one of its enol forms, in bulk water and at the vacuum-water interface. In solution, the cis-isocohumulone molecules bind to the surface of the LTP molecule. The mechanism of binding appears to be purely hydrophobic in nature via desolvation of the protein surface. Binding of hop acids to the LTP leads to a small change in the 3-D conformation of the protein, but no change in the proportion of secondary structure present in helices, even though there is a significant degree of hop acid binding to the helical regions. At the vacuum-water interface, cis-isocohumulone shows a high surface activity and adsorbs rapidly at the interface. LTP then shows a preference to bind to the preadsorbed hop acid layer at the interface rather than to the bare water-vacuum interface. The free energy of adsorption of LTP at the hop-vacuum-water interface is more favorable than for adsorption at the vacuum-water interface. Our results support the view that hop iso-alpha-acids promote beer foam stability by forming bridges between separate adsorbed protein molecules, thus strengthening the adsorbed protein layer and reducing foam breakdown by lamellar phase drainage. The results also suggest a second mechanism may also occur, whereby the concentration of protein at the interface is increased via enhanced protein adsorption to adsorbed hop acid layers. This too would increase foam stability through its effect on the stabilizing protein layer around the foam bubbles.

Comparison of the adsorbed conformation of barley lipid transfer protein at the decane-water and vacuum-water interface: a molecular dynamics simulation.

Molecular dynamics simulation is used to model the adsorption of the barley lipid transfer protein (LTP) at the decane-water and vacuum-water interfaces. Adsorption at both surfaces is driven by displacement of water molecules from the interfacial region. LTP adsorbed at the decane surface exhibits significant changes in its tertiary structure, and penetrates a considerable distance into the decane phase. At the vacuum-water interface LTP shows small conformational changes away from its native structure and does not penetrate into the vacuum space. Modification of the conformational stability of LTP by reduction of its four disulphide bonds leads to an increase in conformational entropy of the molecules, which reduces the driving force for adsorption. Evidence for changes in the secondary structure are also observed for native LTP at the decane-water interface and reduced LTP at the vacuum-water interface. In particular, intermittent formation of short (six-residue) regions of beta-sheet is found in these two systems. Formation of interfacial beta-sheet in adsorbed proteins has been observed experimentally, notably in the globular milk protein beta-lactoglobulin and lysozyme.

Sexual risk behaviour, viral load, and perceptions of HIV transmission among homosexually active Latino men: an exploratory study.

The objective of the paper was to describe awareness of HIV medications and HIV viral load, and to assess the impact of HIV medications (including highly active antiretroviral therapy) and notions of viral load on sexual risk practices. This was an exploratory cross-sectional study of a non-random sample of 395 homosexually active Latino men in New York City. An anonymous self-administered questionnaire was used focusing on perceptions about HIV/AIDS, HIV treatment medications and viral load, risk perceptions, HIV status, and sexual partners and practices in the past 6 months. HIV-positive participants taking HIV medications, those who knew their viral loads, and those who had undetectable viral loads did not report significantly different frequencies of high-risk (receptive or insertive) unprotected anal intercourse (UAI) in comparison to other HIV-positive individuals in the sample. Perceptions about HIV status (assuming same status) and viral load (penetrating partner or letting partner penetrate one) were related to significantly higher high-risk UAI in the past 6 months. HIV prevention strategies for homosexually active Latino men should not only address the basic issue of assuming similar HIV status but also need to examine misconstructions of the transmissibility of HIV as a result of undetectable viral load.