Cognitive Complaint Types Can Correlate With Cognitive Testing, Perceived Stress, and Symptom Distress in Older Adults With Normal Cognition and Dementia.

OBJECTIVE: We examined how cognitive complaint types (CCTs) correlate with cognitive testing, perceived stress, and symptom distress in older adults with normal cognition and dementia. METHODS: Older adults (n = 259) with normal cognition, mild cognitive impairment, or mild-stage Alzheimer disease completed cognitive testing and self-report measures (Cognitive Difficulties Scale, Global Distress Index, Perceived Stress Scale). Cross-sectional analyses examined: (1) CCT composition by classification method,( 2) CCTs by diagnostic group, (3) correlations of CCTs with cognitive testing scores, and (4) correlations of CCTs with perceived stress and symptom distress. RESULTS: CCTs derived from 2 classification approaches loaded onto 4 factors: memory, attention-concentration (AC), temporal orientation, and praxis. Memory contained complaints about both memory and executive functioning. AC contained both classifications of AC complaints. Complaints about AC (AC1 and AC2) differed by diagnostic group (all P < 0.05). One of 2 classifications of AC (AC1) complaints discerned between impaired and unimpaired long-delay memory scores (both P < 0.05). In multivariable analyses, that same classification of AC (AC1) complaints correlated with higher perceived stress (both P < 0.001) but not symptom distress (both P > 0.05). CONCLUSION: CCTs showed a factor structure that was mostly robust between classification methods; however, some content-divergent CCTs shared factors, suggesting construct overlap. Relatively slight variations in content altered how CCTs correlated with diagnostic groups, perceived stress, and symptom distress. Most CCTs did not discern between impaired and unimpaired cognitive test scores. Research is needed to better understand CCTs as clinical markers and targets of clinical interventions.

Awareness of diagnosis predicts changes in quality of life in individuals with mild cognitive impairment and mild stage dementia.

BACKGROUND AND OBJECTIVE: This observational study examined how awareness of diagnosis predicted changes in cognition and quality of life (QOL) 1 year later in older adults with normal cognition and dementia diagnoses. RESEARCH DESIGN AND METHODS: Older adults (n = 259) with normal cognition, mild cognitive impairment (MCI), or mild stage Alzheimer's disease (AD) completed measures of diagnostic awareness, cognition, and multiple domains of QOL. We compared 1-year change in cognition and QOL by diagnostic group and diagnostic awareness. RESULTS: Patients who were unaware of their diagnosis at baseline showed average decreases in both satisfaction with daily life (QOL-AD; paired mean difference (PMD) = -0.9, p < 0.05) and physical functioning (SF-12 PCS; PMD = -2.5, p < 0.05). In contrast, patients aware of their diagnosis at baseline showed no statistically discernable changes in most QOL domains (all p > 0.05). Of patients aware of their diagnosis at baseline (n = 111), those who were still aware (n = 84) showed a decrease in mental functioning at follow up (n = 27; SF-12 MCS). Change in MoCA scores in patients unaware of their diagnosis was similar to that in patients aware of their diagnosis, -1.4 points (95% CI -2.6 to -0.6) and -1.7 points (95% CI -2.4 to -1.1) respectively. DISCUSSION AND IMPLICATIONS: Awareness of one's diagnosis of MCI or AD, not the severity of cognitive impairment, may predict changes in patients' mental functioning, expectations of their memory, satisfaction with daily life, and physical functioning. The findings may help clinicians anticipate the types of threats to wellbeing that a patient might encounter and identify key domains for monitoring.

A confirmatory factor analysis of the novice nurse practitioner role transition scale.

As interest in supporting new nurse practitioners' (NPs) transition to practice increases, those interested in measuring the concept will need an instrument with evidence of reliability and validity. The Novice NP Role Transition (NNPRT) Scale is the first instrument to measure the concept. The preliminary exploratory factor analysis revealed a five-factor structure: organizational alignment, mentorship, sense of purpose, perceived competence and self-confidence, and compensation. Using a cross-sectional design and data from 210 novice NPs, the purpose of this study was to confirm the NNPRT Scale's internal factor structure via confirmatory factor analysis (CFA). The sample was primarily female (97.5%), White (75.9%), and certified in primary care (53.5%). The CFA confirmed the five-factor structure, and model fit was improved by moving and omitting items (χ2 [619] = 1277.799, p < 0.001; Root Mean Square Error of Approximation = 0.071 [0.066-0.077]). The final NNPRT Scale includes 37-items, and internal consistency reliability was calculated at 0.95. Convergent validity evidence was supported by a positive, significant correlation with receiving a formal orientation in the first NP position; a negative, significant correlation with turnover intention; and a lack of a relationship with years of prior registered nurse experience. The NNPRT Scale is an instrument with sound evidence of reliability and validity. The NNPRT Scale will be useful for researchers, administrators, and clinicians looking to explore factors that affect NNPRT, as well as by clinicians and administrators implementing programs to support novice NPs' transition to practice.

Relationships Between Cognitive Complaints and Quality of Life in Older Adults With Mild Cognitive Impairment, Mild Alzheimer Disease Dementia, and Normal Cognition.

PURPOSE: To examine in persons with varying degrees of cognitive impairment the relationship between self-reports of cognitive complaints and quality of life (QOL). METHODS: Older adults (n=259) with normal cognition, mild cognitive impairment (MCI), and mild stage Alzheimer disease (AD) dementia completed tests of cognition and self-report questionnaires about QOL and 3 kinds of cognitive complaints: cognitive difficulties, distress from cognitive difficulties, and believing you had more memory problems than most people. Bivariate, multivariable, and multivariate regression analyses assessed relationships between domains of QOL and each cognitive complaint. RESULTS: Bivariate and multivariable analyses controlling for severity of cognitive and functional impairment found that cognitive complaints were related to relatively lower quality of daily life (QOL-AD, Dementia Quality of Life Scale), greater depression (GDS), more anxiety (BAI), higher perceived stress (PSS), and lower general mental well-being (SF-12 MCS). DISCUSSION: Cognitive complaints have robust associations with QOL. These findings have implications for AD prevention trials and management of clinical populations.

Fit for purpose and modern validity theory in clinical outcomes assessment.

PURPOSE: The US Food and Drug Administration (FDA), as part of its regulatory mission, is charged with determining whether a clinical outcome assessment (COA) is "fit for purpose" when used in clinical trials to support drug approval and product labeling. In this paper, we will provide a review (and some commentary) on the current state of affairs in COA development/evaluation/use with a focus on one aspect: How do you know you are measuring the right thing? In the psychometric literature, this concept is referred to broadly as validity and has itself evolved over many years of research and application. REVIEW: After a brief introduction, the first section will review current ideas about "fit for purpose" and how it has been viewed by FDA. This section will also describe some of the unique challenges to COA development/evaluation/use in the clinical trials space. Following this, we provide an overview of modern validity theory as it is currently understood in the psychometric tradition. This overview will focus primarily on the perspective of validity theorists such as Messick and Kane whose work forms the backbone for the bulk of high-stakes assessment in areas such as education, psychology, and health outcomes. CONCLUSIONS: We situate the concept of fit for purpose within the broader context of validity. By comparing and contrasting the approaches and the situations where they have traditionally been applied, we identify areas of conceptual overlap as well as areas where more discussion and research are needed.

What features of stigma do the public most commonly attribute to Alzheimer's disease dementia? Results of a survey of the U.S. general public.

INTRODUCTION: Understanding the prevalence of beliefs, attitudes, and expectations about Alzheimer's disease dementia in the public could inform strategies to mitigate stigma. METHODS: Random sample of 317 adults from the U.S. public was analyzed to understand reactions toward a man with mild-stage Alzheimer's disease dementia. RESULTS: In adjusted analyses, over half of respondents expected the person to be discriminated against by employers (55.3%; 95% confidence interval [CI] = 47.0-65.2) and be excluded from medical decision-making (55.3%; 95% CI = 46.9-65.4). Almost half expected his health insurance would be limited based on data in the medical record (46.6%; 95% CI = 38.0-57.2), a brain imaging result (45.6%, 95% CI = 37.0-56.3), or genetic test result (44.7%; 95% CI = 36.0-55.4). DISCUSSION: Public education and policies are needed to address concerns about employment and insurance discrimination. Studies are needed to discover how advances in diagnosis and treatment may change Alzheimer's disease stigma.

Development and initial testing of the Penn Parkinson's Daily Activities Questionnaire.

OBJECTIVE: The aim of this work was to describe the development and psychometric analysis of the Penn Parkinson's Daily Activities Questionnaire. The questionnaire is an item response theory-based tool for rating cognitive instrumental activities of daily living in PD. METHODS: Candidate items for the Penn Parkinson's Daily Activities Questionnaire were developed through literature review and focus groups of patients and knowledgeable informants. Item selection and calibration of item-response theory parameters were performed using responses from a cohort of PD patients and knowledgeable informants (n = 388). In independent cohorts of PD patients and knowledgeable informants, assessments of test-retest reliability (n = 50), and construct validity (n = 68) of the questionnaire were subsequently performed. Construct validity was assessed by correlating questionnaire scores with measures of motor function, cognition, an existing activities of daily living measure, and directly observed daily function. RESULTS: Fifty items were retained in the final questionnaire item bank. Items were excluded owing to redundancy, difficult reading level, and when item-response theory parameters could not be calculated. Test-retest reliability was high (intraclass correlation coefficient = 0.97; P < 0.001). The questionnaire correlated strongly with cognition (r = 0.68; P < 0.001) and directly observed daily function (r = 0.87; P < 0.001), but not with motor impairment (r = 0.08; P = 0.53). The questionnaire score accurately discriminated between PD patients with and without dementia (receiver operating characteristic curve = 0.91; 95% confidence interval: 0.85-0.97). CONCLUSIONS: The Penn Parkinson's Daily Activities Questionnaire shows strong evidence of reliability and validity. Item response theory-based psychometric analysis suggests that this questionnaire can discriminate across a range of daily functions.

Why are spousal caregivers more prevalent than nonspousal caregivers as study partners in AD dementia clinical trials?

OBJECTIVES: Most Alzheimer disease (AD) caregivers are not spouses and yet most AD dementia trials enroll spousal study partners. This study examines the association between caregiver relationship to the patient and willingness to enroll in an AD clinical trial and how caregiver burden and research attitudes modify willingness. DESIGN: Interviews with 103 AD caregivers who met criteria for ability to serve as a study partner. RESULTS: A total of 54% of caregivers were spouses or domestic partners and the remaining were adult children. Willingness to enroll a patient in a clinical trial was associated with being a spouse [odds ratio (OR)=2.53, P=0.01], increasing age (OR=1.39, P=0.01), and increasing scores on the Research Attitudes Questionnaire (OR=1.39, P<0.001). No measures of caregiver burden or patient health were significant predictors of willingness. In multivariate models both research attitudes (OR=1.37, P<0.001) and being a spouse, as opposed to an adult child, (OR=2.06, P=0.048) were independently associated with willingness to participate. CONCLUSIONS: Spousal caregivers had both a higher willingness to participate and a more positive attitude toward research. Caregiver burden had no association with willingness to participate. The strongest predictor of willingness was research attitudes.

Analysis of Surgeon and Program Characteristics Associated with Success on American Board of Surgery Exam Outcomes.

BACKGROUND: Existing research exploring predictors of success on American Board of Surgery (ABS) exams focused on either resident or residency program characteristics, but limited studies focus on both. This study examines relationships between both resident and program characteristics and ABS Qualifying (QE) and Certifying Exam (CE) outcomes. STUDY DESIGN: Multilevel logistic regression was used to analyze the relationship between resident and program characteristics and ABS QE and CE 1st attempt pass and eventual certification. Resident characteristics were gender, IMG status, and prior performance, measured by 1st attempt USMLE Step 2 CK and Step 3 scaled scores. Program characteristics were size, %female, %International Medical Graduate (IMG), and program type. The sample included surgeons with QE and CE data from 2007-2019 and matched USMLE scores. RESULTS: Controlling for other variables, prior medical performance positively related to all ABS exam outcomes. The relationships between USMLE scores and success on ABS exams varied but were generally strong. Other resident characteristics that predicted ABS exam outcomes were gender and IMG (QE 1st attempt pass). The only program characteristic that significantly predicted ABS outcomes was %IMG (QE and CE 1st attempt pass). Despite statistical significance, gender, IMG, and %IMG translated to small differences in predicted probabilities of ABS exam success. CONCLUSION: This study highlights resident and program characteristics that predict success on ABS exams. USMLE scores consistently and strongly related to ABS exam success, providing evidence that USMLE scores relate to future high-stakes consequences like board certification. After controlling for prior performance, gender, IMG, and program %IMG significantly related to ABS exam success, but effects were small.

Differential Item Functioning Analysis of United States Medical Licensing Examination Step 1 Items.

PURPOSE: Previous studies have examined and identified demographic group score differences on United States Medical Licensing Examination (USMLE) Step examinations. It is necessary to explore potential etiologies of such differences to ensure fairness of examination use. Although score differences are largely explained by preceding academic variables, one potential concern is that item-level bias may be associated with remaining group score differences. The purpose of this 2019-2020 study was to statistically identify and qualitatively review USMLE Step 1 exam questions (items) using differential item functioning (DIF) methodology. METHOD: Logistic regression DIF was used to identify and classify the effect size of DIF on Step 1 items meeting minimum sample size criteria. After using DIF to flag items statistically, subject matter expert (SME) review was used to identify potential reasons why items may have performed differently between racial and gender groups, including characteristics such as content, format, wording, context, or stimulus materials. USMLE SMEs reviewed items to identify the group difference they believed was present, if any; articulate a rationale behind the group difference; and determine whether that rationale would be considered construct relevant or construct irrelevant. RESULTS: All identified DIF rationales were relevant to the constructs being assessed and therefore did not reflect item bias. Where SME-generated rationales aligned with statistical differences (flags), they favored self-identified women on items tagged to women's health content categories and were judged to be construct relevant. CONCLUSIONS: This study did not find evidence to support the hypothesis that group-level performance differences beyond those explained by prior academic performance variables are driven by item-level bias. Health professions examination programs have an obligation to assess for group differences, and when present, investigate to what extent, if any, measurement bias plays a role.

Revisiting Retake Policy: Analyzing the Success Rates of Examinees With Multiple Attempts on the United States Medical Licensing Examination.

PURPOSE: A testing program's decision making on retake policy considers a number of factors, including fairness to examinees, examination security, examination purpose, and classification accuracy. For high-stakes licensure and certification examinations charged with protection of the public, this includes balancing fairness issues inherent in the potential for false negatives with the public protection need to minimize false positives. Since 2012, the United States Medical Licensing Examination (USMLE) has maintained a policy of ≤ 6 attempts on any examination component. The purpose of this study was to empirically examine the appropriateness of existing USMLE retake policy. METHOD: A sample of over 300,000 examinees who were administered each Step examination between 2008 and 2018 was examined to address how pass rates (per attempt and cumulatively) differed for examinees and examinee subgroups via graphic depictions of pass rates, the impact of restricting the number of attempts on examinees and examinee subgroups via logistic regression analyses, and the impact of restricting the number of attempts on classification error via classification error analyses. RESULTS: Repeaters displayed consistently lower pass rates on subsequent attempts. Regardless of Step, most passing examinees did so by their fourth attempt. Models predicting the log odds of examinees taking ≥ 5 attempts showed a significant effect of gender, race, and medical school status, although the associated sample sizes are small. Misclassification analyses showed that, as attempts increased, the percentage of false passers increased and percentage of false failers decreased. CONCLUSIONS: These considerations led to the USMLE's decision to reduce its attempt limit from 6 to 4 on any Step or Step component effective July 2021. This change balances the competing interests of fairness and examination access to the examinee population, while simultaneously maintaining a minimum standard consistent with the mission of the USMLE program to protect the public.

Development of the novice nurse practitioner role transition scale: An exploratory factor analysis.

BACKGROUND: Novice nurse practitioner role transition (NNPRT) can be described as stressful and turbulent, leading to decreased job satisfaction and increased intent to leave. No published instrument exists to measure NNPRT. Thus, researchers, educators, and administrators are limited in their ability to measure the concept and therefore understand the factors that lead to a successful, or unsuccessful, role transition experience. An instrument with evidence of validity and reliability is needed to conduct large-scale and systematic examinations of NNPRT. PURPOSE: The purpose of this study was to develop and examine the initial factor structure of a novel instrument that measures NNPRT. METHODS: Initial item development was guided by concept analysis, literature review, and qualitative data. Face and content validity were established from expert review. Using pilot data from 89 novice nurse practitioners (NPs), an exploratory factor analysis (EFA) was conducted to examine the instrument's internal factor structure. RESULTS: The NNPRT Scale includes 40 items that measure an individual's perception of their role transition experience. The EFA revealed a five-factor structure: organizational alignment, mentorship, sense of purpose, perceived competence and self-confidence, and compensation. IMPLICATIONS FOR PRACTICE: In an evolving health care system, provider well-being is at the center of workforce, educational, and organizational conversations. Understanding how to optimize the workforce and prepare NPs for health care delivery is increasingly important. The NNPRT Scale will allow for large-scale examinations of the factors that influence NP role transition, as well as assess interventions to prepare and support novice NPs' transitions.

Examining Demographics, Prior Academic Performance, and United States Medical Licensing Examination Scores.

PURPOSE: To examine whether demographic differences exist in United States Medical Licensing Examination (USMLE) scores and the extent to which any differences are explained by students' prior academic achievement. METHOD: The authors completed hierarchical linear modeling of data for U.S. and Canadian allopathic and osteopathic medical graduates testing on USMLE Step 1 during or after 2010, and completing USMLE Step 3 by 2015. Main outcome measures were computer-based USMLE examinations: Step 1, Step 2 Clinical Knowledge, and Step 3. Test-taker characteristics included sex, self-identified race, U.S. citizenship status, English as a second language, and age at first Step 1 attempt. Covariates included composite Medical College Admission Test (MCAT) scores, undergraduate grade point average (GPA), and previous USMLE scores. RESULTS: A total of 45,154 examinees from 172 medical schools met the inclusion criteria. The sample was 67% white and 48% female; 3.7% non-U.S. citizens; and 7.4% with English as a second language. Hierarchical linear models examined demographic variables with and without covariates including MCAT scores and GPA. All Step examinations showed significant differences by gender after adding covariates, varying by Step. Racial differences were observed for each Step, attenuated by the addition of covariates. CONCLUSIONS: Demographic differences in USMLE performance were tempered by previous examination performance and undergraduate performance. Additional research is required to identify factors that contribute to demographic differences, can aid educators' identification of students who would benefit from assistance preparing for USMLE, and can assist residency program directors in assessing performance measures while meeting diversity goals.

Awareness of Mild Cognitive Impairment and Mild Alzheimer's Disease Dementia Diagnoses Associated With Lower Self-Ratings of Quality of Life in Older Adults.

OBJECTIVE: This study examined how awareness of diagnostic label impacted self-reported quality of life (QOL) in persons with varying degrees of cognitive impairment. METHOD: Older adults (n = 259) with normal cognition, Mild Cognitive Impairment (MCI), or mild Alzheimer's disease dementia (AD) completed tests of cognition and self-report questionnaires that assessed diagnosis awareness and multiple domains of QOL: cognitive problems, activities of daily living, physical functioning, mental wellbeing, and perceptions of one's daily life. We compared measures of QOL by cognitive performance, diagnosis awareness, and diagnostic group. RESULTS: Persons with MCI or AD who were aware of their diagnosis reported lower average satisfaction with daily life (QOL-AD), basic functioning (BADL Scale), and physical wellbeing (SF-12 PCS), and more difficulties in daily life (DEM-QOL) than those who were unaware (all p ≤ .007). Controlling for gender, those expecting their condition to worsen over time reported greater depression (GDS), higher stress (PSS), lower quality of daily life (QOL-AD, DEM-QOL), and more cognitive difficulties (CDS) compared to others (all p < .05). DISCUSSION: Persons aware of their diagnostic label-either MCI or AD-and its prognosis report lower QOL than those unaware of these facts about themselves. These relationships are independent of the severity of cognitive impairment.

Performance on the Cardiovascular In-Training Examination in Relation to the ABIM Cardiovascular Disease Certification Examination.

The American College of Cardiology In-Training Exam (ACC-ITE) is incorporated into most U.S. training programs, but its relationship to performance on the American Board of Internal Medicine Cardiovascular Disease (ABIM CVD) Certification Examination is unknown. ACC-ITE scores from third-year fellows from 2011 to 2014 (n = 1,918) were examined. Covariates for regression analyses included sex, age, medical school country, U.S. Medical Licensing Examination Step, and ABIM Internal Medicine Certification Examination scores. A secondary analysis examined fellows (n = 511) who took the ACC-ITE in the first and third years. ACC-ITE scores were the strongest predictor of ABIM CVD scores (p < 0.0001), and the most significant predictor of passing (p < 0.0001). The change in ACC-ITE scores from first to third year was a strong predictor of the ABIM CVD score (p < 0.001). The ACC-ITE is strongly associated with performance on the ABIM CVD Certification Examination.

Bayesian Prior Choice in IRT Estimation Using MCMC and Variational Bayes.

This study investigated the impact of three prior distributions: matched, standard vague, and hierarchical in Bayesian estimation parameter recovery in two and one parameter models. Two Bayesian estimation methods were utilized: Markov chain Monte Carlo (MCMC) and the relatively new, Variational Bayesian (VB). Conditional (CML) and Marginal Maximum Likelihood (MML) estimates were used as baseline methods for comparison. Vague priors produced large errors or convergence issues and are not recommended. For both MCMC and VB, the hierarchical and matched priors showed the lowest root mean squared errors (RMSEs) for ability estimates; RMSEs of difficulty estimates were similar across estimation methods. For the standard errors (SEs), MCMC-hierarchical displayed the largest values across most conditions. SEs from the VB estimation were among the lowest in all but one case. Overall, VB-hierarchical, VB-matched, and MCMC-matched performed best. VB with hierarchical priors are recommended in terms of their accuracy, and cost and (subsequently) time effectiveness.

Identifying an appropriate measurement modeling approach for the Mini-Mental State Examination.

The Mini-Mental State Examination (MMSE) is a 30-item, dichotomously scored test of general cognition. A number of benefits could be gained by modeling the MMSE in an item response theory (IRT) framework, as opposed to the currently used classical additive approach. However, the test, which is built from groups of items related to separate cognitive subdomains, may violate a key assumption of IRT: local item independence. This study aimed to identify the most appropriate measurement model for the MMSE: a unidimensional IRT model, a testlet response theory model, or a bifactor model. Local dependence analysis using nationally representative data showed a meaningful violation of the local item independence assumption, indicating multidimensionality. In addition, the testlet and bifactor models displayed superior fit indices over a unidimensional IRT model. Statistical comparisons showed that the bifactor model fit MMSE respondent data significantly better than the other models considered. These results suggest that application of a traditional unidimensional IRT model is inappropriate in this context. Instead, a bifactor model is suggested for future modeling of MMSE data as it more accurately represents the multidimensional nature of the scale. (PsycINFO Database Record

The Penn Parkinson's Daily Activities Questionnaire-15: Psychometric properties of a brief assessment of cognitive instrumental activities of daily living in Parkinson's disease.

INTRODUCTION: To describe the psychometric properties of the Penn Parkinson's Daily Activities Questionnaire-15 (PDAQ-15), a 15-item measure of cognitive instrumental activities of daily living for Parkinson's disease (PD) patients derived from the original 50-item PDAQ. METHODS: PDAQ-15 items were chosen by expert consensus. Knowledgeable informants of PD participants (n = 161) completed the PDAQ-15. Knowledgeable informants were defined as an individual having regular contact with the PD participant. PD participants were assigned a diagnosis of normal cognition, mild cognitive impairment, or dementia based on expert consensus. RESULTS: PDAQ-15 scores correlated strongly with global cognition (Dementia Rating Scale-2, r = 0.71, p < 0.001) and a performance-based functional measure (Direct Assessment of Functional Status, r = 0.83; p < 0.001). PDAQ-15 scores accurately discriminated between non-demented PD participants (normal cognition/mild cognitive impairment) and PD with dementia (ROC curve area = 0.91), participants with and without any cognitive impairment (normal cognition versus mild cognitive impairment/dementia, ROC curve area = 0.85) and between participants with mild cognitive impairment and dementia (ROC curve area = 0.84). CONCLUSIONS: The PDAQ-15 shows good discriminant validity across cognitive stages, correlates highly with global cognitive performance, and appears suitable to assess daily cognitive functioning in PD.

Does success on the American Board of Surgery general surgery qualifying examination guarantee certifying examination success?

OBJECTIVE: The purpose of this study was to explore the relationship between qualifying examination (QE) and certifying examination (CE) results and to determine whether an appropriate cut-point on the QE would predict success on the CE. DESIGN: The scaled American Board of Surgery (ABS) QE scores of all first-time examinees from 2006 to 2010 were retrospectively matched to their first-time CE pass/fail decisions. Contingency tables illustrating the QE-CE relationship were constructed and appropriate correlational statistics were computed. A receiver operating characteristic (ROC) curve analysis was constructed, with sensitivity and 1-specificity calculated for each possible QE cut-point used to predicted CE pass/fail classifications. Additionally, the area under the curve (AUC) was calculated. PARTICIPANTS: All first-time American Board of Surgery examinees for the Surgery Qualifying Examination from 2006 to 2010. RESULTS: A total of 4385 surgeons were analyzed, with QE scores averaging 82.1 ± 5.8 (range, 58-99) and 82.8% of surgeons passing the CE on their first attempt. Contingency tables suggest a moderate relationship between QE and CE performance, although correlation indexes are low (phi = 0.13, point-biserial = 0.23). For the ROC analysis, the AUC = 0.674 (95% CI; 0.654-0.695) provides a better than chance pass/fail classification (p < 0.001), yet does not meet the minimum threshold for acceptability as a predictive test. No QE cut-point accurately predicted CE pass/fail decisions. CONCLUSIONS: While a moderate relationship between QE scores and CE performance is evident, correlations suggest that the 2 examinations measure different abilities. The low AUC value on the ROC analysis, along with poor predictability at all possible cut-points, show that no appropriate cut-point on the QE predicts success on the CE. These data add to the validity of both tests by providing evidence that distinct latent traits are being measured by both tests.

Measuring how people view biomedical research: Reliability and validity analysis of the Research Attitudes Questionnaire.

With increasing numbers of studies on research ethics and a need to improve the recruitment of research subjects, the ability to measure attitudes toward biomedical research has become important. The Research Attitudes Questionnaire is a significant predictor of the public's attitudes toward and willingness to participate in research, yet limited data are available on its psychometric properties. This study establishes the scale's internal consistency and dimensionality using a large Internet-based sample from the United States. One item was removed due to a poor item-total correlation, and three additional items were removed which formed a reverse-wording measurement artifact factor. With improved internal consistency and dimensionality, the seven-item version has the advantages of shorter administration time and improved psychometric properties.