The predictive validity of MCAT scores and undergraduate GPA for COMLEX-USA licensure exam performance of students enrolled in osteopathic medical schools.

CONTEXT: Osteopathic (Doctor of Osteopathic Medicine [DO]) medical students account for more than 25 % of all medical students in the United States. OBJECTIVES: This study examined the predictive validity of Medical College Admission Test (MCAT) total scores and cumulative undergraduate grade point averages (UGPAs) for performance on the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 1 and Level 2-CE (Cognitive Evaluation) licensure examinations administered by the National Board of Osteopathic Medical Examiners (NBOME). Additionally, the study examined the degree to which MCAT total scores and UGPAs provide comparable prediction of student performance by key sociodemographic variables. METHODS: This study involved a collaborative effort between the Association of American Medical Colleges (AAMC), the American Association of Colleges of Osteopathic Medicine (AACOM) and the NBOME. Data were examined for 39 accredited DO-granting medical schools in the United States during the 2017 application cycle. Researchers utilized three regression models that included MCAT total scores, cumulative UGPA, and combined MCAT total scores and cumulative UGPA to determine predictive validity. Researchers also examined the comparability of prediction for sociodemographic variables by examining the differences between observed and predicted error for both scores and pass/fail success rates. RESULTS: Medium to large correlations were discernible between MCAT total scores, UGPA, and COMLEX-USA examination outcomes. For both COMLEX-USA Level 1 and Level 2-CE scores and pass/fail outcomes, MCAT scores alone provided superior predictive value to UGPA alone. However, MCAT scores and UGPA utilized in conjunction provided the best predictive value. When predicting both licensure examination scores and pass/fail outcomes by sociodemographic variables, all three models provided comparable predictive accuracy. CONCLUSIONS: Findings from this comprehensive study of DO-granting medical schools provide evidence for the value-added benefit of taking MCAT scores and UGPA into consideration, particularly when these measures are utilized in conjunction. Further, findings provide evidence indicating that individuals from different sociodemographic backgrounds who enter medical school with similar MCAT scores and UGPA perform similarly on licensure examination outcome measures.

A validity study of COMLEX-USA Level 3 with the new test design.

CONTEXT: The National Board of Osteopathic Medical Examiners (NBOME) administers the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA), a three-level examination designed for licensure for the practice of osteopathic medicine. The examination design for COMLEX-USA Level 3 (L3) was changed in September 2018 to a two-day computer-based examination with two components: a multiple-choice question (MCQ) component with single best answer and a clinical decision-making (CDM) case component with extended multiple-choice (EMC) and short answer (SA) questions. Continued validation of the L3 examination, especially with the new design, is essential for the appropriate interpretation and use of the test scores. OBJECTIVES: The purpose of this study is to gather evidence to support the validity of the L3 examination scores under the new design utilizing sources of evidence based on Kane's validity framework. METHODS: Kane's validity framework contains four components of evidence to support the validity argument: Scoring, Generalization, Extrapolation, and Implication/Decision. In this study, we gathered data from various sources and conducted analyses to provide evidence that the L3 examination is validly measuring what it is supposed to measure. These include reviewing content coverage of the L3 examination, documenting scoring and reporting processes, estimating the reliability and decision accuracy/consistency of the scores, quantifying associations between the scores from the MCQ and CDM components and between scores from different competency domains of the L3 examination, exploring the relationships between L3 scores and scores from a performance-based assessment that measures related constructs, performing subgroup comparisons, and describing and justifying the criterion-referenced standard setting process. The analysis data contains first-attempt test scores for 8,366 candidates who took the L3 examination between September 2018 and December 2019. The performance-based assessment utilized as a criterion measure in this study is COMLEX-USA Level 2 Performance Evaluation (L2-PE). RESULTS: All assessment forms were built through the automated test assembly (ATA) procedure to maximize parallelism in terms of content coverage and statistical properties across the forms. Scoring and reporting follows industry-standard quality-control procedures. The inter-rater reliability of SA rating, decision accuracy, and decision consistency for pass/fail classifications are all very high. There is a statistically significant positive association between the MCQ and the CDM components of the L3 examination. The patterns of associations, both within the L3 subscores and with L2-PE domain scores, fit with what is being measured. The subgroup comparisons by gender, race, and first language showed expected small differences in mean scores between the subgroups within each category and yielded findings that are consistent with those described in the literature. The L3 pass/fail standard was established through implementation of a defensible criterion-referenced procedure. CONCLUSIONS: This study provides some additional validity evidence for the L3 examination based on Kane's validity framework. The validity of any measurement must be established through ongoing evaluation of the related evidence. The NBOME will continue to collect evidence to support validity arguments for the COMLEX-USA examination series.

The Predictive Value of the Residency AOBFP In-Service Exam, Produced and Administered by ACOFP.

BACKGROUND AND OBJECTIVES: Board certification is acknowledged as the mainstay for ensuring quality physician-delivered health care within medical specialties. The American College of Osteopathic Family Physicians (ACOFP) administers the American Osteopathic Board of Family Physicians' (AOBFP) In-Service Examination (ISE) to provide residents and program directors with a formative examination to assess competency and preparation for successful completion of the AOBFP certifying examination (CE). Unique assessment processes are integral to monitoring development of the osteopathic family physician throughout training and into practice, and to verify their competency for the safety and protection of the public. This study sought to investigate whether performance on the AOBFP ISE predicted performance on the AOBFP CE, and thereby successfully equipped residents to safely enter medical practice. METHODS: In 2020, data from 1,893 PGY-1 through PGY-3 residents (2016-2018), whose ISE scores could be matched with scores on the AOBFP initial board CE, were analyzed for this study. RESULTS: Correlations among ISE administrations across 3 years of postgraduate medical education were in the mid-to-high .6 range; the ISE scores correlated with CE scores in the mid .4 to high .5 range. Less reliable measures of positive predictive value were 0.99, and sensitivity was 0.91. CONCLUSIONS: Results suggest that ISE administrations during residency training are effective in developing remediation strategies for subsequent successful CE performance. The inclusion of osteopathic principles in the AOBFP CE necessitates inclusion of osteopathic content in resident training exams like ISE.

Meaningful use of COMSAE Phase 1 in preparation for COMLEX-USA Level 1.

CONTEXT: The Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) is a three level national standardized licensure examination for the practice of osteopathic medicine. The Comprehensive Medical Self Assessment Examination (COMSAE) is a three phase self assessment tool designed to gauge the base knowledge and ability of candidates preparing for COMLEX-USA. OBJECTIVES: To investigate how COMSAE Phase 1 (Phase 1) was used by candidates and how completing Phase 1 impacted their performance on the COMLEX-USA Level 1 (Level 1) examination. METHODS: Using data from the 2018-2019 administration of Level 1 and Phase 1 examinations, we counted the frequency of the unique Phase 1 forms taken by the candidates and calculated the correlation between the candidates' first attempt Phase 1 scores and the number of Phase 1 forms taken. We then calculated the correlation between the Level 1 scores and the Phase 1 scores. Next, we applied a multilevel regression model to examine the candidates' score improvement on the multiple Phase 1 forms taken. Finally, we investigated the effect of practicing through Phase 1 on the candidates' Level 1 performance using logistic regression models. RESULTS: The majority of candidates took one (2,414; 33.9%) to two (2,196; 30.8%) timed Phase 1 forms prior to the Level 1 examination. There was a significant negative correlation (r=-0.48, t(6,505)=-44.05, p<0.001) between the candidates' first attempt Phase 1 scores and the number of Phase 1 forms taken. There was a strong and positive correlation (r=0.66 to 0.74, p<0.001) between Phase 1 and Level 1 scores. With other variables controlled, on average, candidates' Phase 1 scores increased 23.2 points on one attempt from the previous attempt. Having the most recent Phase 1 score controlled, a greater number of Phase 1 forms taken was associated with an improvement on the Level 1 performance. CONCLUSIONS: The significant correlation between Phase 1 and Level 1 performance provided validity evidence for Phase 1. Moreover, our results suggested that candidates, especially those with lower performance on their initial Phase 1 attempt, might improve their Level 1 performance by taking multiple Phase 1 forms to monitor their academic improvement and gauge their readiness for Level 1.

Evidence-Based Redesign of the COMLEX-USA Series.

To ensure that the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) reflects the evolving practice of osteopathic medicine, the National Board of Osteopathic Medical Examiners has developed new content and format specifications for an enhanced, competency-based examination program to be implemented with COMLEX-USA Level 3 in 2018. This article summarizes the evidence-based design processes that served as the foundation for blueprint development and the evidence supporting its validity. An overview is provided of the blueprint's 2 dimensions: Competency Domains and Clinical Presentations. The authors focus on the evidence that supports interpretation of test scores for the primary and intended purpose of COMLEX-USA, which is osteopathic physician licensure. Important secondary uses and the educational and catalytic effect of assessments are also described. This article concludes with the National Board of Osteopathic Medical Examiners' plans to ensure that the COMLEX-USA series remains current and meets the needs of its stakeholders-the patients who seek care from osteopathic physicians.

The Use of COMLEX-USA and USMLE for Residency Applicant Selection.

BACKGROUND: The Comprehensive Osteopathic Medical Licensing Examination (COMLEX-USA) and the United States Medical Licensing Examination (USMLE) are recognized by all state medical licensing boards in the United States, and the Federation of State Medical Boards has supported the validity of both examinations for medical licensure. Many osteopathic medical students take both examinations. OBJECTIVE: The purpose of this study was to investigate performance on COMLEX-USA Level 1 and USMLE Step 1 of students from colleges of osteopathic medicine where the majority of students took both examinations. METHODS: Data were collected on the entering classes of 2010 and 2011. Relationships between the COMLEX-USA Level 1 and the USMLE Step 1 were quantified using Pearson correlations. The correlation between outcomes on the 2 examinations was evaluated using the phi coefficient. A contingency table was constructed to look at first-attempt outcomes (pass/fail). RESULTS: Data for 2010 and 2011 were collected from 3 osteopathic medical schools, with 795 of 914 students (87%) taking both examinations. The correlation between first-attempt COMLEX-USA Level 1 and USMLE Step 1 scores was statistically significant across and within all 3 schools. The overall correlation was r(795) = 0.84 (P < .001). Pass/fail status on the 2 examinations was moderately correlated (φ = 0.39, P < .01). CONCLUSIONS: Our study found a strong association between COMLEX Level 1 and USMLE Step 1 performance. Additional studies to accurately compare scores on these examinations are warranted.

Osteopathic Manipulative Treatment Technique Scores on the COMLEX-USA Level 2-PE: An Analysis of the Skills Assessed.

CONTEXT: The National Board of Osteopathic Medical Examiners uses the Osteopathic Manipulative Treatment (OMT) Global Rating Tool to score candidate performance in OMT during standardized patient encounters. To the authors' knowledge, no validity research has been published on the underlying constructs measured by this instrument to date. OBJECTIVE: To evaluate whether the OMT Global Rating Tool measures 1 holistic skill or multiple skill sets. METHODS: Structural equation models were fit to data from first-time test-takers who took the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) between July 7, 2014, and April 30, 2015. Two theoretical models were evaluated: a model that posited a single underlying skill set, and a model that posited 2 related but distinct skill sets. RESULTS: The 1-factor and 2-factor models were fit to data from 4673 third- and fourth-year osteopathic medical students. Fit statistics indicated that the data were best fit by a model representing 2 factors: pre- and post-OMT technique assessment and OMT technique (r=0.47). CONCLUSION: The OMT Global Rating Tool seems to measure 2 distinct but related skills. Although the results of this study are applicable to broad populations and cannot be used to provide skill-specific subscores, it might be helpful to remediation efforts to target pre- and post-OMT technique assessments and OMT techniques as distinct areas for intervention.

Preparation Strategies of Osteopathic Medical Students for the COMLEX-USA Level 2-PE.

CONTEXT: Since 2002, osteopathic medical schools have made curricular changes to further enhance the clinical skills of their students, to prepare them for residency training, and to pass the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). OBJECTIVE: To report how students at osteopathic medical schools prepare for the COMLEX-USA Level 2-PE, and to investigate the effect of these techniques on examination performance. METHODS: A standardized survey was given to students before the beginning of their examination to assess the preparation of osteopathic medical students for the COMLEX-USA Level 2-PE, such as coursework, orientation materials, and standardized patient (SP) encounters. Surveys that were completed by first-time test takers during the 2013-2014 and 2014-2015 test cycles were included in this study. RESULTS: Of 9120 surveys administered, 8733 were completed, achieving a response rate of 95.8%. Of those 8733 respondents, 8706 students (99.7%) reported having SP encounters during the first and second year of medical school, and 7379 (84%) reported having at least 1 SP encounter in years 3 and 4. Of 8733 students, 6079 (70%) reported receiving feedback from an osteopathic physician on their SP encounters, and 6049 (69%) and 6253 (72%) reported having viewed the COMLEX-USA Level 2-PE orientation video online and having read the examination's orientation guide, respectively. The largest difference in preparation between students who passed the COMLEX-USA Level 2-PE and students who did not was a prerequisite SP examination at their school, with 5574 students (68.9%) who passed reporting having participated compared with 364 students (56.5%) who failed. None of the differences in clinical skills training and test preparation was associated with statistically significant differences in pass or fail status. CONCLUSION: Osteopathic medical students use a variety of methods to enhance their clinical skills in preparation for the COMLEX-USA Level 2-PE, with universal use of SP programs since the COMLEX-USA Level 2-PE was implemented in 2004. Educators should continue to foster ways to develop students' clinical skills that reflect new advances in education and assessment to ensure that future osteopathic physicians can demonstrate competency in fundamental clinical skills before beginning postgraduate training.

Consistency of interrater scoring of student performances of osteopathic manipulative treatment on COMLEX-USA Level 2-PE.

CONTEXT: Assessment of osteopathic manipulative treatment (OMT) is included in the National Board of Osteopathic Medical Examiners' Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). The scores earned for OMT should be equivalent among all raters regardless of which technique is scored or which rater is scoring the performance. As a quality assurance measure, selected examination dates and the encounters within the administration of COMLEX-USA Level 2-PE are scored by 2 raters: first by a "live" rater and next by a quality assurance rater. Neither rater knows if he or she is the first or second rater. OBJECTIVE: To compare candidate's scores recorded for OMT on COMLEX-USA Level 2-PE to determine whether differences exist among raters and techniques scored. METHODS: The authors evaluated candidate performances that took place from July through November 2012. For each performance, 2 raters scored the same technique or different techniques using the OMT scoring rubric. Discrepancies between scores were compared using t tests. Statistical significance was set at P<.05 for most analyses. RESULTS: Of the 708 performances, there was no statistically significant difference in scoring whether the OMT raters scored the same technique or different techniques when the students performed more than 1. There were no statistically significant differences between these results and instances when only a single technique was performed and scored. CONCLUSION: The present study provides reliability evidence for the use of the global OMT scoring tool in the evaluation of OMT in COMLEX-USA Level 2-PE.

Keyboard data entry use among osteopathic medical students and residents.

CONTEXT: Candidates taking the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) are currently evaluated on their ability to document clinical findings using a handwritten postencounter note. However, keyboard data entry is increasingly used for medical documentation. OBJECTIVE: To determine the use and perception of keyboard data entry among osteopathic medical students and residents in educational and clinical settings. METHODS: A Web-based survey regarding frequency of and preference for keyboard data entry was distributed to 9801 osteopathic medical students, 17,268 osteopathic residents, and 34 clinical deans of colleges of osteopathic medicine (COMs). In addition, 31 COMs' clinical skills center directors were contacted to participate in a telephone survey about the use of keyboard data entry in their centers. RESULTS: A total of 1711 students, 1198 residents, 14 clinical deans, and 17 clinical skills center directors responded to the surveys. The majority of students (872 [51%]) reported using electronic keyboard data entry at their COM's clinical skills center for postencounter notes. Among respondents, 379 students (23%), 77 residents (9%), and 1 clinical dean reported that electronic keyboard data entry is never or rarely used during clinical rotations. Most trainees (1592 students [93%], 864 residents [94%]) reported that they were either comfortable or very comfortable with typing. Given the option of recording methods for SOAP (subjective, objective, assessment, plan) note findings on the COMLEX-USA Level 2-PE, 7 clinical deans were unsure of their students' preferences, while the remaining favored keyboard data entry (5) over handwriting (2). The majority of student and resident respondents would choose keyboard data entry (1009 [60%] and 511 [55%], respectively). CONCLUSION: Osteopathic medical students and residents are comfortable with typing; they are exposed to and would prefer using an electronic form of entry for medical documentation. These results support a conversion from written postencounter notes to keyboard data entry of notes on the COMLEX-USA Level 2-PE.

Patterns of misrepresentation of clinical findings on patient notes during the COMLEX-USA Level 2-PE.

CONTEXT: A health care team depends on the accurate documentation of the patient-physician encounter, be it written or electronic. If documentation is inaccurate or incomplete, patient care may be adversely affected. Previous studies have identified factors that influence documentation errors, such as fatigue, carelessness, and being overworked. More research, however, is needed into the patterns of errors and, by extension, these patterns' potential effect on patient care. Insights about these areas would be valuable to practicing physicians, as well as to medical educators, who could incorporate such insights into the training of medical students. OBJECTIVE: To identify potential patterns of commissive documentation errors on SOAP notes during the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE); specifically, to calculate the frequency with which specific items are misrepresented in SOAP (subjective, objective, assessment, plan) notes and to explore patterns of misrepresentation in relation to the documentation of history taking and physical examination. METHODS: Among the 12,510 candidates tested between July 2007 and June 2010, there were 24 candidates who failed the COMLEX-USA Level 2-PE because they misrepresented clinical findings on SOAP notes. These errors of commission (over-documentation) were categorized as errors in either history-taking or physical examination, and then subcategorized and analyzed to determine if meaningful patterns in the documentation of these errors occurred. RESULTS: A total of 662 errors were recorded among the 24 candidates. History-taking items were more often misrepresented (n=378) than physical examination items (n=284). Patient history-a subcategory that included past medical, surgical, medication, allergy, and family histories-was more likely to be misrepresented than other subcategories under history taking (eg, "history of present illness" and "associated symptoms") and many physical examination subcategories. Results of the analysis indicated that the misrepresentations were not evenly distributed (chi2(10)=323.1; P<.01): candidates whose SOAP notes contained misrepresentations were statistically more likely to misrepresent in some categories rather than in others. CONCLUSION: Physicians and medical students should pay particular attention to patient history, lifestyle history, and neurologic examination to minimize the risk of documentation errors.

An evaluation of cost and appropriateness of care as recommended by candidates on a national clinical skills examination.

BACKGROUND: The National Board of Osteopathic Medical Examiners administers the COMLEX-USA Level 2-PE, an assessment of clinical skills of osteopathic medical students. This evaluation includes developing a patient care plan. PURPOSE: Based on one simulated case, we investigated the appropriateness and cost of care and quantified their relationship to performance. METHODS: Four hundred sixty-seven postencounter notes were coded for appropriateness using expert physician judgments and for cost of care using Centers for Medicare and Medicaid Services data. Various outcome measures were correlated with physician scores. RESULTS: In this case, candidates recommended an average of 5.6 interventions with an average cost of $227 and appropriateness rating of 2.4 on a 1 (indicated) to 4 (potentially dangerous) scale. Total cost and inappropriateness of actions were negatively correlated with candidate scores (r = -.208, p < .0001 and r = -.318, p < .0001, respectively). CONCLUSIONS: Results from this investigation provide some evidence to support the validity of physician note ratings of patient care plans and demonstrate the need to include these principles in medical education.

Pass/fail patterns of candidates who failed COMLEX-USA level 2-PE because of misrepresentation of clinical findings on postencounter notes.

CONTEXT: In 2007, The National Board of Osteopathic Medical Examiners (NBOME) instituted a policy to address the accuracy and integrity of postencounter written documentation recorded during the Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). This policy was instituted not only to protect the integrity of the examination, but also to highlight that overdocumentation of clinical findings not obtained during patient encounters may jeopardize patient safety. OBJECTIVE: To investigate overall and domain pass/fail patterns of candidates who misrepresented clinical findings with regard to past and subsequent performance on COMLEX-USA Level 2-PE. Specifically, to investigate what percentage of candidates failed because of misrepresentation on first attempts and how they performed on subsequent administrations, as well as the previous performance patterns of candidates who failed because of misrepresentation on examination retakes. METHODS: Historical records from NBOME's COMLEX-USA Level 2-PE database (testing cycles 2007-2008, 2008-2009, and 2009-2010) were used to analyze overall and domain pass/fail patterns of candidates who failed at least once because of misrepresentation of clinical findings. RESULTS: Of the 24 candidates who failed because of misrepresentation of postencounter (SOAP) notes, 20 candidates (83%) were first-time examinees. Four candidates (17%) were repeating the examination, 2 of whom were making a third attempt to pass. Among these 20 candidates who failed because of misrepresentation of clinical findings on their first attempt, 19 passed on their next attempt. At the time of study analysis, all but 2 candidates eventually passed the examination in subsequent attempts. CONCLUSION: Among candidates found to have misrepresented clinical findings on postencounter written documentation on COMLEX-USA Level 2-PE, no pattern existed between their past or subsequent performance with regard to overall or domain pass/fail results. The vast majority of these candidates passed the examination on subsequent administrations, and none failed twice because of misrepresentation. Consequences of misrepresentation of clinical findings on COMLEX-USA Level 2-PE are severe and may serve to both raise awareness and prevent these types of errors in the future.

Relationship between standardized patient checklist item accuracy and performing arts experience.

INTRODUCTION: : It is not known whether a Standardized Patient's (SP's) performing arts background could affect his or her accuracy in recording candidate performance on a high-stakes clinical skills examination, such as the Comprehensive Osteopathic Medical Licensing Examination Level 2 Performance Evaluation. The purpose of this study is to investigate the differences in recording accuracy of history and physical checklist items between SPs who identify themselves as performing artists and SPs with no performance arts experience. METHODS: : Forty SPs identified themselves as being performing artists or nonperforming artists. A sample of SP live examination ratings were compared with a second set of ratings obtained after video review (N = 1972 SP encounters) over 40 cases from the 2008-2009 testing cycle. Differences in SP checklist recording accuracy were tested as a function of performing arts experience. RESULTS: : Mean overall agreement rates, both uncorrected and corrected for chance agreement, were very high (0.94 and 0.79, respectively, at the overall examination level). There was no statistically significant difference between the two groups with respect to any of the mean accuracy measures: history taking (z = -0.422, P = 0.678), physical examination (z = -1.453, P = 0.072), and overall data gathering (z = -0.812, P = 0.417) checklist items. CONCLUSION: : Results suggest that SPs with or without a performing arts background complete history taking and physical examination checklist items with high levels of precision. Therefore, SPs with and without performing arts experience can be recruited for high-stakes SP-based clinical skills examinations without sacrificing examination integrity or scoring accuracy.

Relationship between encounter time and candidate performance on COMLEX-USA Level 2-PE.

BACKGROUND: The National Board of Osteopathic Medical Examiners uses a standardized timing format for all osteopathic medical candidates across all stations in the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). Examination standardization, and therefore validity, is weakened if time administered affects student performance. OBJECTIVE: To investigate the relationship between the amount of time students spent with standardized patients (SPs) and their clinical skill component scores. METHODS: Data were analyzed from 249 candidates in 21 test sessions from the 2007-2008 testing cycle. Trained SPs recorded the time each examinee entered and exited each encounter. Outcome measures of the examination included pass (1 if pass; 0 if fail) performance outcome, pass by domain (biomedical/biomechanical and humanistic), and pass by component scores (data gathering, osteopathic manipulative treatment, and patient note scores). Analysis of variance (ANOVA), followed by pairwise comparisons, was used to compare encounter times across cases. RESULTS: A total of 2988 SP encounters were analyzed. Examinees spent an average 12.1 minutes in each patient encounter. Candidates spent less time with cases classified as health promotion and disease prevention than with cases that primarily involved a problem visit, but amount of time spent with cases representing various content areas was not significantly different (P>.05). On average, candidates spent the least amount of time (11.95 minutes) with gastrointestinal cases and the most time (12.26 minutes) with patients who had complaints relating to the neuromusculoskeletal system. A trivial relationship was shown between the continuous variable encounter time as the average amount of time a student spends in the encounter and the dichotomous clinical skill outcome of the examination (r=.03). The relationship between these 2 variables was consistent when examined separately for the humanistic domain (r=-.10) and the biomedical/biomechanical domain (r=.12). The strongest correlation was between encounter time and the data-gathering component score (r=.28). CONCLUSION: Time spent in the encounter was not influential in overall candidate performance in the COMLEX-USA Level 2-PE. Candidates spent a similar amount of time with cases of different content areas. This finding suggests that 14 minutes is sufficient for this osteopathic high-stakes performance evaluation.

Patient note fabrication and consequences of unprofessional behavior in a high-stakes clinical skills licensing examination.

BACKGROUND: The National Board of Osteopathic Medical Examiners (NBOME) administers the Comprehensive Osteopathic Medical Licensing Examination USA Level 2-PE (COMLEX-USA Level 2-PE) and has developed a process that links the competencies of written communication and professionalism by tracking fabrication in the postencounter SOAP (Subjective, Objective, Assessment, Plan) note exercise. METHOD: A process used to identify potential SOAP note fabrication was implemented in the 2007-2008 test cycle for the COMLEX-USA Level 2-PE. RESULTS: A total of 3,753 candidates took the COMLEX-USA Level 2-PE in the 2007-2008 test cycle. Forty-eight candidates were screened, and the NBOME's Subcommittee on SOAP Note Fabrication made failure decisions on eight, leading to a failure rate of 0.2% based on fabrication review. CONCLUSIONS: The NBOME has adopted the stance that postencounter note fabrication represents unprofessional behavior. Screening for and failing candidates who exhibit unprofessional behavior enhances the validity of the examination.