Perspectives of Dermatology Program Directors on the Impact of Step 1 Pass/Fail.

INTRODUCTION: The shift of Step 1 to Pass/Fail has generated several questions and concerns about obtaining residency positions among allopathic and osteopathic students alike. Determining the perspectives of Dermatology Program Directors in regards to post-Step 1 Pass/Fail is critical for students to better prepare for matching into dermatology. METHODS: After receiving Institutional Review Board (IRB) exemption status, the program directors were chosen from 144 Accreditation Council for Graduate Medical Education (ACGME) and 27 American Osteopathic Association (AOA) Dermatology programs using contact information from their respective online website databases. An eight-item survey was constructed on a three-point Likert scale, one free text response, and four demographic questions. The anonymous survey was sent out over the course of three weeks with weekly individualized reminder requests for participation. RESULTS: A total of 54.54% of responders had "Letters of Recommendation" in their top 3. Forty-five percent of responders had "Completed Audition Rotation at Program" in their top 3. And, 38.09% of responders had "USMLE Step 2 CK Scores" in their top 3. CONCLUSION: Approximately 50% of responders agreed that all medical students will have more difficulty matching dermatology. Based on the survey study, Dermatology program directors want to focus more on letters of recommendation, audition rotations, and Step 2 CK scores. Because each field seems to prioritize different aspects of an application, students should attempt to gain as much exposure to different fields such as through research and shadowing to narrow down their ideal specialties. Consequently, the student will have more time to tailor their applications to what residency admissions are looking for.

Modeling relationships between traditional preadmission measures and clinical skills performance on a medical licensure examination.

Medical schools employ a variety of preadmission measures to select students most likely to succeed in the program. The Medical College Admission Test (MCAT) and the undergraduate college grade point average (uGPA) are two academic measures typically used to select students in medical school. The assumption that presently used preadmission measures can predict clinical skill performance on a medical licensure examination was evaluated within a validity argument framework (Kane 1992). A hierarchical generalized linear model tested relationships between the log-odds of failing a high-stakes medical licensure performance examination and matriculant academic and non-academic preadmission measures, controlling for student-and school-variables. Data includes 3,189 matriculants from 22 osteopathic medical schools tested in 2009-2010. Unconditional unit-specific model expected average log-odds of failing the examination across medical schools is -3.05 (se = 0.11) or 5%. Student-level estimated coefficients for MCAT Verbal Reasoning scores (0.03), Physical Sciences scores (0.05), Biological Sciences scores (0.04), uGPA(science) (0.07), and uGPA(non-science) (0.26) lacked association with the log-odds of failing the COMLEX-USA Level 2-PE, controlling for all other predictors in the model. Evidence from this study shows that present preadmission measures of academic ability are not related to later clinical skill performance. Given that clinical skill performance is an important part of medical practice, selection measures should be developed to identify students who will be successful in communication and be able to demonstrate the ability to systematically collect a medical history, perform a physical examination, and synthesize this information to diagnose and manage patient conditions.

An evaluation of cost and appropriateness of care as recommended by candidates on a national clinical skills examination.

BACKGROUND: The National Board of Osteopathic Medical Examiners administers the COMLEX-USA Level 2-PE, an assessment of clinical skills of osteopathic medical students. This evaluation includes developing a patient care plan. PURPOSE: Based on one simulated case, we investigated the appropriateness and cost of care and quantified their relationship to performance. METHODS: Four hundred sixty-seven postencounter notes were coded for appropriateness using expert physician judgments and for cost of care using Centers for Medicare and Medicaid Services data. Various outcome measures were correlated with physician scores. RESULTS: In this case, candidates recommended an average of 5.6 interventions with an average cost of $227 and appropriateness rating of 2.4 on a 1 (indicated) to 4 (potentially dangerous) scale. Total cost and inappropriateness of actions were negatively correlated with candidate scores (r = -.208, p < .0001 and r = -.318, p < .0001, respectively). CONCLUSIONS: Results from this investigation provide some evidence to support the validity of physician note ratings of patient care plans and demonstrate the need to include these principles in medical education.

Survey of Osteopathic Medical Students Regarding Physician Shadowing Experiences Before and During Medical School Training.

INTRODUCTION: Shadowing a physician is an observational experience which includes a student observing a licensed healthcare provider caring for patients. Shadowing is commonly done by students before and during medical school, but little is known about the nature or extent of these extra-curricular observational experiences. OBJECTIVE: We hypothesized that shadowing experiences were common yet variable. We investigated the prevalence, nature, and perceived value of medical student experiences with shadowing physicians (both before and during medical school). METHODS: This survey-based study was non-experimental with a cross-sectional convenience sample of osteopathic medical students about their shadowing experiences before and during medical school. The survey was sent to all matriculated osteopathic medical students (OMS1-4) for the 2017 to 2018 academic year from two medical schools: Philadelphia College of Osteopathic Medicine (PCOM) (1084 total students) and PCOM-Georgia (554 total students). The final survey instrument included three sections: demographics (6 questions), pre-medical shadowing experiences (21 questions), and medical student shadowing experiences (24 questions). RESULTS: Respondents (357) identified themselves as OMS1 (96), OMS2 (89), OMS3 (73), OMS4 (95) and other (2, OMS5) with enrollment at PCOM-Philadelphia (242) and PCOM-Georgia (115). Among survey respondents, 339 (95.5%) reported shadowing a physician as a pre-medical student, and 110 (30.8%) reported shadowing (outside of their required clinical rotations) a physician during medical school. Requirements to participate were inconsistent; fewer than 50% of shadowing experiences required Health Insurance Portability and Accountability Act of 1996 (HIPAA) training, proof of vaccination, or purified protein derivative (PPD) documentation. In addition to observation, pre-medical and medical students, respectively, participated in history taking (44 [13%], 47 [42.7%]), physical examinations (45 [13.3%], 44 [40%]) and procedures (13, [3.8%], 20 [18.2%]) during their shadowing experiences. Motivations to participate in shadowing varied between pre-medical and medical student experiences, but both groups mentioned their desire to learn more about a particular discipline, obtain letters of recommendation, and gain patient care experience. Students recommended both pre-medical (273 [80.5%]) and medical school (93 [84.5%]) shadowing to future students. CONCLUSION: Shadowing remains a common and important tool for students to learn about patient care, medicine and careers. The nature of each shadowing experience and participation requirements are quite variable. Measures to ensure patient safety, confidentiality, liability and supervision are inconsistently applied. Promoting guidelines, as well as codes of conduct, for shadowing could serve as a helpful resource for students, academic advisors and supervising clinicians.

Introduction to Clerkship: Bridging the Gap Between Preclinical and Clinical Medical Education.

BACKGROUND: Despite a diverse range of curricular advancements to address the difficult transition from classroom learning to clinical training during medical education, hurdles persist. A 4-week course was designed at the Philadelphia College of Osteopathic Medicine (PCOM) to make this transition easier. OBJECTIVES: To determine whether PCOM students' comfort and preparedness increased after taking a 4-week clinical transition course before third-year clinical clerkships, and to determine whether faculty perceptions of student preparedness and comfort were improved after participation in the course compared with previous third-year students. METHODS: Second-year osteopathic medical students at PCOM participated in a 4-week course, Introduction to Clinical Clerkship (I2C). The course included 16 small-group exercises, which all took place before students began their third-year clerkship rotations. The exercises in the course extended beyond the skills learned during their classroom years. Students were given a pre- and postcourse survey to evaluate their comfort level with 58 different aspects of clinical practice. Participating faculty were surveyed to evaluate their perception of student preparedness and comfort compared with previous third-year students who had not undergone the exercise. RESULTS: After completing the I2C course, third-year osteopathic medical students (n=232) reported increased comfort with 57 of the 58 learning objectives and each of the 5 coded clinical competency areas (patient assessment, effective communication, hospital logistics, procedural skills, and core knowledge) (P<.01). Preceptors reported that students who completed the I2C course were more prepared (54.5%) and more comfortable (63.4%) with clinical duties, as compared with their recollections of previous third-year osteopathic medical students. CONCLUSION: Within the 5 competencies, students on average felt more comfortable and were perceived by faculty as better prepared than previous students who had not taken the I2C course. The establishment of a preclinical transition exercise appears to help bridge the gap between the preclinical and clinical years. This learning model allows medical students to feel both more comfortable and better prepared throughout the transition from classroom learning to clinical rotations.

Blended Learning Educational Format for Third-Year Pediatrics Clinical Rotation.

CONTEXT: Traditional medical education is shifting to incorporate learning technologies and online educational activities with traditional face-to-face clinical instruction to engage students, especially at remote clinical training sites. OBJECTIVE: To describe and evaluate the effectiveness of the blended learning format (combining online and face-to-face instruction) for third-year osteopathic medical students during their pediatric rotation. METHODS: Third-year medical students who completed the 4-week clerkship in pediatrics during the 2014-2015 academic year were divided into a standard learning group and a blended learning group with online activities (discussion boards, blogs, virtual patient encounters, narrated video presentations, and online training modules). Comprehensive Osteopathic Medical Achievement Test scores and final course grades were compared between the standard learning and blended learning groups. Students in the blended learning group completed a postsurvey regarding their experiences. RESULTS: Of 264 third-year students who completed the 4-week clerkship in pediatrics during the 2014-2015 academic year, 78 (29.5%) participated in the blended learning supplement with online activities. Of 53 students who completed the postsurvey in the blended learning group, 44 (83.0%) agreed or strongly agreed that "The integration of e-learning and face-to-face learning helped me learn pediatrics." Open-ended comments supported this overall satisfaction with the course format; however, 26 of 100 comments reflected a desire to increase the amount of clinical exposure and face-to-face time with patients. No statistical differences were seen between the standard learning (n=186) and blended learning (n=78) groups with regard to Comprehensive Osteopathic Medical Achievement Test scores (P=.321). Compared with the standard learning group, more students in the blended learning group received a final course grade of honors (P=.015). CONCLUSION: Results of this study support the use of blended learning in a clinical training environment. As more medical educators use blended learning, it is important to investigate the best balance between learning with technology and learning in a face-to-face setting. Online activities may enhance but should never fully replace face-to-face learning with real patients.

Web-based objective structured clinical examination with remote standardized patients and Skype: resident experience.

OBJECTIVE: Using Skype and remote standardized patients (RSPs), investigators sought to evaluate user acceptance of a web-based objective structured clinical examination (OSCE) among resident physicians. METHODS: After participating in four web-based clinical encounters addressing pain with RSPs, 59 residents from different training programs, disciplines and geographic locations completed a 52-item questionnaire regarding their experience with Skype and RSPs. Open-ended responses were solicited as well. RESULTS: The majority of participants (97%) agreed or strongly agreed the web-based format was convenient and a practical learning exercise, and 90% agreed or strongly agreed the format was effective in teaching communication skills. Although 93% agreed or strongly agreed they could communicate easily with RSPs using Skype, 80% preferred traditional face-to-face clinical encounters, and 58% reported technical difficulties during the encounters. Open-ended written responses supported survey results. CONCLUSION: Findings from this study expose challenges with technology and human factors, but positive experiences support the continued investigation of web-based OSCEs as a synchronous e-learning initiative for teaching and assessing doctor-patient communication. Such educational programs are valuable but unlikely to replace face-to-face encounters with patients. PRACTICE IMPLICATIONS: This web-based OSCE program provides physician learners with additional opportunity to improve doctor-patient communication.

Keyboard data entry use among osteopathic medical students and residents.

CONTEXT: Candidates taking the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) are currently evaluated on their ability to document clinical findings using a handwritten postencounter note. However, keyboard data entry is increasingly used for medical documentation. OBJECTIVE: To determine the use and perception of keyboard data entry among osteopathic medical students and residents in educational and clinical settings. METHODS: A Web-based survey regarding frequency of and preference for keyboard data entry was distributed to 9801 osteopathic medical students, 17,268 osteopathic residents, and 34 clinical deans of colleges of osteopathic medicine (COMs). In addition, 31 COMs' clinical skills center directors were contacted to participate in a telephone survey about the use of keyboard data entry in their centers. RESULTS: A total of 1711 students, 1198 residents, 14 clinical deans, and 17 clinical skills center directors responded to the surveys. The majority of students (872 [51%]) reported using electronic keyboard data entry at their COM's clinical skills center for postencounter notes. Among respondents, 379 students (23%), 77 residents (9%), and 1 clinical dean reported that electronic keyboard data entry is never or rarely used during clinical rotations. Most trainees (1592 students [93%], 864 residents [94%]) reported that they were either comfortable or very comfortable with typing. Given the option of recording methods for SOAP (subjective, objective, assessment, plan) note findings on the COMLEX-USA Level 2-PE, 7 clinical deans were unsure of their students' preferences, while the remaining favored keyboard data entry (5) over handwriting (2). The majority of student and resident respondents would choose keyboard data entry (1009 [60%] and 511 [55%], respectively). CONCLUSION: Osteopathic medical students and residents are comfortable with typing; they are exposed to and would prefer using an electronic form of entry for medical documentation. These results support a conversion from written postencounter notes to keyboard data entry of notes on the COMLEX-USA Level 2-PE.

Patterns of misrepresentation of clinical findings on patient notes during the COMLEX-USA Level 2-PE.

CONTEXT: A health care team depends on the accurate documentation of the patient-physician encounter, be it written or electronic. If documentation is inaccurate or incomplete, patient care may be adversely affected. Previous studies have identified factors that influence documentation errors, such as fatigue, carelessness, and being overworked. More research, however, is needed into the patterns of errors and, by extension, these patterns' potential effect on patient care. Insights about these areas would be valuable to practicing physicians, as well as to medical educators, who could incorporate such insights into the training of medical students. OBJECTIVE: To identify potential patterns of commissive documentation errors on SOAP notes during the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE); specifically, to calculate the frequency with which specific items are misrepresented in SOAP (subjective, objective, assessment, plan) notes and to explore patterns of misrepresentation in relation to the documentation of history taking and physical examination. METHODS: Among the 12,510 candidates tested between July 2007 and June 2010, there were 24 candidates who failed the COMLEX-USA Level 2-PE because they misrepresented clinical findings on SOAP notes. These errors of commission (over-documentation) were categorized as errors in either history-taking or physical examination, and then subcategorized and analyzed to determine if meaningful patterns in the documentation of these errors occurred. RESULTS: A total of 662 errors were recorded among the 24 candidates. History-taking items were more often misrepresented (n=378) than physical examination items (n=284). Patient history-a subcategory that included past medical, surgical, medication, allergy, and family histories-was more likely to be misrepresented than other subcategories under history taking (eg, "history of present illness" and "associated symptoms") and many physical examination subcategories. Results of the analysis indicated that the misrepresentations were not evenly distributed (chi2(10)=323.1; P<.01): candidates whose SOAP notes contained misrepresentations were statistically more likely to misrepresent in some categories rather than in others. CONCLUSION: Physicians and medical students should pay particular attention to patient history, lifestyle history, and neurologic examination to minimize the risk of documentation errors.

Clinical skills assessment of procedural and advanced communication skills: performance expectations of residency program directors.

BACKGROUND: High stakes medical licensing programs are planning to augment and adapt current examinations to be relevant for a two-decision point model for licensure: entry into supervised practice and entry into unsupervised practice. Therefore, identifying which skills should be assessed at each decision point is critical for informing examination development, and gathering input from residency program directors is important. METHODS: Using data from previously developed surveys and expert panels, a web-delivered survey was distributed to 3,443 residency program directors. For each of the 28 procedural and 18 advanced communication skills, program directors were asked which clinical skills should be assessed, by whom, when, and how. Descriptive statistics were collected, and Intraclass Correlations (ICC) were conducted to determine consistency across different specialties. RESULTS: Among 347 respondents, program directors reported that all advanced communication and some procedural tasks are important to assess. The following procedures were considered 'important' or 'extremely important' to assess: sterile technique (93.8%), advanced cardiovascular life support (ACLS) (91.1%), basic life support (BLS) (90.0%), interpretation of electrocardiogram (89.4%) and blood gas (88.7%). Program directors reported that most clinical skills should be assessed at the end of the first year of residency (or later) and not before graduation from medical school. A minority were considered important to assess prior to the start of residency training: demonstration of respectfulness (64%), sterile technique (67.2%), BLS (68.9%), ACLS (65.9%) and phlebotomy (63.5%). DISCUSSION: Results from this study support that assessing procedural skills such as cardiac resuscitation, sterile technique, and phlebotomy would be amenable to assessment at the end of medical school, but most procedural and advanced communications skills would be amenable to assessment at the end of the first year of residency training or later. CONCLUSIONS: Gathering data from residency program directors provides support for developing new assessment tools in high-stakes licensing examinations.

Frequency of specific osteopathic manipulative treatment modalities used by candidates while taking COMLEX-USA Level 2-PE.

CONTEXT: As one of the key measures of clinical skills assessment, the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) is used to rate candidates' performance of osteopathic manipulative treatment (OMT). OBJECTIVE: To evaluate the frequency of specific OMT modalities used by COMLEX-USA Level 2-PE candidates and to report frequent use on the basis of clinical presentation. METHODS: With enhanced scoring procedures in place, OMT physician-examiners identified and documented specific OMT techniques that were performed by candidates at the 28 colleges of osteopathic medicine and branch campuses that had eligible students at that time. A frequency analysis for OMT techniques, according to clinical content axis, was applied to all candidates (N=4757) for the 2010-2011 testing cycle. RESULTS: Students used a wide range of specific OMT techniques. Candidates performed 24,202 instances of specific OMT techniques, including 10,471 myofascial/soft tissue (43.3%), 3942 muscle energy (16.3%), 1676 sinus drainage (6.9%), 1476 inhibition (6.1%), 1221 fascial release (5.0%), 1171 rib raising (4.8%), 918 lymphatic (3.8%), and 866 counterstrain (3.6%). A few students (<0.01%) used high-velocity, low-amplitude (HVLA), a technique that is prohibited from use on the COMLEX-USA Level 2-PE because of the potential hazard of repeatedly treating a standardized patient with thrust technique to the same segment in a given day. Additional techniques included functional, facilitated positional release, balanced ligamentous tension, and visceral. Use of techniques also varied according to the clinical presentation of the standardized patient (ie, cardiovascular, respiratory, neuromusculoskeletal, gastrointestinal, or other) and chronicity (ie, acute or chronic). CONCLUSION: Findings contributed to a better understanding of the types of OMT techniques being used by candidates taking COMLEX-USA Level 2-PE. Frequency of use of specific OMT modalities varied according to clinical case presentation.

Competency-based classification of COMLEX-USA cognitive examination test items.

CONTEXT: The Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) currently assesses osteopathic medical knowledge via a series of 3 progressive cognitive examinations and 1 clinical skills assessment. In 2009, the National Board of Osteopathic Medical Examiners created the Fundamental Osteopathic Medical Competencies (FOMC) document to outline the essential competencies required for the practice of osteopathic medicine. OBJECTIVES: To measure the distribution and extent to which cognitive examination items of the current series of COMLEX-USA assess knowledge of each of the medical competencies included in the FOMC document. METHODS: Eight graduate medical education panelists with expertise in competency-based assessment reviewed 1046 multiple-choice examination items extracted from the 3 COMLEX-USA cognitive examinations (Level 1, Level 2-Cognitive Evaluation, and Level 3) used during the 2008-2009 testing cycle. The 8 panelists individually judged each item to classify it as 1 of the 6 fundamental osteopathic medical competencies described in the FOMC document. RESULTS: Panelists made 8368 judgments. The majority of the sample examination items were classified as either patient care (3343 [40%]) or medical knowledge (4236 [51%]). Panelists also reported these 2 competencies as being the easiest to define, teach, and assess. The frequency of medical knowledge examination items decreased throughout the COMLEX-USA series (69%, 43%, 40%); conversely, items classified as interpersonal and communication skills, systems-based practice, practice-based learning and improvement, and professionalism increased throughout the 3-examination series. CONCLUSION: Results indicate that knowledge of each of the 6 competencies is being assessed to some extent with the current COMLEX-USA format. These findings provide direction for the enhancement of existing examinations and development of new assessment tools.

Pass/fail patterns of candidates who failed COMLEX-USA level 2-PE because of misrepresentation of clinical findings on postencounter notes.

CONTEXT: In 2007, The National Board of Osteopathic Medical Examiners (NBOME) instituted a policy to address the accuracy and integrity of postencounter written documentation recorded during the Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). This policy was instituted not only to protect the integrity of the examination, but also to highlight that overdocumentation of clinical findings not obtained during patient encounters may jeopardize patient safety. OBJECTIVE: To investigate overall and domain pass/fail patterns of candidates who misrepresented clinical findings with regard to past and subsequent performance on COMLEX-USA Level 2-PE. Specifically, to investigate what percentage of candidates failed because of misrepresentation on first attempts and how they performed on subsequent administrations, as well as the previous performance patterns of candidates who failed because of misrepresentation on examination retakes. METHODS: Historical records from NBOME's COMLEX-USA Level 2-PE database (testing cycles 2007-2008, 2008-2009, and 2009-2010) were used to analyze overall and domain pass/fail patterns of candidates who failed at least once because of misrepresentation of clinical findings. RESULTS: Of the 24 candidates who failed because of misrepresentation of postencounter (SOAP) notes, 20 candidates (83%) were first-time examinees. Four candidates (17%) were repeating the examination, 2 of whom were making a third attempt to pass. Among these 20 candidates who failed because of misrepresentation of clinical findings on their first attempt, 19 passed on their next attempt. At the time of study analysis, all but 2 candidates eventually passed the examination in subsequent attempts. CONCLUSION: Among candidates found to have misrepresented clinical findings on postencounter written documentation on COMLEX-USA Level 2-PE, no pattern existed between their past or subsequent performance with regard to overall or domain pass/fail results. The vast majority of these candidates passed the examination on subsequent administrations, and none failed twice because of misrepresentation. Consequences of misrepresentation of clinical findings on COMLEX-USA Level 2-PE are severe and may serve to both raise awareness and prevent these types of errors in the future.

An investigation of construct validity of humanistic clinical skills on a medical licensure examination.

OBJECTIVE: To test construct validity of humanistic clinical skills measured by a medical licensure performance examination using multitrait-multimethod confirmatory factor analysis. METHODS: Two hundred and twenty-seven third- and fourth-year undergraduate osteopathic medical students in 2007-2008 were randomly sampled. The Global Patient Assessment Tool (GPAT), designed to assess professionalism, interpersonal relationship, and doctor-patient communication was tested under two measurement methods: standard examination ratings and peer performance ratings of the same examinee. Two concurrent validity factors (data gathering and written patient notes) were included. RESULTS: Convergent validity was supported under the two methods of scoring. Moderate to strong correlations among trait factors indicated weak discriminate validity. Method effects were indicated. The relationship between GPAT and two concurrent validity factors indicate measures of relatively different constructs. CONCLUSION: Evidence of construct validity for the GPAT indicate scores should be interpreted as measuring a humanistic clinical skills construct consisting of homogeneous measures of professionalism, interpersonal relationship, and physician-patient communication. Findings are consistent with the interpretation and use of the GPAT as an important part of medical licensure examinations. PRACTICAL IMPLICATIONS: The implications to medical licensure standardized-patient examinations are discussed in terms of the GPAT as a valid measure of medical students' humanistic clinical skills.

Relationship between encounter time and candidate performance on COMLEX-USA Level 2-PE.

BACKGROUND: The National Board of Osteopathic Medical Examiners uses a standardized timing format for all osteopathic medical candidates across all stations in the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). Examination standardization, and therefore validity, is weakened if time administered affects student performance. OBJECTIVE: To investigate the relationship between the amount of time students spent with standardized patients (SPs) and their clinical skill component scores. METHODS: Data were analyzed from 249 candidates in 21 test sessions from the 2007-2008 testing cycle. Trained SPs recorded the time each examinee entered and exited each encounter. Outcome measures of the examination included pass (1 if pass; 0 if fail) performance outcome, pass by domain (biomedical/biomechanical and humanistic), and pass by component scores (data gathering, osteopathic manipulative treatment, and patient note scores). Analysis of variance (ANOVA), followed by pairwise comparisons, was used to compare encounter times across cases. RESULTS: A total of 2988 SP encounters were analyzed. Examinees spent an average 12.1 minutes in each patient encounter. Candidates spent less time with cases classified as health promotion and disease prevention than with cases that primarily involved a problem visit, but amount of time spent with cases representing various content areas was not significantly different (P>.05). On average, candidates spent the least amount of time (11.95 minutes) with gastrointestinal cases and the most time (12.26 minutes) with patients who had complaints relating to the neuromusculoskeletal system. A trivial relationship was shown between the continuous variable encounter time as the average amount of time a student spends in the encounter and the dichotomous clinical skill outcome of the examination (r=.03). The relationship between these 2 variables was consistent when examined separately for the humanistic domain (r=-.10) and the biomedical/biomechanical domain (r=.12). The strongest correlation was between encounter time and the data-gathering component score (r=.28). CONCLUSION: Time spent in the encounter was not influential in overall candidate performance in the COMLEX-USA Level 2-PE. Candidates spent a similar amount of time with cases of different content areas. This finding suggests that 14 minutes is sufficient for this osteopathic high-stakes performance evaluation.

Relationships between high-stakes clinical skills exam scores and program director global competency ratings of first-year pediatric residents.

BACKGROUND: Responding to mandates from the Accreditation Council for Graduate Medical Education (ACGME) and American Osteopathic Association (AOA), residency programs have developed competency-based assessment tools. One such tool is the American College of Osteopathic Pediatricians (ACOP) program directors' annual report. High-stakes clinical skills licensing examinations, such as the Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE), also assess competency in several clinical domains. OBJECTIVE: The purpose of this study is to investigate the relationships between program director competency ratings of first-year osteopathic residents in pediatrics and COMLEX-USA Level 2-PE scores from 2005 to 2009. METHODS: The sample included all 94 pediatric first-year residents who took COMLEX-USA Level 2-PE and whose training was reviewed by the ACOP for approval of training between 2005 and 2009. Program director competency ratings and COMLEX-USA Level 2-PE scores (domain and component) were merged and analyzed for relationships. RESULTS: Biomedical/biomechanical domain scores were positively correlated with overall program director competency ratings. Humanistic domain scores were not significantly correlated with overall program director competency ratings, but did show moderate correlation with ratings for interpersonal and communication skills. The six ACGME or seven AOA competencies assessed empirically by the ACOP program directors' annual report could not be recovered by principal component analysis; instead, three factors were identified, accounting for 86% of the variance between competency ratings. DISCUSSION: A few significant correlations were noted between COMLEX-USA Level 2-PE scores and program director competency ratings. Exploring relationships between different clinical skills assessments is inherently difficult because of the heterogeneity of tools used and overlap of constructs within the AOA and ACGME core competencies.

Relationship between standardized patient checklist item accuracy and performing arts experience.

INTRODUCTION: : It is not known whether a Standardized Patient's (SP's) performing arts background could affect his or her accuracy in recording candidate performance on a high-stakes clinical skills examination, such as the Comprehensive Osteopathic Medical Licensing Examination Level 2 Performance Evaluation. The purpose of this study is to investigate the differences in recording accuracy of history and physical checklist items between SPs who identify themselves as performing artists and SPs with no performance arts experience. METHODS: : Forty SPs identified themselves as being performing artists or nonperforming artists. A sample of SP live examination ratings were compared with a second set of ratings obtained after video review (N = 1972 SP encounters) over 40 cases from the 2008-2009 testing cycle. Differences in SP checklist recording accuracy were tested as a function of performing arts experience. RESULTS: : Mean overall agreement rates, both uncorrected and corrected for chance agreement, were very high (0.94 and 0.79, respectively, at the overall examination level). There was no statistically significant difference between the two groups with respect to any of the mean accuracy measures: history taking (z = -0.422, P = 0.678), physical examination (z = -1.453, P = 0.072), and overall data gathering (z = -0.812, P = 0.417) checklist items. CONCLUSION: : Results suggest that SPs with or without a performing arts background complete history taking and physical examination checklist items with high levels of precision. Therefore, SPs with and without performing arts experience can be recruited for high-stakes SP-based clinical skills examinations without sacrificing examination integrity or scoring accuracy.

Five-year summary of COMLEX-USA level 2-PE examinee performance and survey data.

The authors present data on examination format and examinee demographics, performance, and survey results for the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) from the first five testing cycles (2004-2005 to 2008-2009). First-time examinees in the 2004-2005 testing cycle had a pass rate of 96.1%, compared with a pass rate of 94.7% for first-time examinees in the 2008-2009 testing cycle. Pass rates were fairly consistent across all testing cycles. Based on postexamination survey results from all testing cycles, the majority of examinees reported that the cases in COMLEX-USA Level 2-PE represented appropriate challenges for fourth-year osteopathic medical students. The majority of examinees also reported that comprehensive standardized patient-based examinations and exercises were administered through their colleges of osteopathic medicine. In addition, survey results indicated overall satisfaction among examinees with the administration of COMLEX-USA Level 2-PE.

Correlation between an email based board review program and American Board of Pediatrics General Pediatrics Certifying Examination scores.

OBJECTIVE: To investigate the impact of a weekly email based board review course on individual resident performance on the American Board of Pediatrics (ABP) General Pediatrics Certifying Examination for pediatric residents and, specifically, residents with low ABP In-training Examination (ITE) scores. METHODS: Weekly board-type questions were emailed to all pediatric residents from 2004-2007. Responses to board-type questions were tracked, recorded, and correlated with ITE scores and ABP General Pediatrics Certifying Examination Scores. RESULTS: With regard to total number of questions answered, only total number of questions answered correctly had a significant positive correlation with standard board scores (n = 71, r = 0.24, p = 0.047). For "at risk" residents with ITE scores

Patient note fabrication and consequences of unprofessional behavior in a high-stakes clinical skills licensing examination.

BACKGROUND: The National Board of Osteopathic Medical Examiners (NBOME) administers the Comprehensive Osteopathic Medical Licensing Examination USA Level 2-PE (COMLEX-USA Level 2-PE) and has developed a process that links the competencies of written communication and professionalism by tracking fabrication in the postencounter SOAP (Subjective, Objective, Assessment, Plan) note exercise. METHOD: A process used to identify potential SOAP note fabrication was implemented in the 2007-2008 test cycle for the COMLEX-USA Level 2-PE. RESULTS: A total of 3,753 candidates took the COMLEX-USA Level 2-PE in the 2007-2008 test cycle. Forty-eight candidates were screened, and the NBOME's Subcommittee on SOAP Note Fabrication made failure decisions on eight, leading to a failure rate of 0.2% based on fabrication review. CONCLUSIONS: The NBOME has adopted the stance that postencounter note fabrication represents unprofessional behavior. Screening for and failing candidates who exhibit unprofessional behavior enhances the validity of the examination.