Acute Care of At-Risk Newborns (ACoRN): quantitative and qualitative educational evaluation of the program in a region of China.

BACKGROUND: The Acute Care of at-Risk Newborns (ACoRN) program was developed in Canada for trained health care providers for the identification and management of newborns who are at-risk and/or become unwell in the first few hours or days after birth. The ACoRN process follows an 8-step framework that enables the evaluation and management of babies irrespective of the experience or expertise of the caregiving individual or team. This study assesses the applicability of the program to Chinese pediatric practitioners. METHODS: Course content and educational materials were translated from English into Chinese by bilingual neonatal practitioners. Confidence and knowledge questionnaires were developed and reviewed for face and content validity by a team of ACoRN instructors. Bilingual Chinese instructors were trained at the tertiary perinatal centre in Hangzhou Zhejiang to deliver the course at 15 level II county hospitals. Participants completed pre- and post-course confidence and knowledge questionnaires and provided feedback through post-course focus groups. RESULTS: 216 physicians and nurses were trained. Confidence and knowledge relating to neonatal stabilization improved significantly following the courses. Participants rated course utility and function between 4.2 and 4.6/5 on all items. Pre/post measures of confidence were significantly correlated with post course knowledge. Focus group data supported the perceived value of the program and recommended course adjustments to include pre-course reading, and increased content related to simulation, communication skills, and management of respiratory illness and jaundice. CONCLUSIONS: ACoRN, a Canadian educational program, appears to be well received by Chinese health care providers and results in improved knowledge and confidence. International program adaptation for use by health care professionals requires structured and systematic evaluation to ensure that the program meets the needs of learners, reflects their learning styles, and can be applied in their setting.

Long-term outcomes for surgeons from 3- and 4-year medical school curricula.

BACKGROUND: New approaches are needed to ensure that surgical trainees attain competence in a timely way. Traditional solutions have focused on the years spent in surgic al training. We sought to examine the outcomes of graduates from 3-year versus 4-year medical schools for differences in surgeon performance based on multisource feedback data. METHODS: We used data from the College of Physicians and Surgeons of Alberta's Physician Achievement Review program to determine curricular outcomes. Data for each surgeon included assessments from 25 patients, 8 medical colleagues and 8 nonphysician coworkers (e.g., nurses), and a self-assessment. We used these data to compare 72 physicians from a 3-year school matched with graduates from 4-year schools. The instruments were assessed for evidence of validity and reliability. We compared the groups using 1-way analysis of covariance and multivariate analysis of covariance, with years since graduation as a covariate, and a Cohen d effect size calculation to assess the magnitude of the change. RESULTS: Data for 216 surgeons indicated that there was evidence for instrument validity and reliability. No significant differences were found based on the length of the undergraduate program for any of the questionnaires or factors within the questionnaires. CONCLUSION: Reconsideration might be given to the time spent in medical school before surgical training if training in the specialty and career years are to be maximized. This assumes that students are able to make informed career decisions based on clerkship and other experiences in a 3-year setting.

The development of an instrument to assess clinical teaching with linkage to CanMEDS roles: A psychometric analysis.

BACKGROUND: Assessment of clinical teaching by learners is of value to teachers, department heads, and program directors, and must be comprehensive and feasible. AIMS: To review published evaluation instruments with psychometric evaluations and to develop and psychometrically evaluate an instrument for assessing clinical teaching with linkages to the CanMEDS roles. METHOD: We developed a 19-item questionnaire to reflect 10 domains relevant to teaching and the CanMEDS roles. A total of 317 medical learners assessed 170 instructors. Fourteen (4.4 %) clinical clerks, 229 (72.3%) residents, and 53 (16.7%) fellows assessed 170 instructors. Twenty-one (6.6%) did not specify their position. RESULTS: A mean number of eight raters assessed each instructor. The internal consistency reliability of the 19-item instrument was Cronbach's α = 0.95. The generalizability coefficient (Ep(2)) analysis indicated that the raters achieved Ep(2) of 0.95. The factor analysis showed three factors that accounted for 67.97% of the total variance. The three factors together, with the variance accounted for and their internal consistency reliability, are teaching skills (variance = 53.25s%; Cronbach's α = 0.92), Patient interaction (variance = 8.56%; Cronbach's α = 0.91), and professionalism (variance = 6.16%; Cronbach's α = 0.86). The three factors are intercorrelated (correlations = 0.48, 0.58, 0.46; p < 0.01). CONCLUSION: It is feasible to assess clinical teaching with the 19-item instrument that has demonstrated evidence of both validity and reliability.

Mentorship for the physician recruited from abroad to Canada for rural practice.

BACKGROUND: Mentoring is one way to help physicians new to a country assimilate. AIM: This study examined the feasibility and focus of a mentoring program from the perspective of medical leaders (leaders) and physicians new to Canada (physicians). METHODS: Focus groups with 23 physicians were held in six regional centers. Face-to-face interviews were held with 10 leaders. They were asked to discuss how a mentoring program might be helpful and how a program might be designed and evaluated. RESULTS: Both leaders and physicians recognized that mentorship would support the physician socially, professionally, and emotionally. They told us that mentorship programs should be structured carefully to build trust, allow mentors and mentees some selection, be in geographic proximity where possible, and have transparent rules. While leaders felt that evaluation would be an important part of a mentorship program, the physicians disagreed noting that it would change the nature of the program. Leaders stated that the ultimate evaluation of the program's success would be found in retention numbers. CONCLUSION: Physicians new to a country need support. Mentorship is a feasible approach to support new comers that may result in more efficient and effective integration, enculturation, and higher levels of retention.

Push, pull, and plant: the personal side of physician immigration to alberta, Canada.

BACKGROUND AND OBJECTIVES: The global migration of physicians has led many international physicians to enter practice in Alberta, Canada. The study was designed to explore the personal side of migration and transition experiences of these international medical graduates (IMGs). METHODS: A qualitative study using telephone interviews and a semi-structured interview guide was used to interview 19 IMGs who are currently practicing and have held Part V, restricted or temporary practice licenses for less than 7 years. RESULTS: Three major themes were identified. The first was the "push" from their own country of origin and their perception that moving to Alberta would be better for them. Professional opportunities in their home country had been affected by changing policies, lack of infrastructure, and personal/family safety issues culminating in highly stressful work environments. The second was "pull." An improvement in the quality of personal life was associated with geographical, educational, recreational, and spiritual aspects of daily living for participants and their families in their new environment. The third theme was "plant"ie, factors that encouraged them to stay in Alberta. CONCLUSIONS: This study demonstrates the continued relevance of push and pull theory in understanding IMG physician migration. Our findings in this study indicate that remaining in place, or "being planted" is conditional on political, social, and economic aspects.

Assessment of radiology physicians by a regulatory authority.

PURPOSE: To determine whether it is possible to develop a feasible, valid, and reliable multisource feedback program for radiologists. MATERIALS AND METHODS: Surveys with 38, 29, and 20 items were developed to assess individual radiologists by eight radiologic colleagues (peers), eight referring physicians, and eight co-workers (eg, technicians), respectively, by using five-point scales along with an "unable to assess" category. Radiologists completed a self-assessment on the basis of the peer questionnaire. Items addressed key competencies related to clinical competence, collegiality, professionalism, workplace behavior, and self-management. The study was approved by the University of Calgary Conjoint Health Ethics Research Board. RESULTS: Data from 190 radiologists were available. The mean numbers of respondents per physician were 7.5 of eight (1259 of 1520, 83%), 7.15 of eight (1337 of 1520, 88%), and 7.5 of eight (1420 of 1520, 93%) for peers, referring physicians, and co-workers, respectively. The internal consistency reliability indicated all instruments had a Cronbach alpha of more than 0.95. The generalizability coefficient analysis indicated that the peer, referring physicians, and co-worker instruments achieved a generalizability coefficient of 0.88, 0.79, and 0.87, respectively. The factor analysis indicated that four factors on the colleague questionnaire accounted for 70% of the total variance: clinical competence, collegiality, professional development, and workplace behavior. For the referring physician survey, three factors accounted for 64.1% of the variance: professional development, professional consultation, and professional responsibility. Two factors on the co-worker questionnaire accounted for 63.2% of the total variance: professional responsibility and patient interaction. CONCLUSION: The psychometric examination of the data suggests that the instruments developed to assess radiologists are a feasible way to assess radiology practice and provide evidence for validity and reliability.

Changes in performance: a 5-year longitudinal study of participants in a multi-source feedback programme.

OBJECTIVES: Multi-source feedback (MSF) enables performance data to be provided to doctors from patients, co-workers and medical colleagues. This study examined the evidence for the validity of MSF instruments for general practice, investigated changes in performance for doctors who participated twice, 5 years apart, and determined the association between change in performance and initial assessment and socio-demographic characteristics. METHODS: Data for 250 doctors included three datasets per doctor from, respectively, 25 patients, eight co-workers and eight medical colleagues, collected on two occasions. RESULTS: There was high internal consistency (alpha > 0.90) and adequate generalisability (Ep(2) > 0.70). D study results indicate adequate generalisability coefficients for groups of eight assessors (medical colleagues, co-workers) and 25 patient surveys. Confirmatory factor analyses provided evidence for the validity of factors that were theoretically expected, meaningful and cohesive. Comparative fit indices were 0.91 for medical colleague data, 0.87 for co-worker data and 0.81 for patient data. Paired t-test analysis showed significant change between the two assessments from medical colleagues and co-workers, but not between the two patient surveys. Multiple linear regressions explained 2.1% of the variance at time 2 for medical colleagues, 21.4% of the variance for co-workers and 16.35% of the variance for patient assessments, with professionalism a key variable in all regressions. CONCLUSIONS: There is evidence for the construct validity of the instruments and for their stability over time. Upward changes in performance will occur, although their effect size is likely to be small to moderate.

What multisource feedback factors influence physician self-assessments? A five-year longitudinal study.

BACKGROUND: Multisource feedback, in which medical colleagues, patients, coworkers, and the physician involved provide data, is a tool to inform physician practice. Its impact on physicians' self-assessment through two iterations is unknown. METHOD: Data from 250 family physicians in Alberta who participated in two iterations, five years apart-1999 and 2006--allowed the authors to determine the change in self-assessment scores, using a t test. A multiple regression was used to account for the variance in the scores from the second self-assessment by the data from the multisource feedback and sociodemographics from the first iteration. RESULTS: Physicians rated themselves higher in the second iteration. The linear regression model accounted for 27.4% of the variance in the ratings at the second iteration and incorporated data from the self-assessment. CONCLUSIONS: Physician self-assessment seems driven by stable perceptions that physicians hold about themselves and that may be slow to change.

International medical graduates: learning for practice in Alberta, Canada.

INTRODUCTION: There is little known about the learning that is undertaken by physicians who graduate from a World Health Organization-listed medical school outside Canada and who migrate to Canada to practice. What do physicians learn and what resources do they access in adapting to practice in Alberta, a province of Canada? METHODS: Telephone interviews with a theoretical sample of 19 IMG physicians were analyzed using a grounded theory constant comparative approach to develop categories, central themes, and a descriptive model. RESULTS: The physicians described two types of learning: learning associated with studying for Canadian examinations required to remain and practice in the province and learning that was required to succeed at clinical work in a new setting. This second type of learning included regulations and systems, patient expectations, new disease profiles, new medications, new diagnostic procedures, and managing the referral process. The physicians "settled" into their new setting with the help of colleagues; the Internet, personal digital assistants (PDAs), and computers; reading; and continuing medical education programs. Patients both stimulated learning and were a resource for learning. DISCUSSION: Settling into Alberta, Canada, physicians accommodated and adjusted to their settings with learning activities related to the clinical problems and situations that presented themselves. Collegial support in host communities appeared to be a critical dimension in how well physicians adjusted. The results suggest that mentoring programs may be a way of facilitating settlement.

Roles and responsibilities of family physicians on geriatric health care teams: Health care team members' perspectives.

OBJECTIVE: To examine the beliefs and attitudes of FPs and health care professionals (HCPs) regarding FPs' roles and responsibilities on interdisciplinary geriatric health care teams. DESIGN: Qualitative study using focus groups. SETTING: Calgary Health Region. PARTICIPANTS: Seventeen FPs and 22 HCPs working on geriatric health care teams. METHOD: Four 90-minute focus groups were conducted with FPs, followed by 2 additional 90-minute focus groups with HCPs. The FP focus groups discussed 4 vignettes of typical teamwork scenarios. Discussions were transcribed and the 4 researchers analyzed and coded themes and subthemes and developed the HCP focus group questions. These questions asked about HCPs' expectations of FPs on teams, experiences with FPs on teams, and perspectives on optimal roles on teams. Several meetings were held to determine themes and subthemes. MAIN FINDINGS: Family physicians identified patient centredness, role delineation for team members, team dynamics, and team structure as critical to team success. Both FPs and HCPs had a continuum of beliefs about the role FPs should play on teams, including whether FPs should be autonomous or collaborative decision makers, the extent to which FPs should work within or outside teams, whether FPs should be leaders or simply members of teams, and the level of responsibility implied or explicit in their roles. CONCLUSION: Comments from FPs and HCPs identified intraprofessional and interprofessional tensions that could affect team practice and impede the development of high-functioning teams. It will be important, as primary care reform continues, to help FPs and HCPs learn how to work together effectively on teams so that patients receive the best possible care.

Assessing outcomes through congruence of course objectives and reflective work.

INTRODUCTION: Course outcomes have been assessed by examining the congruence between statements of commitment to change (CTCs) and course objectives. Other forms of postcourse reflective exercises (for example, impact and unmet-needs statements) have not been examined for congruence with course objectives or their utility in assessing course outcomes. This study assessed the congruence of course objectives and statements of commitment to change, effects on practice, unmet-needs, and the utility of supplementing CTCs with other forms of reflective work in course evaluations. METHODS: A 3-module course on Alzheimer's disease and other dementias provided end-of-course CTC statements, follow-up data, and statements of effects on practice and unmet needs. Statements were aligned to module objectives and analyzed descriptively. RESULTS: Of the 932 physicians who registered for 1 of the 3 modules, 404 provided CTCs, 302 provided impact statements, and 265 provided unmet-needs statements. Sixty percent of the CTCs could be assigned to an objective for their module, and between 14% and 25% of CTCs were assigned to objectives for another module. Three-quarters of CTCs were fully or partially implemented. Physicians did not have an opportunity to implement the new content in 70% of nonimplemented CTCs. Fewer impact and unmet-needs statements were congruent with course objectives than CTCs. CONCLUSIONS: Commitment-to-change statements had more congruence with objectives than did impact or unmet-needs statements. These latter statements, particularly those that could not be assigned to an objective, may reinforce and supplement the information provided by CTC analyses.

Permanent small groups: group dynamics, learning, and change.

INTRODUCTION: The concept of "communities of practice," a special facet of social constructivist learning theory, provides a new template against which we can examine the learning that goes on within permanent small groups of physicians. We interviewed participants and facilitators about the dynamics of these groups, their learning in conjunction with these groups, and the role the facilitator played to see the extent to which they captured the essence of communities of practice. METHODS: Semistructured interviews were conducted with physicians known to be participants or facilitators of small groups that met regularly. A constant comparative method was used for data gathering and analysis leading to coded themes, categories, and subcategories. The coding schemas were tested, the analyses were reviewed, and data were recoded as necessary. To ensure accuracy, interviewees were provided with a preliminary copy of the manuscript to ensure that the interpretation of the data was appropriately handled. RESULTS: Interviews were conducted with 10 facilitators and 22 group members representing 24 different groups of physicians. The groups appeared to function as communities of practice in which the members were supportive of each other's learning and respectful of one another, reporting little conflict. Members preferred to agree to disagree rather than pursue a "right" answer or consensus. Most of the discussion focused on scientific information and the way in which their colleagues approached common problems. Practice refinement rather than new directions in patient care appeared to be the goal. The facilitators in these groups played a key role in providing administrative support for the group and often the energy needed to sustain them. DISCUSSION: Small groups that meet regularly provide a supportive network to share knowledge and validate clinical experience. There is some evidence that the groups have the potential to become communities of practice but do not actually achieve that level of sharing. Research needs to be done to determine how these groups could become more powerful as communities of practice and vehicles for more substantive learning and change.

Stillbirth and newborn mortality in India after helping babies breathe training.

OBJECTIVE: This study evaluated the effectiveness of Helping Babies Breathe (HBB) newborn care and resuscitation training for birth attendants in reducing stillbirth (SB), and predischarge and neonatal mortality (NMR). India contributes to a large proportion of the worlds annual 3.1 million neonatal deaths and 2.6 million SBs. METHODS: This prospective study included 4187 births at >28 weeks' gestation before and 5411 births after HBB training in Karnataka. A total of 599 birth attendants from rural primary health centers and district and urban hospitals received HBB training developed by the American Academy of Pediatrics, using a train-the-trainer cascade. Pre-post written trainee knowledge, posttraining provider performance and skills, SB, predischarge mortality, and NMR before and after HBB training were assessed by using χ(2) and t-tests for categorical and continuous variables, respectively. Backward stepwise logistic regression analysis adjusted for potential confounding. RESULTS: Provider knowledge and performance systematically improved with HBB training. HBB training reduced resuscitation but increased assisted bag and mask ventilation incidence. SB declined from 3.0% to 2.3% (odds ratio [OR] 0.76, 95% confidence interval [CI] 0.59-0.98) and fresh SB from 1.7% to 0.9% (OR 0.54, 95% CI 0.37-0.78) after HBB training. Predischarge mortality was 0.1% in both periods. NMR was 1.8% before and 1.9% after HBB training (OR 1.09, 95% CI 0.80-1.47, P = .59) but unknown status at 28 days was 2% greater after HBB training (P = .007). CONCLUSIONS: HBB training reduced SB without increasing NMR, indicating that resuscitated infants survived the neonatal period. Monitoring and community-based assessment are recommended.

Helping Babies Breathe: global neonatal resuscitation program development and formative educational evaluation.

OBJECTIVES: To develop an educational program designed to train health care providers in resource limited settings to carry out neonatal resuscitation. We analyzed facilitator and learner perceptions about the course, examined skill performance, and assessed the quality of instruments used for learner evaluation as part of the formative evaluation of the educational program Helping Babies Breathe. METHODS: Multiple stakeholders and a Delphi panel contributed to program development. Training of facilitators and learners occurred in global field test sites. Course evaluations and focus groups provided data on facilitator and learner perceptions. Knowledge and skill assessments included pre/post scores from multiple choice questions (MCQ) and post-training assessment of bag and mask skills, as well as 2 objective structured clinical evaluations (OSCE). RESULTS: Two sites (Kenya and Pakistan) trained 31 facilitators and 102 learners. Participants expressed high satisfaction with the program and high self-efficacy with respect to neonatal resuscitation. Assessment of participant knowledge and skills pre/post-program demonstrated significant gains; however, the majority of participants could not demonstrate mastery of bag and mask ventilation on the post-training assessment without additional practice. CONCLUSIONS: Participants in a program for neonatal resuscitation in resource-limited settings demonstrated high satisfaction, high self-efficacy and gains in knowledge and skills. Mastery of ventilation skills and integration of skills into case management may not be achievable in the classroom setting without additional practice, continued learning, and active mentoring in the workplace. These findings were used to revise program structure, materials and assessment tools.

Moving into medical practice in a new community: the transition experience.

INTRODUCTION: Physicians undertake many transitions during the course of a medical career. The purpose of this study was to explore the experiences of physicians who moved to a new community. METHODS: A semistructured interview format was used to explore transitional experiences, including reasons for moving; the role of colleagues, learning, and organizational structures; how various mediating factors affected perceptions; and how the experience affected the physicians personally. We used qualitative methods in which data were collected, coded, and analyzed concurrently. RESULTS: 20 physicians from family medicine, internal medicine, and pediatrics described their experiences. Both the professional context and the geographic location affected physicians' perceptions of the move. Both internal and external mediating factors appeared to influence how physicians experienced and adjusted to the move. Physicians who joined functioning units appeared to have fewer problems. The physicians who had more difficulty were physicians who did not come to a specific job, often coming as the result of a spousal move; did not have a professional network in the city; had not sorted out licensure requirements; and were entering community (not institutional) practice. DISCUSSION: This study demonstrates the critical nature of institutional support structures to integrate the newcomer, collegial relationships within the workplace, and the importance of family and friends in mediating the adjustment period. Consideration should be given to structured mentorship or peer-buddy programs and longitudinal educational programs (eg, rounds) that may enable physicians to establish networks and gain practical local knowledge quickly.

Learning to practice in Canada: the hidden curriculum of international medical graduates.

INTRODUCTION: There is movement of physicians internationally. In some cases, physicians are recruited from low-income countries to wealthier countries like Canada to provide medical services in underresourced communities. This needs assessment examined the clinical medicine learning challenges faced by international medical graduates (IMGs) from the perspective of both the IMGs and medical leaders (eg, Vice President-Medical for a Health Region). METHODS: Focus groups with 25 IMGs were held in 6 regional centers. Face-to-face interviews were held with 10 medical leaders. Participants were asked about the learning associated with patient management, patient referral, and investigation, for billing and insurance, and learning about new systems of care. Qualitative data were analyzed to determine how well the perspectives on learning were aligned. RESULTS: IMGs and medical leaders recognized that learning and support were needed by physicians without previous experience in Canada. They had similar lists of learning issues. Although medical leaders believed the new information was explicit, readily available, and could be learned from short explanations and lists; IMGs found that guidelines and expectations were implicit, confusing, and contradictory. There were mediating influences in the form of orientation programs, other IMGs, and "how to" lists in some cases, which helped the newcomer. DISCUSSION: There was concordance about aspects of the learning that was required between IMGs and medical leaders. There was little agreement about the approach to learning or a recognition that the learning tasks were complicated.

An analysis of long-term outcomes of the impact of curriculum: a comparison of the three- and four-year medical school curricula.

PURPOSE: To determine the long-term effects of curriculum length on physician competence, the authors compared the performance of graduates from the University of Calgary (U of C; a school with a three-year curriculum) with matched samples from the University of Alberta (U of A) and from other Canadian schools with a four-year curriculum. METHOD: The authors used data from the College of Physicians and Surgeons of Alberta, Physician Achievement Review (PAR) program to determine curricular outcomes. The authors analyzed PAR program data, comprising reviews from medical colleagues, nonphysician coworkers (e.g., nurses, pharmacists), patients, and the physicians themselves, for 166 physicians from U of C, U of A, and other universities. They compared groups using one-way analysis of covariance (ANCOVA) and multivariate analysis of covariance (MANCOVA), with years since graduation as a covariate, and a Cohen d effect size calculation to assess the magnitude of the differences. RESULTS: The authors analyzed review data for 498 physicians. The results of ANCOVA showed that no significant differences existed among schools for the self and the patient aggregate mean questionnaire scores. Aggregate mean questionnaire scores from the medical colleague and coworker surveys were significant, albeit with a small effect size. MANCOVA showed small but significant differences among schools on the aggregate mean factor scores for medical colleague, coworker, and patient questionnaires. CONCLUSIONS: Although differences among schools exist, they are small. They suggest at least equivalent performance for graduates of three- and four-year medical schools who practice in Alberta.

The assessment of pathologists/laboratory medicine physicians through a multisource feedback tool.

CONTEXT: There is increasing interest in ensuring that physicians demonstrate the full range of Accreditation Council for Graduate Medical Education competencies. OBJECTIVE: To determine whether it is possible to develop a feasible and reliable multisource feedback instrument for pathologists and laboratory medicine physicians. DESIGN: Surveys with 39, 30, and 22 items were developed to assess individual physicians by 8 peers, 8 referring physicians, and 8 coworkers (eg, technologists, secretaries), respectively, using 5-point scales and an unable-to-assess category. Physicians completed a self-assessment survey. Items addressed key competencies related to clinical competence, collaboration, professionalism, and communication. RESULTS: Data from 101 pathologists and laboratory medicine physicians were analyzed. The mean number of respondents per physician was 7.6, 7.4, and 7.6 for peers, referring physicians, and coworkers, respectively. The reliability of the internal consistency, measured by Cronbach alpha, was > or = .95 for the full scale of all instruments. Analysis indicated that the medical peer, referring physician, and coworker instruments achieved a generalizability coefficient of .78, .81, and .81, respectively. Factor analysis showed 4 factors on the peer questionnaire accounted for 68.8% of the total variance: reports and clinical competency, collaboration, educational leadership, and professional behavior. For the referring physician survey, 3 factors accounted for 66.9% of the variance: professionalism, reports, and clinical competency. Two factors on the coworker questionnaire accounted for 59.9% of the total variance: communication and professionalism. CONCLUSIONS: It is feasible to assess this group of physicians using multisource feedback with instruments that are reliable.

Assessment of psychiatrists in practice through multisource feedback.

OBJECTIVE: To assess the feasibility and evidence for the reliability and validity of a set of questionnaires for psychiatrists, given that multisource feedback (MSF) or 360 degrees evaluation allows medical colleagues, coworkers, and patients to provide feedback about competencies to enhance physician improvement in intended directions. METHOD: Surveys with 40, 22, 38, and 37 items were developed to assess psychiatrists by 25 patients, 8 coworkers, 8 psychiatrist colleagues, and self, respectively, using a 5-point agreement scale with an unable-to-assess category. Items addressed key competencies related to communication skills, professionalism, collegiality, and self-management. Feasibility was assessed with response rates for each instrument. Validity was assessed with a table of specifications, the percentage of participants unable to assess the psychiatrist for each item, and exploratory factor analyses to determine which items grouped together into scales. Reliability was assessed by Cronbach's alpha and generalizability coefficients. RESULTS: A sample of 101 psychiatrists provided data. A total of 2456 patients (24.32/25.00 per psychiatrist), 744 coworkers (7.37/8.00 per psychiatrist), 764 colleagues (7.56/8.00 per psychiatrist), and 101 self forms were analyzed. The overall internal consistency reliability of the instruments was a Cronbach's alpha of 0.98, 0.96, and 0.98 for patient, coworker, and medical colleague surveys, respectively. The generalizability coefficient for the patient, coworker, and medical colleague was 0.78, 0.82, and 0.81, respectively. CONCLUSION: It is possible to develop a feasible MSF program for psychiatrists with evidence of reliability and validity that can provide feedback about key clinical competencies.