Training and validation of standardized patients for unannounced assessment of physicians' management of depression.

OBJECTIVE: Standardized patients (SPs) have been developed to measure practitioner performance in actual practice settings, but results have not been fully validated for psychiatric disorders. This study describes the process of creating reliable and valid SPs for unannounced assessment of general-practitioners' management of depression disorders in Iran. METHOD: Ten psychology and nursing students (potential SPs) took part in a five-session course involving training in dialogue and body language. Five scenarios, along with corresponding checklists representing common presentations of mood disorders in primary-care settings, were developed by an expert group. The SPs' role-play performance of their respective scenario was videotaped and scored independently by three psychiatrists according to an observational rating scale to assess validity. The role-play was repeated after 1 week with the same scenario and the same doctor, to assess test-retest reliability. The reliability of each checklist to be used by the SPs was assessed by testing interrater reliability between groups of SPs. RESULTS: The cutoff score for the SPs' portrayal validity was 90% or above for all SPs. Mean interrater reliability for the checklists was acceptable for the SPs watching the same videos and filling in the checklists, while the mean kappa for assessing concurrent validity in filling in the checklists was lower. The test-retest performance for assessing reliability resulted in a mean kappa of 0.72. All SPs except one, who was not recruited, performed acceptably well. CONCLUSION: The authors have demonstrated a thorough validation of the technique of using standardized patients in the portrayal of depressive disorders in primary-care settings in Iran, which creates confidence in employing this technique to evaluate doctors' performance, for example, after an educational intervention. Similar methods of validation can be used for SPs' portrayal of other psychiatric disorders.

Effect of a peer-educational intervention on provider knowledge and reported performance in family planning services: a cluster randomized trial.

BACKGROUND: Peer education is an interactive method of teaching or learning which is widely used for educating school and college students, in a variety of different forms. However, there are few studies on its effectiveness for in-service education. The aim of this study was to evaluate the effect of an educational programme including peer discussions, based on a needs assessment, on the providers' knowledge and reported performance in family planning services. METHODS: An educational programme was designed and applied in a random selection of half of in-charges of the 74 family health units (intervention group) in Tabriz at a regular monthly meeting. The other half constituted the control group. The programme included eight pages of written material and a two-hour, face-to-face discussion session with emphasis on the weak areas identified through a needs assessment questionnaire. The educated in-charges were requested to carry out a similar kind of programme with all peers at their health facilities within one month. All in-charges received one self-administered questionnaire containing knowledge questions one month after the in-charge education (follow-up I: 61 responses), and another one containing knowledge and self-reported performance questions 26 months later (follow-up II: 61 responses). Also, such tests were done for the peers facilitated by the in-charges one (105 responses) and 27 months (114 responses) after the peer discussions. Multiple linear regression was used for comparing mean total scores, and Chi square for comparing proportions between control and intervention groups, after defining facility as the unit of randomization. RESULTS: The mean total percentage scores of knowledge (percent of maximal possible score) in the intervention group were significantly higher than in the control group, both at follow-up I (63%) and at follow-up II (57%); with a difference of 16 (95% CI: 11, 22) and 5 (95% CI: 0.4, 11) percentage units, respectively. Only two of the nine reported performance items were significantly different among the non in-charges in the intervention group at follow-up II. CONCLUSIONS: The educational programme including peer discussions using existing opportunities with no need for additional absence from the workplace might be a useful complement to formal large group education for the providers.