Evaluation techniques in Punjab, Pakistan: eight years of reforms in health professional education.

Pakistan, the most populated country in the WHO Eastern Mediterranean region has a population of over 170 million, spread over five provinces and four federally administered areas. It has a growth rate of 1.9%. Punjab is the most populous and developed province with an estimated population in 2010 of 81 million. In 2008, Punjab's development index of 0.60 and a literacy rate of 80% were the highest in the country. In Pakistan, the number of doctors and nurses has risen from 48 to 71 per 100,000 and from 16 to 30 per 100,000, respectively between 1990 and 2003. The major challenge, still, is the imbalance of the population to health-care workers ratio. At the time of creation of Pakistan, King Edward Medical College was the only fully functioning medical college. Over the years, as a result of health reform initiatives, a number of government medical colleges were established in the country. University of Health Sciences, Lahore was established in 2002, having sole jurisdiction over all medical, dental and allied health institutes in the province with the aim of moving medical education towards an outcome-based patient and community-oriented competency-driven system. This paper attempts to clarify how initiatives and reforms in the evaluation process have helped the UHS realise its aims. Evaluation in all branches of higher education has long been taken as a means to an end. The focus of UHS on teacher-training, introduction of behavioural sciences as a compulsory subject and setting up an outcome-based evaluation process, has established a knowledge-acquisition medical education atmosphere. The challenges in the future relate to sustainability through capacity-building and staying abreast with the Best Evidence Medical Education practices worldwide, implementing them to fit our local needs and resources.

Relationship of awards in multiple choice questions and structured answer questions in the undergraduate years and their effectiveness in evaluation.

BACKGROUND: A number of evaluation tools for assessing the cognitive and affective domains in accordance with Bloom's taxonomy are available for summative assessment. At the University of Health Sciences, Lahore, Multiple Choice Questions (MCQs) and Structured Answer Questions (SAQs) are used for the evaluation of the cognitive domain at all six hierarch levels of taxonomy using the tables of specifications to ensure content validity. The rationale of having two evaluation tools seemingly similar in their evaluative competency yet differing in feasibility of construction, administration and marking is being challenged in this study. METHODS: The MCQ and SAQ awards of the ten percent sample population amounting to 985 students in fifteen Medical and Dental Colleges across Punjab were entered into SPSS-15 and correlated according to the cognitive and affective level of assessment in relation to the Bloom's taxonomy and their grouping in the Tables of Specifications, using parametric tests. 3494 anonymously administered questionnaires were analyzed using ethnograph. RESULTS: No statistically significant difference was found in the mean marks obtained by the students when MCQs and SAQs were compared according to their groupings in the Tables of Specifications at all levels of cognitive hierarchical testing. End-of-yearcognitive level testing targets set were not met and more questions were set at the lower cognitive testing levels. Expenses incurred in setting MCQs and SAQs were comparable but conduct and assessment costs for MCQs and SAQs were 6% and 94% of the total respectively. In both MCQs and SAQs students performed better at higher cognitive testing levels whereas the SAQs and MCQs were able to marginally test the lower levels of affective domain only. Student's feedback showed that attempting MCQs required critical thinking, experience and practice. CONCLUSION: MCQs are more cost effective means at levels of cognitive domain assessment.

Mental and neurological health research priorities setting in developing countries.

INTRODUCTION: A multi-region consultation process designed to generate locally produced regional and global research priorities on mental and neurological health in low- and middle-income countries. METHODS: Between 2003 and 2005, priority setting exercises on MNH research, using the systematic combined approach matrix (CAM) were held in the six regions of the developing world. One regional meeting per region was convened, and a global meeting was organized before and after the regional exercises. RESULTS: During regional meetings, regional agendas were created listing both research priorities and local problems in MNH. During global meetings, a global research agenda was established and four crucial areas of research priorities were identified: awareness and advocacy, enhancement of research capacity, training for service delivery, and development of evidence based policy. CONCLUSIONS: The combined matrix approach enabled the development of regional and global MNH research agendas, derived from bottom up consultations within and between low- and middle-income countries. Collaboration between regions with similar priorities was instituted. Such research agendas are designed to assist policy-makers and donors in the allocation of scarce resources, but they require regular review to reflect changing needs.

The International Consortium on Mental Health Policy and Services: objectives, design and project implementation.

The concept of the burden of disease, introduced and estimated for a broad range of diseases in the World Bank report of 1993 illustrated that mental and neurological disorders not only entail a higher burden than cancer, but are responsible, in developed and developing countries, for more than 15% of the total burden of all diseases. As a consequence, over the past decade, mental disorders have ranked increasingly highly on the international agenda for health. However, the fact that mental health and nervous system disorders are now high on the international health agenda is by no means a guarantee that the fate of patients suffering from these disorders in developing countries will improve. In most developing countries the treatment gap for mental and neurological disorders is still unacceptably high. To address this problem, an international network of collaborating institutions in low-income countries has been set up. The establishment and the achievements of this network--the International Consortium on Mental Health Policy and Services--are reported. Sixteen institutions in developing countries collaborate (supported by a small number of scientific resource centres in industrialized nations) in projects on applied mental health systems research. Over a two-year period, the network produced the key elements of a national mental health policy; provided tools and methods for assessing a country's current mental health status (context, needs and demands, programmes, services and care and outcomes); established a global network of expertise, i.e., institutions and experts, for use by countries wishing to reform their mental health policy, services and care; and generated guidelines and examples for upgrading mental health policy with due regard to the existing mental health delivery system and demographic, cultural and economic factors.

The mental health policy template: domains and elements for mental health policy formulation.

Mental disorders are a major and rising cause of disease burden in all countries. Even when resources are available, many countries do not have the policy and planning frameworks in place to identify and deliver effective interventions. The World Health Organization (WHO) and the World Bank have emphasized the need for ready access to the basic tools for mental health policy formulation, implementation and sustained development. The Analytical Studies on Mental Health Policy and Service Project, undertaken in 1999-2001 by the International Consortium for Mental Health Services and funded by the Global Forum for Health Research aims to address this need through the development of a template for mental health policy formulation. A mental health policy template has been developed based on an inventory of the key elements of a successful mental health policy. These elements have been validated against a review of international literature, a study of existing mental health policies and the results of extensive consultations with experts in the six WHO regions of the world. The Mental Health Policy Template has been revised and its applicability will be tested in a number of developing countries during 2001-2002. The Mental Health Policy Template and the work of the Consortium for Mental Health Services will be presented and the future role of the template in mental health policy development and reform in developing countries will be discussed.

The appropriateness and use of focus group methodology across international mental health communities.

The ability to interpret collected data across international mental health communities often proves to be difficult. The following paper reports on the use and appropriateness of focus group methodology in helping to clarify issues that could help substantiate data collection and comparison across different cultures and regions. Field tests of the focus group methodology were undertaken in different regions and this paper describes an overview of the final field test in Sofia, Bulgaria. The findings and experiences with utilizing this methodology were incorporated in subsequent data collections.

The mental health country profile: background, design and use of a systematic method of appraisal.

This article describes the construction and use of a systematic structured method of mental health country situation appraisal, in order to help meet the need for conceptual tools to assist planners and policy makers develop and audit policy and implementation strategies. The tool encompasses the key domains of context, needs, resources, provisions and outcomes, and provides a framework for synthesizing key qualitative and quantitative information, flagging up gaps in knowledge, and for reviewing existing policies. It serves as an enabling tool to alert and inform policy makers, professionals and other key stakeholders about important issues which need to be considered in mental health policy development. It provides detailed country specific information in a systematic format, to facilitate global sharing of experiences of mental health reform and strategies between policy makers and other stakeholders. Lastly, it is designed to be a capacity building tool for local stakeholders to enhance situation appraisal, and multisectorial policy development and implementation.

Pakistan mental health country profile.

The Republic of Pakistan is a South East Asian country with a population of over 140.7 million. Its population is fast growing and the majority (70%) live in rural areas with a feudal or tribal value system. The economy is dependent on agriculture and 35% of the population live below the poverty line. Islam is the main religion and 'mental illnesses' are stigmatized and widely perceived to have supernatural causes. The traditional healers along with psychiatric services are the main mental health service providers. The number of trained mental health professionals is small as compared to the population demands and specialist services are virtually non-existent. Lack of data on prevalence of various mental illnesses and monitory constraints are the major hurdles in the development of mental health services. A number of innovative programmes to develop indigenous models of care like the 'Community Mental Health Programme' and 'Schools Mental Health Programme' have been developed. These programmes have been found effective in reducing stigma and increase awareness of mental illness amongst the adults and children living in rural areas. Efforts by the government and mental health professionals have led to the implementation of a 'National Mental Health Policy' and 'Mental Health Act' in 2001. These aim at integrating mental health services with the existing health services, improving mental health care delivery and safeguarding the rights of mentally ill people. A favourable political will and the help of international institutions like the World Health Organization are required to achieve these aims.

Recruitment of consultant psychiatrists from low- and middle-income countries.

The UK's 2-year International Fellowship Programme for consultant doctors has inadvertently highlighted the long-standing issues of the costs and benefits of such recruitment for the countries of origin, and of whether it is ethical for rich countries to recruit health personnel not only from other rich countries but also from low- and middle-income countries. The 'brain drain' from poor to rich countries has been recognised for decades; it occurs in the health sector as well as other sectors, such as education, science and engineering. It has had serious ramifications for the health service infrastructure in low-income countries, where poverty, morbidity, disability and mortality are increasing rather than decreasing, and it is a matter of serious concern for both the World Health Organization and the International Monetary Fund (Carrington & Detragiache, 1998; Lee, 2003).

Demand for, access to and use of community mental health care: lessons from a demonstration project in India and Pakistan.

BACKGROUND: A widely promoted model of mental health care and prevention appropriate to many low-income countries is one that is integrated into the local primary health care system. AIMS: To examine the influence of health-seeking behaviours (demand-side factors) and the access to/availability of services (supply-side factors) on local service utilisation patterns for people with common mental disorders. METHOD: Two rural catchment populations outside Bangalore (India) and Rawalpindi (Pakistan), one with the standard primary health care system, the other with additional mental health care training and support, were screened for common mental disorders. Diagnosed cases were interviewed about their use of and perceptions of local health care services (repeated three months later). RESULTS: Individuals' use of integrated mental health and other care was modest. Principal (self-rated) supply-side factors were the cost of care, distance from treatment centre, a perception that care would not be effective, and concerns regarding stigma. Perceptions improved over three months, accompanied by an increased preference for public over private providers, but this was not restricted to the integrated care localities. CONCLUSION: The use (and therefore effectiveness) of mental health services integrated into primary care is influenced by the health-seeking behaviours and perceptions of the local population. Efforts to integrate mental health into primary care need to be accompanied by educational activities in order to increase awareness, reduce stigma and draw attention to the availability of effective treatment.