How to use polarity thinkingTM to manage tensions between accountability and learner agency when using a multipurpose portfolio?

Portfolios are often implemented to target multiple purposes, e.g. assessment, accountability and/or self-regulated learning. However, in educational practice, it appears to be difficult to combine different purposes in one portfolio, as interdependencies between the purposes can cause tensions. This paper explored directions to manage tensions that are inextricably linked to multipurpose portfolio use. We used a systems thinking methodology, that was based on the polarity thinkingTM framework. This framework provides a step-by-step approach to chart a polarity map® that can help to balance the tensions present in specific settings. We followed the steps of the framework to chart a polarity map for multipurpose portfolio use. Based on literature and our prior research, we selected one overarching polarity: accountability and learner agency. This polarity seems responsible for multiple tensions related to multipurpose portfolio use. We formulated values (potential benefits) and fears (tensions that can arise) of the two poles of this polarity. Then, we organised a session with stakeholders who work with the portfolio of the Dutch General Practice speciality programme. Together we formulated action steps and early warnings that can help to balance accountability and learner agency during multipurpose portfolio use. In addition to previous recommendations concerning portfolio use, we advocate that it is important to create a shared frame of reference between all involved with the multipurpose portfolio. During this process, the acknowledgement and discussion of tensions related to multipurpose portfolio use are vital.

A practical approach to programmatic assessment design.

Assessment of complex tasks integrating several competencies calls for a programmatic design approach. As single instruments do not provide the information required to reach a robust judgment of integral performance, 73 guidelines for programmatic assessment design were developed. When simultaneously applying these interrelated guidelines, it is challenging to keep a clear overview of all assessment activities. The goal of this study was to provide practical support for applying a programmatic approach to assessment design, not bound to any specific educational paradigm. The guidelines were first applied in a postgraduate medical training setting, and a process analysis was conducted. This resulted in the identification of four steps for programmatic assessment design: evaluation, contextualisation, prioritisation and justification. Firstly, the (re)design process starts with sufficiently detailing the assessment environment and formulating the principal purpose. Key stakeholders with sufficient (assessment) expertise need to be involved in the analysis of strengths and weaknesses and identification of developmental needs. Central governance is essential to balance efforts and stakes with the principal purpose and decide on prioritisation of design decisions and selection of relevant guidelines. Finally, justification of assessment design decisions, quality assurance and external accountability close the loop, to ensure sound underpinning and continuous improvement of the assessment programme.

Psychometric qualities of questionnaires for the assessment of otitis media impact.

BACKGROUND: The assessment of impact and evaluation of treatment effects in chronic otitis media (OM) calls for a much broader approach than just examining the presence of middle ear effusion or hearing loss. It is increasingly recognised that this condition may result in a comprised quality of life. Several studies have used proxy completed questionnaires to objectify the illness experience associated with chronic OM. OBJECTIVE OF REVIEW: To review questionnaires which have been developed to describe the effects of chronic OM on the daily functioning of children. Psychometric properties have been evaluated, in addition to discriminative and evaluative qualities. TYPE OF REVIEW: A systematic review of publications pertaining to developed questionnaires related with chronic OM. SEARCH STRATEGY: Systematic literature searches of PubMed (1966-January 2007) and EMBASE (1989-January 2007) were conducted, supplemented by using free text words to identify publications after January 2005. RESULTS: The included 15 questionnaires were developed for children with recurrent or persistent OM, describing functional health status (FHS), while two questionnaires also evaluate the effect of tympanostomy tubes insertion. The questionnaires generally cover six impact areas (physical symptoms, child development, educational performance, emotional/practical burden and general health status) with physical symptoms being the most prominant. CONCLUSIONS: The OM8-30, OMO-22 and OM-6 adequately reflect the multidimensional aspects of FHS in chronic OM. The OMO-22 and OM8-30 show the best psychometric properties for the discrimination of impact severity between children, while the OM-6 was found to have the best qualities for the evaluation of clinical change. Clinical applicability is crucial for the assessment of FHS in chronic OM, but requires a trade-off with necessary psychometric properties.

The initial development of an instrument for the description of "otitis media with effusion specific behavior" in young children.

The objective of this study was a preliminary evaluation of a newly developed questionnaire concerning behavior of young children with recurrent Otitis Media with Effusion (OME) and documenting behavioral change after tympanostomy tube insertion. A sample of 95 parent-child pairs were interviewed using an age specific questionnaire: after diagnosing OME and scheduling for surgery, at the day care theatre a few weeks later, and 6 weeks after the intervention. The levels of internal consistency were varying from still insufficient to fairly good for the different subscales of the construct "OME-specific behavior" and these levels were comparable for both age specific questionnaires. As expected, parental report proved to be stable during time, but behavioral changes were reported after tympanostomy tubes insertion. The questionnaires seem to assess a decline in behavior expected to be specific for OME. Implications for future research are discussed, while considering the influence of parental concerns on behavioral reports.

Response shift in the measurement of quality of life in hearing impaired adults after hearing aid fitting.

UNLABELLED: REASON FOR THE STUDY: Response shift is the change in the meaning of one's self-evaluation of a target construct, like quality of life (QOL). The objective of this study was to investigate whether response shift in the measurement of generic and specific QOL occurred in persons with a relatively mild health condition. For this purpose hearing impairment was used as a research model. MAJOR FINDINGS: Response shift effects were observed in the scores on the dimensions of hearing related QOL. In the scores on overall hearing related QOL, and in the scores on the generic control items, no response shift occurred. CONCLUSIONS: This study showed that response shift effects can take place in a relatively mild condition as well. The occurrence of response shift in QOL ratings over time could have large implications for the estimation of the effectiveness of medical interventions and for the use of these estimations in cost-effectiveness analyses. After a successful treatment the conventional change could be an underestimation of the effectiveness of the treatment, although it has also been argued that psychological adaptation is a welcome capacity of human beings, and that then-test changes do no justice to this capacity.