Using co-creation and multi-criteria decision analysis to close service gaps for underserved populations.

BACKGROUND: Navigating treatment pathways remains a challenge for populations with complex needs due to bottlenecks, service gaps and access barriers. The application of novel methods may be required to identify and remedy such problems. OBJECTIVE: To demonstrate a novel approach to identifying persistent service gaps, generating potential solutions and prioritizing action. DESIGN: Co-creation and multi-criteria decision analysis in the context of a larger, mixed methods study. SETTING AND PARTICIPANTS: Community-dwelling sample of older women living alone (OWLA), residing in Melbourne, Australia (n = 13-37). Convenience sample of (n = 11) representatives from providers and patient organizations. INTERVENTIONS: Novel interventions co-created to support health, well-being and independence for OWLA and bridge missing links in pathways to care. MAIN OUTCOME MEASURES: Performance criteria, criterion weights , performance ratings, summary scores and ranks reflecting the relative value of interventions to OWLA. RESULTS: The co-creation process generated a list of ten interventions. Both OWLA and stakeholders considered a broad range of criteria when evaluating the relative merits of these ten interventions and a "Do Nothing" alternative. Combining criterion weights with performance ratings yielded a consistent set of high priority interventions, with "Handy Help," "Volunteer Drivers" and "Exercise Buddies" most highly ranked by both OWLA and stakeholder samples. DISCUSSION AND CONCLUSIONS: The present study described and demonstrated the use of multi-criteria decision analysis to prioritize a set of novel interventions generated via a co-creation process. Application of this approach can add community voice to the policy debate and begin to bridge the gap in service provision for underserved populations.

Economic evaluation of simulated and traditional clinical placements in occupational therapy education.

INTRODUCTION: This economic evaluation complements results of the randomised controlled trial that established non-inferiority of the learning outcomes of a one-week simulated clinical placement (SCP) in occupational therapy qualifying degrees in comparison to an equivalent traditional clinical placement (TCP). This companion study presents detailed cost analyses of two placement alternatives and a cost-benefit study to assess the value for money of SCP. An economic evaluation of simulated versus traditional placements has not previously been conducted in Australia. METHODS: Nine SCP/TCP rounds were conducted by six Australian universities. Costs were collected using study-specific instruments. Public health sector costs were sourced from available literature. Willingness-to-pay for SCP/TCP was estimated using both a Discrete Choice Experiment and a Contingent Valuation method. These methods were employed to assess a comparative 'value' of SCP/TCP from the perspective of heads of occupational therapy departments (N = 28), who were asked to put a monetary value on the broader range of benefits associated with SCP/TCP. RESULTS: From the universities' perspective the average cost per student ranged from AUD$460 to AUD$1511 for simulated and AUD$144 to AUD$1112 for traditional placement. From the health care sector perspective, the difference in costs favoured simulated placements for four implementations and traditional placements for five. In the Discrete Choice Experiment respondents preferred traditional rather than simulated placement and would pay additional AUD$533. The estimated monetary value of simulated placements from a contingent valuation ranged from AUD$200 to AUD$1600. CONCLUSIONS: For universities that procure TCPs predominately at public health care facilities and sustain high administrative overheads, the SCP program could be a cost-saving alternative. From a broader value-for-money perspective, respondents favoured TCP over SCP, yet placed importance on placement availability and opportunity to demonstrate competence for students during the placement. Results should be interpreted with caution and further research with larger sample sizes is required.

Uncertainty and the Undervaluation of Services for Severe Health States in Cost-Utility Analyses.

OBJECTIVES: To test the hypothesis that the "severity effect"-the preference for more than utility-maximizing expenditure on severe health states-may be the result of, or exacerbated by, the uncertainty associated with the chance of contracting the illness that causes the severe health state. METHODS: Survey respondents were asked to imagine that they will contract one of two illnesses and asked to allocate a budget between two insurance policies, each of which provides services for the treatment of one of the illnesses. A person's final health state varied with the amount of insurance purchased for the illness that occurred and therefore with the level of treatment. The relative cost of the two policies was altered and the selected levels of insurance compared with the levels that would be provided by a health authority that sought to maximize total utility or quality-adjusted life-years from its own budget. RESULTS: Respondents selected more than utility-maximizing insurance for protection against severe health states. A number of psychological factors that affect measurement under uncertainty do not affect utility as currently measured. This difference may explain the present results and also explain the "severity paradox" that personal preferences as presently measured imply less expenditure on severe health states than do "social preferences" for the treatment of strangers. CONCLUSIONS: Uncertainty alters preferences. Incorporating these preferences in decision making would result in greater spending on severe health states.

Sharing and the Provision of "Cost-Ineffective" Life-Extending Services to Less Severely Ill Patients.

BACKGROUND: Cost-utility analysis prioritizes services using cost, life-years, and the health state utility of the life-years. Nevertheless, a significant body of evidence suggests that the public would prefer more variables to be considered in decision making and at least some sharing of the budget with services for severe conditions that are not cost-effective because of their high cost. OBJECTIVES: To examine whether this preference for sharing persists for less severe conditions when both cost effectiveness and illness severity would indicate that resources should be allocated to other services. METHODS: Survey respondents were asked to divide a budget between two patients facing life-threatening illnesses. The severity of the illnesses differed and the price of treatment was varied. RESULTS: Sharing occurred in all scenarios including scenarios in which the illness was less severe and services were not cost-effective. Results are consistent with behavior commonly observed in other contexts. CONCLUSIONS: Results suggest that sharing per se is important and that the public would support some funding of cost-ineffective services for less severe health problems.

"Fibromyalgia and quality of life: mapping the revised fibromyalgia impact questionnaire to the preference-based instruments".

BACKGROUND: The revised version of the Fibromyalgia Impact Questionnaire (FIQR) is one of the most widely used specific questionnaires in FM studies. However, this questionnaire does not allow calculation of QALYs as it is not a preference-based measure. The aim of this study was to develop mapping algorithm which enable FIQR scores to be transformed into utility scores that can be used in the cost utility analyses. METHODS: A cross-sectional survey was conducted. One hundred and 92 Spanish women with Fibromyalgia were asked to complete four general quality of life questionnaires, i.e. EQ-5D-5 L, 15D, AQoL-8D and SF-12, and one specific disease instrument, the FIQR. A direct mapping approach was adopted to derive mapping algorithms between the FIQR and each of the four multi-attribute utility (MAU) instruments. Health state utility was treated as the dependent variable in the regression analysis, whilst the FIQR score and age were predictors. RESULTS: The mean utility scores ranged from 0.47 (AQoL-8D) to 0.69 (15D). All correlations between the FIQR total score and MAU instruments utility scores were highly significant (p < 0.0001) with magnitudes larger than 0.5. Although very slight differences in the mean absolute error were found between ordinary least squares (OLS) estimator and generalized linear model (GLM), models based on GLM were better for EQ-5D-5 L, AQoL-8D and 15D. CONCLUSION: Mapping algorithms developed in this study enable the estimation of utility values from scores in a fibromyalgia specific questionnaire.

Assessing the validity of the ICECAP-A capability measure for adults with depression.

BACKGROUND: Effectiveness and cost-effectiveness are increasingly important considerations in determining which mental health services are funded. Questions have been raised concerning the validity of generic health status instruments used in economic evaluation for assessing mental health problems such as depression; measuring capability wellbeing offers a possible alternative. The aim of this study is to assess the validity of the ICECAP-A capability instrument for individuals with depression. METHODS: Hypotheses were developed using concept mapping. Validity tests and multivariable regression analysis were applied to data from a cross-sectional dataset to assess the performance of ICECAP-A in individuals who reported having a primary condition of depression. The ICECAP-A was collected alongside instruments used to measure: 1. depression using the depression scale of the Depression, Anxiety and Stress Scale (DASS-D of DASS-21); 2. mental health using the Kessler Psychological Distress Scale (K10); 3. generic health status using a common measure collected for use in economic evaluations, the five level version of EQ-5D (EQ-5D-5L). RESULTS: Hypothesised associations between the ICECAP-A (items and index scores) and depression constructs were fully supported in statistical tests. In the multivariable analysis, instruments designed specifically to measure depression and mental health explained a greater proportion of the variation in ICECAP-A than the EQ-5D-5L. CONCLUSION: The ICECAP-A instrument appears to be suitable for assessing outcome in adults with depression for resource allocation purposes. Further research is required on its responsiveness and use in economic evaluation. Using a capability perspective when assessing cost-effectiveness could potentially re-orientate resource provision across physical and mental health care services.

Deriving population norms for the AQoL-6D and AQoL-8D multi-attribute utility instruments from web-based data.

OBJECTIVES: (i) to demonstrate a method which ameliorates the problem of self-selection in the estimation of population norms from web-based data and (ii) to use the method to calculate population norms for two multi-attribute utility (MAU) instruments, the AQoL-6D and AQoL-8D, and population norms for the sub-scales from which they are constructed. METHODS: A web-based survey administered the AQoL-8D MAU instrument (which subsumes the AQoL-6D questionnaire), to members of the public along with the AQoL-4D which has extant population norms. Age, gender and the AQoL-4D were used as post-stratification auxiliary variables to construct weights to ameliorate the potential effects of self-selection associated with web-based surveys. The weights were used to estimate unbiased population norms. Standard errors from the weighted samples were calculated using Jackknife estimation. RESULTS: For both AQoL-6D and AQoL-8D, physical health dimensions decline significantly with age. In contrast, for the majority of the psycho-social dimensions there is a significant U-shaped profile. The net effect is a shallow U-shaped relationship between age and both the AQoL-6D and AQoL-8D utilities. This contrasts with the almost monotonic decline in the utilities derived from the AQoL-4D and SF-6D MAU instruments. CONCLUSIONS: Post-stratification weights were used to ameliorate potential bias in the derivation of norms from web-based data for the AQoL-6D and AQoL-8D. The methods may be used generally to obtain norms when suitable auxiliary variables are available. The inclusion of an enlarged psycho-social component in the two instruments significantly alters the demographic profile.

Estimating Utility Weights for the Vision Related Quality of Life Index.

PURPOSE: The VisQoL instrument was constructed as a vision-specific addition to the AQoL-6D multi-attribute utility instrument. The composite instrument, the AQoL-7D, was assigned utility scores that are the basis for now estimating VisQoL utilities when it is used as a stand-alone instrument. This study aimed to construct mapping functions that allow utility scores to be assigned to the Vision Related Quality of Life (VisQoL) instrument, a stand-alone vision-specific quality-of-life measure. METHODS: A sample of 164 patients completed the AQoL-7D, which includes the VisQoL. Mapping algorithms between VisQoL and AQoL-7D were then derived using two econometric methods, ordinary least squares estimator and generalized linear model (GLM). Two model specifications were considered with either six VisQoL raw item values or VisQoL overall dimension value as the key independent variables. The predictive performance of each method on each model specification was assessed using the mean absolute error (MAE) and intraclass correlation coefficient (ICC). Both internal and external validation tests (using a second, independent sample of 164 patients) were performed. RESULTS: The mapping algorithms derived from the GLM had superior properties to the ordinary least squares-based algorithms in both internal and external validation tests. The ICC values ranged from 0.851 to 0.913, and the MAE ranged from 0.043 to 0.052 for two model specifications, based on two econometric methods. However, predicted utilities tend to over-predict/under-predict the lowest/highest observed utility. CONCLUSIONS: Mapping algorithms predicting AQoL-7D utility based on six VisQoL items or VisQoL dimension value have been developed. The algorithm can be used to estimate quality adjusted life years. This allows the VisQoL to be used in cost utility analyses.

Why do multi-attribute utility instruments produce different utilities: the relative importance of the descriptive systems, scale and 'micro-utility' effects.

PURPOSE: Health state utilities measured by the major multi-attribute utility instruments differ. Understanding the reasons for this is important for the choice of instrument and for research designed to reconcile these differences. This paper investigates these reasons by explaining pairwise differences between utilities derived from six multi-attribute utility instruments in terms of (1) their implicit measurement scales; (2) the structure of their descriptive systems; and (3) 'micro-utility effects', scale-adjusted differences attributable to their utility formula. METHODS: The EQ-5D-5L, SF-6D, HUI 3, 15D and AQoL-8D were administered to 8,019 individuals. Utilities and unweighted values were calculated using each instrument. Scale effects were determined by the linear relationship between utilities, the effect of the descriptive system by comparison of scale-adjusted values and 'micro-utility effects' by the unexplained difference between utilities and values. RESULTS: Overall, 66 % of the differences between utilities was attributable to the descriptive systems, 30.3 % to scale effects and 3.7 % to micro-utility effects. DISCUSSION: Results imply that the revision of utility algorithms will not reconcile differences between instruments. The dominating importance of the descriptive system highlights the need for researchers to select the instrument most capable of describing the health states relevant for a study. CONCLUSIONS: Reconciliation of inconsistent utilities produced by different instruments must focus primarily upon the content of the descriptive system. Utility weights primarily determine the measurement scale. Other differences, attributable to utility formula, are comparatively unimportant.

Assessing outcomes for cost-utility analysis in depression: comparison of five multi-attribute utility instruments with two depression-specific outcome measures.

BACKGROUND: Many mental health surveys and clinical studies do not include a multi-attribute utility instrument (MAUI) that produces quality-adjusted life-years (QALYs). There is also some question about the sensitivity of the existing utility instruments to mental health. AIMS: To compare the sensitivity of five commonly used MAUIs (Assessment of Quality of Life - Eight Dimension Scale (AQoL-8D), EuroQoL-five dimension (EQ-5D-5L), Short Form 6D (SF-6D), Health Utilities Index Mark 3 (HUI3), 15D) with that of disease-specific depression outcome measures (Depression Anxiety Stress Scales (DASS-21) and the Kessler Psychological Distress Scale (K10)) and develop 'crosswalk' transformation algorithms between the measures. METHOD: Individual data from 917 people with self-report depression collected as part of the International Multi-Instrument Comparison Survey. RESULTS: All the MAUIs discriminated between the levels of severity measured by the K10 and the DASS-21. The AQoL-8D had the highest correlation with the disease-specific measures and the best goodness-of-fit transformation properties. CONCLUSIONS: The algorithms developed in this study can be used to determine cost-effectiveness of services or interventions where utility measures are not collected.

Mapping of incontinence quality of life (I-QOL) scores to assessment of quality of life 8D (AQoL-8D) utilities in patients with idiopathic overactive bladder.

BACKGROUND: The Incontinence Quality of Life (I-QOL) questionnaire is a commonly used and validated incontinence specific QOL instrument. The objective of this study is to develop an algorithm to map I-QOL to the Assessment of Quality of Life (AQoL) 8D utility instrument in patients with idiopathic overactive bladder (iOAB). METHODS: I-QOL and AQoL-8D scores were collected in a survey of 177 Australian adults with urinary incontinence due to iOAB. Three statistical methods were used for estimation, namely ordinary least squares (OLS) regression, the robust MM-estimator, and the generalised linear models (GLM). Each included a range of explanatory variables. Model performance was assessed using key goodness-of-fit measures in the validation dataset. RESULTS: The I-QOL total score and AQoL-8D utility scores were positively correlated (r = 0.50, p < 0.0001). Similarly, the three sub-scales of the I-QOL were correlated with the eight dimensions and two super-dimensions of the AQoL-8D. The GLM estimator, with I-QOL total score as the explanatory variable exhibited the best precision (MAE = 0.15 and RMSE = 0.18) with a mapping function given by AQoL-8D = exp(-1.28666 + 1.011072*I-QOL/100). CONCLUSIONS: The mapping algorithm developed in this study allows the derivation of AQoL-8D utilities from I-QOL scores. The algorithm allows the calculation of preference-based QOL scores for use in cost-utility analyses to assess the impact of interventions in urinary incontinence.

An instrument for measuring the social willingness to pay for health state improvement.

This paper describes an instrument for measuring the social value of changes in health status, the Relative Social Willingness to Pay. It is a unique combination of measurement attributes designed to minimise cognitive complexity and provide an additional option for measuring 'social value'. Similar to the person trade-off (PTO), it adopts a social perspective and asks respondents to evaluate programmes on behalf of society. Unlike the PTO, trade-offs between the options use dollars, not numbers of patients. Respondents are not, however, asked for their personal willingness to pay. Rather, the opportunity cost of funds spent on one service is as an offsetting reduction in funds for a second service. The amount spent on each service therefore indicates relative, not absolute, value. However, the two services combine to produce one Quality adjusted life year which allows the calculation of a Quality adjusted life year-like unit of social value on a 0-1 scale. A three-stage survey was used to test the instrument's reliability, validity and sensitivity to the framing of the main question. Results indicate that the Relative Social Willingness to Pay produces values similar to but less than the PTO and time trade-off techniques.

Modelling utility weights for the Assessment of Quality of Life (AQoL)-8D.

PURPOSE: The objective of this paper is to describe the four-stage methodology used to obtain utility scores for the Assessment of Quality of Life (AQoL)-8D, a 35-item 8 dimension multi-attribute utility instrument, which was created to achieve a high degree of sensitivity to psycho-social health. METHODS: Data for the analyses were obtained from a representative group of 347 members of the Australian public and from 323 mental health patients each of whom provided VAS and time trade-off valuations of multiple health states. Data were used initially to create multiplicative scoring algorithms for each of the instrument's 8 dimensions and for the overall instrument. Each of the algorithms was then subject to a second-stage econometric 'correction'. RESULTS: Algorithms were successfully created for each of the AQoL-8D's dimensions, for physical and mental 'super-dimensions' and for the overall AQoL-8D instrument. The final AQoL-8D algorithm has good predictive power with respect to the TTO valuations. CONCLUSIONS: The AQoL-8D is a suitable instrument for researchers conducting cost utility analyses generally but, in particular, for the analysis of services affecting psycho-social health.

Construction of the descriptive system for the Assessment of Quality of Life AQoL-6D utility instrument.

BACKGROUND: Multi attribute utility (MAU) instruments are used to include the health related quality of life (HRQoL) in economic evaluations of health programs. Comparative studies suggest different MAU instruments measure related but different constructs. The objective of this paper is to describe the methods employed to achieve content validity in the descriptive system of the Assessment of Quality of Life (AQoL)-6D, MAU instrument. METHODS: The AQoL program introduced the use of psychometric methods in the construction of health related MAU instruments. To develop the AQoL-6D we selected 112 items from previous research, focus groups and expert judgment and administered them to 316 members of the public and 302 hospital patients. The search for content validity across a broad spectrum of health states required both formative and reflective modelling. We employed Exploratory Factor Analysis and Structural Equation Modelling (SEM) to meet these dual requirements. RESULTS AND DISCUSSION: The resulting instrument employs 20 items in a multi-tier descriptive system. Latent dimension variables achieve sensitive descriptions of 6 dimensions which, in turn, combine to form a single latent QoL variable. Diagnostic statistics from the SEM analysis are exceptionally good and confirm the hypothesised structure of the model. CONCLUSIONS: The AQoL-6D descriptive system has good psychometric properties. They imply that the instrument has achieved construct validity and provides a sensitive description of HRQoL. This means that it may be used with confidence for measuring health related quality of life and that it is a suitable basis for modelling utilities for inclusion in the economic evaluation of health programs.

Predicting time trade-off health state valuations of adolescents in four Pacific countries using the Assessment of Quality-of-Life (AQoL-6D) instrument.

OBJECTIVES: Pacific Obesity Prevention in Communities (OPIC) is a community-based intervention project targeting adolescent obesity in Australia, New Zealand, Fiji, and Tonga. The Assessment of Quality of Life Mark 2 (AQoL-6D) instrument was completed by 15,481 adolescents to obtain a description of the quality of life associated with adolescent overweight and obesity, and a corresponding utility score for use in a cost-utility analysis of the interventions. This article describes the recalibration of this utility instrument for adolescents in each country. METHODS: The recalibration was based on country-specific time trade-off (TTO) data for 30 multiattribute health states constructed from the AQoL-6D descriptive system. Senior secondary students, in a classroom setting, responded to 10 health state scenarios each. These TTO interviews were conducted for 24 groups, comprising 279 students in the four countries resulting in 2790 completed TTO scores. The TTO scores were econometrically transformed by regressing the TTO scores upon predicted scores from the AQoL-6D to produce country-specific algorithms. The latter incorporated country-specific "corrections" to the Australian adult utility weights in the original AQoL. RESULTS: This article reports two methodological elements not previously reported. The first is the econometric modification of an extant multi-attribute utility instrument to accommodate cultural and other group-specific differences in preferences. The second is the use of the TTO technique with adolescents in a classroom group setting. Significant differences in utility scores were found between the four countries. CONCLUSION: Statistical results indicate that the AQoL-6D can be validly used in the economic evaluation of both the OPIC interventions and other adolescent programs.

Co-creation of services to maintain independence and optimise well-being: Learnings from Australia's Older Women Living Alone (OWLA) project.

For many populations at risk of social isolation, including Older Women Living Alone (OWLA), existing services to maintain independence and optimise well-being are difficult to access, unsuitable or unavailable. Co-creation is a strategy to develop 'person-centred' services that meet the needs of individuals. We adapted an existing framework for co-creation and used participatory action research methods, supported by an evidence base comprising a systematic review, analysis of routinely collected data and interviews, to develop person-centred services for OWLA. This approach achieved co-creation through an iterative process of consultation and review, involving a series of facilitated discussions with women living alone and stakeholders. A total of 13 women living alone, aged ≥55 years, and 11 stakeholders representing service providers and advocacy groups, were recruited to participate in these discussions. Sessions with between three and five OWLA, were held across Melbourne. The information was compiled and presented to service stakeholders in a single facilitated forum, held in central Melbourne. Smaller facilitated sessions with OWLA followed, to review and discuss the collated service stakeholder input. The information from these OWLA sessions were again compiled and directed back to the service stakeholders for consideration and further discussion. The two groups came together for a final forum to prioritise the co-created ten services that they believed would be feasible and would address unmet need to support OWLA maintain independence. The process of co-creation was time-consuming and required considerable preparation to facilitate input from the target population. Small groups, gathering at convenient local locations, with transport support were essential in removing barriers to participation. However, co-creation was a viable method of eliciting the women's preferences and developing services more likely to meet their needs.

Do quality-adjusted life years take account of lost income? Evidence from an Australian survey.

The procedures used in cost utility analysis for eliciting quality of life weights have generally omitted any instruction concerning the level of consumption in a health state, despite the fact that some health states preclude the possibility of normal employment. This introduces ambiguity into the interpretation of quality of life (QoL) scores, and project ranking is sensitive to the subsequent treatment of consumption in the analysis. This article reports the results of a study that questioned 131 respondents to a time trade-off (TTO) interview about their assumptions concerning consumption and the amount of thought given to consumption. Results indicate that, without prompting, most assumed unchanged consumption, implying little bias in existing studies.

Correction to: Are Quality-Adjusted Life Years a Good Proxy Measure of Individual Capabilities?

Open Access This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0International License.

Does a patient's health potential affect the social valuation of health services?

BACKGROUND: Patients with a permanent impairment may be unable to reach full health. Consequently health services which cure illnesses which are unrelated to the impairment may increase health less than services for patients with no impairment. While it has been argued that this should not lead to discrimination against impaired patients there is little evidence to determine whether this equity-efficiency trade-off is consistent with social values. OBJECTIVES: To measure the effect of permanent impairment upon the social valuation of services for unrelated illnesses. METHODS: Social valuations of services for illnesses associated with mobility, depression or pain were assessed and compared for patients with and without a permanent impairment using the Relative Social Willingness to Pay (RS-WTP) instrument. The maximum valuation of services for impaired patients was also compared with the maximum utility which could be gained when utility was measured using three multi attribute utility instruments. RESULTS: Curing the illness of impaired patients was valued 8-11 percent less than the cure of patients with no impairment. Discrimination decreased as the severity of the illness increased. Valuation of health states using the utility instruments implied significantly greater discrimination than the social valuations using the RS-WTP instrument. CONCLUSIONS: Health services are valued less highly when a patient's health potential is impaired. However discrimination is significantly less than would occur if the value of the services were limited to the value of the health state causing the impairment. The argument for disregarding a patient's limited health potential when resources are allocated therefore receives some support from social valuations but the case for completely equal treatment depends upon additional ethical arguments.

Does the use of the proportional shortfall help align the prioritisation of health services with public preferences?

It has been proposed that equity may be included in the economic evaluation of health services using the 'proportional shortfall' (PS)-the proportion of a person's QALY expectation that they would lose because of an illness. The present paper reports the results of a population survey designed to test whether PS helped to explain people's preferences for health services and whether it did this better than the absolute shortfall or the equity related variables that PS seeks to replace. Survey respondents were asked to allocate 100 votes between 13 scenarios and a standard scenario. Variation in the allocation of votes was explained by health gain and different combinations of the equity variables. Differences in votes for the comparisons were significantly related to differences in PS but the relationship was weaker than between votes and the age related variables. Cases were identified where PS suggested a priority ordering of services which was strongly rejected by respondents. It is concluded that the use of PS is unlikely to improve the alignment of priorities with public preferences.