Imagining Resilient Courts: from COVID-19 to the Future of Canada's Court System.

The coronavirus disease 2019 (COVID-19) pandemic has challenged an array of democratic institutions in complex and unprecedented ways. Little academic work, however, has considered the pandemic's impact on Canada's courts. This article aims to partially fill that gap by exploring the Canadian court system's response to COVID-19 and the prospects for administering justice amid disasters, all through the lens of resilience. After taking a forensic look at how the court system has managed the challenges brought on by COVID-19, we argue that features of resilience such as self-organization, flexibility, learning, and reflexive planning can contribute to the administration of justice during future shocks. We propose that the business of judging during shocks can become more integral to the business as usual of court systems. Imagining such a resilient court can be a way to step from COVID-19 to the future of Canada's court system.

La pandémie de la maladie à coronavirus 2019 (COVID-19) a mis au défi un grand nombre d'institutions démocratiques, de manière complexe et inédite. Or, très peu de recherches universitaires se sont intéressées à l'impact de la pandémie sur les tribunaux canadiens. Cet article vise à combler partiellement cette lacune en explorant la réponse du système judiciaire canadien à la COVID-19 et l'administration de la justice pendant les moments de crise, dans l'optique de la résilience. Un regard attentif à la manière dont le système judiciaire a géré les défis occasionnés par la COVID-19 nous permet de voir que les aspects de résilience tels que l'auto-organisation, la flexibilité, l'apprentissage et la planification réflexive peuvent contribuer à l'administration de la justice lors de futurs chocs. Nous proposons donc que les procédures adoptées pendant les temps de crise deviennent une pratique plus courante. Un tel tribunal résilient serait une façon de passer au travers de la COVID-19 et d'aider le système judiciaire canadien à se propulser dans l'avenir.

Computing PROPr Utility Scores for PROMIS® Profile Instruments.

OBJECTIVES: The Patient-Reported Outcomes Measurement Information System® (PROMIS) Profile instruments measure health status on 8 PROMIS domains. The PROMIS-Preference (PROPr) score provides a preference-based summary score for health states defined by 7 PROMIS domains. The Profile and PROPr share 6 domains; PROPr has 1 unique domain (Cognitive Function-Abilities), and the Profile has 2 unique domains (Anxiety and Pain Intensity). We produce an equation for calculating PROPr utility scores with Profile data. METHODS: We used data from 3982 members of US online survey panels who have scores on all 9 PROMIS domains. We used a 70%/30% split for model fit/validation. Using root-mean-square error and mean error on the utility scale, we compared models for predicting the missing Cognitive Function score via (A) the population average; (B) a score representing excellent cognitive function; (C) a score representing poor cognitive function; (D) a score predicted from linear regression of the 8 profile domains; and (E) a score predicted from a Bayesian neural network of the 8 profile domains. RESULTS: The mean errors in the validation sample on the PROPr scale (which ranges from -0.022 to 1.00) for the models were: (A) 0.025, (B) 0.067, (C) -0.23, (D) 0.018, and (E) 0.018. The root-mean-square errors were: (A) 0.097, (B) 0.12, (C) 0.29, (D) 0.095, and (E) 0.094. CONCLUSION: Although the Bayesian neural network had the best root-mean-square error for producing PROPr utility scores from Profile instruments, linear regression performs almost as well and is easier to use. We recommend the linear model for producing PROPr utility scores for PROMIS Profiles.

Perceptions of Clinical Experience and Scientific Evidence in Medical Decision Making: A Survey of a Stratified Random Sample of Swedish Health Care Professionals.

BACKGROUND: Evidence-based medicine recognizes that clinical expertise gained through experience is essential to good medical practice. However, it is not known what beliefs clinicians hold about how personal clinical experience and scientific knowledge contribute to their clinical decision making and how those beliefs vary between professions, which themselves vary along relevant characteristics, such as their evidence base. DESIGN: We investigate how years in the profession influence health care professionals' beliefs about science and their clinical experience through surveys administered to random samples of Swedish physicians, nurses, occupational therapists, dentists, and dental hygienists. The sampling frame was each profession's most recent occupational registry. RESULTS: Participants (N = 1,627, 46% response rate) viewed science as more important for decision making, more certain, and more systematic than experience. Differences among the professions were greatest for systematicity, where physicians saw the largest gap between the 2 types of knowledge across all levels of professional experience. The effect of years in the profession varied; it had little effect on assessments of importance across all professions but otherwise tended to decrease the difference between assessments of science and experience. Physicians placed the greatest emphasis on science over clinical experience among the 5 professions surveyed. CONCLUSIONS: Health care professions appear to share some attitudes toward professional knowledge, despite the variation in the age of the professions and the scientific knowledge base available to practitioners. Training and policy making about clinical decision making might improve by accounting for the ways in which knowledge is understood across the professions. HIGHLIGHTS: Study participants, representing 5 health care professions-medicine, nursing, occupational therapy, dentistry, and dental hygiene-viewed science as more important for decision making, more certain, and more systematic than their personal clinical experience.Of all the professions represented in the study, physicians saw the greatest differences between the 2 types of knowledge.The effect of years of professional experience varied but tended to be small, attenuating the differences seen between science and clinical experience.

Assessing How Consumers Interpret and Act on Results From At-Home COVID-19 Self-test Kits: A Randomized Clinical Trial.

IMPORTANCE: The US Food and Drug Administration (FDA) authorized SARS-CoV-2 rapid at-home self-test kits for individuals with and without symptoms. How appropriately users interpret and act on the results of at-home COVID-19 self-tests is unknown. OBJECTIVE: To assess how users of at-home COVID-19 self-test kits interpret and act on results when given instructions authorized by the FDA, instructions based on decision science principles, or no instructions. DESIGN, SETTING, AND PARTICIPANTS: A randomized clinical trial was conducted of 360 adults in the US who were recruited in April 2021 to complete an online survey on their interpretation of at-home COVID-19 self-test results. Participants were given 1 of 3 instruction types and were presented with 1 of 4 risk scenarios. Participants were paid $5 and had a median survey completion time of 8.7 minutes. Data analyses were performed from June to July 2021. INTERVENTION: Participants were randomized to receiving either the FDA-authorized instructions (authorized), the intervention instructions (intervention), or no instructions (control), and to 1 of 4 scenarios: 3 with a high pretest probability of infection (COVID-19 symptoms and/or a close contact with COVID-19) and 1 with low pretest probability (no symptoms and no contact). The intervention instructions were designed using decision science principles. MAIN OUTCOMES AND MEASURES: Proportion of participants in the high pretest probability scenarios choosing to quarantine per federal recommendations and perceived probabilities of infection given a negative or positive COVID-19 test result. A Bonferroni correction accounted for multiple comparisons (3 instruction types × 4 scenarios; α = 0.004). RESULTS: After excluding 22 individuals who completed the survey too quickly, the responses of 338 participants (median [IQR] age, 38 [31 to 48] years; 154 (46%) women; 215 (64%) with a college degree or higher) were included in the study analysis. Given a positive test result, 95% (322 of 338; 95% CI, 0.92 to 0.97) of the total participants appropriately chose to quarantine regardless of which instructions they had received. Given a negative test result, participants in the high pretest probability scenarios were more likely to fail to quarantine appropriately with the authorized instructions (33%) than with the intervention (14%; 95% CI for the 19% difference, 6% to 31%; P = .004) or control (24%; 95% CI for the 9% difference, -4% to 23%; P = .02). In the low pretest probability scenario, the proportion choosing unnecessary quarantine was higher with the authorized instructions (31%) than with the intervention (22%; 95% CI for the 9% difference, -14% to 31%) or control (10%; 95% CI for the 21% difference, 0.5% to 41%)-neither comparison was statistically significant (P = .05 and P = .20 respectively). CONCLUSIONS AND RELEVANCE: The findings of this randomized clinical trial indicate that at-home COVID-19 self-test kit users relying on the authorized instructions may not follow the Centers for Disease Control and Prevention's quarantine recommendations, producing unintended risks and unnecessary disruptions. Redesigned instructions that follow decision science principles may improve compliance. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04758299.

Certainty and systematicity of practice-derived evidence matter for its relative importance in professional decision-making: Survey results on the role of proven experience in Swedish medicine, nursing, OT, dentistry, and dental hygiene.

Background: High-quality healthcare decisions need to balance input from science and clinical practice. When two sources of evidence - such as scientific and practice-derived evidence - are compared, integrated, or need to stand-in for one another, they need to be comparable on similar dimensions. Since 1891, Swedish physicians have been operating under a legal requirement to base their healthcare decisions on science and "proven experience" (approximately clinical expertise), and today all healthcare personnel in Sweden fall under this legal requirement. Objectives: We investigated the dynamics between these two kinds of evidence with respect to importance, systematicity, and certainty by studying Swedish healthcare professionals. Design: Survey to professionals; document studies of political discourse. Method: In this study, a survey was sent to simple random samples of Swedish professionals in medicine, nursing, occupational therapy, dentistry, and dental hygiene, asking about the roles of science and proven experience in medical decision making. Outcome measures were how important, certain, and systematic science and proven experience are for successful medical decision making. Participants: The sampling frame was each profession's most recent occupational registry accessed by the Swedish federal statistical agency. 3500 surveys were distributed. 1626 surveys were returned. 26 participants were removed prior to analysis (exclusion criteria: more than one profession indicated, missing certificate, and mistake in stratum). The final sample consisted of 295 physicians, 300 nurses, 365 occupational therapists, 339 dentists, and 301 hygienists. 162 responses in questions used as variables in the analyses were either uninterpretable or empty; those were replaced with the modal response for a given participant's profession on a given question. Results: In the study, proven experience's perceived importance for clinical decision making is positively correlated with its certainty and systematicity, and an increased certainty and systematicity is positively correlated with a diminished difference in importance between science and proven experience for almost all professions surveyed in this study. Conclusions: Proven experience has an evidentiary role in clinical decision making, and this role depends in part on its certainty and systematicity. Notably, this makes the EBM-based perspective that practice-derived knowledge is primarily of implementation value less plausible.

The epistemic roles of clinical expertise: An empirical study of how Swedish healthcare professionals understand proven experience.

Clinical expertise has since 1891 a Swedish counterpart in proven experience. This study aims to increase our understanding of clinicians' views of their professional expertise, both as a source or body of knowledge and as a skill or quality. We examine how Swedish healthcare personnel view their expertise as captured by the (legally and culturally relevant) Swedish concept of "proven experience," through a survey administered to a simple random sample of Swedish physicians and nurses (2018, n = 560). This study is the first empirical attempt to analyse the notion of proven experience as it is understood by Swedish physicians and nurses. Using statistical techniques for data dimensionality reduction (confirmatory factor analysis and multidimensional scaling), the study provides evidence that the proven experience concept is multidimensional and that a model consisting of three dimensions-for brevity referred to as "test/evidence", "practice", and "being an experienced/competent person"-describes the survey responses well. In addition, our results cannot corroborate the widely held assumption in evidence-based medicine that an important component of clinical expertise consists of experience of patients' preferences.

A secure procedure for early career scientists to report apparent misconduct.

Early career scientists sometimes observe senior scientists engage in apparent scientific misconduct, but feel powerless to intervene, lest they imperil their careers. We propose a Secure Reporting Procedure that both protects them, when pursuing those concerns, and treats the senior scientists fairly. The proposed procedure is, we argue, consistent with the ethical principles of the scientific community, as expressed in the codes of its professional organizations. However, its implementation will require changes in procedures and regulations. Those efforts will be a small price to pay for protecting the scientific community's integrity and fidelity to its principles. We begin by describing the circumstances motivating the proposal, then sketch its design, and, finally, illustrate next steps in its application in two national settings.

Modeling the relative risk of SARS-CoV-2 infection to inform risk-cost-benefit analyses of activities during the SARS-CoV-2 pandemic.

Risk-cost-benefit analysis requires the enumeration of decision alternatives, their associated outcomes, and the quantification of uncertainty. Public and private decision-making surrounding the COVID-19 pandemic must contend with uncertainty about the probability of infection during activities involving groups of people, in order to decide whether that activity is worth undertaking. We propose a model of SARS-CoV-2 infection probability that can produce estimates of relative risk of infection for diverse activities, so long as those activities meet a list of assumptions, including that they do not last longer than one day (e.g., sporting events, flights, concerts), and that the probability of infection among possible routes of infection (i.e., droplet, aerosol, fomite, and direct contact) are independent. We show how the model can be used to inform decisions facing governments and industry, such as opening stadiums or flying on airplanes; in particular, it allows for estimating the ranking of the constituent components of activities (e.g., going through a turnstile, sitting in one's seat) by their relative risk of infection, even when the probability of infection is unknown or uncertain. We prove that the model is a good approximation of a more refined model in which we assume infections come from a series of independent risks. A linearity assumption governing several potentially modifiable risks factors-such as duration of the activity, density of participants, and infectiousness of the attendees-makes interpreting and using the model straightforward, and we argue that it does so without significantly diminishing the reliability of the model.

Evaluation of PROMIS Preference Scoring System (PROPr) in Patients Undergoing Hemodialysis or Kidney Transplant.

BACKGROUND AND OBJECTIVES: A preference-based health utility score (PROPr) can be calculated using Patient-Reported Outcomes Measurement Information System domain scores. We assessed the construct validity of PROPr among patients treated with KRT (hemodialysis or kidney transplant). DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: We performed a secondary analysis of data collected in multicenter, cross-sectional studies of adults treated with KRT, recruited between April 2016 to March 2020 in Toronto, Canada. All participants provided informed consent. The outcome was the PROPr score. Coadministered outcome variables included the Short-Form Six-Domain (SF-6D) and EuroQol Five-Domain Five-Level (EQ-5D-5L) scores. Socioeconomic and clinical variables included age, sex, diabetes, eGFR, serum albumin, hemoglobin, KRT, and Charlson Comorbidity Index. Construct validity was assessed through correlations between PROPr and SF-6D or EQ-5D-5L, and associations between PROPr and other exposure variables. Health-condition impact estimates (coefficients for health conditions compared with a referent category, e.g., dialysis versus kidney transplant) were calculated using multivariable linear regression. RESULTS: The mean (SD) age of the 524 participants was 57 (17) years, 58% were male, and 45% were White. Median (interquartile range) score was 0.39 (0.24-0.58) for PROPr, 0.69 (0.58-0.86) for SF-6D, and 0.85 (0.70-0.91) for EQ-5D-5L. Large correlations were observed between PROPr versus SF-6D (0.79; 95% confidence interval [95% CI], 0.76 to 0.82) and EQ-5D-5L (0.71; 95% CI, 0.66 to 0.75). Both PROPr and the other utility indices demonstrated health-condition impact in the expected direction. For example, the estimate for PROPr was -0.17 (95% CI, -0.13 to -0.21) for dialysis (versus kidney transplant), -0.05 (95% CI, -0.11 to 0.01; P=0.08) for kidney transplant recipients with an eGFR of <45 versus ≥45 ml/min per 1.73 m2, and -0.28 (95% CI, -0.22 to -0.33) for moderate/severe versus no/mild depressive symptoms. CONCLUSIONS: Our results support the validity of PROPr among patients treated with KRT.

Incorporating Mortality in Health Utility Measures.

The creation of multiattribute health utility systems requires design choices that have profound effects on the utility model, many of which have been documented and studied in the literature. Here we describe one design choice that has, to the best of our knowledge, been unrecognized and therefore ignored. It can emerge in any multiattribute decision analysis in which one or more essential outcomes cannot be described in terms of the multiattribute space. In health applications, the state of being dead is such an outcome. When the remaining health is conceptualized as a multidimensional space, determining the utility of the state of being dead requires using the interval-scale properties of cardinal utility, combined with elicited utilities for the state of being dead and the all-worst state, to produce a utility function in which the state of being dead has a utility of 0 and full health has a utility of 1 (i.e., the quality-adjusted life-year scale). Although previously unrecognized, there are two approaches to accomplish that step, and they produce different results in almost all cases. As a corollary, the choice of approach determines the proportion of states rated as worse than dead by the system. For example, in the Health Utility Index 3 (HUI3), the method used classifies 78% of the 972,000 unique health states in the classification system as worse than dead, and that proportion increases to 85% when the HUI3 is recalculated using the alternative approach. Studies of populations with significant morbidity are the most likely to be sensitive to the design choice. Those who design utility measures should be aware that they are using a researcher degree of freedom when they decide how to scale the state of being dead.

"Science and proven experience": How should the epistemology of medicine inform the regulation of healthcare?

The Swedish medico-legal concept of "science and proven experience" is both legally important and ambiguous. The conceptual uncertainty associated with it can hamper effective assessment of medical evidence in legal proceedings and encourage medical professionals to distrust legal regulation. We examine normative criteria a functioning medico-legal notion should presumably meet, e.g. clarity, acceptability and consistency with existing laws. We also survey healthcare professionals to see how they understand science and proven experience and thus determine the extent to which their understanding meets the normative criteria. The survey suggests that medical professionals feel more certain about "science and proven experience" in the medical context than they do in a legal context. They still have substantial trust in the legal use of the notion, but they do not believe that legal professionals should be allowed to determine the meaning of "science and proven experience" in the legal context. With these results in mind, we argue that the best way to meet the normative criteria and resolve conceptual uncertainty is to specify sub-questions that clarify the notion. We recommend an analytical-deliberative approach that will close the gap between the medical and legal professions' perceptions of how law and medicine relate.

A deterministic linear infection model to inform Risk-Cost-Benefit Analysis of activities during the SARS-CoV-2 pandemic.

Risk-cost-benefit analysis requires the enumeration of decision alternatives, their associated outcomes, and the quantification of uncertainty. Public and private decision-making surrounding the COVID-19 pandemic must contend with uncertainty about the probability of infection during activities involving groups of people, in order to decide whether that activity is worth undertaking. We propose a deterministic linear model of SARS-CoV-2 infection probability that can produce estimates of relative risk for diverse activities, so long as those activities meet a list of assumptions, including that they do not last longer than one day. We show how the model can be used to inform decisions facing governments and industry, such as opening stadiums or flying on airplanes. We prove that the model is a good approximation of a more refined model in which we assume infections come from a series of independent risks. The linearity assumption makes interpreting and using the model straightforward, and we argue that it does so without significantly diminishing the reliability of the model.

A Scoring Algorithm for Deriving Utility Values from the Neuro-QoL for Patients with Multiple Sclerosis.

INTRODUCTION: The Neuro-QoL is a standardized approach to assessing health-related quality of life in people with neurological conditions, including multiple sclerosis (MS). Item banks were developed with item response theory (IRT) methodology so items are calibrated along a continuum of each construct. The purpose of this study was to develop a preference-based scoring algorithm for the Neuro-QoL to derive utilities that could be used in economic modeling. METHODS: With input from neurologists, 6 Neuro-QoL domains were selected based on relevance to MS and used to define health states for a utility elicitation study in the United Kingdom. General population participants and individuals with MS valued the health states and completed questionnaires (including Neuro-QoL short forms). The Neuro-QoL Utility Scoring System (NQU) was derived based on multi-attribute utility theory using data from the general population sample. Single-attribute disutility functions for 6 Neuro-QoL domains were estimated using isotonic regression with linear interpolation and then combined with a multiplicative model. NQU validity was assessed using MS participant data. RESULTS: Interviews were completed with 203 general population participants (50.2% female; mean age = 45.0 years) and 62 participants with MS (62.9% female; mean age = 46.1 years). Mean (SD) NQU scores were 0.94 (0.06) and 0.82 (0.13) for the general population and MS samples, respectively. The NQU demonstrated known-groups validity by differentiating among subgroups categorized based on level of disability. The NQU demonstrated convergent validity via correlations with generic measures (0.66 and 0.63 with EQ-5D-5L and Health Utilities Index Mark 3, respectively; both P < 0.001). DISCUSSION: With the NQU, utilities can be derived from any MS treatment group, subgroup, or patient sample who completes items from 6 Neuro-QoL domains. Because the Neuro-QoL is frequently used with MS patients, the NQU greatly expands the options for quantifying outcomes in cost-utility analyses conducted to inform allocation of resources for MS treatment.

Exclusion Criteria as Measurements I: Identifying Invalid Responses.

Background. In a systematic review, Engel et al. found large variation in the exclusion criteria used to remove responses held not to represent genuine preferences in health state valuation studies. We offer an empirical approach to characterizing the similarities and differences among such criteria. Setting. Our analyses use data from an online survey that elicited preferences for health states defined by domains from the Patient-Reported Outcomes Measurement Information System (PROMIS®), with a U.S. nationally representative sample (N = 1164). Methods. We use multidimensional scaling to investigate how 10 commonly used exclusion criteria classify participants and their responses. Results. We find that the effects of exclusion criteria do not always match the reasons advanced for applying them. For example, excluding very high and very low values has been justified as removing aberrant responses. However, people who give very high and very low values prove to be systematically different in ways suggesting that such responses may reflect different processes. Conclusions. Exclusion criteria intended to remove low-quality responses from health state valuation studies may actually remove deliberate but unusual ones. A companion article examines the effects of the exclusion criteria on societal utility estimates.

Exclusion Criteria as Measurements II: Effects on Utility Functions.

Background. Researchers often justify excluding some responses in studies eliciting valuations of health states as not representing respondents' true preferences. Here, we examine the effects of applying 8 common exclusion criteria on societal utility estimates. Setting. An online survey of a US nationally representative sample (N = 1164) used the standard gamble method to elicit preferences for health states defined by 7 health domains from the Patient-Reported Outcomes Measurement Information System (PROMIS®). Methods. We estimate the impacts of applying 8 commonly used exclusion criteria on mean utility values for each domain, using beta regression, a form of analysis suited to double-bounded scales, such as utility. Results. Exclusion criteria have varied effects on the utility functions for the different PROMIS health domains. As a result, applying those criteria would have varied effects on the value of treatments (and side effects) that change health status on those domains. Limitations. Although our method could be applied to any health utility judgments, the present estimates reflect the features of the study that produced them. Those features include the selected health domains, standard gamble method, and an online format that excluded some groups (e.g., visually impaired and illiterate individuals). We also examined only a subset of all possible exclusion criteria, selected to represent the space of possibilities, as characterized in a companion article. Conclusions. Exclusion criteria can affect estimates of the societal utility of health states. We use those effects, in conjunction with the results of the companion article, to make suggestions for selecting exclusion criteria in future studies.

Defining the representativeness heuristic in trauma triage: A retrospective observational cohort study.

BACKGROUND: Under-triage of severely injured patients presenting to non-trauma centers (failure to transfer to a trauma center) remains problematic despite quality improvement efforts. Insights from the behavioral science literature suggest that physician heuristics (intuitive judgments), and in particular the representativeness heuristic (pattern recognition), may contribute to under-triage. However, little is known about how the representativeness heuristic is instantiated in practice. METHODS: A multi-disciplinary group of experts identified candidate characteristics of "representative" severe trauma cases (e.g., hypotension). We then reviewed the charts of patients with moderate-to-severe injuries who presented to nine non-trauma centers in western Pennsylvania from 2010-2014 to assess the association between the presence of those characteristics and triage decisions. We tested bivariate associations using χ2 and Fisher's Exact method and multivariate associations using random effects logistic regression. RESULTS: We identified 235,605 injured patients with 3,199 patients (1%) having moderate-to-severe injuries. Patients had a median age of 78 years (SD 20.1) and mean Injury Severity Score of 10.9 (SD 3.3). Only 759 of these patients (24%) were transferred to a trauma center as recommended by the American College of Surgeons clinical practice guidelines. Representative characteristics occurred in 704 patients (22%). The adjusted odds of transfer were higher in the presence of representative characteristics compared to when they were absent (aOR 1.7, 95% CI: 1.4-2.0, p < 0.001). CONCLUSIONS: Most moderate-to-severely injured patients present without the characteristics representative of severe trauma. Presence of these characteristics is associated with appropriate transfer, suggesting that modifying physicians' heuristics in trauma may improve triage patterns.

Cross-sectional validation of the PROMIS-Preference scoring system.

OBJECTIVES: The PROMIS-Preference (PROPr) score is a recently developed summary score for the Patient-Reported Outcomes Measurement Information System (PROMIS). PROPr is a preference-based scoring system for seven PROMIS domains created using multiplicative multi-attribute utility theory. It serves as a generic, societal, preference-based summary scoring system of health-related quality of life. This manuscript evaluates construct validity of PROPr in two large samples from the US general population. METHODS: We utilized 2 online panel surveys, the PROPr Estimation Survey and the Profiles-Health Utilities Index (HUI) Survey. Both included the PROPr measure, patient demographic information, self-reported chronic conditions, and other preference-based summary scores: the EuroQol-5D (EQ-5D-5L) and HUI in the PROPr Estimation Survey and the HUI in the Profiles-HUI Survey. The HUI was scored as both the Mark 2 and the Mark 3. Known-groups validity was evaluated using age- and gender-stratified mean scores and health condition impact estimates. Condition impact estimates were created using ordinary least squares regression in which a summary score was regressed on age, gender, and a single health condition. The coefficient for the health condition is the estimated effect on the preference score of having a condition vs. not having it. Convergent validity was evaluated using Pearson correlations between PROPr and other summary scores. RESULTS: The sample consisted of 983 respondents from the PROPr Estimation Survey and 3,000 from the Profiles-HUI survey. Age- and gender-stratified mean PROPr scores were lower than EQ-5D and HUI scores, with fewer subjects having scores corresponding to perfect health on the PROPr. In the PROPr Estimation survey, all 11 condition impact estimates were statistically significant using PROPr, 8 were statistically significant by the EQ-5D, 7 were statistically significant by HUI Mark 2, and 9 were statistically significant by HUI Mark 3. In the Profiles-HUI survey, all 21 condition impact estimates were statistically significant using summary scores from all three scoring systems. In these samples, the correlations between PROPr and the other summary measures ranged from 0.67 to 0.70. CONCLUSIONS: These results provide evidence of construct validity for PROPr using samples from the US general population.

Estimation of a Preference-Based Summary Score for the Patient-Reported Outcomes Measurement Information System: The PROMIS®-Preference (PROPr) Scoring System.

BACKGROUND: Health-related quality of life (HRQL) preference-based scores are used to assess the health of populations and patients and for cost-effectiveness analyses. The National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS®) consists of patient-reported outcome measures developed using item response theory. PROMIS is in need of a direct preference-based scoring system for assigning values to health states. OBJECTIVE: To produce societal preference-based scores for 7 PROMIS domains: Cognitive Function-Abilities, Depression, Fatigue, Pain Interference, Physical Function, Sleep Disturbance, and Ability to Participate in Social Roles and Activities. SETTING: Online survey of a US nationally representative sample ( n = 983). METHODS: Preferences for PROMIS health states were elicited with the standard gamble to obtain both single-attribute scoring functions for each of the 7 PROMIS domains and a multiplicative multiattribute utility (scoring) function. RESULTS: The 7 single-attribute scoring functions were fit using isotonic regression with linear interpolation. The multiplicative multiattribute summary function estimates utilities for PROMIS multiattribute health states on a scale where 0 is the utility of being dead and 1 the utility of "full health." The lowest possible score is -0.022 (for a state viewed as worse than dead), and the highest possible score is 1. LIMITATIONS: The online survey systematically excludes some subgroups, such as the visually impaired and illiterate. CONCLUSIONS: A generic societal preference-based scoring system is now available for all studies using these 7 PROMIS health domains.

An Approach to Reconciling Competing Ethical Principles in Aggregating Heterogeneous Health Preferences.

BACKGROUND: Health-related quality of life (HRQL) scores are used extensively to quantify the effectiveness of medical interventions. Societal preference-based HRQL scores aim to produce societal valuations of health by aggregating valuations from individuals in the general population, where each aggregation procedure embodies different ethical principles, as explained in social choice theory. METHODS: Using the Health Utilities Index as an exemplar, we evaluate societal preference-based HRQL measures in the social choice theory framework. RESULTS: We find that current preference aggregation procedures are typically justified in terms of social choice theory. However, by convention, they use only one of many possible aggregation procedures (the mean). Central to the choice of aggregation procedure is how to treat preference heterogeneity, which can affect analyses that rely on HRQL scores, such as cost-effectiveness analyses. We propose an analytical-deliberative framework for choosing one (or a set of) aggregation procedure(s) in a socially credible way, which we believe to be analytically sound and empirically tractable, but leave open the institutional mechanism needed to implement it. CONCLUSIONS: Socially acceptable decisions about aggregating heterogeneous preferences require eliciting stakeholders' preferences among the set of analytically sound procedures, representing different ethical principles. We describe a framework for eliciting such preferences for the creation of HRQL scores, informed by social choice theory and behavioral decision research.