Chained Deep Learning Using Generalized Cross-Entropy for Multiple Annotators Classification.

Supervised learning requires the accurate labeling of instances, usually provided by an expert. Crowdsourcing platforms offer a practical and cost-effective alternative for large datasets when individual annotation is impractical. In addition, these platforms gather labels from multiple labelers. Still, traditional multiple-annotator methods must account for the varying levels of expertise and the noise introduced by unreliable outputs, resulting in decreased performance. In addition, they assume a homogeneous behavior of the labelers across the input feature space, and independence constraints are imposed on outputs. We propose a Generalized Cross-Entropy-based framework using Chained Deep Learning (GCECDL) to code each annotator's non-stationary patterns regarding the input space while preserving the inter-dependencies among experts through a chained deep learning approach. Experimental results devoted to multiple-annotator classification tasks on several well-known datasets demonstrate that our GCECDL can achieve robust predictive properties, outperforming state-of-the-art algorithms by combining the power of deep learning with a noise-robust loss function to deal with noisy labels. Moreover, network self-regularization is achieved by estimating each labeler's reliability within the chained approach. Lastly, visual inspection and relevance analysis experiments are conducted to reveal the non-stationary coding of our method. In a nutshell, GCEDL weights reliable labelers as a function of each input sample and achieves suitable discrimination performance with preserved interpretability regarding each annotator's trustworthiness estimation.

Value of willingness to pay for a QALY gained in Iran; a modified chained-approach.

BACKGROUND: Due to the lack of a constant Willingness to Pay per one additional Quality Adjusted Life Years gained based on the preferences of Iran's general public, the cost-effectiveness of health system interventions is unclear and making it challenging to apply economic evaluation to health resources priority setting. METHODS: We have measured this cost-effectiveness threshold with the participation of 2854 individuals from five provinces, each representing an income quintile, using a modified Time Trade-Off-based Chained-Approach. In this online-based empirical survey, to extract the health utility value, participants were randomly assigned to one of two green (21121) and yellow (22222) health scenarios designed based on the earlier validated EQ-5D-3L questionnaire. RESULTS: Across the two health state versions, mean values for one QALY gain (rounded) ranged from $6740-$7400 and $6480-$7120, respectively, for aggregate and trimmed models, which are equivalent to 1.35-1.18 times of the GDP per capita. Log-linear Multivariate OLS regression analysis confirmed that respondents were more likely to pay if their income, disutility, and education level were higher than their counterparts. CONCLUSIONS: In the health system of Iran, any intervention that is with the incremental cost-effectiveness ratio, equal to and less than 7402.12 USD, will be considered cost-effective.

Sample restrictions and the elicitation of a constant willingness to pay per quality adjusted life year.

It is well established that the underlying theoretical assumptions needed to obtain a constant proportional trade-off between a quality adjusted life year (QALY) and willingness to pay (WTP) are restrictive and often empirically violated. In this paper, we set out to investigate whether the proportionality conditions (in terms of scope insensitivity and severity independence) can be satisfied when data is restricted to include only respondents who pass certain consistency criteria. We hypothesize that the more we restrict the data, the better the compliance with the requirement of constant proportional trade-off between WTP and QALY. We revisit the Danish data from the European Value of a QALY survey eliciting individual WTP for a QALY (WTP-Q). Using a "chained approach" respondents were first asked to value a specified health state using the standard gamble (SG) or the time-trade-off (TTO) approach and subsequently asked their WTP for QALY gains of 0.05 and 0.1 (tailored according to the respondent's SG/TTO valuation). Analyzing the impact of the different exclusion criteria on the two proportionality conditions, we find strong evidence against a constant WTP-Q. Restricting our data to include only respondents who pass the most stringent consistency criteria does not impact on the performance of the proportionality conditions for WTP-Q.

Value of a QALY and VSI estimated with the chained approach.

The value of a quality-adjusted life-year (QALY) and the value of a statistical injury (VSI) are important measures within health economics and transport economics. Several studies have, therefore, estimated people's willingness to pay (WTP) for these estimates, but most results show scale insensitivity. The 'original' chained approach (CA) is a method developed to mitigate this problem by combining the contingent valuation (CV) with standard gamble (SG). In contrast to the version of the CA applied by the previous research of the WTP for a QALY, the original version allows the value of major health gains to be estimated without having the respondents express their WTP directly. The objective of this study was to estimate the value of a QALY and VSI in the context of non-fatal road traffic accidents using the original CA to test if the approach, applied to a wide range of health gains, is able to derive valid estimates and a constant value of a QALY which the previous research has not been able to show. Data were collected from a total of 800 individuals in the Swedish adult general population using two web-based questionnaires. The values of a QALY based on trimmed estimates were close to constant at €300,000 irrespective of the size of the QALY gain. The study shows that the original CA method may be a valid method to estimate the value of a QALY and VSI for major health losses. It also supports the use of a higher threshold value for a QALY than that which is currently applied by several health technology assessment agencies in different countries.

Estimating a WTP-based value of a QALY: the 'chained' approach.

A major issue in health economic evaluation is that of the value to place on a quality adjusted life year (QALY), commonly used as a measure of health care effectiveness across Europe. This critical policy issue is reflected in the growing interest across Europe in development of more sound methods to elicit such a value. EuroVaQ was a collaboration of researchers from 9 European countries, the main aim being to develop more robust methods to determine the monetary value of a QALY based on surveys of the general public. The 'chained' approach of deriving a societal willingness-to-pay (WTP) based monetary value of a QALY used the following basic procedure. First, utility values were elicited for health states using the standard gamble (SG) and time trade off (TTO) methods. Second, a monetary value to avoid some risk/duration of that health state was elicited and the implied WTP per QALY estimated. We developed within EuroVaQ an adaptation to the 'chained approach' that attempts to overcome problems documented previously (in particular the tendency to arrive at exceedingly high WTP per QALY values). The survey was administered via Internet panels in each participating country and almost 22,000 responses achieved. Estimates of the value of a QALY varied across question and were, if anything, on the low side with the (trimmed) 'all country' mean WTP per QALY ranging from $18,247 to $34,097. Untrimmed means were considerably higher and medians considerably lower in each case. We conclude that the adaptation to the chained approach described here is a potentially useful technique for estimating WTP per QALY. A number of methodological challenges do still exist, however, and there is scope for further refinement.