Simulated patients and their reality: An inquiry into theory and method.

Simulated standardized patients (SSP) have emerged as close to a 'gold standard' for measuring the quality of clinical care. This method resolves problems of patient mix across healthcare providers and allows care to be benchmarked against preexisting standards. Nevertheless, SSPs are not real patients. How, then, should data from SSPs be considered relative to clinical observations with 'real' patients in a given health system? Here, we reject the proposition that SSPs are direct substitutes for real patients and that the validity of SSP studies therefore relies on their ability to imitate real patients. Instead, we argue that the success of the SSP methodology lies in its counterfactual manipulations of the possibilities available to real careseekers - especially those paths not taken up by them - through which real responses can be elicited from real providers. Using results from a unique pilot study where SSPs returned to providers for follow-ups when asked, we demonstrate that the SSP method works well to elicit responses from the provider through conditional manipulations of SSP behavior. At the same time, observational methods are better suited to understand what choices real people make, and how these can affect the direction of diagnosis and treatment. A combination of SSP and observational methods can thus help parse out how quality of care emerges for the "patient" as a shared history between care-seeking individuals and care providers.

Variations in the quality of tuberculosis care in urban India: A cross-sectional, standardized patient study in two cities.

BACKGROUND: India has the highest burden of tuberculosis (TB). Although most patients with TB in India seek care from the private sector, there is limited evidence on quality of TB care or its correlates. Following our validation study on the standardized patient (SP) method for TB, we utilized SPs to examine quality of adult TB care among health providers with different qualifications in 2 Indian cities. METHODS AND FINDINGS: During 2014-2017, pilot programs engaged the private health sector to improve TB management in Mumbai and Patna. Nested within these projects, to obtain representative, baseline measures of quality of TB care at the city level, we recruited 24 adults to be SPs. They were trained to portray 4 TB "case scenarios" representing various stages of disease and diagnostic progression. Between November 2014 and August 2015, the SPs visited representatively sampled private providers stratified by qualification: (1) allopathic providers with Bachelor of Medicine, Bachelor of Surgery (MBBS) degrees or higher and (2) non-MBBS providers with alternative medicine, minimal, or no qualifications. Our main outcome was case-specific correct management benchmarked against the Standards for TB Care in India (STCI). Using ANOVA, we assessed variation in correct management and quality outcomes across (a) cities, (b) qualifications, and (c) case scenarios. Additionally, 2 micro-experiments identified sources of variation: first, quality in the presence of diagnostic test results certainty and second, provider consistency for different patients presenting the same case. A total of 2,652 SP-provider interactions across 1,203 health facilities were analyzed. Based on our sampling strategy and after removing 50 micro-experiment interactions, 2,602 interactions were weighted for city-representative interpretation. After weighting, the 473 Patna providers receiving SPs represent 3,179 eligible providers in Patna; in Mumbai, the 730 providers represent 7,115 eligible providers. Correct management was observed in 959 out of 2,602 interactions (37%; 35% weighted; 95% CI 32%-37%), primarily from referrals and ordering chest X-rays (CXRs). Unnecessary medicines were given to nearly all SPs, and antibiotic use was common. Anti-TB drugs were prescribed in 118 interactions (4.5%; 5% weighted), of which 45 were given in the case in which such treatment is considered correct management. MBBS and more qualified providers had higher odds of correctly managing cases than non-MBBS providers (odds ratio [OR] 2.80; 95% CI 2.05-3.82; p < 0.0001). Mumbai non-MBBS providers had higher odds of correct management than non-MBBS in Patna (OR 1.79; 95% CI 1.06-3.03), and MBBS providers' quality of care did not vary between cities (OR 1.15; 95% CI 0.79-1.68; p = 0.4642). In the micro-experiments, improving diagnostic certainty had a positive effect on correct management but not across all quality dimensions. Also, providers delivered idiosyncratically consistent care, repeating all observed actions, including mistakes, approximately 75% of the time. The SP method has limitations: it cannot account for patient mix or care-management practices reflecting more than one patient-provider interaction. CONCLUSIONS: Quality of TB care is suboptimal and variable in urban India's private health sector. Addressing this is critical for India's plans to end TB by 2025. For the first time, we have rich measures on representative levels of care quality from 2 cities, which can inform private-sector TB interventions and quality-improvement efforts.

Efficiency of HIV services in Nigeria: Determinants of unit cost variation of HIV counseling and testing and prevention of mother-to-child transmission interventions.

BACKGROUND: Like most countries with a substantial HIV burden, Nigeria continues to face challenges in reaching coverage targets of HIV services. A fundamental problem is stagnated funding in recent years. Improving efficiency is therefore paramount to effectively scale-up HIV services. In this study, we estimated the facility-level average costs (or unit costs) of HIV Counseling and Testing (HCT) and Prevention of Mother-to-Child Transmission (PMTCT) services and characterized determinants of unit cost variation. We investigated the role of service delivery modalities and the link between facility-level management practices and unit cost variability along both services' cascades. METHODS: We conducted a cross-sectional, observational, micro-costing study in Nigeria between December 2014 and May 2015 in 141 HCT, and 137 PMTCT facilities, respectively. We retrospectively collected relevant input quantities (personnel, supplies, utilities, capital, and training), input prices, and output data for the year 2013. Staff costs were adjusted using time-motion methods. We estimated the facility-level average cost per service along the HCT and PMTCT service cascades and analyzed their composition and variability. Through linear regressions analysis, we identified aspects of service delivery and management practices associated with unit costs variations. RESULTS: The weighted average cost per HIV-positive client diagnosed through HCT services was US$130. The weighted average cost per HIV-positive woman on prophylaxis in PMTCT services was US$858. These weighted values are estimates of nationally representative unit costs in Nigeria. For HCT, the facility-level unit costs per client tested and per HIV-positive client diagnosed were US$30 and US$1,364, respectively; and the median unit costs were US$17 and US$245 respectively. For PMTCT, the facility-level unit costs per woman tested, per HIV-positive woman diagnosed, and per HIV-positive woman on prophylaxis were US$46, US$2,932, and US$3,647, respectively, and the median unit costs were US$24, US$1,013 and US$1,448, respectively. Variability in costs across facilities was principally explained by the number of patients, integration of HIV services, task shifting, and the level of care. DISCUSSION: Our findings demonstrate variability in unit costs across facilities. We found evidence consistent with economies of scale and scope, and efficiency gains in facilities implementing task-shifting. Our results could inform program design by suggesting ways to improve resource allocation and efficiently scale-up the HIV response in Nigeria. Some of our findings might also be relevant for other settings.

Use of standardised patients to assess quality of tuberculosis care: a pilot, cross-sectional study.

BACKGROUND: Existing studies of the quality of tuberculosis care have relied on recall-based patient surveys, questionnaire surveys of knowledge, and prescription or medical record analysis, and the results mostly show the health-care provider's knowledge rather than actual practice. No study has used standardised patients to assess clinical practice. Therefore we aimed to assess quality of care for tuberculosis using such patients. METHODS: We did a pilot, cross-sectional validation study of a convenience sample of consenting private health-care providers in low-income and middle-income areas of Delhi, India. We recruited standardised patients in apparently good health from the local community to present four cases (two of presumed tuberculosis and one each of confirmed tuberculosis and suspected multidrug-resistant tuberculosis) to a randomly allocated health-care provider. The key objective was to validate the standardised-patient method using three criteria: negligible risk and ability to avoid adverse events for providers and standardised patients, low detection rates of standardised patients by providers, and data accuracy across standardised patients and audio verification of standardised-patient recall. We also used medical vignettes to assess providers' knowledge of presumed tuberculosis. Correct case management was benchmarked using Standards for Tuberculosis Care in India (STCI). FINDINGS: Between Feb 2, and March 28, 2014, we recruited and trained 17 standardised patients who had 250 interactions with 100 health-care providers, 29 of whom were qualified in allopathic medicine (ie, they had a Bachelor of Medicine & Surgery [MBBS] degree), 40 of whom practised alternative medicine, and 31 of whom were informal health-care providers with few or no qualifications. The interactions took place between April 1, and April 23, 2014. The proportion of detected standardised patients was low (11 [5%] detected out of 232 interactions among providers who completed the follow-up survey), and standardised patients' recall correlated highly with audio recordings (r=0·63 [95% CI 0·53-0·79]), with no safety concerns reported. The mean consultation length was 6 min (95% CI 5·5-6·6) with a mean of 6·18 (5·72-6·64) questions or examinations completed, representing 35% (33-38) of essential checklist items. Across all cases, only 52 (21% [16-26]) of 250 were correctly managed. Correct management was higher among MBBS-qualified doctors than other types of health-care provider (adjusted odds ratio 2·41 [95% CI 1·17-4·93]; p=0·0166). Of the 69 providers who completed the vignette, knowledge in the vignettes was more consistent with STCI than their actual clinical practice-eg, 50 (73%) ordered a chest radiograph or sputum test during the vignette compared with seven (10%) during the standardised-patient interaction; OR 0·04 (95% CI 0·02-0·11); p<0·0001. INTERPRETATION: Standardised patients can be successfully implemented to assess tuberculosis care. Our data suggest a big gap between private provider knowledge and practice. Additional work is needed to substantiate our pilot data, understand the know-do gap in provider behaviour, and to identify the best approach to measure and improve the quality of tuberculosis care in India. FUNDING: Grand Challenges Canada, the Bill & Melinda Gates Foundation, Knowledge for Change Program, and the World Bank Development Research Group.

Assessing cost and technical efficiency of HIV prevention interventions in sub-Saharan Africa: the ORPHEA study design and methods.

BACKGROUND: Scaling up services to achieve HIV targets will require that countries optimize the use of available funding. Robust unit cost estimates are essential for the better use of resources, and information on the heterogeneity in the unit cost of delivering HIV services across facilities - both within and across countries - is critical to identifying and addressing inefficiencies. There is limited information on the unit cost of HIV prevention services in sub-Saharan Africa and information on the heterogeneity within and across countries and determinants of this variation is even more scarce. The "Optimizing the Response in Prevention: HIV Efficiency in Africa" (ORPHEA) study aims to add to the empirical body of knowledge on the cost and technical efficiency of HIV prevention services that decision makers can use to inform policy and planning. METHODS/DESIGN: ORPHEA is a cross-sectional observational study conducted in 304 service delivery sites in Kenya, Rwanda, South Africa, and Zambia to assess the cost, cost structure, cost variability, and the determinants of efficiency for four HIV interventions: HIV testing and counselling (HTC), prevention of mother-to-child transmission (PMTCT), voluntary medical male circumcision (VMMC), and HIV prevention for sex workers. ORPHEA collected information at three levels (district, facility, and individual) on inputs to HIV prevention service production and their prices, outputs produced along the cascade of services, facility-level characteristics and contextual factors, district-level factors likely to influence the performance of facilities as well as the demand for HIV prevention services, and information on process quality for HTC, PMTCT, and VMMC services. DISCUSSION: ORPHEA is one of the most comprehensive studies on the cost and technical efficiency of HIV prevention interventions to date. The study applied a robust methodological design to collect comparable information to estimate the cost of HTC, PMTCT, VMMC, and sex worker prevention services in Kenya, Rwanda, South Africa, and Zambia, the level of efficiency in the current delivery of these services, and the key determinants of efficiency. The results of the study will be important to decision makers in the study countries as well as those in countries facing similar circumstances and contexts.