Patient choice, entry, and the quality of primary care: Evidence from Swedish reforms.

Policies aiming to spur quality competition among health care providers are ubiquitous, but their impact on quality is ex ante ambiguous, and credible empirical evidence is lacking in many contexts. This study contributes to the sparse literature on competition and primary care quality by examining recent competition enhancing reforms in Sweden. The reforms aimed to stimulate patient choice and entry of private providers across the country but affected markets differently depending on the initial market structure. We exploit the heterogeneous impact of the reforms in a difference-in-differences strategy, contrasting more and less exposed markets over the period 2005-2013. Although the reforms led to substantially more entry of new providers in more exposed markets, the effects on primary care quality were modest: We find small improvements of patients' overall satisfaction with care, but no consistently significant effects on avoidable hospitalisation rates or satisfaction with access to care. We find no evidence of economically meaningful quality reductions on any outcome measure.

Can pay-for-performance to primary care providers stimulate appropriate use of antibiotics?

Antibiotic resistance is a major threat to public health worldwide. As the healthcare sector's use of antibiotics is an important contributor to the development of resistance, it is crucial that physicians only prescribe antibiotics when needed and that they choose narrow-spectrum antibiotics, which act on fewer bacteria types, when possible. Inappropriate use of antibiotics is nonetheless widespread, not least for respiratory tract infections (RTI), a common reason for antibiotics prescriptions. We examine if pay-for-performance (P4P) presents a way to influence primary care physicians' choice of antibiotics. During 2006-2013, 8 Swedish healthcare authorities adopted P4P to make physicians select narrow-spectrum antibiotics more often in the treatment of children with RTI. Exploiting register data on all purchases of RTI antibiotics in a difference-in-differences analysis, we find that P4P significantly increased the share of narrow-spectrum antibiotics. There are no signs that physicians gamed the system by issuing more prescriptions overall.

Does risk-adjusted payment influence primary care providers' decision on where to set up practices?

BACKGROUND: Providing equal access to health care is an important objective in most health care systems. It is especially pertinent in systems like the Swedish primary care market, where private providers are free to establish themselves in any part of the country. To improve equity in access to care, 15 out 21 county councils in Sweden have implemented risk-adjusted capitation based on the Care Need Index, which increases capitation to primary care centers with a large share of patients with unfavorable socioeconomic and demographic characteristics. Our aim is to estimate the effects of using care-need adjusted capitation on the supply of private primary care centers. METHOD: We use a dataset that combines information on all primary care centers in Sweden during 2005-2013, the payment system and other conditions for establishing new primary care centers used in the county councils, and demographic, geographic, and socioeconomic variables for low-level geographic areas. To estimate the effects of care-need adjusted capitation, we use difference-in-differences models, contrasting the development over time between areas with and without risk-adjusted capitation, and with high and low Care Need Index values. RESULTS: Risk-adjusted capitation significantly increases the number of private primary care centers in areas with relatively high Care Need Index values. The adjustment results in a changed distribution of private centers within county councils; the total number of private centers does not increase in county councils using care-need adjusted capitation. The effects are furthermore increasing over the first three years after the implementation of such capitation, and concentrated to the lower and middle range of the group of areas with high index values. CONCLUSIONS: Risk-adjusted capitation based on the Care Need Index increases the supply of private primary care centers in areas with unfavorable socioeconomic and demographic characteristics. More generally, this result indicates that risk-adjusted capitation can significantly affect private providers' establishment decisions.

Protocol: Interventions aimed at preventing out-of-home placement of children: A systematic review.

This is the protocol for a Campbell systematic review. The objectives are as follows. The aim of the present review is to synthesize evidence on the effectiveness of interventions for at-risk families aimed at preventing the out-of-home placement of children or increasing the likelihood that children are reunited with their birth families following temporary care arrangements. The review has two objectives: (1) To assess the effectiveness of interventions for at-risk families with children aged between 0 and 17 years old on measures of out-of-home placement and on secondary outcomes. (2) To identify factors that modify intervention effectiveness (e.g., prior placements, parental risk factors such as substance abuse, mental health issues, age, minority status, child risk factors such as disabilities, age, and gender).

Service learning for improving academic success in students in grade K to 12: A systematic review.

Background: School-based service-learning is a teaching strategy that explicitly links community service to academic instruction. It is distinctive from traditional voluntarism or community service in that it intentionally connects service activities with curriculum concepts and includes structured time for reflection. Service learning, by connecting education to real world issues and allowing students to address problems they identify, may be particularly efficacious as it increases engagement and motivates students, in particular students who might not respond well to more traditional teaching methods. Objectives: The main objective was to answer the following research question: What are the effects of service learning on academic success, neither employed, nor in education or training (NEET) status post compulsory school, personal and social skills, and risk behaviour of students in primary and secondary education (grades kindergarten to 12)? Further, we wanted to investigate study-level summaries of participant characteristics (e.g., gender, age or socioeconomic level) and quality of the service learning programme. Search Methods: We identified relevant studies through electronic searches of bibliographic databases, governmental and grey literature repositories, hand search in specific targeted journals, citation tracking, and Internet search engines. The database searches were carried out in November 2019 and other resources were searched in October 2020. We searched to identify both published and unpublished literature, and reference lists of included studies and relevant reviews were searched. Selection Criteria: The intervention was service learning which can be described as a curriculum-based community service that integrates classroom instruction (such as classroom discussions, presentations, or directed writing) with community service activities. We included children in primary and secondary education (grades kindergarten to 12) in general education. Our primary focus was on measures of academic success and NEET status. A secondary focus was on measures of personal and social skills, and risk behaviour (such as drug and alcohol use, violent behaviour, sexual risk taking). All study designs that used a well-defined control group were eligible for inclusion. Studies that utilised qualitative approaches were not included. Data Collection and Analysis: The total number of potentially relevant studies constituted 13,719 hits. A total of 37 studies met the inclusion criteria. The 37 studies analysed 30 different populations. Only 10 studies (analysing nine different populations) could be used in the data synthesis. Eighteen studies could not be used in the data synthesis as they were judged to have critical risk of bias and, in accordance with the protocol, were excluded from the meta-analysis on the basis that they would be more likely to mislead than inform. Five studies did not provide enough information enabling us to calculate an effects size and standard error, and one study did not provide enough information to assess risk of bias. Finally, two clusters of studies used the same data sets, resulting in an additional three studies we did not use in the data synthesis. Meta-analysis of all outcomes were conducted on each conceptual outcome separately. All analyses were inverse variance weighted using random effects statistical models incorporating both the sampling variance and between study variance components into the study level weights. Random effects weighted mean effect sizes were calculated using 95% confidence intervals. We carried out a sensitivity analysis to examine the impact of correcting for clustered assignment of treatments. Main Results: The 10 studies (analysing nine different populations) used for meta analysis were all from the United States. The timespan in which included studies were carried out was 33 years, from 1980 to 2013; on average the intervention year was 2007. The average number of participants in the analysed service learning interventions was 937, ranging from 18 to 3556 and the average number of controls was 927, ranging from 20 to 3395. At most, the results from three studies could be pooled in any of the meta-analyses. All the meta-analyses showed a weighted average that favoured the intervention group except the pregnancy outcome. None of them was statistically significant except the weighted average of the two studies reporting math test results. The random effects weighted standardised mean difference was 0.09 [95% confidence interval (CI): -0.02 to 0.21] for students' general grade point average; 0.04 (95% CI: -0.08 to 0.16) for reading; 0.21 (95% CI: 0.09 to 0.33) for math; 0.03 (95% CI: -0.10 to 0.16) for days absent from school; 0.13 (95% CI: -0.14 to 0.40) for self-esteem; 0.07 (95% CI: -0.04 to 0.18) for locus of control. The random effects weighted odds ratio was 1.05 (95% CI: 0.63 to 1.74) for pregnancy and 0.96 (95% CI: 0.74 to 1.25) for sexual risk behaviour. In addition, a number of other outcomes were reported in a single study only. There were no appreciable changes in the results as indicated by the sensitivity analysis. We did not find any adverse effects. Authors' Conclusions: In this review, we aimed to find evidence of the effectiveness of service learning on students' academic success, personal and social skills, and risk behaviour. However, the evidence was inconclusive. We found only few randomised controlled trials and the risk of bias in the included non-randomised studies was very high. All available evidence used in the data synthesis was US-based. The majority of studies available for meta-analysis reported on a very limited number of outcomes; in particular few reported results on students' academic success even though the outcome was collected. Further, the majority of studies used in the meta-analyses reported implementation problems. These considerations point to the need for more rigorously conducted studies performed outside the United States, reporting a larger number of outcomes. It would be natural to consider conducting a series of randomised controlled trial with specific allocation to implementation of high-quality service learning as guided by the eight standards: (1) Meaningful service, (2) Link to curriculum, (3) Reflection, (4) Diversity, (5) Youth voice, (6) Community partnerships, (7) Progress monitoring and (8) Sufficient duration and intensity. Specific attention would also have to be paid to stringency in terms of conducting a well-designed randomised trial with low risk of bias and ensuring that the sample sizes are large enough to enable sufficient power.

Adult/child ratio and group size in early childhood education or care to promote the development of children aged 0-5 years: A systematic review.

Background: Worldwide, a large number of infants, toddlers, and preschoolers are enroled in formal non-parental early childhood education or care (ECEC). Theoretically, lower adult/child ratios (fewer children per adult) and smaller group sizes are hypothesised to be associated with positive child outcomes in ECEC. A lower adult/child ratio and a smaller group size may increase both the extent and quality of adult/child interactions during the day. Objectives: The objective of this review is to synthesise data from studies to assess the impact of adult/child ratio and group size in ECEC on measures of process characteristics of quality of care and on child outcomes. Search Methods: Relevant studies were identified through electronic searches of bibliographic databases, governmental and grey literature repositories, Internet search engines, hand search of specific targeted journals, citation tracking and contact to experts. The primary searches were carried out up to September 2020. Additional searches were carried out in February 2022. Selection Criteria: The intervention was changes to adult/child ratio and group size in ECEC with children aged 0-5 years old. All study designs that used a well-defined control group were eligible for inclusion. Data Collection and Analysis: The total number of potential relevant studies constituted 14,060 hits. A total of 31 studies met the inclusion criteria and were critically appraised by the review authors. The 31 studies analysed 26 different populations. Only 12 studies analysing 8 different populations (N = 4300) could be used in the data synthesis. Included studies were published between 1968 and 2019, and the average publication year was 1992. We used random-effects meta-analysis, applying both robust-variance estimation and restricted maximum likelihood procedures to synthesise effect sizes. We conducted separate analyses for process quality measures and language and literacy measures. Main Results: The meta-analysis using measures of process quality as the outcome included 84 effect sizes, 5 studies, and 6256 observations. The weighted average effect size was positive but not statistically significant (effect size [ES] = 0.10, 95% confidence interval [CI] = [-0.07, 0.27]) using robust-variance estimation. The adjusted degrees of freedom were below 4 (df = 1.5), meaning that the results were unreliable. Similarly, the low number of studies made the estimation of heterogeneity statistics difficult. The I 2 and τ 2 estimates were both 0, and the Q-statistic 2.3 (p = 0.69). We found a similar, but statistically significant, weighted average effect size using a restricted maximum likelihood procedure (ES = 0.10, 95% CI = [0.004, 0.20]), and similar low levels of heterogeneity (Q = 0.7, I 2 = 0%, τ 2 = 0). The meta-analysis of language and literacy outcomes is based on three studies exploring different changes to group size and/or adult/child ratio in ECEC. The meta-analysis of language and literacy measures included 12 effect sizes, 3 studies, and 14,625 observations. The weighted average effect size was negative but not statistically significant (ES = -0.04, 95% CI = [-0.61, 0.53]) using the robust variance estimation procedure. The adjusted degrees of freedom were again below 4 (df = 1.9) and the results were unreliable. The heterogeneity statistics indicated substantial heterogeneity (Q = 9.3, I 2 = 78.5%, τ 2 = 0.07). The restricted maximum likelihood procedure yielded similar results (ES = -0.06, 95% CI = [-0.57, 0.46], Q = 6.1, I 2 = 64.3%, τ 2 = 0.03). Authors' Conclusions: The main finding of the present review is that there are surprisingly few quantitative studies exploring the effects of changes to adult/child ratio and group size in ECEC on measures of process quality and on child outcomes. The overall quality of the included studies was low, and only two randomised studies were used in the meta-analysis. The risk of bias in the majority of included studies was high, also in studies used in the meta-analysis. Due to the limited number of studies that could be used in the data synthesis, we were unable to explore the effects of adult/child ratio and group size separately. No study that examined the effects of changes of the adult/child-ratio and/or group size on socio-emotional child outcomes could be included in the meta-analysis. No high quality study examined the effects of large changes in adult/child ratio and group size on measures of process quality, or explored effects for children younger than 2 years. We included few studies (3) in the meta-analysis that investigated measures of language and literacy and results for these outcomes were inconclusive. In one specification, we found a small statistically significant effect on process quality, suggesting that fewer children per adult and smaller group sizes do increase the process quality in ECEC. Caution regarding the interpretation must be exerted due to the heterogeneity of the study designs, the limited number of studies, and the generally high risk of bias within the included studies. Results of the present review have implications for both research and practice. First, findings from the present review tentatively support the theoretical hypothesis that lower adult/child ratios (fewer children per adult) and smaller group sizes beneficially influence process quality in ECEC. This hypothesis is reflected in the existence of standards and regulation on the minimum requirements regarding adult/child ratios and maximum group size in ECEC. However, the research literature to date provides little guidance on what the appropriate adult/child ratios and group sizes are. Second, findings from the present review may be seen as a testimony to the urgent need for more contemporary high-quality research exploring the effects of changes in adult/child ratio and group size in ECEC on measures of process quality and child developmental and socio-emotional outcomes.

PROTOCOL: Testing frequency and student achievement: A systematic review.

This is the protocol for a Campbell review. Our primary research question is: What are the effects of different testing frequencies on student achievement? Our secondary research question is: What are the effects of different testing frequencies on measures of students' testing anxiety? Our third research question is: How are the effects of different testing frequencies on student achievement and testing anxiety moderated by subject, grade, type of test, duration of the intervention, and gender?

PROTOCOL: School-based language, math, and reading interventions for executive functions in children and adolescents: A systematic review.

This is the protocol for a Campbell systematic review. Our primary objective for this systematic review is to examine if preschool and school-based interventions aimed at improving language, literacy, and/or mathematical skills increase children's and adolescents' executive functions. As a secondary objective, we will examine how the effects of language, literacy, and mathematics interventions on executive functions are moderated by the subject of the intervention, child age or grade, the type of EF measured, and the at-risk status of participants. We will also explore how the effects are moderated by other study characteristics, and estimate the effects of the included interventions on language, literacy, and mathematical skills.

Targeted school-based interventions for improving reading and mathematics for students with or at risk of academic difficulties in Grades K-6: A systematic review.

Background: Low levels of numeracy and literacy skills are associated with a range of negative outcomes later in life, such as reduced earnings and health. Obtaining information about effective interventions for children with or at risk of academic difficulties is therefore important. Objectives: The main objective was to assess the effectiveness of interventions targeting students with or at risk of academic difficulties in kindergarten to Grade 6. Search Methods: We searched electronic databases from 1980 to July 2018. We searched multiple international electronic databases (in total 15), seven national repositories, and performed a search of the grey literature using governmental sites, academic clearinghouses and repositories for reports and working papers, and trial registries (10 sources). We hand searched recent volumes of six journals and contacted international experts. Lastly, we used included studies and 23 previously published reviews for citation tracking. Selection Criteria: Studies had to meet the following criteria to be included: Population: The population eligible for the review included students attending regular schools in kindergarten to Grade 6, who were having academic difficulties, or were at risk of such difficulties. Intervention: We included interventions that sought to improve academic skills, were conducted in schools during the regular school year, and were targeted (selected or indicated). Comparison: Included studies used an intervention-control group design or a comparison group design. We included randomised controlled trials (RCT); quasi-randomised controlled trials (QRCT); and quasi-experimental studies (QES). Outcomes: Included studies used standardised tests in reading or mathematics. Setting: Studies carried out in regular schools in an OECD country were included. Data Collection and Analysis: Descriptive and numerical characteristics of included studies were coded by members of the review team. A review author independently checked coding. We used an extended version of the Cochrane Risk of Bias tool to assess risk of bias. We used random-effects meta-analysis and robust-variance estimation procedures to synthesise effect sizes. We conducted separate meta-analyses for tests performed within three months of the end of interventions (short-term effects) and longer follow-up periods. For short-term effects, we performed subgroup and moderator analyses focused on instructional methods and content domains. We assessed sensitivity of the results to effect size measurement, outliers, clustered assignment of treatment, risk of bias, missing moderator information, control group progression, and publication bias. Results: We found in total 24,414 potentially relevant records, screened 4247 of them in full text, and included 607 studies that met the inclusion criteria. We included 205 studies of a wide range of intervention types in at least one meta-analysis (202 intervention-control studies and 3 comparison designs). The reasons for excluding studies from the analysis were that they had too high risk of bias (257), compared two alternative interventions (104 studies), lacked necessary information (24 studies), or used overlapping samples (17 studies). The total number of student observations in the analysed studies was 226,745. There were 93% RCTs among the 327 interventions we included in the meta-analysis of intervention-control contrasts and 86% were from the United States. The target group consisted of, on average, 45% girls, 65% minority students, and 69% low-income students. The mean Grade was 2.4. Most studies included in the meta-analysis had a moderate to high risk of bias.The overall average effect sizes (ES) for short-term and follow-up outcomes were positive and statistically significant (ES = 0.30, 95% confidence interval [CI] = [0.25, 0.34] and ES = 0.27, 95% CI = [0.17, 0.36]), respectively). The effect sizes correspond to around one third to one half of the achievement gap between fourth Grade students with high and low socioeconomic status in the United States and to a 58% chance that a randomly selected score of an intervention group student is greater than the score of a randomly selected control group student.All measures indicated substantial heterogeneity across short-term effect sizes. Follow-up outcomes pertain almost exclusively to studies examining small-group instruction by adults and effects on reading measures. The follow-up effect sizes were considerably less heterogeneous than the short-term effect sizes, although there was still statistically significant heterogeneity.Two instructional methods, peer-assisted instruction and small-group instruction by adults, had large and statistically significant average effect sizes that were robust across specifications in the subgroup analysis of short-term effects (ES around 0.35-0.45). In meta-regressions that adjusted for methods, content domains, and other study characteristics, they had significantly larger effect sizes than computer-assisted instruction, coaching of personnel, incentives, and progress monitoring. Peer-assisted instruction also had significantly larger effect sizes than medium-group instruction. Besides peer-assisted instruction and small-group instruction, no other methods were consistently significant across the analyses that tried to isolate the association between a specific method and effect sizes. However, most analyses showed statistically significant heterogeneity also within categories of instructional methods.We found little evidence that effect sizes were larger in some content domains than others. Fractions had significantly higher associations with effect sizes than all other math domains, but there were only six studies of interventions targeting fractions. We found no evidence of adverse effects in the sense that no method or domain had robustly negative associations with effect sizes.The meta-regressions revealed few other significant moderators. Interventions in higher Grades tend to have somewhat lower effect sizes, whereas there were no significant differences between QES and RCTs, general tests and tests of subdomains, and math tests and reading tests. Authors' Conclusions: Our results indicate that interventions targeting students with or at risk of academic difficulties from kindergarten to Grade 6 have on average positive and statistically significant short-term and follow-up effects on standardised tests in reading and mathematics. Peer-assisted instruction and small-group instruction are likely to be effective components of such interventions.We believe the relatively large effect sizes together with the substantial unexplained heterogeneity imply that schools can reduce the achievement gap between students with or at risk of academic difficulties and not-at-risk students by implementing targeted interventions, and that more research into the design of effective interventions is needed.

PROTOCOL: Adult/child ratio and group size in early childhood education or care to promote the development of children aged 0-5 years: A systematic review.

This is the protocol for a Campbell review. The objectives are as follows: To synthesize data from studies to assess the impact of adult/child ratio and group size in ECEC on measures of process characteristics of quality of care and on child outcome measures.