AI and XAI second opinion: the danger of false confirmation in human-AI collaboration.

Can AI substitute a human physician's second opinion? Recently the Journal of Medical Ethics published two contrasting views: Kempt and Nagel advocate for using artificial intelligence (AI) for a second opinion except when its conclusions significantly diverge from the initial physician's while Jongsma and Sand argue for a second human opinion irrespective of AI's concurrence or dissent. The crux of this debate hinges on the prevalence and impact of 'false confirmation'-a scenario where AI erroneously validates an incorrect human decision. These errors seem exceedingly difficult to detect, reminiscent of heuristics akin to confirmation bias. However, this debate has yet to engage with the emergence of explainable AI (XAI), which elaborates on why the AI tool reaches its diagnosis. To progress this debate, we outline a framework for conceptualising decision-making errors in physician-AI collaborations. We then review emerging evidence on the magnitude of false confirmation errors. Our simulations show that they are likely to be pervasive in clinical practice, decreasing diagnostic accuracy to between 5% and 30%. We conclude with a pragmatic approach to employing AI as a second opinion, emphasising the need for physicians to make clinical decisions before consulting AI; employing nudges to increase awareness of false confirmations and critically engaging with XAI explanations. This approach underscores the necessity for a cautious, evidence-based methodology when integrating AI into clinical decision-making.

"Going into the black box": a policy analysis of how the World Health Organization uses evidence to inform guideline recommendations.

Background: The World Health Organization (WHO) plays a crucial role in producing global guidelines. In response to previous criticism, WHO has made efforts to enhance the process of guideline development, aiming for greater systematicity and transparency. However, it remains unclear whether these changes have effectively addressed these earlier critiques. This paper examines the policy process employed by WHO to inform guideline recommendations, using the update of the WHO Consolidated HIV Testing Services (HTS) Guidelines as a case study. Methods: We observed guideline development meetings and conducted semi-structured interviews with key participants involved in the WHO guideline-making process. The interviews were recorded, transcribed, and analysed thematically. The data were deductively coded and analysed in line with the main themes from a published conceptual framework for context-based evidence-based decision making: introduction, interpretation, and application of evidence. Results: The HTS guideline update was characterized by an inclusive and transparent process, involving a wide range of stakeholders. However, it was noted that not all stakeholders could participate equally due to gaps in training and preparation, particularly regarding the complexity of the Grading Recommendations Assessment Development Evaluation (GRADE) framework. We also found that WHO does not set priorities for which or how many guidelines should be produced each year and does not systematically evaluate the implementation of their recommendations. Our interviews revealed disconnects in the evidence synthesis process, starting from the development of systematic review protocols. While GRADE prioritizes evidence from RCTs, the Guideline Development Group (GDG) heavily emphasized "other" GRADE domains for which little or no evidence was available from the systematic reviews. As a result, expert judgements and opinions played a role in making recommendations. Finally, the role of donors and their presence as observers during GDG meetings was not clearly defined. Conclusion: We found a need for a different approach to evidence synthesis due to the diverse range of global guidelines produced by WHO. Ideally, the evidence synthesis should be broad enough to capture evidence from different types of studies for all domains in the GRADE framework. Greater structure is required in formulating GDGs and clarifying the role of donors through the process.

Long-term impact of parenting-related leave policies on adolescents' well-being: a systematic review of quasi-experiments.

BACKGROUND: Parenting-related leave policies have gained increasing endorsement across Organisation for Economic Co-operation and Development (OECD) countries in recent decades. Previous reviews have focused on the short-term impacts and found predominantly positive effects on children. Although there is a growing interest in the long-term impact during adolescence and young adulthood, a comprehensive assessment of this aspect is currently lacking. METHODS: We systematically reviewed studies from three electronic databases (Scopus, Web of Science and PubMed), which used quasi-experimental design and examined policies legislating the introduction or expansion of parenting-related leave policies in North America or Europe. We looked at studies focused on well-being beyond the age of 12 and analyzed the findings across different domains of well-being: health, education and labour market outcomes. RESULTS: The quasi-experimental evidence is rather limited. The introduction of leave policies or gender-specific quotas produces substantial benefits in the long run. Further, maternal socioeconomic and educational background appears to play a substantial moderating role between leave and adolescents' well-being. Adolescents with mothers who have higher levels of education have demonstrated a more pronounced advantage from the extended time spent together, thereby accentuating pre-existing disparities. CONCLUSIONS: Though the expansion of already long leaves might not generate significant outcomes, the introduction of leave policies or gender-specific quotas produces substantial long-term benefits. This evidence entails considerable policy implications for countries that lack a national leave policy or offer only short durations of paid leave, such as the USA.

How to increase COVID-19 vaccination among a population with persistently suboptimal vaccine uptake? Evidence from the North Macedonia mobile vaccination and public health advice caravan.

Over three years since the first COVID-19 vaccine was approved, many countries still have suboptimal vaccination rates despite holding great amounts of vaccines. Overall, there is little evidence on which policies are more effective to encourage vaccination, particularly in countries where a large share of the population remains unvaccinated. In this study, we examine the effectiveness of a community-based intervention carried out in March 2022 in North Macedonia, a country with a large and persistent share of the population that remains unvaccinated. The intervention, spearheaded by the Ministry of Health and supported by the World Health Organization and UNICEF, consisted of a mobile caravan offering vaccination and public health advice to different locations across the country on different days. Results from our staggered difference-in-difference model show that the mobile vaccination caravan increased daily vaccination rates by 7.7 vaccines per 100,000 inhabitants during the three weeks after the day of the caravan visit. This corresponds to a 35 % increase with respect to pre-intervention vaccination rates. We estimate a cost-effectiveness of 25.4 US dollars (USD) per additional vaccination induced. These results point to mobile caravan vaccines as an effective and cost-effective strategy to increase COVID-19 vaccination rates, even in a context of persistently suboptimal uptake.