Encouraging voluntary government action via a solar-friendly designation program to promote solar energy in the United States.

SignificanceDue to market and system failures, policies and programs at the local level are needed to accelerate the renewable energy transition. A voluntary environmental program (VEP), such as SolSmart, can encourage local governments to adopt solar-friendly best practices. Unlike previous research, this study uses a national sample, more recent data, and a matched control group for difference-in-differences estimation to quantify the causal impact of a VEP in the public, rather than private, sector. We offer empirical evidence that SolSmart increased installed solar capacity and, with less statistical significance, the number of solar installations. The results inform the design of sustainability-focused VEPs and future research to understand the causal pathways between local governments' voluntary actions and solar market development.

Improving Social Bot Detection Through Aid and Training.

OBJECTIVE: We test the effects of three aids on individuals' ability to detect social bots among Twitter personas: a bot indicator score, a training video, and a warning. BACKGROUND: Detecting social bots can prevent online deception. We use a simulated social media task to evaluate three aids. METHOD: Lay participants judged whether each of 60 Twitter personas was a human or social bot in a simulated online environment, using agreement between three machine learning algorithms to estimate the probability of each persona being a bot. Experiment 1 compared a control group and two intervention groups, one provided a bot indicator score for each tweet; the other provided a warning about social bots. Experiment 2 compared a control group and two intervention groups, one receiving the bot indicator scores and the other a training video, focused on heuristics for identifying social bots. RESULTS: The bot indicator score intervention improved predictive performance and reduced overconfidence in both experiments. The training video was also effective, although somewhat less so. The warning had no effect. Participants rarely reported willingness to share content for a persona that they labeled as a bot, even when they agreed with it. CONCLUSIONS: Informative interventions improved social bot detection; warning alone did not. APPLICATION: We offer an experimental testbed and methodology that can be used to evaluate and refine interventions designed to reduce vulnerability to social bots. We show the value of two interventions that could be applied in many settings.

Show-Me Resilience: Assessing and Reconciling Rural Leaders' Perceptions of Climate Resilience in Missouri.

Rural areas of the United States play a vital role in coping with, adapting to and mitigating climate change, yet they often lag urban areas in climate planning and action. Rural leaders-e.g., policymakers, state/federal agency professionals, non-profit organization leadership, and scholars - are pivotal for driving the programs and policies that support resilient practices, but our understanding of their perspectives on climate resilience writ large is limited. We conducted semi-structured interviews with 23 rural leaders in Missouri to elucidate their conceptualizations of climate resilience and identify catalysts and constraints for climate adaptation planning and action across rural landscapes. We investigated participants' perceptions of the major vulnerabilities of rural communities and landscapes, threats to rural areas, and potential steps for making rural Missouri more resilient in the face of climate change. We found that most rural leaders conceptualized climate resilience as responding to hazardous events rather than anticipating or planning for hazardous trends. The predominant threats identified were flooding and drought, which aligns with climate projections for the Midwest. Participants proposed a wide variety of specific steps to enhance resilience but had the highest agreement about the utility of expanding existing programs. The most comprehensive suite of solutions was offered by participants who conceptualized resilience as involving social, ecological, and economic systems, underscoring the importance of broad thinking for developing more holistic solutions to climate-associated threats and the potential impact of greater collaboration across domains. We highlight and discuss a Missouri-based levee setback project that was identified by participants as a showcase of collaborative resilience-building.

Evaluating the impact of broadband access and internet use in a small underserved rural community.

Having adequate access to the internet at home enhances quality-of-life for households and facilitates economic and social opportunities. Despite increased investment in response to the COVID-19 pandemic, millions of households in the rural United States still lack adequate access to high-speed internet. In this study, we evaluate a wireless broadband network deployed in Turney, a small, underserved rural community in northwest Missouri. In addition to collecting survey data before and after this internet intervention, we collected pre-treatment and post-treatment survey data from comparison communities to serve as a control group. Due to technical constraints, some of Turney's interested participants could not connect to the network, creating an additional comparison group. These comparisons suggest two primary findings, (1) changes in using the internet for employment, education, and health could not be directly attributed to the internet intervention, and (2) the internet intervention was associated with benefits stemming from the ability to use multiple devices at once. This study has implications for the design of future broadband evaluation studies, particularly those examining underserved rather than unserved communities. Recommendations for identifying appropriate outcome variables, executing recruitment strategies, and selecting the timing of surveys are made.

Knowing When to Pass: The Effect of AI Reliability in Risky Decision Contexts.

OBJECTIVE: This study manipulates the presence and reliability of AI recommendations for risky decisions to measure the effect on task performance, behavioral consequences of trust, and deviation from a probability matching collaborative decision-making model. BACKGROUND: Although AI decision support improves performance, people tend to underutilize AI recommendations, particularly when outcomes are uncertain. As AI reliability increases, task performance improves, largely due to higher rates of compliance (following action recommendations) and reliance (following no-action recommendations). METHODS: In a between-subject design, participants were assigned to a high reliability AI, low reliability AI, or a control condition. Participants decided whether to bet that their team would win in a series of basketball games tying compensation to performance. We evaluated task performance (in accuracy and signal detection terms) and the behavioral consequences of trust (via compliance and reliance). RESULTS: AI recommendations improved task performance, had limited impact on risk-taking behavior, and were under-valued by participants. Accuracy, sensitivity (d'), and reliance increased in the high reliability AI condition, but there was no effect on response bias (c) or compliance. Participant behavior was only consistent with a probability matching model for compliance in the low reliability condition. CONCLUSION: In a pay-off structure that incentivized risk-taking, the primary value of the AI recommendations was in determining when to perform no action (i.e., pass on bets). APPLICATION: In risky contexts, designers need to consider whether action or no-action recommendations will be more influential to design appropriate interventions.

Duped by Bots: Why Some are Better than Others at Detecting Fake Social Media Personas.

OBJECTIVE: We examine individuals' ability to detect social bots among Twitter personas, along with participant and persona features associated with that ability. BACKGROUND: Social media users need to distinguish bots from human users. We develop and demonstrate a methodology for assessing those abilities, with a simulated social media task. METHOD: We analyze performance from a signal detection theory perspective, using a task that asked lay participants whether each of 50 Twitter personas was a human or social bot. We used the agreement of two machine learning models to estimate the probability of each persona being a bot. We estimated the probability of participants indicating that a persona was a bot with a generalized linear mixed-effects model using participant characteristics (social media experience, analytical reasoning, and political views) and stimulus characteristics (bot indicator score and political tone) as regressors. RESULTS: On average, participants had modest sensitivity (d') and a criterion that favored responding "human." Exploratory analyses found greater sensitivity for participants (a) with less self-reported social media experience, (b) greater analytical reasoning ability, and (c) who were evaluating personas with opposing political views. Some patterns varied with participants' political identity. CONCLUSIONS: Individuals have limited ability to detect social bots, with greater aversion to mistaking bots for humans than vice versa. Greater social media experience and myside bias appeared to reduce performance, as did less analytical reasoning ability. APPLICATION: These patterns suggest the need for interventions, especially when users feel most familiar with social media.

Setting Priorities in Behavioral Interventions: An Application to Reducing Phishing Risk.

Phishing risk is a growing area of concern for corporations, governments, and individuals. Given the evidence that users vary widely in their vulnerability to phishing attacks, we demonstrate an approach for assessing the benefits and costs of interventions that target the most vulnerable users. Our approach uses Monte Carlo simulation to (1) identify which users were most vulnerable, in signal detection theory terms; (2) assess the proportion of system-level risk attributable to the most vulnerable users; (3) estimate the monetary benefit and cost of behavioral interventions targeting different vulnerability levels; and (4) evaluate the sensitivity of these results to whether the attacks involve random or spear phishing. Using parameter estimates from previous research, we find that the most vulnerable users were less cautious and less able to distinguish between phishing and legitimate emails (positive response bias and low sensitivity, in signal detection theory terms). They also accounted for a large share of phishing risk for both random and spear phishing attacks. Under these conditions, our analysis estimates much greater net benefit for behavioral interventions that target these vulnerable users. Within the range of the model's assumptions, there was generally net benefit even for the least vulnerable users. However, the differences in the return on investment for interventions with users with different degrees of vulnerability indicate the importance of measuring that performance, and letting it guide interventions. This study suggests that interventions to reduce response bias, rather than to increase sensitivity, have greater net benefit.

Quantifying Phishing Susceptibility for Detection and Behavior Decisions.

OBJECTIVE: We use signal detection theory to measure vulnerability to phishing attacks, including variation in performance across task conditions. BACKGROUND: Phishing attacks are difficult to prevent with technology alone, as long as technology is operated by people. Those responsible for managing security risks must understand user decision making in order to create and evaluate potential solutions. METHOD: Using a scenario-based online task, we performed two experiments comparing performance on two tasks: detection, deciding whether an e-mail is phishing, and behavior, deciding what to do with an e-mail. In Experiment 1, we manipulated the order of the tasks and notification of the phishing base rate. In Experiment 2, we varied which task participants performed. RESULTS: In both experiments, despite exhibiting cautious behavior, participants' limited detection ability left them vulnerable to phishing attacks. Greater sensitivity was positively correlated with confidence. Greater willingness to treat e-mails as legitimate was negatively correlated with perceived consequences from their actions and positively correlated with confidence. These patterns were robust across experimental conditions. CONCLUSION: Phishing-related decisions are sensitive to individuals' detection ability, response bias, confidence, and perception of consequences. Performance differs when people evaluate messages or respond to them but not when their task varies in other ways. APPLICATION: Based on these results, potential interventions include providing users with feedback on their abilities and information about the consequences of phishing, perhaps targeting those with the worst performance. Signal detection methods offer system operators quantitative assessments of the impacts of interventions and their residual vulnerability.

Analysis of rural broadband adoption dynamics: A theory-driven agent-based model.

Demand for broadband internet has far outpaced its availability. In addition, the "new normal" imposed by the COVID-19 pandemic has further disadvantaged unserved and underserved areas. To address this challenge, federal and state agencies are funding internet service providers (ISPs) to deploy broadband infrastructure in these areas. To support goals to provide broadband service to as many people as possible as quickly as possible, policymakers and ISPs may benefit from better tools to predict take rates and formulate effective strategies to increase the adoption of high-speed internet. However, there is typically insufficient data available to understand consumer attitudes. We propose using an agent-based model grounded in the Theory of Planned Behavior, a behavioral theory that explains the consumer's decision-making process. The model simulates residential broadband adoption by capturing the effect of market competition, broadband service attributes, and consumer characteristics. We demonstrate the effectiveness of this type of tool via a use case in Missouri to show how simulation results can inform predictions of broadband adoption. In the model, broadband take rates increase as the presence of existing internet users in the area increases and price decreases. With further development, this type of simulation can guide decision-making for infrastructure and digital literacy investment based on demand as well as support the design of market subsidies that aim to reduce the digital divide.

Measure Twice, Change Once: Using Simulation to Support Change Management in Rural Healthcare Delivery.

Barriers to adequate healthcare in rural areas remain a grand challenge for local healthcare systems. In addition to patients' travel burdens, lack of health insurance, and lower health literacy, rural healthcare systems also experience significant resource shortages, as well as issues with recruitment and retention of healthcare providers, particularly specialists. These factors combined result in complex change management-focused challenges for rural healthcare systems. Change management initiatives are often resource intensive, and in rural health organizations already strapped for resources, it may be particularly risky to embark on change initiatives. One way to address these change management concerns is by leveraging socio-technical simulation models to estimate techno-economic feasibility (e.g., is it technologically feasible, and is it economical?) as well as socio-utility feasibility (e.g., how will the changes be utilized?). We present a framework for how healthcare systems can integrate modeling and simulation techniques from systems engineering into a change management process. Modeling and simulation are particularly useful for investigating the amount of uncertainty about potential outcomes, guiding decision-making that considers different scenarios, and validating theories to determine if they accurately reflect real-life processes. The results of these simulations can be integrated into critical change management recommendations related to developing readiness for change and addressing resistance to change. As part of our integration, we present a case study showcasing how simulation modeling has been used to determine feasibility and potential resistance to change considerations for implementing a mobile radiation oncology unit. Recommendations and implications are discussed.

Designing explainable AI to improve human-AI team performance: A medical stakeholder-driven scoping review.

The rise of complex AI systems in healthcare and other sectors has led to a growing area of research called Explainable AI (XAI) designed to increase transparency. In this area, quantitative and qualitative studies focus on improving user trust and task performance by providing system- and prediction-level XAI features. We analyze stakeholder engagement events (interviews and workshops) on the use of AI for kidney transplantation. From this we identify themes which we use to frame a scoping literature review on current XAI features. The stakeholder engagement process lasted over nine months covering three stakeholder group's workflows, determining where AI could intervene and assessing a mock XAI decision support system. Based on the stakeholder engagement, we identify four major themes relevant to designing XAI systems - 1) use of AI predictions, 2) information included in AI predictions, 3) personalization of AI predictions for individual differences, and 4) customizing AI predictions for specific cases. Using these themes, our scoping literature review finds that providing AI predictions before, during, or after decision-making could be beneficial depending on the complexity of the stakeholder's task. Additionally, expert stakeholders like surgeons prefer minimal to no XAI features, AI prediction, and uncertainty estimates for easy use cases. However, almost all stakeholders prefer to have optional XAI features to review when needed, especially in hard-to-predict cases. The literature also suggests that providing both system- and prediction-level information is necessary to build the user's mental model of the system appropriately. Although XAI features improve users' trust in the system, human-AI team performance is not always enhanced. Overall, stakeholders prefer to have agency over the XAI interface to control the level of information based on their needs and task complexity. We conclude with suggestions for future research, especially on customizing XAI features based on preferences and tasks.

Don't touch that dial: Psychological reactance, transparency, and user acceptance of smart thermostat setting changes.

Automation inherently removes a certain amount of user control. If perceived as a loss of freedom, users may experience psychological reactance, which is a motivational state that can lead a person to engage in behaviors to reassert their freedom. In an online experiment, participants set up and communicated with a hypothetical smart thermostat. Participants read notifications about a change in the thermostat's setting. Phrasing of notifications was altered across three dimensions: strength of authoritative language, deviation of temperature change from preferences, and whether or not the reason for the change was transparent. Authoritative language, temperatures outside the user's preferences, and lack of transparency induced significantly higher levels of reactance. However, when the system presented a temperature change outside of the user's preferences, reactance was mitigated and user acceptance was higher if the thermostat's operations were transparent. Providing justification may be less likely to induce psychological reactance and increase user acceptance. This supports efforts to use behavioral approaches, such as demand response, to increase sustainability and limit the impacts of climate change.

Techno-Economic Feasibility Analysis of a Fully Mobile Radiation Oncology System Using Monte Carlo Simulation.

PURPOSE: Disparities in radiation oncology (RO) can be attributed to geographic location, socioeconomic status, race, sex, and other societal factors. One potential solution is to implement a fully mobile (FM) RO system to bring radiotherapy to rural areas and reduce barriers to access. We use Monte Carlo simulation to quantify techno-economic feasibility with uncertainty, using two rural Missouri scenarios. METHODS: Recently, a semimobile RO system has been developed by building an o-ring linear accelerator (linac) into a mobile coach that is used for temporary care, months at a time. Transitioning to a more FM-RO system, which changes location within a given day, presents technical challenges including logistics and quality assurance. This simulation includes cancer census in both northern and southeastern Missouri, multiple treatment locations within a given day, and associated expenditures and revenues. A subset of patients with lung, breast, and rectal diseases, treated with five fractions, was simulated in the FM-RO system. RESULTS: The FM-RO can perform all necessary quality assurance tests as suggested in national medical physics guidelines within 1.5 hours, thus demonstrating technological feasibility. In northern and southeastern Missouri, five-fraction simulations' net incomes were, in US dollars (USD), $1.55 ± 0.17 million (approximately 74 patients/year) and $3.65 USD ± 0.25 million (approximately 98 patients/year), respectively. The number of patients seen had the highest correlation with net income as well as the ability to break-even within the simulation. The model does not account for disruptions in care or other commonly used treatment paradigms, which may lead to differences in estimated economic return. Overall, the mobile system achieved a net benefit, even for the most negative simulation scenarios. CONCLUSION: Our simulations suggest technologic success and economic viability for a FM-RO system within rural Missouri and present an interesting solution to address other geographic disparities in access to radiotherapy.

Reducing Kidney Discard With Artificial Intelligence Decision Support: the Need for a Transdisciplinary Systems Approach.

PURPOSE OF REVIEW: A transdisciplinary systems approach to the design of an artificial intelligence (AI) decision support system can more effectively address the limitations of AI systems. By incorporating stakeholder input early in the process, the final product is more likely to improve decision-making and effectively reduce kidney discard. RECENT FINDINGS: Kidney discard is a complex problem that will require increased coordination between transplant stakeholders. An AI decision support system has significant potential, but there are challenges associated with overfitting, poor explainability, and inadequate trust. A transdisciplinary approach provides a holistic perspective that incorporates expertise from engineering, social science, and transplant healthcare. A systems approach leverages techniques for visualizing the system architecture to support solution design from multiple perspectives. SUMMARY: Developing a systems-based approach to AI decision support involves engaging in a cycle of documenting the system architecture, identifying pain points, developing prototypes, and validating the system. Early efforts have focused on describing process issues to prioritize tasks that would benefit from AI support.