Managing extreme AI risks amid rapid progress.

Preparation requires technical research and development, as well as adaptive, proactive governance.

Regulating advanced artificial agents.

Governance frameworks should address the prospect of AI systems that cannot be safely tested.

Judging facts, judging norms: Training machine learning models to judge humans requires a modified approach to labeling data.

As governments and industry turn to increased use of automated decision systems, it becomes essential to consider how closely such systems can reproduce human judgment. We identify a core potential failure, finding that annotators label objects differently depending on whether they are being asked a factual question or a normative question. This challenges a natural assumption maintained in many standard machine-learning (ML) data acquisition procedures: that there is no difference between predicting the factual classification of an object and an exercise of judgment about whether an object violates a rule premised on those facts. We find that using factual labels to train models intended for normative judgments introduces a notable measurement error. We show that models trained using factual labels yield significantly different judgments than those trained using normative labels and that the impact of this effect on model performance can exceed that of other factors (e.g., dataset size) that routinely attract attention from ML researchers and practitioners.

Spurious normativity enhances learning of compliance and enforcement behavior in artificial agents.

How do societies learn and maintain social norms? Here we use multiagent reinforcement learning to investigate the learning dynamics of enforcement and compliance behaviors. Artificial agents populate a foraging environment and need to learn to avoid a poisonous berry. Agents learn to avoid eating poisonous berries better when doing so is taboo, meaning the behavior is punished by other agents. The taboo helps overcome a credit assignment problem in discovering delayed health effects. Critically, introducing an additional taboo, which results in punishment for eating a harmless berry, further improves overall returns. This "silly rule" counterintuitively has a positive effect because it gives agents more practice in learning rule enforcement. By probing what individual agents have learned, we demonstrate that normative behavior relies on a sequence of learned skills. Learning rule compliance builds upon prior learning of rule enforcement by other agents. Our results highlight the benefit of employing a multiagent reinforcement learning computational model focused on learning to implement complex actions.

Toward a 21st-century health care system: recommendations for health care reform.

The coverage, cost, and quality problems of the U.S. health care system are evident. Sustainable health care reform must go beyond financing expanded access to care to substantially changing the organization and delivery of care. The FRESH-Thinking Project (www.fresh-thinking.org) held a series of workshops during which physicians, health policy experts, health insurance executives, business leaders, hospital administrators, economists, and others who represent diverse perspectives came together. This group agreed that the following 8 recommendations are fundamental to successful reform: 1. Replace the current fee-for-service payment system with a payment system that encourages and rewards innovation in the efficient delivery of quality care. The new payment system should invest in the development of outcome measures to guide payment. 2. Establish a securely funded, independent agency to sponsor and evaluate research on the comparative effectiveness of drugs, devices, and other medical interventions. 3. Simplify and rationalize federal and state laws and regulations to facilitate organizational innovation, support care coordination, and streamline financial and administrative functions. 4. Develop a health information technology infrastructure with national standards of interoperability to promote data exchange. 5. Create a national health database with the participation of all payers, delivery systems, and others who own health care data. Agree on methods to make de-identified information from this database on clinical interventions, patient outcomes, and costs available to researchers. 6. Identify revenue sources, including a cap on the tax exclusion of employer-based health insurance, to subsidize health care coverage with the goal of insuring all Americans. 7. Create state or regional insurance exchanges to pool risk, so that Americans without access to employer-based or other group insurance could obtain a standard benefits package through these exchanges. Employers should also be allowed to participate in these exchanges for their employees' coverage. 8. Create a health coverage board with broad stakeholder representation to determine and periodically update the affordable standard benefit package available through state or regional insurance exchanges.