Faster sorting algorithms discovered using deep reinforcement learning.

Fundamental algorithms such as sorting or hashing are used trillions of times on any given day1. As demand for computation grows, it has become critical for these algorithms to be as performant as possible. Whereas remarkable progress has been achieved in the past2, making further improvements on the efficiency of these routines has proved challenging for both human scientists and computational approaches. Here we show how artificial intelligence can go beyond the current state of the art by discovering hitherto unknown routines. To realize this, we formulated the task of finding a better sorting routine as a single-player game. We then trained a new deep reinforcement learning agent, AlphaDev, to play this game. AlphaDev discovered small sorting algorithms from scratch that outperformed previously known human benchmarks. These algorithms have been integrated into the LLVM standard C++ sort library3. This change to this part of the sort library represents the replacement of a component with an algorithm that has been automatically discovered using reinforcement learning. We also present results in extra domains, showcasing the generality of the approach.

Effects of subtotal pancreatectomy and long-term glucose and lipid overload on insulin secretion and glucose homeostasis in minipigs.

INTRODUCTION: Nowadays, there are no strong diabetic pig models, yet they are required for various types of diabetes research. Using cutting-edge techniques, we attempted to develop a type 2 diabetic minipig model in this study by combining a partial pancreatectomy (Px) with an energetic overload administered either orally or parenterally. METHODS: Different groups of minipigs, including Göttingen-like (GL, n = 17) and Ossabaw (O, n = 4), were developed. Prior to and following each intervention, metabolic assessments were conducted. First, the metabolic responses of the Göttingen-like (n = 3) and Ossabaw (n = 4) strains to a 2-month High-Fat, High-Sucrose diet (HFHSD) were compared. Then, other groups of GL minipigs were established: with a single Px (n = 10), a Px combined with a 2-month HFHSD (n = 6), and long-term intraportal glucose and lipid infusions that were either preceded by a Px (n = 4) or not (n = 4). RESULTS: After the 2-month HFHSD, there was no discernible change between the GL and O minipigs. The pancreatectomized group in GL minipigs showed a significantly lower Acute Insulin Response (AIR) (18.3 ± 10.0 IU/mL after Px vs. 34.9 ± 13.7 IU/mL before, p < .0005). In both long-term intraportal infusion groups, an increase in the Insulinogenic (IGI) and Hepatic Insulin Resistance Indexes (HIRI) was found with a decrease in the AIR, especially in the pancreatectomized group (IGI: 4.2 ± 1.9 after vs. 1.5 ± 0.8 before, p < .05; HIRI (×10-5 ): 12.6 ± 7.9 after vs. 3.8 ± 4.3 before, p < .05; AIR: 24.4 ± 13.7 µIU/mL after vs. 43.9 ± 14.5 µIU/mL before, p < .005). Regardless of the group, there was no fasting hyperglycemia. CONCLUSIONS: In this study, we used pancreatectomy followed by long-term intraportal glucose and lipid infusions to develop an original minipig model with metabolic syndrome and early signs of glucose intolerance. We reaffirm the pig's usefulness as a preclinical model for the metabolic syndrome but without the fasting hyperglycemia that characterizes diabetes mellitus.

Oral metformin transiently lowers post-prandial glucose response by reducing the apical expression of sodium-glucose co-transporter 1 in enterocytes.

Metformin (MET) is the most prescribed antidiabetic drug, but its mechanisms of action remain elusive. Recent data point to the gut as MET's primary target. Here, we explored the effect of MET on the gut glucose transport machinery. Using human enterocytes (Caco-2/TC7 cells) in vitro, we showed that MET transiently reduced the apical density of sodium-glucose transporter 1 (SGLT1) and decreased the absorption of glucose, without changes in the mRNA levels of the transporter. Administered 1 h before a glucose challenge in rats (Wistar, GK), C57BL6 mice and mice pigs, oral MET reduced the post-prandial glucose response (PGR). This effect was abrogated in SGLT1-KO mice. MET also reduced the luminal clearance of 2-(18F)-fluoro-2-deoxy-D-glucose after oral administration in rats. In conclusion, oral metformin transiently lowers post-prandial glucose response by reducing the apical expression of SGLT1 in enterocytes, which may contribute to the clinical effects of the drug.

Primary Graft Function and 5 Year Insulin Independence After Pancreas and Islet Transplantation for Type 1 Diabetes: A Retrospective Parallel Cohort Study.

In islet transplantation (ITx), primary graft function (PGF) or beta cell function measured early after last infusion is closely associated with long term clinical outcomes. We investigated the association between PGF and 5 year insulin independence rate in ITx and pancreas transplantation (PTx) recipients. This retrospective multicenter study included type 1 diabetes patients who underwent ITx in Lille and PTx in Nantes from 2000 to 2022. PGF was assessed using the validated Beta2-score and compared to normoglycemic control subjects. Subsequently, the 5 year insulin independence rates, as predicted by a validated PGF-based model, were compared to the actual rates observed in ITx and PTx patients. The study enrolled 39 ITx (23 ITA, 16 IAK), 209 PTx recipients (23 PTA, 14 PAK, 172 SPK), and 56 normoglycemic controls. Mean[SD] PGF was lower after ITx (ITA 22.3[5.2], IAK 24.8[6.4], than after PTx (PTA 38.9[15.3], PAK 36.8[9.0], SPK 38.7[10.5]), and lower than mean beta-cell function measured in normoglycemic control: 36.6[4.3]. The insulin independence rates observed at 5 years after PTA and PAK aligned with PGF predictions, and was higher after SPK. Our results indicate a similar relation between PGF and 5 year insulin independence in ITx and solitary PTx, shedding new light on long-term transplantation outcomes.

Competition-level code generation with AlphaCode.

Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent transformer-based neural network models show impressive code generation abilities yet still perform poorly on more complex tasks requiring problem-solving skills, such as competitive programming problems. Here, we introduce AlphaCode, a system for code generation that achieved an average ranking in the top 54.3% in simulated evaluations on recent programming competitions on the Codeforces platform. AlphaCode solves problems by generating millions of diverse programs using specially trained transformer-based networks and then filtering and clustering those programs to a maximum of just 10 submissions. This result marks the first time an artificial intelligence system has performed competitively in programming competitions.

Discovering faster matrix multiplication algorithms with reinforcement learning.

Improving the efficiency of algorithms for fundamental computations can have a widespread impact, as it can affect the overall speed of a large amount of computations. Matrix multiplication is one such primitive task, occurring in many systems-from neural networks to scientific computing routines. The automatic discovery of algorithms using machine learning offers the prospect of reaching beyond human intuition and outperforming the current best human-designed algorithms. However, automating the algorithm discovery procedure is intricate, as the space of possible algorithms is enormous. Here we report a deep reinforcement learning approach based on AlphaZero1 for discovering efficient and provably correct algorithms for the multiplication of arbitrary matrices. Our agent, AlphaTensor, is trained to play a single-player game where the objective is finding tensor decompositions within a finite factor space. AlphaTensor discovered algorithms that outperform the state-of-the-art complexity for many matrix sizes. Particularly relevant is the case of 4 × 4 matrices in a finite field, where AlphaTensor's algorithm improves on Strassen's two-level algorithm for the first time, to our knowledge, since its discovery 50 years ago2. We further showcase the flexibility of AlphaTensor through different use-cases: algorithms with state-of-the-art complexity for structured matrix multiplication and improved practical efficiency by optimizing matrix multiplication for runtime on specific hardware. Our results highlight AlphaTensor's ability to accelerate the process of algorithmic discovery on a range of problems, and to optimize for different criteria.

The challenge of HLA donor specific antibodies in the management of pancreatic islet transplantation: an illustrative case-series.

Islet transplantation is a unique paradigm in organ transplantation, since multiple donors are required to achieve complete insulin-independence. Preformed or de novo Donor Specific Antibodies (DSA) may target one or several donor islets, which adds complexity to the analysis of their impact. Adult patients with type 1 diabetes transplanted with pancreatic islets between 2005 and 2018 were included in a single-center observational study. Thirty-two recipients with available sera tested by solid-phase assays for anti-HLA antibodies during their whole follow-up were analyzed. Twenty-five recipients were islet-transplantation-alone recipients, and 7 islet-after-kidney recipients. Seven recipients presented with DSA at any time during follow-up (two with preformed DSA only, one with preformed and de novo DSA, 4 with de novo DSA only). Only islet-transplantation-alone recipients presented with de novo DSA. Three clinical trajectories were identified according to: 1/the presence of preformed DSA, 2/early de novo DSA or 3/late de novo DSA. Only late de novo DSA were associated with unfavorable outcomes, depicted by a decrease of the ß-score. Islet transplantation with preformed DSA, even with high MFI values, is associated with favorable outcomes in our experience. On the contrary, de novo DSA, and especially late de novo DSA, may be associated with allograft loss.

Innovative transdermal delivery of insulin using gelatin methacrylate-based microneedle patches in mice and mini-pigs.

Painless and controlled on-demand drug delivery is the ultimate goal for the management of various chronic diseases, including diabetes. To achieve this purpose, microneedle patches are gaining increased attention. While degradable microneedle (MN) arrays are widely employed, the use of non-dissolving MN patches remains a challenge to overcome. In this study, we demonstrate that crosslinking gelatin methacrylate with polyethylene glycol diacrylate (PEGDA) is potent for engineering non-dissolving MN arrays. Incorporation of MoS2 nanosheets as a photothermal component into MN hydrogels results in MNs featuring on-demand release properties. An optimized MoS2-MN array patch formed using a hydrogel solution containing 500 µg mL-1 of MoS2 and photochemically crosslinked for 5 min shows required mechanical behavior under a normal compressive load to penetrate the stratum corneum of mice or pig skin and allows the delivery of macromolecular therapeutics such as insulin upon swelling. Using ex vivo and in vivo models, we show that the MoS2-MN patches can be used for loading and releasing insulin for therapeutic purposes. Indeed, transdermal administration of insulin loaded into MoS2-MN patches reduces blood glucose levels in C57BL/6 mice and mini-pigs comparably to subcutaneously injected insulin. We believe that this on-demand delivery system might alter the current insulin therapies and might be a potential approach for delivery of other proteins.

Photothermal Activatable Mucoadhesive Fiber Mats for On-Demand Delivery of Insulin via Buccal and Corneal Mucosa.

Electrospun fiber mats loaded with therapeutics have gained considerable attention as a versatile tool in the biomedical field. While these bandages are largely based on fast-dissolving polymers to release the cargo, stimuli-responsive fiber mats have the advantages of providing a timely and spatially controlled drug delivery platform, which can be refilled and reused several times. These benefits make electrospun fiber patches original platforms for painless and convenient on-demand hormone release. Because of the high need of more convenient and non-invasive methods for delivering insulin, a hormone that is currently used to treat hundred million people with diabetes worldwide, we have investigated the tremendous potential of reduced graphene oxide modified poly(acrylic acid) based fiber mats as an original platform for buccal and corneal insulin delivery on-demand. The PAA@rGO hydrogel-like fibers rendered water-insoluble by incorporating ß-cyclodextrin, followed by thermal cross-linking, which showed adequate tensile strength along with high adsorption capacity of insulin at pH 7 and good recyclability. The fiber mats maintained good fibrous morphology and high loading efficiency even after five loading-release cycles. The mucoadhesive nature of the fibers allowed their application for insulin delivery via the eye cornea and the buccal mouth lining, as evidenced in ex vivo studies. Insulin loaded PAA@rGO hydrogel-like fibers showed an insulin flux via buccal lining of pigs of 16.6 ± 2.9 µg cm-2 h-1 and 24.3 ± 3.1 µg cm-2 h-1 for porcine cornea. Testing on healthy adult volunteers confirmed the excellent, mucoadhesive nature of the bandage, with three out of six volunteers feeling completely comfortable (note 8.3) while wearing the patches in the buccal cavity.

Long-Term Analgesia following a Single Application of Fentanyl Transdermal Solution in Pigs.

INTRODUCTION: In animal research, obtaining efficient and constant pain control is regulatory but challenging. The gold standard pain management consists of opioid analgesic administration, such as buprenorphine or fentanyl extended-release patches. However, as in all drugs with a short half-life time, repeated buprenorphine administrations are needed, leading to multiple injections that affect the research protocol. On the other hand, fentanyl patch efficacy is discussed in some species. These elements highlight the need of an optimal formulation of analgesic drugs for laboratory animals. In this study, we investigated how Recuvyra®, a fentanyl transdermal solution (FTS), validated in dog perioperative pain management, could provide sustained analgesia after a single topical administration in pigs in a surgical context. METHODS: A total of 11 minipigs were used in this study. As a preliminary experiment, two different doses were tested as a single application on five pigs: two pigs at full dose (2.6 mg/kg) and three pigs at half dose (1.3 mg/kg). Plasma fentanyl dosages were performed during 4 consecutive days, using liquid chromatography with tandem mass spectrometry detection. The efficacy of FTS was then evaluated in a perioperative period. Six minipigs benefited from a surgical intervention comprising a laparotomy. The FTS was blotted on the skin in a single application 20 min before the surgical incision and plasma fentanyl dosages, clinical examination (body weight, food intake, heart rate, and body temperature) and pain assessment were performed for 7 consecutive days. RESULTS: During the preliminary experiment, all fentanyl concentrations reached the minimum effective concentration (MEC) extrapolated in pigs (fentanylemia ≥0.2 ng/mL) throughout the 4 days. The half dose was chosen for the next step of the study. After the surgical intervention, all plasma fentanyl concentrations remained above the MEC up to 7 days post administration. Pig clinical examinations and pain evaluations showed efficient and constant pain control at the half dose, and few adverse events were observed. DISCUSSION AND CONCLUSION: This study confirms the pharmacological and clinical efficacy of FTS at 1.3 mg/kg in pigs throughout at least 7 postoperative days following laparotomy. The clinical analgesic effect of FTS appears more efficient and well-tolerated than the one observed with repeated injections of buprenorphine. This analgesic drug formulation could be universally used in animal research to provide optimal perioperative pain management and long-term analgesia.

Characterization of one anastomosis gastric bypass and impact of biliary and common limbs on bile acid and postprandial glucose metabolism in a minipig model.

The alimentary limb has been proposed to be a key driver of the weight-loss-independent metabolic improvements that occur upon bariatric surgery. However, the one anastomosis gastric bypass (OAGB) procedure, consisting of one long biliary limb and a short common limb, induces similar beneficial metabolic effects compared to Roux-en-Y Gastric Bypass (RYGB) in humans, despite the lack of an alimentary limb. The aim of this study was to assess the role of the length of biliary and common limbs in the weight loss and metabolic effects that occur upon OAGB. OAGB and sham surgery, with or without modifications of the length of either the biliary limb or the common limb, were performed in Gottingen minipigs. Weight loss, metabolic changes, and the effects on plasma and intestinal bile acids (BAs) were assessed 15 days after surgery. OAGB significantly decreased body weight, improved glucose homeostasis, increased postprandial GLP-1 and fasting plasma BAs, and qualitatively changed the intestinal BA species composition. Resection of the biliary limb prevented the body weight loss effects of OAGB and attenuated the postprandial GLP-1 increase. Improvements in glucose homeostasis along with changes in plasma and intestinal BAs occurred after OAGB regardless of the biliary limb length. Resection of only the common limb reproduced the glucose homeostasis effects and the changes in intestinal BAs. Our results suggest that the changes in glucose metabolism and BAs after OAGB are mainly mediated by the length of the common limb, whereas the length of the biliary limb contributes to body weight loss.NEW & NOTEWORTHY Common limb mediates postprandial glucose metabolism change after gastric bypass whereas biliary limb contributes to weight loss.

Mastering Atari, Go, chess and shogi by planning with a learned model.

Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess1 and Go2, where a perfect simulator is available. However, in real-world problems, the dynamics governing the environment are often complex and unknown. Here we present the MuZero algorithm, which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. The MuZero algorithm learns an iterable model that produces predictions relevant to planning: the action-selection policy, the value function and the reward. When evaluated on 57 different Atari games3-the canonical video game environment for testing artificial intelligence techniques, in which model-based planning approaches have historically struggled4-the MuZero algorithm achieved state-of-the-art performance. When evaluated on Go, chess and shogi-canonical environments for high-performance planning-the MuZero algorithm matched, without any knowledge of the game dynamics, the superhuman performance of the AlphaZero algorithm5 that was supplied with the rules of the game.

Ten-Year Outcome of Islet Alone or Islet After Kidney Transplantation in Type 1 Diabetes: A Prospective Parallel-Arm Cohort Study.

OBJECTIVE: The long-term outcome of allogenic islet transplantation is unknown. The aim of this study was to evaluate the 10-year outcome of islet transplantation in patients with type 1 diabetes and hypoglycemia unawareness and/or a functioning kidney graft. RESEARCH DESIGN AND METHODS: We enrolled in this prospective parallel-arm cohort study 28 subjects with type 1 diabetes who received islet transplantation either alone (ITA) or after a kidney graft (IAK). Islet transplantation consisted of two or three intraportal infusions of allogenic islets administered within (median [interquartile range]) 68 days (43-92). Immunosuppression was induced with interleukin-2 receptor antibodies and maintained with sirolimus and tacrolimus. The primary outcome was insulin independence with A1C ≤6.5% (48 mmol/mol). Secondary outcomes were patient and graft survival, severe hypoglycemic events (SHEs), metabolic control, and renal function. RESULTS: The primary outcome was met by (Kaplan-Meier estimates [95% CI]) 39% (22-57) and 28% (13-45) of patients 5 and 10 years after islet transplantation, respectively. Graft function persisted in 82% (62-92) and 78% (57-89) of case subjects after 5 and 10 years, respectively, and was associated with improved glucose control, reduced need for exogenous insulin, and a marked decrease of SHEs. ITA and IAK had similar outcomes. Primary graft function, evaluated 1 month after the last islet infusion, was significantly associated with the duration of graft function and insulin independence. CONCLUSIONS: Islet transplantation with the Edmonton protocol can provide 10-year markedly improved metabolic control without SHEs in three-quarters of patients with type 1 diabetes, kidney transplanted or not.

The GLP1R Agonist Liraglutide Reduces Hyperglucagonemia Induced by the SGLT2 Inhibitor Dapagliflozin via Somatostatin Release.

The newest classes of anti-diabetic agents include sodium-glucose cotransporter 2 (SGLT2) inhibitors and glucagon-like peptide 1 receptor (GLP1R) agonists. The SGLT2 inhibitor dapagliflozin reduces glucotoxicity by glycosuria but elevates glucagon secretion. The GLP1R agonist liraglutide inhibits glucagon; therefore, we hypothesize that the cotreatment of dapagliflozin with liraglutide could reduce hyperglucagonemia and hyperglycemia. Here we use five complementary models: human islet cultures, healthy mice, db/db mice, diet-induced obese (DIO) mice, and somatostatin receptor-2 (SSTR2) KO mice. A single administration of liraglutide and dapagliflozin in combination improves glycemia and reduces dapagliflozin-induced glucagon secretion in diabetic mice. Chronic treatment with liraglutide and dapagliflozin produces a sustainable reduction of glycemia compared with each drug alone. Moreover, liraglutide reduces dapagliflozin-induced glucagon secretion by enhancing somatostatin release, as demonstrated by SSTR2 inhibition in human islets and in mice. Collectively, these data provide mechanistic insights into how intra-islet GLP1R activation is critical for the regulation of glucose homeostasis.

Preclinical Assessment of Leptin Transport into the Cerebrospinal Fluid in Diet-Induced Obese Minipigs.

OBJECTIVE: A minipig model was employed to explore the changes in endogenous leptin transport into the central nervous system and in hypothalamic sensitivity to exogenous leptin when individuals are placed on high-fat diet (HFD) compared with standard diet. METHODS: Serum and cerebrospinal fluid (CSF) leptin concentrations during 10 weeks of HFD versus standard diet and exogenous leptin-induced STAT3 phosphorylation in the hypothalamus of minipigs were assessed, and the hypothalamic leptin-sensitive cells were characterized by immunofluorescence. RESULTS: The efficiency of the passage of endogenous blood-borne leptin into the CSF (measured as the log [CSF:serum leptin ratio]) decreased over time in minipigs fed a HFD (ß = -0.04 ± 0.005 per kilogram of weight gain in HFD; P < 0.0001), while it remained stable in minipigs fed a standard diet. However, the ability of peripherally administered leptin to activate its receptor in hypothalamic neurons was preserved in obese minipigs at 10 weeks of HFD. CONCLUSIONS: Together, these data are consistent with the existence of an early-onset tranport deficiency for endogenous circulating leptin into the brain in individuals developing obesity, preceding the acquisition of hypothalamic leptin resistance. Although additional studies are required to identify the underlying mechanisms, our study paves the way for the development of new preclinical pharmacological models targeting the restoration of the shuttling of peripheral leptin into the central nervous system to manage obesity.

Safety of Islet Autotransplantation After Pancreatectomy for Adenocarcinoma.

BACKGROUND: Total pancreatectomy with intraportal islet autotransplantation (TPIAT) rather than partial pancreatectomy could represent a major shift in the management of patients with resectable pancreatic ductal adenocarcinoma (PDAC) when risks of postoperative pancreatic fistula are well identified. This approach provides a theoretical risk of tumor cell dissemination when islet cells are transplanted into the portal vein. Our objective was to explore the safety of TPIAT in PDAC in a mouse preclinical model of subcutaneous xenotransplantation of human cells isolated from pancreatic specimen during partial pancreatectomy performed for PDAC. METHODS: Patients requiring pancreatectomy for PDAC were prospectively included. Immunocompromised mice were transplanted with pancreatic cells isolated from the nonmalignant part of the surgical specimen (experimental group). Results were compared with pancreatic tumor implants (control group). Pancreatic grafts were explanted at 6 weeks for histological analyses. RESULTS: Nine patients were included, and 31 mice were transplanted. In the experimental group, explants were microscopically devoid of tumor cell, and no metastasis was observed. In the control group, all explants were composed of tumor. CONCLUSIONS: We report in a preclinical model the absence of local and distant spreading of malignant cells after pancreatic islets xenograft isolated from PDAC patients. These data supports the oncological safety of TPIAT as valuable alternative to partial pancreatectomy for PDAC patients with a high risk of postoperative pancreatic fistula.

Single Low Dose of Human Recombinant Antithrombin (ATryn) has no Impact on Endotoxin-Induced Disseminated Intravascular Coagulation: An Experimental Randomized Open Label Controlled Study.

BACKGROUND: Antithrombin (AT) III physiological levels are decreased during septic shock and supplementation therapy could therefore be beneficial. OBJECTIVE: We hypothesized that the use of recombinant human AT could reduce disseminated intravascular coagulation (DIC) occurrence. METHODS: We conducted a randomized open label controlled experimental study. Ten female "Large White" pigs were challenged with i.v. infusion of Escherichia coli endotoxin. Two groups of 5 pigs were randomly assigned to receive either recombinant human AT 100 U/kg over 30 min (ATryn group) or 0.9% saline (control group). AT III levels, coagulation, hemostasis, inflammation parameters, hemodynamics, and microcirculatory parameters were measured over a 5-h period. Immediately after euthanasia, kidneys were withdrawn for histology evaluation. Statistical analysis was performed with nonparametric tests and Dunn's test for multiple comparisons. RESULTS: AT III activity was significantly higher in the ATryn group than in the control group from 60% (213% [203-223] vs. 104% [98-115], P = 0.008, respectively) to 300 min (115% [95-124] vs. 79% [67-93], P = 0.03). Recombinant human AT supplementation had no impact on hemodynamics, microcirculatory parameters, and sequential changes of coagulation parameters (platelet count, fibrinogen level, thrombin-AT complexes, and von Willebrand factor). Interleukin 6 and tumor necrosis factor α values were statistically the same for both groups throughout the study. Percentage of thrombosed glomeruli and percentage of thrombosed capillary in glomerulus were not significantly different between both groups. CONCLUSIONS: In our model of endotoxic shock, a single low dose of recombinant human AT did not prevent DIC occurrence, severity, inflammatory profile, or hemodynamic alterations.

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play.

The game of chess is the longest-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. By contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go by reinforcement learning from self-play. In this paper, we generalize this approach into a single AlphaZero algorithm that can achieve superhuman performance in many challenging games. Starting from random play and given no domain knowledge except the game rules, AlphaZero convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.