SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations.

OpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics-based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this article is to present SofaGym, an open-source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins. The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in an RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics-based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications; we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL, and planning and point out research perspectives using the platform.

Interaction of osmium(ii) redox probes with DNA: insights from theory.

In the course of developing ultrasensitive and quantitative electrochemical point-of-care analytical tools for genetic detection of infectious diseases, osmium(ii) metallointercalators were revealed to be suitable and efficient redox probes to monitor the in vitro DNA amplification [Defever etal, Anal. Chem., 2011, 83, 1815-1821]. In this work, we thus propose a complete computational protocol in order to evaluate the affinity between Os(ii) complexes with double-stranded DNA. This protocol is based on molecular dynamics, with the parametrization of the GAFF force field for the Os(ii) complexes presenting an octahedral environment with polypyridine ligands, and QM/QM' calculations to evaluate the binding energy. For three Os(ii) probes and different binding sites, molecular dynamics simulations and interaction energies calculated at the QM/QM' level are successively discussed and compared to experimental data in order to identify the most stable binding sites. The computational protocol we propose should then be used to design more efficient Os(ii) metallointercalators.

Real-time electrochemical LAMP: a rational comparative study of different DNA intercalating and non-intercalating redox probes.

We present a comparative study of ten redox-active probes for use in real-time electrochemical loop-mediated isothermal amplification (LAMP). Our main objectives were to establish the criteria that need to be fulfilled for minimizing some of the current limitations of the technique and to provide future guidelines in the search for ideal redox reporters. To ensure a reliable comparative study, each redox probe was tested under similar conditions using the same LAMP reaction and the same entirely automatized custom-made real-time electrochemical device (designed for electrochemically monitoring in real-time and in parallel up to 48 LAMP samples). Electrochemical melt curve analyses were recorded immediately at the end of each LAMP reaction. Our results show that there are a number of intercalating and non-intercalating redox compounds suitable for real-time electrochemical LAMP and that the best candidates are those able to intercalate strongly into ds-DNA but not too much to avoid inhibition of the LAMP reaction. The strongest intercalating redox probes were finally shown to provide higher LAMP sensitivity, speed, greater signal amplitude, and cleaner-cut DNA melting curves than the non-intercalating molecules.

Kinetic rotating droplet electrochemistry: a simple and versatile method for reaction progress kinetic analysis in microliter volumes.

Here, we demonstrate a new generic, affordable, simple, versatile, sensitive, and easy-to-implement electrochemical kinetic method for monitoring, in real time, the progress of a chemical or biological reaction in a microdrop of a few tens of microliters, with a kinetic time resolution of ca. 1 s. The methodology is based on a fast injection and mixing of a reactant solution (1-10 µL) in a reaction droplet (15-50 µL) rapidly rotated over the surface of a nonmoving working electrode and on the recording of the ensuing transient faradaic current associated with the transformation of one of the components. Rapid rotation of the droplet was ensured mechanically by a rotating rod brought in contact atop the droplet. This simple setup makes it possible to mix reactants efficiently and rotate the droplet at a high spin rate, hence generating a well-defined hydrodynamic steady-state convection layer at the underlying stationary electrode. The features afforded by this new kinetic method were investigated for three different reaction schemes: (i) the chemical oxidative deprotection of a boronic ester by H2O2, (ii) a biomolecular binding recognition between a small target and an aptamer, and (iii) the inhibition of the redox-mediated catalytic cycle of horseradish peroxidase (HRP) by its substrate H2O2. For the small target/aptamer binding reaction, the kinetic and thermodynamic parameters were recovered from rational analysis of the kinetic plots, whereas for the HRP catalytic/inhibition reaction, the experimental amperometric kinetic plots were reproduced from numerical simulations. From the best fits of simulations to the experimental data, the kinetics rate constants primarily associated with the inactivation/reactivation pathways of the enzyme were retrieved. The ability to perform kinetics in microliter-size samples makes this methodology particularly attractive for reactions involving low-abundance or expensive reagents.

Towards a better understanding of pelvic system disorders using numerical simulation.

Genital prolapse is a pathologic hyper-mobility of the organs that forms the pelvic system. Although this is common condition, the pathophysiology of this disorder is not well known. In order to improve the understanding of its origins, we recreate--virtually--this biomechanical pathology using numerical simulation. The approach builds on a finite element model with parameters measured on several fresh cadavers. The meshes are created from a MRI of a healthy woman and the simulation includes the mechanical interactions between organs (contacts, ligaments, adhesion...). The model is validated through comparison of functional mobilities of the pelvic system observed on a dynamic MRI. We then propose to modify, step by step, the model and its parameters to produce a pathologic situation and have a better understanding of the process. It is not a formal proof but the numerical experiments reinforce the clinical hypothesis on the multifactorial origins of the pathology.

Homogeneous electrochemical monitoring of exonuclease III activity and its application to nucleic acid testing by target recycling.

A simple and fast electrochemical nucleic acid assay based on a target recycling strategy and the release of a double-stranded DNA intercalating redox probe upon digestion of a specific duplex by exonuclease III is demonstrated.

Simple and highly enantioselective electrochemical aptamer-based binding assay for trace detection of chiral compounds.

A new electrochemical methodology is reported for monitoring in homogeneous solution the enantiospecific binding of a small chiral analyte to an aptamer. The principle relies on the difference of diffusion rates between the targeted molecule and the aptamer/target complex, and thus on the ability to more easily electrochemically detect the former over the latter in a homogeneous solution. This electrochemical detection strategy is significant because, in contrast to the common laborious and time-consuming heterogeneous binding approaches, it is based on a simple and fast homogeneous binding assay which does not call for an aptamer conformational change upon ligand binding. The methodology is here exemplified with the specific chiral recognition of trace amounts of l- or d-tyrosinamide by a 49-mer d- or l-deoxyribooligonucleotide receptor. Detection as low as 0.1% of the minor enantiomer in a nonracemic mixture can be achieved in a very short analysis time (<1 min). The assay finally combines numerous attractive features including simplicity, rapidity, low cost, flexibility, low volume samples (few microliters), and homogeneous format.

Real-time electrochemical monitoring of isothermal helicase-dependent amplification of nucleic acids.

We described an electrochemical method to monitor in real-time the isothermal helicase-dependent amplification of nucleic acids. The principle of detection is simple and well-adapted to the development of portable, easy-to-use and inexpensive nucleic acids detection technologies. It consists of monitoring a decrease in the electrochemical current response of a reporter DNA intercalating redox probe during the isothermal DNA amplification. The method offers the possibility to quantitatively analyze target nucleic acids in less than one hour at a single constant temperature, and to perform at the end of the isothermal amplification a DNA melt curve analysis for differentiating between specific and non-specific amplifications. To illustrate the potentialities of this approach for the development of a simple, robust and low-cost instrument with high throughput capability, the method was validated with an electrochemical system capable of monitoring up to 48 real-time isothermal HDA reactions simultaneously in a disposable microplate consisting of 48-electrochemical microwells. Results obtained with this approach are comparable to that obtained with a well-established but more sophisticated and expensive fluorescence-based method. This makes for a promising alternative detection method not only for real-time isothermal helicase-dependent amplification of nucleic acid, but also for other isothermal DNA amplification strategies.

Real-time electrochemical PCR with a DNA intercalating redox probe.

The proof-of-principle of a nonoptical real-time PCR method based on the electrochemical monitoring of a DNA intercalating redox probe that becomes considerably less easily electrochemically detectable once intercalated to the amplified double-stranded DNA is demonstrated. This has been made possible thanks to the finding of a redox intercalator that (i) strongly and specifically binds to the amplified double-stranded DNA, (ii) does not significantly inhibit PCR, (iii) is chemically stable under PCR cycling, and (iv) is sensitively detected by square wave voltammetry during PCR cycling. Among the different DNA intercalating redox probes that we have investigated, namely, methylene blue, Os[(bpy)(2)phen](2+), Os[(bpy)(2)DPPZ](2+), Os[(4,4'-dimethyl-bpy)(2)DPPZ](2+) and Os[(4,4'-diamino-bpy)(2)DPPZ](2+) (with bpy = 2,2'-bipyridine, phen = phenanthroline, and DPPZ = dipyrido[3,2-a:2',3'-c]phenazine), the one and only compound with which it has been possible to demonstrate the proof-of-concept is the Os[(bpy)(2)DPPZ](2+). In terms of analytical performances, the methodology described here compares well with optical-based real-time PCRs, offering finally the same advantages than the popular and routinely used SYBR Green-based real-time fluorescent PCR, but with the additional incomes of being potentially much cheaper and easier to integrate in a hand-held miniaturized device.

Real-time electrochemical monitoring of the polymerase chain reaction by mediated redox catalysis.

We described the proof-of-principle of a nonoptical real-time PCR that uses cyclic voltammetry for indirectly monitoring the amplified DNA product generated in the PCR reaction solution after each PCR cycle. To enable indirect measurement of the amplicon produced throughout PCR, we monitor electrochemically the progressive consumption (i.e., the decrease of concentration) of free electroactive deoxynucleoside triphosphates (dNTPs) used for DNA synthesis. This is accomplished by exploiting the fast catalytic oxidation of native deoxyguanosine triphosphate (dGTP) or its unnatural analogue 7-deaza-dGTP by the one-electron redox catalysts Ru(bpy)(3)(3+) (with bpy = 2,2'-bipyridine) or Os(bpy)(3)(3+) generated at an electrode. To demonstrate the feasibility of the method, a disposable array of eight miniaturized self-contained electrochemical cells (working volume of 50 microL) has been developed and implemented in a classical programmable thermal cycler and then tested with the PCR amplification of two illustrated examples of real-world biological target DNA sequences (i.e., a relatively long 2300-bp sequence from the bacterial genome of multidrug-resistant Achromobacter xylosoxidans and a shorter 283-bp target from the human cytomegalovirus). Although the method works with both mediator/base couples, the catalytic peak current responses recorded with the Ru(bpy)(3)(3+)/dGTP couple under real-time PCR conditions are significantly affected by a continuous current drift and interference with the background solvent discharge, thus leading to poorly reproducible data. Much more reproducible and reliable results are finally obtained with the Os(bpy)(3)(3+)/7-deaza-dGTP, a result that is attributed to the much lower anodic potential at which the catalytic oxidation of 7-deaza-dGTP by Os(bpy)(3)(3+) is detected. Under these conditions, an exponential decrease of the catalytic signal as a function of the number of PCR cycles is obtained, allowing definition of a cycle threshold value (C(t)) that correlates inversely with the initial amount of target DNA. A semilogarithmic plot of C(t) with the initial copy number of target DNA gives a standard linear curve similar to that obtained with fluorescent-based real-time PCR. Although the detection limit (10(3) molecules of target DNA in 50 microL) and sensitivity of the electrochemical method is not as high as conventional optical-based real-time PCR, the methodology described here offers many of the advantages of real-time PCR, such as a high dynamic range (over 8-log(10)) and speed, high amplification efficiency (close to 2), and the elimination of post-PCR processing. The method also has the advantage of being very simple, just requiring the use of low-cost single-use electrodes and the addition of a minute amount of redox catalyst into the PCR mixture. Moreover, compared to the other recently developed electrochemical real-time PCR based on solid-phase amplification, the present approach does not require electrode functionalization by a DNA probe. Finally, on account of the relative insensitivity of electrochemical methods to downscaling, the detection scheme is quite promising for use in miniaturized devices and in the development of point-of-care diagnosis applications.

Theory and practice of enzyme bioaffinity electrodes. Chemical, enzymatic, and electrochemical amplification of in situ product detection.

The two articles in this series are dedicated to bioaffinity electrodes with in situ detection of the product of the enzyme label after recognition by its conjugate immobilized on the electrode. Part 1 was devoted to direct electrochemical detection, whereas the present contribution deals with homogeneous chemical and enzymatic amplification of the primary electrochemical signal. The theoretical relationships that are established for these modes of amplification are applied to the avidin-biotin recognition in a system that involves alkaline phosphatase as enzyme label and 4-amino-2,6-dichloro-phenyl phosphate as substrate, generating 2,6-dichloro-4-aminophenol as electrochemically active product. Chemical amplification then results from the addition of NADH, which reduces the 2,6-dichloro-quinonimine resulting from the electrochemical oxidation of 2,6-dichloro-4-aminophenol. An increased amplification is obtained when the reduction of 2,6-dichloro-quinonimine involves diaphorase in solution with NADH as substrate. The excellent agreement between theoretical predictions and experimental data required a detailed theoretical analysis and the independent determination of the key kinetic parameters of the system. The theoretical analysis was extended to monolayer and multilayered films of auxiliary enzyme as well as to electrochemical amplification by means of closely spaced dual electrodes so as to offer a rational comparative panorama of the amplification capabilities of the various possible strategies. Confinement of the profile of the product, and/or its oxidized form, in the vicinity the electrode surface appears as a key parameter of amplification.

Theory and practice of enzyme bioaffinity electrodes. Direct electrochemical product detection.

The use of enzyme labeling techniques to convert biorecognition events into high sensitivity electrochemical signals may follow two different strategies. One, in which the current is the electrocatalytic response of a redox couple serving as cosubstrate to a redox enzyme label and another that consists in the detection of an electrochemically active product of the enzyme label. The theoretical relationships that link, in the latter case, the electrochemical current response to the amount of recognized labeled target analyte are established for steady-state diffusion-convection chronoamperometric regimes. Two governing parameters thus emerge. One measures the Michaelis-Menten competition in the enzyme kinetics. The other characterizes the competition between the enzymatic kinetics and the diffusion of the substrate. The electrochemical response is finally related to the labeled target analyte concentration in solution through the recognition isotherm. The direct electrochemical product detection thus provides a route to the characteristics of the recognition isotherm, which serves as a calibration curve in analytical applications. The establishment of further theoretical relationships allows one to surmise the increase in sensitivity that may be obtained by using cyclic voltammetry instead of steady-state chronoamperometry in standard electrochemical cells or by accumulation of the enzyme-product in cells of small volume/surface ratios. The theoretical predictions are tested with the example of the avidin-biotin recognition process in a system that involves alkaline phosphatase as enzyme label and 4-amino-2,6-dichlorophenyl phosphate as substrate, generating 4-amino-2,6-dichlorophenol as electrochemically active product. The advantages of the dichloro-substitution are discussed. The theoretical analysis is a requisite for a rational and realistic discussion of the analytical performances of the steady-state chronoamperometric and cyclic voltammetric approaches. These are shown to compare favorably with the best heterogeneous bioaffinity assays so far reported.

Electrode surface confinement of self-assembled enzyme aggregates using magnetic nanoparticles and its application in bioelectrocatalysis.

Self-assembled enzyme aggregates, prepared from magnetic iron oxide nanoparticles, avidin, and a biotinylated redox enzyme, were shown particularly useful for the simple, fast, and efficient construction of highly enzyme-loaded electrodes with the help of a magnet. The approach was illustrated in the case of the bioelectrocatalytic oxidation of NADH by a diaphorase oxidoreductase in the presence of a ferrocene mediator. Two different self-assembling procedures were tested, taking advantage of the spontaneous aggregation of the nanoparticles in the presence of avidin and also of the multivalency binding of biotinylated diaphorase toward avidin. Activities of the bound and unbound diaphorase were systematically controlled allowing determination of the number of active biotinylated diaphorase per nanoparticle incorporated within each magnetic enzyme aggregate. An active enzyme loading capacity of up to 2.35 nmol mg-1 was found for the best nanostructured enzyme assembly, which is 200 times better than for commercialized magnetic micrometer-sized beads coated with streptavidin and saturated with diaphorase. With the help of a permanent magnet, the magnetic enzyme aggregates were finally magnetically collected as a film on the surface of a small screen-printed carbon electrode and the catalytic currents recorded by cyclic voltammetry. From the analysis of the steady-state catalytic current responses and the kinetic rate constants of biotinylated diaphorase, it was possible to determine the enzyme concentration within the magnetic films. Owing to the high enzyme loading in the aggregates of nanoparticles (i.e., 130 microM), the catalytic current responses were definitely higher than the ones measured at an electrode coated with a closed-packed monolayer of diaphorase or at an electrode covered with a film of magnetic micrometer-sized streptavidin beads saturated with diaphorase.

High amplification rates from the association of two enzymes confined within a nanometric layer immobilized on an electrode: modeling and illustrating example.

Electrochemical responses (e.g., chronoamperometric) obtained with an immobilized enzyme that produces an electroactive species may be used to quantitate the amount of enzyme or the concentration of its substrate. It is shown, on theoretical and experimental bases, that product-to-substrate coupling with a second enzyme co-immobilized with the first within one or within a small number of monolayers, allows high amplification rates (higher than 1000), avoids membrane transport limitations, and lends itself to precise kinetic analyses that provide guidelines for optimization of the analytical sensitivity. Very large amplification factors, as large as several thousands, can be reached experimentally, in agreement with appropriately derived theoretical predictions, thus opening the route to the rational design of high-performance substrate sensing or affinity assays applications.

Redox enzymes immobilized on electrodes with solution cosubstrates. General procedure for simulation of time-resolved catalytic responses.

In view of the existing and potential applications of electrochemical enzymatic catalysis with redox enzymes immobilized on the electrode surface in biosensors, a numerical calculation procedure for simulating their cyclic voltammetric responses is presented. It is applicable to systems involving a redox cosubstrate in solution. The cosubstrates, substrates, products, and inhibitors are assumed to diffuse linearly (planar electrode) between the electrode and the solution. The reactions in which the various forms of the immobilized enzyme participate may be as numerous and intricate as required by the simulation with no other restriction than the computing time. They may, at will, follow or not follow Michaelis-Menten kinetics. Slow charge-transfer cosubstrates are treated in the framework of Butler-Volmer kinetic law.

Electrochemistry of immobilized redox enzymes: kinetic characteristics of NADH oxidation catalysis at diaphorase monolayers affinity immobilized on electrodes.

In the class of NADH:acceptor oxidoreductases, the diaphorase from Bacillus stearothermophilusis a particularly promising enzyme for sensing NADH, and indirectly a great number of analytes, when coupled with a NAD-dependent dehydrogenase as well as for the design of mono- and multienzyme affinity sensors. The design and rational optimization of such systems require devising immobilization procedures that prevent dramatic losses of the enzymatic activity and a full kinetic characterization of the immobilized enzyme system. Two immobilization procedures are described, which involve recognition of the biotinylated diaphorase by a monolayer of neutravidin adsorbed on the electrode surface either directly or through the intermediacy of a monolayer of biotinylated rabbit immunoglobulin. Thorough kinetic characterization of the two systems is derived from cyclic voltammetric responses. A precise estimate of the enzyme coverages is obtained after comparing the enzyme kinetics of the immobilized and the homogeneous system.

Polymer-cushioned lipid bilayers in porous alumina.

Using small-angle neutron scattering (SANS) and cyclic voltammetry (CV), we show that model biological membranes can be deposited on a polymer cushion confined in highly regular porous alumina. The thicknesses of the dilute polymer cushion chemically bound to the alumina and of the supported bilayer are obtained for two polyethylene glycol cushions (PEG(5000) and PEG(20000)) and for a cushion made of chains bearing a lipid anchor at their free end (DSPE-PEG(3400)). The bilayers are studied well below and well above the chain melting temperature of the lipid mixture (DMPC/DMPE: 80/20), using a coenzyme (Ubiquinone, UQ(10)) as a redox probe for the voltammetry experiments. Analysis of the SANS form factor of the bilayers shows that the bilayer thickness can be extracted in this particular geometry. Using PEG chains grafted at a low surface density (D < 2R(g)), the thickness of the complete molecular construction is obtained by CV, which shows (after subtracting the bilayer thickness) that the polymer cushion thickness can be varied from 50 to 150 Angstroms. The values obtained with three different chain lengths, are in perfect agreement with the radius derived from the Flory theory.

An intestinal surgery simulator: real-time collision processing and visualization.

This research work is aimed toward the development of a VR-based trainer for colon cancer removal. It enables the surgeons to interactively view and manipulate the concerned virtual organs as during a real surgery. First, we present a method for animating the small intestine and the mesentery (the tissue that connects it to the main vessels) in real-time, thus enabling user interaction through virtual surgical tools during the simulation. We present a stochastic approach for fast collision detection in highly deformable, self-colliding objects. A simple and efficient response to collisions is also introduced in order to reduce the overall animation complexity. Second, we describe a new method based on generalized cylinders for fast rendering of the intestine. An efficient curvature detection method, along with an adaptive sampling algorithm, is presented. This approach, while providing improved tessellation without the classical self-intersection problem, also allows for high-performance rendering thanks to the new 3D skinning feature available in recent GPUs. The rendering algorithm is also designed to ensure a guaranteed frame rate. Finally, we present the quantitative results of the simulations and describe the qualitative feedback obtained from the surgeons.