Application of self-supervised approaches to the classification of X-ray diffraction spectra during phase transitions.

Spectroscopy and X-ray diffraction techniques encode ample information on investigated samples. The ability of rapidly and accurately extracting these enhances the means to steer the experiment, as well as the understanding of the underlying processes governing the experiment. It improves the efficiency of the experiment, and maximizes the scientific outcome. To address this, we introduce and validate three frameworks based on self-supervised learning which are capable of classifying 1D spectral curves using data transformations preserving the scientific content and only a small amount of data labeled by domain experts. In particular, in this work we focus on the identification of phase transitions in samples investigated by x-ray powder diffraction. We demonstrate that the three frameworks, based either on relational reasoning, contrastive learning, or a combination of the two, are capable of accurately identifying phase transitions. Furthermore, we discuss in detail the selection of data augmentation techniques, crucial to ensure that scientifically meaningful information is retained.

Megahertz pulse trains enable multi-hit serial femtosecond crystallography experiments at X-ray free electron lasers.

The European X-ray Free Electron Laser (XFEL) and Linac Coherent Light Source (LCLS) II are extremely intense sources of X-rays capable of generating Serial Femtosecond Crystallography (SFX) data at megahertz (MHz) repetition rates. Previous work has shown that it is possible to use consecutive X-ray pulses to collect diffraction patterns from individual crystals. Here, we exploit the MHz pulse structure of the European XFEL to obtain two complete datasets from the same lysozyme crystal, first hit and the second hit, before it exits the beam. The two datasets, separated by <1 µs, yield up to 2.1 Å resolution structures. Comparisons between the two structures reveal no indications of radiation damage or significant changes within the active site, consistent with the calculated dose estimates. This demonstrates MHz SFX can be used as a tool for tracking sub-microsecond structural changes in individual single crystals, a technique we refer to as multi-hit SFX.

Gold Standard for macromolecular crystallography diffraction data.

Macromolecular crystallography (MX) is the dominant means of determining the three-dimensional structures of biological macromolecules. Over the last few decades, most MX data have been collected at synchrotron beamlines using a large number of different detectors produced by various manufacturers and taking advantage of various protocols and goniometries. These data came in their own formats: sometimes proprietary, sometimes open. The associated metadata rarely reached the degree of completeness required for data management according to Findability, Accessibility, Interoperability and Reusability (FAIR) principles. Efforts to reuse old data by other investigators or even by the original investigators some time later were often frustrated. In the culmination of an effort dating back more than two decades, a large portion of the research community concerned with high data-rate macromolecular crystallography (HDRMX) has now agreed to an updated specification of data and metadata for diffraction images produced at synchrotron light sources and X-ray free-electron lasers (XFELs). This 'Gold Standard' will facilitate the processing of data sets independent of the facility at which they were collected and enable data archiving according to FAIR principles, with a particular focus on interoperability and reusability. This agreed standard builds on the NeXus/HDF5 NXmx application definition and the International Union of Crystallo-graphy (IUCr) imgCIF/CBF dictionary, and it is compatible with major data-processing programs and pipelines. Just as with the IUCr CBF/imgCIF standard from which it arose and to which it is tied, the NeXus/HDF5 NXmx Gold Standard application definition is intended to be applicable to all detectors used for crystallography, and all hardware and software developers in the field are encouraged to adopt and contribute to the standard.

Author Correction: Membrane protein megahertz crystallography at the European XFEL.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Visualization of protein crystals by high-energy phase-contrast X-ray imaging.

For the extraction of the best possible X-ray diffraction data from macromolecular crystals, accurate positioning of the crystals with respect to the X-ray beam is crucial. In addition, information about the shape and internal defects of crystals allows the optimization of data-collection strategies. Here, it is demonstrated that the X-ray beam available on the macromolecular crystallography beamline P14 at the high-brilliance synchrotron-radiation source PETRA III at DESY, Hamburg, Germany can be used for high-energy phase-contrast microtomography of protein crystals mounted in an optically opaque lipidic cubic phase matrix. Three-dimensional tomograms have been obtained at X-ray doses that are substantially smaller and on time scales that are substantially shorter than those used for diffraction-scanning approaches that display protein crystals at micrometre resolution. Adding a compound refractive lens as an objective to the imaging setup, two-dimensional imaging at sub-micrometre resolution has been achieved. All experiments were performed on a standard macromolecular crystallography beamline and are compatible with standard diffraction data-collection workflows and apparatus. Phase-contrast X-ray imaging of macromolecular crystals could find wide application at existing and upcoming low-emittance synchrotron-radiation sources.

Membrane protein megahertz crystallography at the European XFEL.

The world's first superconducting megahertz repetition rate hard X-ray free-electron laser (XFEL), the European XFEL, began operation in 2017, featuring a unique pulse train structure with 886 ns between pulses. With its rapid pulse rate, the European XFEL may alleviate some of the increasing demand for XFEL beamtime, particularly for membrane protein serial femtosecond crystallography (SFX), leveraging orders-of-magnitude faster data collection. Here, we report the first membrane protein megahertz SFX experiment, where we determined a 2.9 Å-resolution SFX structure of the large membrane protein complex, Photosystem I, a > 1 MDa complex containing 36 protein subunits and 381 cofactors. We address challenges to megahertz SFX for membrane protein complexes, including growth of large quantities of crystals and the large molecular and unit cell size that influence data collection and analysis. The results imply that megahertz crystallography could have an important impact on structure determination of large protein complexes with XFELs.

Scientific instrument Femtosecond X-ray Experiments (FXE): instrumentation and baseline experimental capabilities.

The European X-ray Free-Electron Laser (EuXFEL) delivers extremely intense (>1012 photons pulse-1 and up to 27000 pulses s-1), ultrashort (<100 fs) and transversely coherent X-ray radiation, at a repetition rate of up to 4.5 MHz. Its unique X-ray beam parameters enable novel and groundbreaking experiments in ultrafast photochemistry and material sciences at the Femtosecond X-ray Experiments (FXE) scientific instrument. This paper provides an overview of the currently implemented experimental baseline instrumentation and its performance during the commissioning phase, and a preview of planned improvements. FXE's versatile instrumentation combines the simultaneous application of forward X-ray scattering and X-ray spectroscopy techniques with femtosecond time resolution. These methods will eventually permit exploitation of wide-angle X-ray scattering studies and X-ray emission spectroscopy, along with X-ray absorption spectroscopy, including resonant inelastic X-ray scattering and X-ray Raman scattering. A suite of ultrafast optical lasers throughout the UV-visible and near-IR ranges (extending up to mid-IR in the near future) with pulse length down to 15 fs, synchronized to the X-ray source, serve to initiate dynamic changes in the sample. Time-delayed hard X-ray pulses in the 5-20 keV range are used to probe the ensuing dynamic processes using the suite of X-ray probe tools. FXE is equipped with a primary monochromator, a primary and secondary single-shot spectrometer, and a timing tool to correct the residual timing jitter between laser and X-ray pulses.

The Karabo distributed control system.

The Karabo distributed control system has been developed to address the challenging requirements of the European X-ray Free Electron Laser facility, including complex and custom-made hardware, high data rates and volumes, and close integration of data analysis for distributed processing and rapid feedback. Karabo is a pluggable, distributed application management system forming a supervisory control and data acquisition environment as part of a distributed control system. Karabo provides integrated control of hardware, monitoring, data acquisition and data analysis on distributed hardware, allowing rapid control feedback based on complex algorithms. Services exist for access control, data logging, configuration management and situational awareness through alarm indicators. The flexible framework enables quick response to the changing requirements in control and analysis, and provides an efficient environment for development, and a single interface to make all changes immediately available to operators and experimentalists.

Radiation-damage investigation of a DNA 16-mer.

In macromolecular crystallography, a great deal of effort has been invested in understanding radiation-damage progression. While the sensitivity of protein crystals has been well characterized, crystals of DNA and of DNA-protein complexes have not thus far been studied as thoroughly. Here, a systematic investigation of radiation damage to a crystal of a DNA 16-mer diffracting to 1.8 Šresolution and held at 100 K, up to an absorbed dose of 45 MGy, is reported. The RIDL (Radiation-Induced Density Loss) automated computational tool was used for electron-density analysis. Both the global and specific damage to the DNA crystal as a function of dose were monitored, following careful calibration of the X-ray flux and beam profile. The DNA crystal was found to be fairly radiation insensitive to both global and specific damage, with half of the initial diffraction intensity being lost at an absorbed average diffraction-weighted dose, D1/2, of 19 MGy, compared with 9 MGy for chicken egg-white lysozyme crystals under the same beam conditions but at the higher resolution of 1.4 Å. The coefficient of sensitivity of the DNA crystal was 0.014 Å2 MGy-1, which is similar to that observed for proteins. These results imply that the significantly greater radiation hardness of DNA and RNA compared with protein observed in a DNA-protein complex and an RNA-protein complex could be due to scavenging action by the protein, thereby protecting the DNA and RNA in these studies. In terms of specific damage, the regions of DNA that were found to be sensitive were those associated with some of the bound calcium ions sequestered from the crystallization buffer. In contrast, moieties farther from these sites showed only small changes even at higher doses.

MHz data collection of a microcrystalline mixture of different jack bean proteins.

We provide a detailed description of a serial femtosecond crystallography (SFX) dataset collected at the European X-ray free-electron laser facility (EuXFEL). The EuXFEL is the first high repetition rate XFEL delivering MHz X-ray pulse trains at 10 Hz. The short spacing (<1 µs) between pulses requires fast flowing microjets for sample injection and high frame rate detectors. A data set was recorded of a microcrystalline mixture of at least three different jack bean proteins (urease, concanavalin A, concanavalin B). A one megapixel Adaptive Gain Integrating Pixel Detector (AGIPD) was used which has not only a high frame rate but also a large dynamic range. This dataset is publicly available through the Coherent X-ray Imaging Data Bank (CXIDB) as a resource for algorithm development and for data analysis training for prospective XFEL users.

Megahertz serial crystallography.

The new European X-ray Free-Electron Laser is the first X-ray free-electron laser capable of delivering X-ray pulses with a megahertz inter-pulse spacing, more than four orders of magnitude higher than previously possible. However, to date, it has been unclear whether it would indeed be possible to measure high-quality diffraction data at megahertz pulse repetition rates. Here, we show that high-quality structures can indeed be obtained using currently available operating conditions at the European XFEL. We present two complete data sets, one from the well-known model system lysozyme and the other from a so far unknown complex of a ß-lactamase from K. pneumoniae involved in antibiotic resistance. This result opens up megahertz serial femtosecond crystallography (SFX) as a tool for reliable structure determination, substrate screening and the efficient measurement of the evolution and dynamics of molecular structures using megahertz repetition rate pulses available at this new class of X-ray laser source.

Calibration of rotation axes for multi-axis goniometers in macromolecular crystallography.

The installation of multi-axis goniometers such as the ESRF/EMBL miniKappa goniometer system has allowed the increased use of sample reorientation in macromolecular crystallography. Old and newly appearing data collection methods require precision and accuracy in crystal reorientation. The proper use of such multi-axis systems has necessitated the development of rapid and easy to perform methods for establishing and evaluating device calibration. A new diffraction-based method meeting these criteria has been developed for the calibration of the motors responsible for rotational motion. This method takes advantage of crystal symmetry by comparing the orientations of a sample rotated about a given axis and checking that the magnitude of the real rotation fits the calculated angle between these two orientations. Hence, the accuracy and precision of rotational motion can be assessed. This rotation calibration procedure has been performed on several beamlines at the ESRF and other synchrotrons. Some resulting data are presented here for reference.

Megahertz data collection from protein microcrystals at an X-ray free-electron laser.

X-ray free-electron lasers (XFELs) enable novel experiments because of their high peak brilliance and femtosecond pulse duration. However, non-superconducting XFELs offer repetition rates of only 10-120 Hz, placing significant demands on beam time and sample consumption. We describe serial femtosecond crystallography experiments performed at the European XFEL, the first MHz repetition rate XFEL, delivering 1.128 MHz X-ray pulse trains at 10 Hz. Given the short spacing between pulses, damage caused by shock waves launched by one XFEL pulse on sample probed by subsequent pulses is a concern. To investigate this issue, we collected data from lysozyme microcrystals, exposed to a ~15 µm XFEL beam. Under these conditions, data quality is independent of whether the first or subsequent pulses of the train were used for data collection. We also analyzed a mixture of microcrystals of jack bean proteins, from which the structure of native, magnesium-containing concanavalin A was determined.

Data Analysis WorkbeNch (DAWN).

Synchrotron light source facilities worldwide generate terabytes of data in numerous incompatible data formats from a wide range of experiment types. The Data Analysis WorkbeNch (DAWN) was developed to address the challenge of providing a single visualization and analysis platform for data from any synchrotron experiment (including single-crystal and powder diffraction, tomography and spectroscopy), whilst also being sufficiently extensible for new specific use case analysis environments to be incorporated (e.g. ARPES, PEEM). In this work, the history and current state of DAWN are presented, with two case studies to demonstrate specific functionality. The first is an example of a data processing and reduction problem using the generic tools, whilst the second shows how these tools can be targeted to a specific scientific area.

Predicting the X-ray lifetime of protein crystals.

Radiation damage is a major cause of failure in macromolecular crystallography experiments. Although it is always best to evenly illuminate the entire volume of a homogeneously diffracting crystal, limitations of the available equipment and imperfections in the sample often require a more sophisticated targeting strategy, involving microbeams smaller than the crystal, and translations of the crystal during data collection. This leads to a highly inhomogeneous distribution of absorbed X-rays (i.e., dose). Under these common experimental conditions, the relationship between dose and time is nonlinear, making it difficult to design an experimental strategy that optimizes the radiation damage lifetime of the crystal, or to assign appropriate dose values to an experiment. We present, and experimentally validate, a predictive metric diffraction-weighted dose for modeling the rate of decay of total diffracted intensity from protein crystals in macromolecular crystallography, and hence we can now assign appropriate "dose" values to modern experimental setups. Further, by taking the ratio of total elastic scattering to diffraction-weighted dose, we show that it is possible to directly compare potential data-collection strategies to optimize the diffraction for a given level of damage under specific experimental conditions. As an example of the applicability of this method, we demonstrate that by offsetting the rotation axis from the beam axis by 1.25 times the full-width half maximum of the beam, it is possible to significantly extend the dose lifetime of the crystal, leading to a higher number of diffracted photons, better statistics, and lower overall radiation damage.

The use of a mini-κ goniometer head in macromolecular crystallography diffraction experiments.

Most macromolecular crystallography (MX) diffraction experiments at synchrotrons use a single-axis goniometer. This markedly contrasts with small-molecule crystallography, in which the majority of the diffraction data are collected using multi-axis goniometers. A novel miniaturized κ-goniometer head, the MK3, has been developed to allow macromolecular crystals to be aligned. It is available on the majority of the structural biology beamlines at the ESRF, as well as elsewhere. In addition, the Strategy for the Alignment of Crystals (STAC) software package has been developed to facilitate the use of the MK3 and other similar devices. Use of the MK3 and STAC is streamlined by their incorporation into online analysis tools such as EDNA. The current use of STAC and MK3 on the MX beamlines at the ESRF is discussed. It is shown that the alignment of macromolecular crystals can result in improved diffraction data quality compared with data obtained from randomly aligned crystals.

The use of workflows in the design and implementation of complex experiments in macromolecular crystallography.

The automation of beam delivery, sample handling and data analysis, together with increasing photon flux, diminishing focal spot size and the appearance of fast-readout detectors on synchrotron beamlines, have changed the way that many macromolecular crystallography experiments are planned and executed. Screening for the best diffracting crystal, or even the best diffracting part of a selected crystal, has been enabled by the development of microfocus beams, precise goniometers and fast-readout detectors that all require rapid feedback from the initial processing of images in order to be effective. All of these advances require the coupling of data feedback to the experimental control system and depend on immediate online data-analysis results during the experiment. To facilitate this, a Data Analysis WorkBench (DAWB) for the flexible creation of complex automated protocols has been developed. Here, example workflows designed and implemented using DAWB are presented for enhanced multi-step crystal characterizations, experiments involving crystal reorientation with kappa goniometers, crystal-burning experiments for empirically determining the radiation sensitivity of a crystal system and the application of mesh scans to find the best location of a crystal to obtain the highest diffraction quality. Beamline users interact with the prepared workflows through a specific brick within the beamline-control GUI MXCuBE.

Translation calibration of inverse-kappa goniometers in macromolecular crystallography.

Precise and convenient crystal reorientation is of experimental importance in macromolecular crystallography (MX). The development of multi-axis goniometers, such as the ESRF/EMBL mini-κ, necessitates the corresponding development of calibration procedures that can be used for the setup, maintenance and troubleshooting of such devices. While traditional multi-axis goniometers require all rotation axes to intersect the unique point of the sample position, recently developed miniaturized instruments for sample reorientation in MX are not as restricted. However, the samples must always be re-centred following a change in orientation. To overcome this inconvenience and allow the use of multi-axis goniometers without the fundamental restriction of having all axes intersecting in the same point, an automatic translation correction protocol has been developed for such instruments. It requires precise information about the direction and location of the rotation axes. To measure and supply this information, a general, easy-to-perform translation calibration (TC) procedure has also been developed. The TC procedure is routinely performed on most MX beamlines at the ESRF and some results are presented for reference.

MxCuBE: a synchrotron beamline control environment customized for macromolecular crystallography experiments.

The design and features of a beamline control software system for macromolecular crystallography (MX) experiments developed at the European Synchrotron Radiation Facility (ESRF) are described. This system, MxCuBE, allows users to easily and simply interact with beamline hardware components and provides automated routines for common tasks in the operation of a synchrotron beamline dedicated to experiments in MX. Additional functionality is provided through intuitive interfaces that enable the assessment of the diffraction characteristics of samples, experiment planning, automatic data collection and the on-line collection and analysis of X-ray emission spectra. The software can be run in a tandem client-server mode that allows for remote control and relevant experimental parameters and results are automatically logged in a relational database, ISPyB. MxCuBE is modular, flexible and extensible and is currently deployed on eight macromolecular crystallography beamlines at the ESRF. Additionally, the software is installed at MAX-lab beamline I911-3 and at BESSY beamline BL14.1.