A Python package based on robust statistical analysis for serial crystallography data processing.

The term robustness in statistics refers to methods that are generally insensitive to deviations from model assumptions. In other words, robust methods are able to preserve their accuracy even when the data do not perfectly fit the statistical models. Robust statistical analyses are particularly effective when analysing mixtures of probability distributions. Therefore, these methods enable the discretization of X-ray serial crystallography data into two probability distributions: a group comprising true data points (for example the background intensities) and another group comprising outliers (for example Bragg peaks or bad pixels on an X-ray detector). These characteristics of robust statistical analysis are beneficial for the ever-increasing volume of serial crystallography (SX) data sets produced at synchrotron and X-ray free-electron laser (XFEL) sources. The key advantage of the use of robust statistics for some applications in SX data analysis is that it requires minimal parameter tuning because of its insensitivity to the input parameters. In this paper, a software package called Robust Gaussian Fitting library (RGFlib) is introduced that is based on the concept of robust statistics. Two methods are presented based on the concept of robust statistics and RGFlib for two SX data-analysis tasks: (i) a robust peak-finding algorithm and (ii) an automated robust method to detect bad pixels on X-ray pixel detectors.

Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography.

X-ray crystallography has witnessed a massive development over the past decade, driven by large increases in the intensity and brightness of X-ray sources and enabled by employing high-frame-rate X-ray detectors. The analysis of large data sets is done via automatic algorithms that are vulnerable to imperfections in the detector and noise inherent with the detection process. By improving the model of the behaviour of the detector, data can be analysed more reliably and data storage costs can be significantly reduced. One major requirement is a software mask that identifies defective pixels in diffraction frames. This paper introduces a methodology and program based upon concepts of machine learning, called robust mask maker (RMM), for the generation of bad-pixel masks for large-area X-ray pixel detectors based on modern robust statistics. It is proposed to discriminate normally behaving pixels from abnormal pixels by analysing routine measurements made with and without X-ray illumination. Analysis software typically uses a Bragg peak finder to detect Bragg peaks and an indexing method to detect crystal lattices among those peaks. Without proper masking of the bad pixels, peak finding methods often confuse the abnormal values of bad pixels in a pattern with true Bragg peaks and flag such patterns as useful regardless, leading to storage of enormous uninformative data sets. Also, it is computationally very expensive for indexing methods to search for crystal lattices among false peaks and the solution may be biased. This paper shows how RMM vastly improves peak finders and prevents them from labelling bad pixels as Bragg peaks, by demonstrating its effectiveness on several serial crystallography data sets.

Megahertz pulse trains enable multi-hit serial femtosecond crystallography experiments at X-ray free electron lasers.

The European X-ray Free Electron Laser (XFEL) and Linac Coherent Light Source (LCLS) II are extremely intense sources of X-rays capable of generating Serial Femtosecond Crystallography (SFX) data at megahertz (MHz) repetition rates. Previous work has shown that it is possible to use consecutive X-ray pulses to collect diffraction patterns from individual crystals. Here, we exploit the MHz pulse structure of the European XFEL to obtain two complete datasets from the same lysozyme crystal, first hit and the second hit, before it exits the beam. The two datasets, separated by <1 µs, yield up to 2.1 Å resolution structures. Comparisons between the two structures reveal no indications of radiation damage or significant changes within the active site, consistent with the calculated dose estimates. This demonstrates MHz SFX can be used as a tool for tracking sub-microsecond structural changes in individual single crystals, a technique we refer to as multi-hit SFX.

Observations of phase changes in monoolein during high viscous injection.

Serial crystallography of membrane proteins often employs high-viscosity injectors (HVIs) to deliver micrometre-sized crystals to the X-ray beam. Typically, the carrier medium is a lipidic cubic phase (LCP) media, which can also be used to nucleate and grow the crystals. However, despite the fact that the LCP is widely used with HVIs, the potential impact of the injection process on the LCP structure has not been reported and hence is not yet well understood. The self-assembled structure of the LCP can be affected by pressure, dehydration and temperature changes, all of which occur during continuous flow injection. These changes to the LCP structure may in turn impact the results of X-ray diffraction measurements from membrane protein crystals. To investigate the influence of HVIs on the structure of the LCP we conducted a study of the phase changes in monoolein/water and monoolein/buffer mixtures during continuous flow injection, at both atmospheric pressure and under vacuum. The reservoir pressure in the HVI was tracked to determine if there is any correlation with the phase behaviour of the LCP. The results indicated that, even though the reservoir pressure underwent (at times) significant variation, this did not appear to correlate with observed phase changes in the sample stream or correspond to shifts in the LCP lattice parameter. During vacuum injection, there was a three-way coexistence of the gyroid cubic phase, diamond cubic phase and lamellar phase. During injection at atmospheric pressure, the coexistence of a cubic phase and lamellar phase in the monoolein/water mixtures was also observed. The degree to which the lamellar phase is formed was found to be strongly dependent on the co-flowing gas conditions used to stabilize the LCP stream. A combination of laboratory-based optical polarization microscopy and simulation studies was used to investigate these observations.

Stability, flow alignment and a phase transition of the lipidic cubic phase during continuous flow injection.

Continuous flow injection is a key technology for serial crystallography measurements of protein crystals suspended in the lipidic cubic phase (LCP). To date, there has been little discussion in the literature regarding the impact of the injection process itself on the structure of the lipidic phase. This is despite the fact that the phase of the injection matrix is critical for the flow properties of the stream and potentially for sample stability. Here we report small-angle X-ray scattering measurements of a monoolein:water mixture during continuous delivery using a high viscosity injector. We observe both an alignment and modification of the LCP as a direct result of the injection process. The orientation of the cubic lattice with respect to the beam was estimated based on the anisotropy of the diffraction pattern and does not correspond to a single low order zone axis. The solvent fraction was also observed to impact the stability of the cubic phase during injection. In addition, depending on the distance traveled by the lipid after exiting the needle, the phase is observed to transition from a pure diamond phase (Pn3m) to a mixture containing both gyriod (Ia3d) and lamellar (Lα) phases. Finite element modelling of the observed phase behaviour during injection indicates that the pressure exerted on the lipid stream during extrusion accounts for the variations in the phase composition of the monoolein:water mixture.

Data reduction for serial crystallography using a robust peak finder.

A peak-finding algorithm for serial crystallography (SX) data analysis based on the principle of 'robust statistics' has been developed. Methods which are statistically robust are generally more insensitive to any departures from model assumptions and are particularly effective when analysing mixtures of probability distributions. For example, these methods enable the discretization of data into a group comprising inliers (i.e. the background noise) and another group comprising outliers (i.e. Bragg peaks). Our robust statistics algorithm has two key advantages, which are demonstrated through testing using multiple SX data sets. First, it is relatively insensitive to the exact value of the input parameters and hence requires minimal optimization. This is critical for the algorithm to be able to run unsupervised, allowing for automated selection or 'vetoing' of SX diffraction data. Secondly, the processing of individual diffraction patterns can be easily parallelized. This means that it can analyse data from multiple detector modules simultaneously, making it ideally suited to real-time data processing. These characteristics mean that the robust peak finder (RPF) algorithm will be particularly beneficial for the new class of MHz X-ray free-electron laser sources, which generate large amounts of data in a short period of time.

Analytical analysis of motion separability.

Motion segmentation is an important task in computer vision and several practical approaches have already been developed. A common approach to motion segmentation is to use the optical flow and formulate the segmentation problem using a linear approximation of the brightness constancy constraints. Although there are numerous solutions to solve this problem and their accuracies and reliabilities have been studied, the exact definition of the segmentation problem, its theoretical feasibility and the conditions for successful motion segmentation are yet to be derived. This paper presents a simplified theoretical framework for the prediction of feasibility, of segmentation of a two-dimensional linear equation system. A statistical definition of a separable motion (structure) is presented and a relatively straightforward criterion for predicting the separability of two different motions in this framework is derived. The applicability of the proposed criterion for prediction of the existence of multiple motions in practice is examined using both synthetic and real image sequences. The prescribed separability criterion is useful in designing computer vision applications as it is solely based on the amount of relative motion and the scale of measurement noise.