Mind your prevalence!

Multiple metrics are used when assessing and validating the performance of quantitative structure-activity relationship (QSAR) models. In the case of binary classification, balanced accuracy is a metric to assess the global performance of such models. In contrast to accuracy, balanced accuracy does not depend on the respective prevalence of the two categories in the test set that is used to validate a QSAR classifier. As such, balanced accuracy is used to overcome the effect of imbalanced test sets on the model's perceived accuracy. Matthews' correlation coefficient (MCC), an alternative global performance metric, is also known to mitigate the imbalance of the test set. However, in contrast to the balanced accuracy, MCC remains dependent on the respective prevalence of the predicted categories. For simplicity, the rest of this work is based on the positive prevalence. The MCC value may be underestimated at high or extremely low positive prevalence. It contributes to more challenging comparisons between experiments using test sets with different positive prevalences and may lead to incorrect interpretations. The concept of balanced metrics beyond balanced accuracy is, to the best of our knowledge, not yet described in the cheminformatic literature. Therefore, after describing the relevant literature, this manuscript will first formally define a confusion matrix, sensitivity and specificity and then present, with synthetic data, the danger of comparing performance metrics under nonconstant prevalence. Second, it will demonstrate that balanced accuracy is the performance metric accuracy calibrated to a test set with a positive prevalence of 50% (i.e., balanced test set). This concept of balanced accuracy will then be extended to the MCC after showing its dependency on the positive prevalence. Applying the same concept to any other performance metric and widening it to the concept of calibrated metrics will then be briefly discussed. We will show that, like balanced accuracy, any balanced performance metric may be expressed as a function of the well-known values of sensitivity and specificity. Finally, a tale of two MCCs will exemplify the use of this concept of balanced MCC versus MCC with four use cases using synthetic data. SCIENTIFIC CONTRIBUTION: This work provides a formal, unified framework for understanding prevalence dependence in model validation metrics, deriving balanced metric expressions beyond balanced accuracy, and demonstrating their practical utility for common use cases. In contrast to prior literature, it introduces the derived confusion matrix to express metrics as functions of sensitivity, specificity and prevalence without needing additional coefficients. The manuscript extends the concept of balanced metrics to Matthews' correlation coefficient and other widely used performance indicators, enabling robust comparisons under prevalence shifts.

The Asymmetric Aza-silyl-Prins Reaction: Synthesis of Enantiopure Piperidines.

The design and development of the first asymmetric aza-silyl-Prins reaction is reported, giving rise to valuable and diverse piperidines and pipecolic acid derivatives in both high yields and as single enantiomers. The creation of a novel chiral auxiliary-homoallylic amine for the aza-silyl-Prins reaction is essential to its success.

Synthesis of Azabicycles via Cascade Aza-Prins Reactions: Accessing the Indolizidine and Quinolizidine Cores.

The first detailed studies of intramolecular aza-Prins and aza-silyl-Prins reactions, starting from acyclic materials, are reported. The methods allow rapid and flexible access toward an array of [6,5] and [6,6] aza-bicycles, which form the core skeletons of various alkaloids. On the basis of our findings on the aza-Prins and aza-silyl-Prins cyclizations, herein we present simple protocols for the intramolecular preparation of the azabicyclic cores of the indolizidines and quinolizidines using a one-pot cascade process of N-acyliminium ion formation followed by aza-Prins cyclization and either elimination or carbocation trapping. It is possible to introduce a range of different substituents into the heterocycles through a judicial choice of Lewis acid and solvent(s), with halo-, phenyl-, and amido-substituted azabicyclic products all being accessed through these highly diastereoselective processes.

A detailed investigation of the aza-Prins reaction.

The development of a Lewis acid-promoted aza-Prins reaction to form piperidines and pyrrolidines is described. Indium trichloride has been found to be a highly successful and mild Lewis acid for promoting this reaction. A thorough mechanistic investigation is described, including the factors that influence the formation of the 5- or 6-membered ring product(s).

A versatile indium trichloride mediated Prins-type reaction to unsaturated heterocycles.

A versatile and high-yielding indium trichloride mediated cyclization reaction of silylated homoallylic alcohols, thiols, or amines with aldehydes or epoxides is described as a rapid route to a range of unsaturated heterocycles. The excellent diastereoselectivity observed offers a method of wide scope and generality.