Chemical-protein relation extraction with ensembles of carefully tuned pretrained language models.

The identification of chemical-protein interactions described in the literature is an important task with applications in drug design, precision medicine and biotechnology. Manual extraction of such relationships from the biomedical literature is costly and often prohibitively time-consuming. The BioCreative VII DrugProt shared task provides a benchmark for methods for the automated extraction of chemical-protein relations from scientific text. Here we describe our contribution to the shared task and report on the achieved results. We define the task as a relation classification problem, which we approach with pretrained transformer language models. Upon this basic architecture, we experiment with utilizing textual and embedded side information from knowledge bases as well as additional training data to improve extraction performance. We perform a comprehensive evaluation of the proposed model and the individual extensions including an extensive hyperparameter search leading to 2647 different runs. We find that ensembling and choosing the right pretrained language model are crucial for optimal performance, whereas adding additional data and embedded side information did not improve results. Our best model is based on an ensemble of 10 pretrained transformers and additional textual descriptions of chemicals taken from the Comparative Toxicogenomics Database. The model reaches an F1 score of 79.73% on the hidden DrugProt test set and achieves the first rank out of 107 submitted runs in the official evaluation. Database URL: https://github.com/leonweber/drugprot.

The impact of physical connectedness on body height in Swiss conscripts.

BACKGROUND: Human populations differ in height. Recent evidence suggests that social networks play an important role in the regulation of adolescent growth and adult height. We further investigated the effect of physical connectedness on height. MATERIAL AND METHODS: We considered Switzerland as a geographic network with 169 nodes (district capitals) and 335 edges (connecting roads) and studied effects of connectedness on height in Swiss conscript from 1884 - 1891, 1908 - 1910, and 2004 - 2009. We also created exponential-family random graph models to separate possible unspecific effects of geographic vicinity. RESULTS: In 1884 - 1891, in 1908 - 1910, and in 2004 - 2009, 1(st), 2(nd) and 3(rd) order neighboring districts significantly correlate in height (p < 0.01). The correlations depend on the order of connectedness, they decline with increasing distance. Short stature districts tend to have short, tall stature districts tend to have tall neighbors. Random network analyses suggest direct road effects on height. Whereas in 1884 - 1891, direct road effects were only visible between 1(st) order neighbors, direct road effects extended to 2(nd) and 3(rd) in 1908 - 1910, and in 2004 - 2009, also to 4(th) order neighbors, and might reflect historic improvements in transportation. The spatial correlations did not significantly change when height was controlled for goiter (1884 - 1889) and for median per capita income (2006), suggesting direct road effects also in goiter-allowed-for height and income-allowed-for height. CONCLUSION: Height in a district depends on height of physically connected neighboring districts. The association decreases with increasing distance in the net. The present data suggest that people can be short because their neighbors are short; or tall because their neighbors are tall (community effect on growth). Psycho-biological effects seem to control growth and development within communities that go far beyond our current understanding of growth regulation.