ABSTRACT
Purpose: To develop and evaluate domain-specific and pretrained bidirectional encoder representations from transformers (BERT) models in a transfer learning task on varying training dataset sizes to annotate a larger overall dataset. Materials and Methods: The authors retrospectively reviewed 69 095 anonymized adult chest radiograph reports (reports dated April 2020-March 2021). From the overall cohort, 1004 reports were randomly selected and labeled for the presence or absence of each of the following devices: endotracheal tube (ETT), enterogastric tube (NGT, or Dobhoff tube), central venous catheter (CVC), and Swan-Ganz catheter (SGC). Pretrained transformer models (BERT, PubMedBERT, DistilBERT, RoBERTa, and DeBERTa) were trained, validated, and tested on 60%, 20%, and 20%, respectively, of these reports through fivefold cross-validation. Additional training involved varying dataset sizes with 5%, 10%, 15%, 20%, and 40% of the 1004 reports. The best-performing epochs were used to assess area under the receiver operating characteristic curve (AUC) and determine run time on the overall dataset. Results: The highest average AUCs from fivefold cross-validation were 0.996 for ETT (RoBERTa), 0.994 for NGT (RoBERTa), 0.991 for CVC (PubMedBERT), and 0.98 for SGC (PubMedBERT). DeBERTa demonstrated the highest AUC for each support device trained on 5% of the training set. PubMedBERT showed a higher AUC with a decreasing training set size compared with BERT. Training and validation time was shortest for DistilBERT at 3 minutes 39 seconds on the annotated cohort. Conclusion: Pretrained and domain-specific transformer models required small training datasets and short training times to create a highly accurate final model that expedites autonomous annotation of large datasets.Keywords: Informatics, Named Entity Recognition, Transfer Learning Supplemental material is available for this article. ©RSNA, 2022See also the commentary by Zech in this issue.
ABSTRACT
INTRODUCTION: The aim of this research was to evaluate the impact of a novel tele-rehabilitation system on self-reported functional outcomes compared to usual care during the first three months after stroke. METHODS: A parallel, two-arm, evaluator-blinded, randomised controlled trial was conducted. Adults aged ≥40 years who had suffered a stroke within four weeks of the start of the study were recruited from the general community. The intervention group received access to a novel tele-rehabilitation system and programme for three months. The primary outcome measures utilised were the frequency and limitation total scores of the Late-Life Function and Disability Instrument (LLFDI) at three months. RESULTS: A total of 124 individuals were recruited. The mean differences in the LLDFI frequency and limitation total scores at three months comparing the intervention and control groups were -3.30 (95% confidence interval (CI) -7.81 to 1.21) and -6.90 (95% CI -15.02 to 1.22), respectively. Adjusting for the respective baseline covariates and baseline Barthel Index also showed no significant difference between interventions in the LLFDI outcomes. DISCUSSION: The intervention and control groups self-reported similar improvements in functional outcomes. Tele-rehabilitation may be a viable option to provide post-stroke rehabilitation services in Singapore while reducing barriers to continue rehabilitation conventionally after discharge from hospital and encouraging more participation.
