Swin MAE: Masked autoencoders for small datasets.
Comput Biol Med
; 161: 107037, 2023 07.
Article
in En
| MEDLINE
| ID: mdl-37230020
ABSTRACT
The development of deep learning models in medical image analysis is majorly limited by the lack of large-sized and well-annotated datasets. Unsupervised learning does not require labels and is more suitable for solving medical image analysis problems. However, most unsupervised learning methods must be applied to large datasets. To make unsupervised learning applicable to small datasets, we proposed Swin MAE, a masked autoencoder with Swin Transformer as its backbone. Even on a dataset of only a few thousand medical images, Swin MAE can still learn useful semantic features purely from images without using any pre-trained models. It can equal or even slightly outperform the supervised model obtained by Swin Transformer trained on ImageNet in the transfer learning results of downstream tasks. Compared to MAE, Swin MAE brought a performance improvement of twice and five times for downstream tasks on BTCV and our parotid dataset, respectively. The code is publicly available at https//github.com/Zian-Xu/Swin-MAE.
Key words
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Parotid Gland
/
Problem Solving
Language:
En
Journal:
Comput Biol Med
Year:
2023
Type:
Article
Affiliation country:
China