RESUMO
In oncology, melanoma is a serious concern, often arising from DNA changes caused mainly by ultraviolet radiation. This cancer is known for its aggressive growth, highlighting the necessity of early detection. Our research introduces a novel deep learning framework for melanoma classification, trained and validated using the extensive SIIM-ISIC Melanoma Classification Challenge-ISIC-2020 dataset. The framework features three dilated convolution layers that extract critical feature vectors for classification. A key aspect of our model is incorporating the Off-policy Proximal Policy Optimization (Off-policy PPO) algorithm, which effectively handles data imbalance in the training set by rewarding the accurate classification of underrepresented samples. In this framework, the model is visualized as an agent making a series of decisions, where each sample represents a distinct state. Additionally, a Generative Adversarial Network (GAN) augments training data to improve generalizability, paired with a new regularization technique to stabilize GAN training and prevent mode collapse. The model achieved an F-measure of 91.836% and a geometric mean of 91.920%, surpassing existing models and demonstrating the model's practical utility in clinical environments. These results demonstrate its potential in enhancing early melanoma detection and informing more accurate treatment approaches, significantly advancing in combating this aggressive cancer.