Enhancing Breast Cancer Detection in Ultrasound Images: An Innovative Approach Using Progressive Fine‐Tuning of Vision Transformer Models

Breast cancer is ranked as the second most common cancer among women globally, highlighting the critical need for precise and early detection methods. Our research introduces a novel approach for classifying benign and malignant breast ultrasound images. We leverage advanced deep learning methodologies, mainly focusing on the vision transformer (ViT) model. Our method distinctively features progressive fine-tuning, a tailored process that incrementally adapts the model to the nuances of breast tissue classification. Ultrasound imaging was chosen for its distinct benefits in medical diagnostics. This modality is noninvasive and cost-effective and demonstrates enhanced specificity, especially in dense breast tissues where traditional methods may struggle. Such characteristics make it an ideal choice for the sensitive task of breast cancer detection. Our extensive experiments utilized the breast ultrasound images dataset, comprising 780 images of both benign and malignant breast tissues. The dataset underwent a comprehensive analysis using several pretrained deep learning models, including VGG16, VGG19, DenseNet121, Inception, ResNet152V2, DenseNet169, DenseNet201, and the ViT. The results presented were achieved without employing data augmentation techniques. The ViT model demonstrated robust accuracy and generalization capabilities with the original dataset size, which consisted of 637 images. Each model’s performance was meticulously evaluated through a robust 10-fold cross-validation technique, ensuring a thorough and unbiased comparison. Our findings are significant, demonstrating that the progressive fine-tuning substantially enhances the ViT model’s capability. This resulted in a remarkable accuracy of 94.49% and an AUC score of 0.921, significantly higher than models without fine-tuning. These results affirm the efficacy of the ViT model and highlight the transformative potential of integrating progressive fine-tuning with transformer models in medical image classification tasks. The study solidifies the role of such advanced methodologies in improving early breast cancer detection and diagnosis, especially when coupled with the unique advantages of ultrasound imaging.