Exploring advanced techniques for enhancing CapsNet: custom squashing, pretraining, and routing in large-scale unbalanced data

OpenAlex English

Capsule networks (CapsNet) have emerged as a promising alternative to traditional convolutional neural networks (CNNs) for image classification, due to their ability to capture spatial hierarchies and relationships between parts and wholes. However, CapsNet still faces limitations such as high computational complexity, inefficient routing, and difficulty in learning complex features when trained from scratch. To address these issues, this study presents two enhanced methods to improve the performance of the baseline CapsNet model. The first method integrates pre-trained CapsNet architectures—including GoogLeNet-Inception V3-based CapsNet (GN-CapsNet), Visual Geometry Group-based CapsNet (VGG-CapsNet), and Residual Network-based CapsNet (RES-CapsNet)—to extract robust and complex features. It also employs an improved squash function and a modified dynamic routing mechanism to enhance learning stability, routing efficiency, and pattern recognition. The second method introduces Fire-CapsNet: a lightweight model that uses Fire modules for efficient feature extraction and a custom swish activation function to reduce computational cost while maintaining high accuracy. The performance of these models was evaluated using four benchmark datasets: Bone Marrow (BM), MNIST, Fashion-MNIST, and CIFAR-10. Results demonstrated substantial improvements in classification performance. Specifically, on the BM dataset, the baseline CapsNet model achieved an accuracy of 96.99%, the VGG-CapsNet model 99.31%, the RES-CapsNet model 99.38%, the GN-CapsNet model 99.89%, and the Fire-CapsNet model achieved the highest accuracy of 99.92% in the shortest computation time, highlighting its effectiveness and efficiency for real-world applications. The link to the work is https://github.com/Aminaalr/Exploring-Advanced-Techniques-for-CapsNet .