Abstract

Automatic detection of gastrointestinal (GI) lesions makes endoscopic diagnosis more accurate, but it is difficult because the lesions can look different. A deep learning (DL) method for specific classification makes things more accurate, but for clinical use, a strong model is needed to make sure the location is correct. This study comes up with a better way to divide things up and solves these problems. The goal is to make a custom partition frame with a Residual Swin Transformer Fusion Network (RN-RSTFN) that has been fine-tuned with a Refined Nutcracker. Gaussian Filtering (GF) was used to reduce noise in the images and make them clearer. Then, Z-Score was used to standardize the distribution of pixel intensity. Function Extraction using a histogram of oriented gradients (HOG) helps to get the wound pattern needed for better partition performance. Give the right location for GI ulcers. Endoscopic image datasets that are available to the public from depot and medical institutions include pictures of different GI lesions. Gaussian Filtering (GF) was used to reduce noise in the images and make them clearer. Then, Z-Score was used to standardize the distribution of pixel intensity. The RN-RSTFN that was suggested combines the hierarchical representation and residual learning of the Swin Transformer to improve the boundaries of lesions. When measured using the Dice score and mIOU, Precision, and Recall measures, the RN-RSTFN model showed a lot of improvement in finding GI lesions. The scores were 0.9458, 0.9383, 0.9634, and 0.9596, respectively.

Keywords

Endoscopic, Automatic gastrointestinal (GI), Image Segmentation, Refined Nutcracker-tuned Residual Swin Transformer Fusion Network (RN-RSTFN),

Downloads

Download data is not yet available.

References

  1. A.C. Vasconcelos, M. Dinis-Ribeiro, D. Libânio, Endoscopic resection of early gastric cancer and pre-malignant gastric lesions. Cancers, Cancers, 15(12), (2023) 3084. https://doi.org/10.3390/cancers15123084
  2. M. Sekine, T. Asano, H. Mashima, The diagnosis of small gastrointestinal subepithelial lesions by endoscopic ultrasound-guided fine needle aspiration and biopsy. Diagnostics, 12(4), (2022) 810. https://doi.org/10.3390/diagnostics12040810
  3. W. Wang, X. Yang, X. Li, J. Tang, Convolutional‐capsule network for gastrointestinal endoscopy image classification. International Journal of Intelligent Systems, 37(9), (2022) 5796-5815. https://doi.org/10.1002/int.22815
  4. S.M. Bitar, M. Moussa, The risk factors for the recurrent upper gastrointestinal hemorrhage among acute peptic ulcer disease patients in Syria: A prospective cohort study. Annals of Medicine and Surgery, 74, (2022) 103252. https://doi.org/10.1016/j.amsu.2022.103252
  5. Y. Liu, S. Zuo, Self-supervised monocular depth estimation for gastrointestinal endoscopy. Computer Methods and Programs in Biomedicine, 238, (2023) 107619. https://doi.org/10.1016/j.cmpb.2023.107619
  6. R. Zhang, B. Peng, Y. Liu, X. Liu, J. Huang, K. Suzuki, Y. Nakajima, D. Nemoto, K. Togashi, X. Zhu, Localization of Capsule Endoscope in Alimentary Tract by Computer-Aided Analysis of Endoscopic Images. Sensors, 25(3), (2025) 746. https://doi.org/10.3390/s25030746
  7. Y. Okagawa, S. Abe, M. Yamada, I. Oda, and Y. Saito, Artificial intelligence in endoscopy. Digestive diseases and sciences, 67(5), (2022) 1553-1572. https://doi.org/10.1007/s10620-021-07086-z
  8. A. Nagahara, A. Shiotani, K. Iijima, T. Kamada, Y. Fujiwara, K. Kasugai, M. Kato, K. Higuchi, The role of advanced endoscopy in the management of inflammatory digestive diseases (upper gastrointestinal tract). Digestive Endoscopy, 34(1), (2022) 63-72. https://doi.org/10.1111/den.13982
  9. P.H. Conze, G. Andrade-Miranda, V.K. Singh, V. Jaouen, D. Visvikis, Current and emerging trends in medical image segmentation with deep learning. IEEE Transactions on Radiation and Plasma Medical Sciences, 7(6), (2023) 545-569. https://doi.org/10.1109/TRPMS.2023.3265863
  10. F. Renna, M. Martins, A. Neto, A. Cunha, D. Libânio, M. Dinis-Ribeiro, M. Coimbra, Artificial intelligence for upper gastrointestinal endoscopy: a roadmap from technology development to clinical practice. Diagnostics, 12(5), (2022) 1278. https://doi.org/10.3390/diagnostics12051278
  11. S. Ahmad, J.S. Kim, D.K. Park, T. Whangbo, Automated detection of gastric lesions in endoscopic images by leveraging attention-based yolov7. IEEE Access, 11, (2023) 87166-87177. https://doi.org/10.1109/ACCESS.2023.3296710
  12. J.Y. Nam, H.J. Chung, K.S. Choi, H. Lee, T.J. Kim, H. Soh, E. Ae Kang , S.J. Cho, J. Chul Ye, J. Pil Im, S.G. Kim, J.S. Kim, H. Chung, J.H. Lee, Deep learning model for diagnosing gastric mucosal lesions using endoscopic images: development, validation, and method comparison. Gastrointestinal Endoscopy, 95(2), (2022) 258-268. https://doi.org/10.1016/j.gie.2021.08.022
  13. S. Wang, Q. He, P. Zhang, X. Chen, S. Zuo, Toward Automatic Detection of Gastric Lesion for Upper Gastrointestinal Endoscopy with Neural Network. Journal of Medical Robotics Research, 7(01), (2022) 2141003. https://doi.org/10.1142/S2424905X21410038
  14. P. Pornvoraphat, K. Tiankanon, R. Pittayanon, P. Sunthornwetchapong, P. Vateekul, R. Rerknimitr, Real-time gastric intestinal metaplasia diagnosis tailored for bias and noisy-labeled data with multiple endoscopic imaging. Computers in Biology and Medicine, 154, (2023) 106582. https://doi.org/10.1016/j.compbiomed.2023.106582
  15. M. Vania, B.A. Tama, H. Maulahela, S. Lim, Recent advances in applying machine learning and deep learning to detect upper gastrointestinal tract lesions. IEEE Access, 11, (2023) 66544-66567. https://doi.org/10.1109/ACCESS.2023.3290997
  16. M. Alhajlah, Automated lesion detection in gastrointestinal endoscopic images: leveraging deep belief networks and genetic algorithm-based Segmentation. Multimedia Tools and Applications, (2024) 1-15. https://doi.org/10.1007/s11042-024-20439-w
  17. R. Mantri, R.A.H. Khan, D.T. Mane, An Efficient System for Detection and Classification of Acute Lymphoblastic Leukemia Using Semi-Supervised Segmentation Technique. International Research Journal of Multidisciplinary Technovation, 7(2), (2025) 121-134. https://doi.org/10.54392/irjmt25210
  18. V. Rajinikanth, S. Aslam, S. Kadry, O. Thinnukool, Semi/Fully-Automated Segmentation of Gastric-Polyp Using Aquila-Optimization-Algorithm Enhanced Images. Computers, Materials & Continua, 70(2), (2022). http://dx.doi.org/10.32604/cmc.2022.019786
  19. M. Hanscom, D.R. Cave, Endoscopic capsule robot-based diagnosis, navigation and localization in the gastrointestinal tract. Frontiers in Robotics and AI, 9, (2022) 896028. https://doi.org/10.3389/frobt.2022.896028
  20. D. Bajhaiya, S.N. Unni, Deep learning-enabled detection and localization of gastrointestinal diseases using wireless-capsule endoscopic images. Biomedical Signal Processing and Control, 93, (2024) 106125. https://doi.org/10.1016/j.bspc.2024.106125
  21. L. Ma, X. Su, L. Ma, X. Gao, M. Sun, Deep learning for classification and localization of early gastric cancer in endoscopic images. Biomedical Signal Processing and Control, 79, (2023) 104200. https://doi.org/10.1016/j.bspc.2022.104200
  22. L. Wu, M. Xu, X. Jiang, X. He, H. Zhang, Y. Ai, Q. Tong , P. Lv, B. Lu, M. Guo, M. Huang, L. Ye, L. Shen, H.Yu, Real-time artificial intelligence for detecting focal lesions and diagnosing neoplasms of the stomach by white-light endoscopy (with videos). Gastrointestinal Endoscopy, 95(2), (2022) 269-280. https://doi.org/10.1016/j.gie.2021.09.017
  23. S. Mahmood, M.M.S. Fareed, G. Ahmed, F. Dawood, S. Zikria, A. Mostafa, S.F. Jilani, M. Asad, M. Aslam, A robust deep model for classification of peptic ulcer and other digestive tract disorders using endoscopic images. Biomedicines, 10(9), (2022) 2195. https://doi.org/10.3390/biomedicines10092195
  24. D. Bravo, J. Ruano, M. Jaramillo, S. Medina, M. Gómez, F.A. González, E. Romero, (2024) Automatic endoscopy classification by fusing depth estimations and image information. IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, Greece. https://doi.org/10.1109/ISBI56570.2024.10635452
  25. J. Chen, G. Wang, Y. Ding, Z. Zhang, K. Xia, L. Xu, X. Xu, Development of an AI‐Assisted System for Automatic Recognition and Localization Marking of Colonic Polyps (With Video). Journal of Gastroenterology and Hepatology, 40(7), (2025) 1797-1808. https://doi.org/10.1111/jgh.16980
  26. M.N. Noor, M. Nazir, S.A. Khan, I. Ashraf, O.Y. Song, Localization and classification of gastrointestinal tract disorders using explainable AI from endoscopic images. Applied Sciences, 13(15), (2023) 9031. https://doi.org/10.3390/app13159031
  27. D. Jha, P.H. Smedsrud, M.A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, H.D. Johansen, (2019). Resunet++: An advanced architecture for medical image segmentation. In 2019 IEEE international symposium on multimedia (ISM), IEEE, USA.
  28. Y. Qin, J. Chang, L. Li, M. Wu, Enhancing gastroenterology with multimodal learning: the role of large language model chatbots in digestive endoscopy. Frontiers in Medicine, 12, (2025) 1583514. https://doi.org/10.3389/fmed.2025.1583514
  29. E.M. El-Gammal, W. El-Shafai, T.E. Taha, A.S. El-Fishawy, F.E. Abd El-Samie, A survey of artificial intelligence models for wireless capsule endoscopy videos for superior automatic diagnosis: problems and solutions. Multimedia Tools and Applications, (2025) 1-35. https://doi.org/10.1007/s11042-024-18300-1
  30. R. Zhang, B. Peng, Y. Liu, X. Liu, J. Huang, K. Suzuki, Y. Nakajima, D. Nemoto, K. Togashi, X. Zhu, Localization of Capsule Endoscope in Alimentary Tract by Computer-Aided Analysis of Endoscopic Images. Sensors, 25(3), (2025) 746. https://doi.org/10.3390/s25030746
  31. Kaggle, (2023) Kvasir-SEG Dataset, Kaggle. Available: https://www.kaggle.com/datasets/tanmaydebnath/kvasir-seg-dataset?select=dyed-lifted-polyps
  32. A. Garbaz, Y. Oukdach, S. Charfi, M. El Ansari, L. Koutti, M. Salihoun, Gsac-uformer: Groupwise self-attention convolutional transformer-based unet for medical image segmentation. Cognitive Computation, 17(69), (2025) 1-14. https://doi.org/10.1007/s12559-025-10425-1
  33. N. Rochmawati, C. Fatichah, B. Amaliah, A.B. Raharjo, F. Dumont, E. Thibaudeau, C. Dumas, Deep Learning-Based Lesion Detection in Endoscopy: A Systematic Literature Review. IEEE Access, 13, (2025) 43532 – 43556. https://doi.org/10.1109/ACCESS.2025.3548167
  34. A. Dhali, V. Kipkorir, R. Maity, B.S. Srichawla, J. Biswas, R.B. Rathna, H.R. Bharadwaj, I. Ongidi, T. Chaudhry, G. Morara, M. Waithaka, C. Rugut, M. Lemashon, I. Cheruiyot, D. Ojuka, S. Ray, G. K. Dhali, Artificial intelligence–assisted capsule endoscopy versus conventional capsule endoscopy for detection of small bowel lesions: a systematic review and meta‐analysis. Journal of Gastroenterology and Hepatology, 40(5), (2025) 1105-1118. https://doi.org/10.1111/jgh.16931
  35. X. Guo, L. Pang, P. Chen, Q. Jiang, Y. Zhong, Deep ensemble framework with Bayesian optimization for multi-lesion recognition in capsule endoscopy images. Medical & Biological Engineering & Computing, (2025) 1-16. https://doi.org/10.1007/s11517-025-03380-4
  36. C. Hu, Y. Xia, Z. Zheng, M. Cao, G. Zheng, S. Chen, J. Sun, W. Chen, Q. Zheng, S. Pan, Y. Zhang, J. Chen, P. Yu, J. Xu, J. Xu, Z. Qiu, T. Lin, B. Yun, J. Yao, W. Guo, C. Gao, X. Kong, K. Chen, Z. Wen, G. Zhu, J. Qiao, Y. Pan, H. Li, X. Gong, Z. Ye, W. Ao, L. Zhang, X. Yan, Y. Tong, X. Yang, X. Zheng, S. Fan, J.Cao, C. Yan, K. Xie, S. Zhang, Y. Wang, L. Zheng, Y. Wu, Z. Ge, X. Tian, X. Zhang, Y. Wang, R. Zhang, Y. Wei, W. Zhu, J. Zhang, H. Qiu, M. Su, L. Shi, Z. Xu, L. Zhang, Cheng, X. (2025). AI-based large-scale screening of gastric cancer from noncontrast CT imaging. Nature Medicine, 1-9. https://doi.org/10.1038/s41591-025-03785-6
  37. S. Siddiqui, J.A. Khan, T. Akram, M. Alharbi, J. Cha, D.A. AlHammadi, SNet: A novel convolutional neural network architecture for advanced endoscopic image classification of gastrointestinal disorders. SLAS technology, 33, (2025) 100304. https://doi.org/10.1016/j.slast.2025.100304
  38. P. Patil, V. Narawade, Radiology Image Data Augmentation and Image Enhancement in Respiratory Disease Infection Detection Using Machine Learning Approach. International Research Journal of Multidisciplinary Technovation, 6(2), (2024) 133-155. https://doi.org/10.54392/irjmt24211
  39. T.T. Habe, K. Haataja, P. Toivanen, Precision enhancement in wireless capsule endoscopy: a novel transformer-based approach for real-time video object detection. Frontiers in Artificial Intelligence, 8, (2025) 1529814. https://doi.org/10.3389/frai.2025.1529814
  40. W.A. Mustafa, H. Alquran, Editorial for the Special Issue Advances in Medical Image Processing, Segmentation, and Classification. Diagnostics, 15(9), (2025) 1114. https://doi.org/10.3390/diagnostics15091114