STM Journals Peer-reviewed research from Consortium e-Learning Network

Journal of Image Processing & Pattern Recognition Progress Review Article

SkinSight: Design and Implementation of an Intelligent Skin Type Detection System

  1. Diya Goswami VIT Bhopal University, Bhopal
  2. Sayantika Laskar VIT Bhopal University, Bhopal

Abstract

Identifying an individual’s skin type accurately is essential for creating personalized dermatological treatments and formulating skincare products that genuinely meet user needs. In this project, a real- time skin type classification system is developed using a combination of convolutional neural networks (CNNs) and modern computer vision techniques. The system processes live video streams, isolates the facial region through Haar cascade–based detection, and applies a series of preprocessing steps to enhance clarity and highlight essential skin features. Once the facial area is prepared, the trained CNN model classifies the skin into one of three categories: dry, normal, or oily. During evaluation, the model demonstrates strong accuracy and consistent performance, making it suitable for real-time use on consumer devices or clinical tools. Beyond simple classification, this technology has the potential to support personalized skincare recommendations, assist dermatologists with initial screenings, and improve automated beauty or health applications. Future enhancements may include expanding the model to recognize additional skin concerns, improving robustness under varying lighting conditions, and integrating more advanced feature-extraction methods to further increase reliability.

Keywords

References (30)

  1. Saiwaeo S, Arwatchananukul S, Mungmai L, Preedalikit W, Aunsri N. Human skin type classification using image processing and deep learning approaches. Heliyon. 2023;9(11):e21176. doi:10.1016/j.heliyon.2023.e21176
  2. KARA FB, KARA R, SAKACI ÇELİK S. Skin Type Detection with Deep Learning: A Comparative Analysis. Düzce Üniversitesi Bilim ve Teknoloji Dergisi. 2023;11(2):729-742. doi:10.29130/dubited.930096
  3. Ran J, Dong G, Yi F, Li L, Wu Y. Automatic Measurement of Comprehensive Skin Types Based on Image Processing and Deep Learning. Electronics. 2024;14(1):49. doi:10.3390/electronics14010049
  4. Efata R, Loka WI, Wijaya N, Suhartono D. Facial Skin Type Prediction Based on Baumann Skin Type Solutions Theory Using Machine Learning. TEM Journal. 2023:96-103. doi:10.18421/tem121-13
  5. Goceri E. Convolutional Neural Network Based Desktop Applications to Classify Dermatological Diseases. 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS). 2020:138-143. doi:10.1109/ipas50080.2020.9334956
  6. Goceri E. Automated Skin Cancer Detection: Where We Are and The Way to The Future. 2021 44th International Conference on Telecommunications and Signal Processing (TSP). 2021:48-51. doi:10.1109/tsp52935.2021.9522605
  7. Goceri E, Karakas AA. Comparative evaluations of CNN based networks for skin lesion classification. In: Xiao Y, Abraham AP, Roth J, editors. Proceedings of the IADIS International Conference Computer Graphics, Visualization, Computer Vision and Image Processing 2020; Big Data Analytics, Data Mining and Computational Intelligence 2020; and Theory and Practice in Modern Computing 2020; 2020 Jul 23–25; Virtual Conference. Lisbon: IADIS – International Association for Development of the Information Society; 2020. p. 237–242.
  8. Xie P, Li T, Liu J, Li F, Zhou J, Zuo K. Analyze Skin Histopathology Images Using Multiple Deep Learning Methods. 2021 IEEE 3rd International Conference on Frontiers Technology of Information and Computer (ICFTIC). 2021:374-377. doi:10.1109/icftic54370.2021.9647425
  9. Zhang B, Zhou X, Luo Y, Zhang H, Yang H, Ma J, et al. Opportunities and Challenges: Classification of Skin Disease Based on Deep Learning. Chinese Journal of Mechanical Engineering. 2021;34(1). doi:10.1186/s10033-021-00629-5
  10. Ravnbak MH. Objective determination of Fitzpatrick skin type. Dan Med Bull. 2010;57(8):B4153. PubMed PMID: 20682135.
  11. Yook D, Lee H, Yoo IC. A survey on parallel training algorithms for deep neural networks. J Acoust Soc Korea. 2020;39(6):505–514.
  12. Vinod S, Thomas MV. A Comparative Analysis on Deep Learning Techniques for Skin Cancer Detection and Skin Lesion Segmentation. 2021 International Conference on Communication, Control and Information Sciences (ICCISc). 2021:1-6. doi:10.1109/iccisc52257.2021.9484873
  13. Cho SI, Kim D, Lee H, Um TT, Kim H. Explore highly relevant questions in the Baumann skin type questionnaire through the digital skin analyzer: A retrospective single‐center study in South Korea. Journal of Cosmetic Dermatology. 2023;22(11):3159-3167. doi:10.1111/jocd.15820
  14. Srimaharaj W, Chaising S. Distance-based integration method for human skin type identification. Computers in Biology and Medicine. 2024;178:108575. doi:10.1016/j.compbiomed.2024.108575
  15. Hossain MS, Shahriar GM, Syeed MMM, Uddin MF, Hasan M, Shivam S, et al. Region of interest (ROI) selection using vision transformer for automatic analysis using whole slide images. Scientific Reports. 2023;13(1). doi:10.1038/s41598-023-38109-6
  16. Indriyani, Sudarma IM. Classification of facial skin type using discrete wavelet transform, contrast, local binary pattern and support vector machine. J Theor Appl Inf Technol. 2020 Mar 15;98(5):768– 799.
  17. Cui H, Feng C, Zhang T, Martínez-Ríos V, Martorell P, Tortajada M, et al. Effects of a lotion containing probiotic ferment lysate as the main functional ingredient on enhancing skin barrier: a randomized, self-control study. Scientific Reports. 2023;13(1). doi:10.1038/s41598-023-43336-y
  18. Nusantara TF, Atmaja RD, Azizah A. [Klasifikasi jenis kulit wajah pria berdasarkan tekstur menggunakan metode gray level co-occurrence matrix (GLCM) dan support vector machine]. Classification of men’s face skin types based on the texture using gray level co-occurrence matrix (GLCM) and support vector machine method. eProceedings Eng. 2018 Aug;5(2):2130–2137. [Indonesian].
  19. Baumann L. Understanding and Treating Various Skin Types: The Baumann Skin Type Indicator. Dermatologic Clinics. 2008;26(3):359-373. doi:10.1016/j.det.2008.03.007
  20. Azadeh Noori Hoshyar, Al-Jumaily A, Sulaiman R. Review on automatic early skin cancer detection. 2011 International Conference on Computer Science and Service System (CSSS). 2011:4036-4039. doi:10.1109/csss.2011.5974581
  21. Goceri E. Image Augmentation for Deep Learning Based Lesion Classification from Skin Images. 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS). 2020:144-148. doi:10.1109/ipas50080.2020.9334937
  22. Savchenko A. Probabilistic Neural Network With Complex Exponential Activation Functions in Image Recognition. IEEE Transactions on Neural Networks and Learning Systems. 2020;31(2):651-660. doi:10.1109/tnnls.2019.2908973
  23. Wang Z, Yan M, Chen J, Liu S, Zhang D. Deep learning library testing via effective model generation. Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2020:788-799. doi:10.1145/3368089.3409761
  24. Yao R, Li J, Hui M, Bai L, Wu Q. Feature Selection Based on Random Forest for Partial Discharges Characteristic Set. IEEE Access. 2020;8:159151-159161. doi:10.1109/access.2020.3019377
  25. Ding DM, Tu Y, Man MQ, Wu WJ, Lu FY, Li X, et al. Association between lactic acid sting test scores, self‐assessed sensitive skin scores and biophysical properties in Chinese females. International Journal of Cosmetic Science. 2019;41(4):398-404. doi:10.1111/ics.12550
  26. Ahuja R, Sharma SC. Exploiting machine learning and feature selection algorithms to predict instructor performance in higher education. J Inf Sci Eng. 2021;37(5):993–1009. doi:10.6688/JISE.20210937(5).0001.
  27. Liu J, Zhang X, Li Y, Wang J, Kim HJ. Deep Learning-Based Reasoning With Multi-Ontology for IoT Applications. IEEE Access. 2019;7:124688-124701. doi:10.1109/access.2019.2937353
  28. Department of Computer Science and Engineering, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India., Bhadula* S, Sharma S, Department of Computer Science and Engineering, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India., Juyal P, Department of Computer Science and Engineering, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India., et al. Machine Learning Algorithms based Skin Disease Detection. International Journal of Innovative Technology and Exploring Engineering. 2019;9(2):4044-4049. doi:10.35940/ijitee.b7686.129219
  29. Kolkur MS, Kalbande DR, Kharkar V. Machine learning approaches to multi-class human skin disease detection. Int J Comput Intell Res. 2018;14(1):29–39.
  30. Badr M, Elkasaby A, Alrahmawy M, El-Metwally S. A Multi-model Deep Learning Architecture for Diagnosing Multi-class Skin Diseases. Journal of Imaging Informatics in Medicine. 2024;38(3):1776-1795. doi:10.1007/s10278-024-01300-w
Support