STM Journals Peer-reviewed research from Consortium e-Learning Network

International Journal of Algorithms Design and Analysis Review Review Article

ASL Mobile Translator with CNN Algorithm

  1. Sasikumar M. Department of Computer Applications, People’s Education Society University, Bengaluru
  2. Lekha Achuth Department of Computer Applications, People’s Education Society University, Bengaluru

Abstract

There are around 63 million people in India with speech and hearing disabilities, and the number goes all the way up to 300 million across the world. All of them face issues in their day-to-day life as they can only have conversations with gestures. American Sign Language (ASL) mobile translator with convolutional neural network (CNN) algorithm is an easy-to-use mobile application, which uses complex images and video recognizing models built with an open-source tool called OpenCV. It efficiently recognizes gestures being shown to the camera and translates it to voice message, which can be understood by a normal person. Being a mobile application, it is portable and can be carried anywhere.

Keywords

References (12)

  1. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, June 27–30, 2016. pp. 770–
  2. Jenkins J, Rashad S. An Innovative Method for Automatic American Sign Language Interpretation using Machine Learning and Leap Motion Controller. 2021 IEEE 12th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON). 2021. doi:10.1109/uemcon53757.2021.9666640
  3. Boulares M, Jemni M. Mobile sign language translation system for deaf community. Proceedings of the International Cross-Disciplinary Conference on Web Accessibility. 2012:1-4. doi:10.1145/2207016.2207049
  4. Bird JJ, Ekárt A, Faria DR. British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language. Sensors. 2020;20(18):5151. doi:10.3390/s20185151
  5. Murthy GR, Jadon RS. A review of vision based hand gestures recognition. Int J Inform Technol Knowledge Manage. 2009; 2 (2): 405–410.
  6. Tawalare SC, Karale NE, Pande S, Khamparia A. Identification of Characters (Digits) Through Customized Convolutional Neural Network. Lecture Notes on Data Engineering and Communications Technologies. 2021:471-480. doi:10.1007/978-981-16-6285-0_38
  7. Srivastava S, Gangwar A, Mishra R, Singh S. Sign Language Recognition System Using TensorFlow Object Detection API. Communications in Computer and Information Science. 2022:634-646. doi:10.1007/978-3-030-96040-7_48
  8. Johnny S, Nirmala SJ. Sign language translator using machine learning. SN Computer Sci. 2022; 3: 1–6.
  9. Al-Saffar AAM, Tao H, Talab MA. Review of deep convolution neural network in image classification. 2017 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET). 2017:26-31. doi:10.1109/icramet.2017.8253139
  10. Szegedy C, Wei Liu, Yangqing Jia, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015:1-9. doi:10.1109/cvpr.2015.7298594
  11. Goyal S, Sharma I, Sharma S. Sign language recognition system for deaf and dumb people. Int J Eng Res Technol. 2013; 2 (4): 382–387.
  12. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Communications of the ACM. 2017;60(6):84-90. doi:10.1145/3065386