Klasifikasi Abjad SIBI (Sistem Bahasa Isyarat Indonesia) menggunakan Mediapipe dengan Metode Deep Learning

Abstract

General public knowledge in Indonesia regarding the Indonesian Sign Language System (SIBI) is quite low. This can prevent deaf and mute people from doing activities in public facilities. In this paper, we propose an alphabetical classification of SIBI sign language using the mediapipe and the Deep Learning method to help deaf and mute people communicate with the public. The methodology of this paper collects a dataset in the form of image data from the hand patterns of each SIBI sign language alphabet by combining a webcam with the help of the open-cv library to retrieve image data. With the mediapipe, the data is extracted from the coordinates of the landmarks in his hand and then normalized for each coordinate. Then the model is trained with a fully connected layer deep learning algorithm. The experimental results obtained an accuracy of 94.32% and a loss of 15.17% on the training data, an accuracy of 93.52% and a loss of 18.91% on the validation data, and an accuracy of 93.94% on the test data. These results show that the mediapipe and the deep learning fully connected layer algorithm successfully detect the right and left hands, and coordinates of landmarks according to the SIBI sign language alphabet correctly