Encouraged to create smooth interactions between the deaf and speech impaired community in everyday life, lecturer at the Faculty of Computer Science (Fasilkom) Universitas Indonesia (UI) Dr. Ir. Erdefi Rakun, M.Sc., created the mobile application “Text to Gesture SIBI”. Designed together with a team of researchers from the Machine Learning & Computer Vision (MLCV) Lab and the Information Retrieval – Natural Language Processing (IR-NLP) Lab, the text translation application into 3D animation for the Indonesian Sign Language System (SIBI) is now available on Playstore (Android OS) or App Store (iOS).
In this application, users can choose one of the two available 3D animated characters, namely Andi and Aini. Andi and Aini will help users translate the desired sentences into SIBI in animated form. Apart from being used as a translator of Indonesian sentences into 3D SIBI animations, this application can also be used as a digital dictionary of all the movements in the SIBI dictionary, and is also useful as a tool for learning how to sign SIBI correctly.
There are several features available in this application, namely a feature to adjust the animation speed so that users can observe SIBI movements according to their needs; zoom in or zoom out feature; feature rotates the 3D model so that movement can be seen from various directions (left, right and front); and a help feature as a guide to using the application. Currently, the “Text to Gesture SIBI” application already has more than 3000 SIBI animations and can translate more than 300,000 words and affixes.
SIBI gestures can be divided into word gestures consisting of main gestures, affix gestures, formation words and task words. Apart from word signals, SIBI also has finger signals consisting of the alphabet and numbers. One of the unique things about SIBI is that it has affix word signs which are formed by combining affix sign movements (prefixes, suffixes, particles) with basic word sign movements.
The technology used to develop the translator application is image processing, animation creation, Natural Language Processing (NLP), and mobile applications. To develop a text translation application into 3D animation, the steps required include building a database of hand and mouth movements for all words in the SIBI dictionary; the process of analyzing input sentences into word components or affixes in the sentence; and generate animations for each word component or affix taken from the SIBI movement database.
The development of the SIBI 3D text to animation translator application involved Fasilkom UI students from both Bachelor and Master levels, namely Wikan Setiaji, IGM Surya A. Darmana, Sultan Muzahidin, and Misael Jonathan, as well as teachers and Special School (SLB) students. Santi Rama, SLB Negeri 3 Jakarta, and SLB Negeri 12 Jakarta. The application development team was also assisted by Arya Sidharta who helped perfect the Movement database and Narendra Arief who perfected the mobile application.
The “Text to Gesture SIBI” application has been officially registered in the Intellectual Property Rights (Haki) of the Ministry of Law and Human Rights of the Republic of Indonesia with patent number IDS000003290-1 in 2020. The title of the patent is “Method for Generating SIBI Gestures (Indonesian Gesture System) from Indonesian Sentence Text on the Electronical Device.” In 2021, the character Andi has also been registered in Haki with the registration number 000253062, while the character Aini has the registration number 000253061.
Apart from developing a text to 3D animation translation application, Dr. Erdefi Rakun in 2022 also produced a scientific article entitled “SIBI (Sign System Indonesian Language) Text-to-3D Animation Translation Mobile Application” which was published in the International Journal of Advanced Computer Science and Applications Volume 13 Issue 9 by Science and Information (SAI ) Organization Limited.
Dr. Erdefi hopes that what he has done together with the team can be of benefit to the millions of deaf and speech impaired people in Indonesia in particular. The “Text to Gesture SIBI” application can be downloaded via the link http://cs.ui.ac.id/SIBI.IOS for iOS or http://cs.ui.ac.id/SIBI.Android for Android systems.