Edcomtech: Jurnal Kajian Teknologi Pendidikan Vol .
No .
Month Year, pp.
page-page https://doi.
org/10.
17977/um039x pISSN: 2548-9879 .
eISSN: 2599-2139 .
http://journal2.
id/index.
php/edcomtech Design of Voice Translator Application for Multinational Sign Language: A Notion for Global Communication for People with Hearing Issues AU Murtiasih Murtiasih1.
Ediyanto Ediyanto2*.
Lukman Hakim3 Department of Special Education.
Faculty of Education.
Universitas Negeri Malang.
Malang, 65114.
Indonesia Department of Special Education.
Faculty of Education.
Universitas Negeri Malang.
Malang, 65114.
Indonesia Informatics Engineering Study Program.
Faculty of Engineering.
Universitas Yudharta Pasuruan.
Pasuruan 67162.
Indonesia *Email corresponding author: ediyanto.
fip@um.
Article InfoAU Abstrak Article history:
Received.
Revised.
Accepted .
Published .
Komunikasi merupakan sebuah aktivitas penyampaian informasi melalui pertukaran pikiran, pesan, atau informasi melalui ucapan, visual, sinyal, tulisan, atau perilaku.
Terdapat empat kategori utama dalam melakukan komunikasi, diantaranya ialah tertulis, visual, verbal, dan nonverbal.
Salah satu bentuk komunikasi nonverbal ialah Bahasa isyarat yang merupakan bahasa utama dan merupakan salah satu bentuk komunikasi yang paling penting bagi Komunikasi yang terjalin antara masyarakat dengar dengan penyandang tunarungu tidak dapat dilakukan secara mudah, perlu adanya kesepakatan makna terhadap bahasa isyarat yang digunakan pada saat berkomunikasi, hal tersebut bertujuan agar tidak terjadi kesalahan dalam menafsirkan informasi yang disampaikan.
Dari masalah yang muncul, penulis menawarkan solusi yaitu dengan mengembangkan aplikasi penerjemah seperti google translate untuk memfasilitasi komunikasi antar individu yang menggunakan bahasa yang berbeda.
Dengan menggunakan aplikasi ini, individu dapat memasukkan pesan asli yang disampaikan dan nantinya aplikasi akan menerjemahkan ke dalam memungkinkan pemahaman dan komunikasi yang lebih baik.
How to cite: Last name.
First name.
Article Edcomtech: Jurnal Kajian Teknologi Pendidikan.
Vol.
, pageAepage.
https://doi.
org/10.
17977/um039x A The Author.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.
0 International License Kata Kunci: Komunikasi.
Teknologi.
Tunarungu.
Artificial Intelligence Abstract Communication involves conveying information through the exchange of thoughts, messages, or information via speech, visuals, signals, writing, or behavior.
Four central communication categories include written, visual, verbal, and Among those categories, sign language is a crucial form of nonverbal communication, serving as the primary language Communication between hearing individuals and those with hearing issues is not always Accordingly, it is essential to establish a clear understanding of the sign language to avoid misinterpretation of the conveyed information.
To address these challenges, this study designs a translator application, similar to Google Translate, that facilitates communication between individuals who speak different sign languages.
By using this application, individuals can input the original message, then the application will translate the message into the desired language, thereby Keywords: Communication, technology, hearing impairment.
Artificial Intelligence AU
INTRODUCTION
Communication is the delivery of information through the exchange of ideas, messages, or signals via speech, visuals, writing, or behavior (Ginting, 2.
As a social being, human movements and behavior serve as forms of communication, thereby, communication is an inseparable aspect of everyday life (Sari, 2.
Research from Desi Damayani describes communication as a crucial human activity.
Meanwhile, effective communication requires feedback in order to achieve goals in both organizational and human life.
Additionally, communication necessitates the effort to share, inform, converse, exchange ideas, relate, and establish friendships (Pohan.
Desi Damayani, 2.
There are four primary types of communication encompassing written, visual, verbal, and nonverbal communication.
Written communication involves the use of symbols, such as letters and numbers, through writing, typing, or printing to convey information.
Meanwhile, visual communication facilitates communication through photos, art, drawings, charts, sketches, and graphs to convey information.
Among those types of communication, verbal communication serves as the most commonly utilized communication in society, such as in conducting presentations, video conferences, telephone calls, and meetings.
Nonverbal communication, on the other hand, utilizes body language, facial expressions, and gestures to communicate information to others (Fisipol, 2.
For non-verbal communication, sign language has been one of the media (PMPK, 2.
Sign language is a critical and essential mode of communication, particularly for individuals with hearing impairments (Sari et al.
, 2.
This language entails a complex system Nasha Hikmatia A, .
, as it employs a combination of lip movements, body language, and hand gestures to convey intentions and information (Alfikri et al.
, 2.
However, the majority of Indonesian lack understanding and willingness to learn sign language, leading to communication limitations when interacting with individuals who have hearing issues (Alfikri et al.
, 2.
Additionally.
Sari notes that sign language is not universally understood or utilized, while the accessibility of information for individuals with hearing disabilities is not always available in every environment (Sari et al.
, 2.
Communication between people with hearing impairment and hearing communities can be challenging without agreement on the meaning of the sign language being used.
This agreement aims to prevent errors in interpreting conveyed information (Kissya, 2.
Following the progression of the era, the use of technology devices has accelerated Accordingly, this technological advancement may also facilitate ease of access to communication using sign language (E.
Nasha Hikmatia A, 2.
For that purpose, a study has formulated a sign language translator application using the Android platform.
The project involved creating a Tensorflow Lite model for the translator and implementing the Convolution Neural Network (CNN) method using 1820 American Sign Language (ASL) Objective evaluations were utilized throughout the research process.
However, in this research, there was a problem of decreasing data accuracy by 73%, because of misprediction on several sign languages (Alfikri et al.
, 2.
Similar research has been conducted by Nasha and Ihsan, who successfully created a real-time voice translator application for Indonesian Sign Language (BISINDO) using the MobilenetV2 architecture.
However, their research did not incorporate a sufficient number of images with diverse backgrounds and Long Short Term Memory (LSTM) in the model.
As a result, the model was unable to detect dynamic cues, which could have led to increased accuracy of the classification (Nasha Hikmatia A, 2.
Another study designed an application to recognize and translate the Indonesian Sign Language System (SIBI) using a leap motion and Hidden Markov Models.
However, the research reported an insufficient ability to recognize every given movement (Deasy, 2.
Furthermore, a study performed by Nasri formulated a translation system for Indonesian speech sounds to the Indonesian sign system (SIBI).
However, that study required further improvement due to its inability to function effectively in crowded environments and accurately capture variations in user volume and speaking speed (Nasri, 2.
Previous research has identified challenges in developing applications that can accurately translate sign language into text and voice.
Therefore, further study that produces advanced translator applications that effectively interpret all types of sign language rather than a single form, similar to Google Translate, remains necessary.
This application can facilitate communication between individuals who speak different languages.
Therefore, this study designs a language translation application that can bridge the gap between nonverbal communication and spoken language, allowing for improved understanding and FORMULATION OF PROBLEMS This research addresses the limitations of sign language as a means of Sign language mostly relies on a combination of gestures and sounds, which may not be comprehensible for those unfamiliar with the sign language.
While sign language may be a valuable tool for communication, it may not always effectively convey messages, especially for those who are unfamiliar with the language.
THEORY
Definition of Deafness Deafness is a condition where a person is unable to hear something, either totally or partially (Haliza, et al.
, 2.
In addition, hearing disability can be defined as the result of a problem that occurs in an individual's hearing system as a result of a person's perception of acoustic stimuli is less than usual, or unable to perceive it at all (Aigerim et al.
, 2.
Hearing barriers themselves can cause an individual to have minimal language input, resulting in individuals having difficulty communicating verbally (Amka & Mirnawati, 2.
It is also conveyed by Prakash et al.
, .
that hearing loss is an extraordinary situation that can limit a child's ability to communicate verbally.
Hearing loss is considered an invisible and hidden obstacle, and is one of the common sensory disorders globally (Ziadat.
, 2.
According to World Health Organization (WHO) statistics in 2018 show that there are 466 people with hearing loss in the world, and estimates that hearing loss will increase to 630 million, 900 million in 2030, and 2050 (World Health Organization, 2.
Characteristics of the Deaf Language and Speech Language skills are very important to have, because doing so can make an individual able to develop themselves in terms of speaking skills.
In addition, an individual who is skilled in speaking can affect a person's ability to convey ideas (Dewi, et al.
, 2.
The direct impact of deafness is the inhibition of verbal communication, both expressively .
and receptively .
nderstanding the speech of other.
, so that it has difficulty in communicating with the hearing community environment which commonly uses verbal language as a means of communication.
This means that it illustrates that the impact on hearing loss affects a person in language because this can cause problems with community acceptance which has an impact on misunderstanding and treatment (Haliza.
Nur.
Eko Kuntarto, 2.
In addition.
Sabaityte & Davidavicius, .
revealed that deafness in individuals can be quite an obstacle as it can limit perception, knowledge and understanding of grammar.
As a result, it was found that an individual is more likely to face problems in developing their academic skills later in life.
One of the most important problems faced in this regard is the problem of establishing a communication model that can facilitate the language development of deaf individuals and allow them to communicate easily with their families and society, as well as ensure that the entire academic life can be maintained in a better way (Soleymani.
Mahmoodabadi.
, & Nouri, 2.
Emotional and Social Aspects Hearing loss can affect individuals' quality of life including educational, social, and economic aspects (Jaiyeola & Adeyemo, 2018.
Punch, 2.
Hearing loss in adulthood impacts individuals causing social isolation, loneliness, exposure to violence, emotional and psychological disturbances, and difficulties in relationships with partners and children (Frajtag, & Jelinic, 2017.
Mulwafu et al.
, 2.
Most deaf people experience depression due to reduced communication competence.
In addition, deaf people claim to feel a lack of understanding and empathy from the surrounding environment, relatives, and friends, causing psychosocial problems (Holman et al.
, 2.
Al-Khateeb et al.
, .
revealed that the overall level of emotional intelligence of deaf students is low.
In line with this, (Luckner & Movahedazarhouligh, 2.
, also said that poor social-emotional skills can lead to social rejection and an increased risk of developing mental health problems that may continue into SOLUTION The solution to the aforementioned problems is the development of translation applications similar to Google Translate to facilitate communication between individuals who speak different languages.
Analogous to Google Translate, this application serves as a machine translation service that can translate text, speech, images, and videos in real-time from one language to another.
This service will be available as websites and mobile applications, making it easily accessible to users.
Using this translation app, individuals can input a message in their original language and receive a translation in their desired language.
This facilitates improved understanding and communication between individuals who do not speak the same language.
DESIGN OF SYSTEM
This section focuses on the system design that consists of technical and functional details of the translator application for addressing the challenges in communicating using sign language.
Following the aforementioned background and issues, the proposed solution is the design of the translator application.
The detailed system design for this application is illustrated in Figure 1.
Figure 1.
Design of the Application Sign Language Input (Vide.
In this section, the user starts the process by uploading a video containing sign language into the application.
Image and Gesture Recognition:
The designed application uses artificial intelligence technology to recognize images and hand gestures in the video.
This technology aids the app in understanding how users convey messages through their hand gestures.
Feature Processing and Extraction:
The relevant information is extracted from the hand gestures, including special gestures that are part of sign language.
This process helps identify patterns and key elements required for Sign Language Modelling:
The application builds a sign language model based on the extracted information.
The model tries to interpret the message from the identified hand gestures.
Machine Learning:
The sign language model is automatically updated and adjusted through machine learning.
This allows the app to become more accurate and adaptive over time.
Text to Voice Conversion:
Once the sign language model identifies the message, a translated text is generated.
This text is then converted into voice so that it can be heard.
Voice Output:
The voice generated from the translated text is delivered to the user.
This allows people who do not understand sign language to understand the message conveyed by the user.
Following this flow, the app serves as an intermediary that facilitates effective communication between individuals who use sign language and those who are not familiar with the language.
This technology provides the possibility to create a more inclusive communicative environment.
CONCLUSION
Communication using mere sign language is insufficient to effectively convey messages, especially when communicating with individuals who are not familiar with the use of sign language.
To overcome this problem, a solution that bridges the gap between gestures and spoken language is necessary.
Therefore, this study offers a design of translator applications similar to Google Translate to facilitate communication between individuals who use different languages.
By using this application, individuals can input their original Then, the application will translate it into the desired language, thus enabling better understanding and communication.
REFERENCES