El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 Volume 24. Nomor 2. Oktober 2025 P-ISSN 2087-3638. E-ISSN 2655-7746 https://ftkjournal-uinmataram. id/index. php/eltsaqafah ANALYZING THE IMPROVEMENT OF STUDENTS' ARABIC LANGUAGE FLUENCY AND PRONUNCIATION USING TALKPAL AI Siti Durotun Naseha University of Hasyim AsyAoari Tebuireng. Indonesia corresponding author: sitinaseha@unhasy. Abdul Ghofur University of Hasyim AsyAoari Tebuireng. Indonesia abdulghofur@unhasy. Iftitaahul Mufarrihah University of Hasyim AsyAoari Tebuireng. Indonesia iftitaahulmufarrihah@unhasy. Rifqi Nur Ahmada University of Hasyim AsyAoari Tebuireng. Indonesia rfqahmada@gmail. Mochammad Fauzan Azizi University of Hasyim AsyAoari Tebuireng. Indonesia fauzanazizi@mhs. Article History Submitted: 11 Sep 2025. Revised: 27 Sep 2025. Accepted: 30 Sep 2025 DOI 10. 20414/tsaqafah. Abstract Mastering fluency and pronunciation in Arabic poses a significant challenge for students, primarily due to a lack of practice and immediate feedback. Artificial intelligence (AI) technology, such as TalkPal AI, offers an interactive solution for practicing conversation. however, its effectiveness needs to be empirically This study aims to analyze the improvement in students' fluency and pronunciation through an intervention using TalkPal AI. The research employed a mixed methods approach with a convergent parallel design. Quantitative data were obtained from oral tests . re-test and post-tes. and analyzed using N-Gain and Paired Sample T-Tests, while qualitative data were collected through open-ended questionnaires and analyzed thematically. The results indicated a significant improvement in both fluency and pronunciation, as demonstrated by a moderate NGain score and the rejection of the null hypothesis (H. in the t-test. The qualitative findings revealed that this improvement was driven by the instant feedback feature, a low-pressure learning environment, and enhanced student confidence. Therefore. TalkPal AI is proven to be an effective learning tool that can comprehensively improve students' speaking competence. Keywords: fluency. TalkPal AI This is an open access article under the CCAeBY-SA license | 253 Analyzing The Improvement A (Naseha. Ghofur. Mufarrihah. Ahmada. Aziz. INTRODUCTION Arabic is one of the international languages that plays a vital role in the fields of religion, education, diplomacy, and commerce. Speaking skill . ahArah al-kalA. is a primary competency that students must master, particularly for those specializing in Arabic studies. However, empirical evidence indicates that many students still struggle to communicate actively, fluently, and with confidence. This is attributed to the lack of interactive conversational practice media, an insufficient language environment, and limited opportunities for intensive speaking practice. The process of learning Arabic in higher education institutions generally remains focused on grammatical rules . awAAoi. and translation, while speaking practice has not yet received optimal emphasis(Lutfiyatun et al. , 2. Consequently, students tend to be more proficient in reading and writing but less fluent in expressing ideas orally. Another challenge is the lecturers' limitation in providing intensive conversational practice to all students equally. Therefore, technological-based learning innovations are necessary to bridge the gap and address the students' need for independent and guided practice in speaking Arabic. Advancements in Artificial Intelligence (AI) technology have unveiled new opportunities in language learning(Rachmayanti & Alatas, 2. One rapidly developing innovation is the utilization of Automatic Speech Recognition (ASR), which enables devices to recognize, assess, and provide automatic feedback on a user's speech. This technology can support students in practicing pronunciation, intonation, and speaking fluency without always relying on instructor TalkPal AI is an AI-based application that has integrated ASR technology into the language learning process(Sakaroni, 2. This application offers an interactive experience through conversation simulations, real-time feedback, and personalized learning tailored to the user's With these features, students have the opportunity to practice speaking Arabic anytime and anywhere, which is expected to significantly enhance their speaking skills. This research aims to analyze the effectiveness of using ASR-based TalkPal AI in improving students' Arabic speaking skills. The study focuses on the extent to which this application can help improve aspects of fluency, pronunciation accuracy, and the ability to structure arguments in conversation. Furthermore, this research also seeks to understand students' responses to their AI technology-based learning experience. The findings of this study are expected to provide both theoretical and practical benefits. Theoretically, it can enrich the body of knowledge on the integration of AI technology, particularly ASR, in Arabic language learning. Practically, it can serve as a reference for lecturers, students, and application developers to optimize technology-based learning media for | 254 This is an open access article under the CCAeBY-SA license El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 enhancing mahArah al-kalAm. Thus, this research is hoped to support the transformation of Arabic language learning towards a more effective and adaptive digital era. LITERATURE REVIEW Mastery of foreign languages, including Arabic, has become an urgent necessity in the context of globalization and international communication, particularly among academics. Students learning Arabic often face significant challenges in achieving adequate fluency and These difficulties stem from the complexity of the Arabic phonological system, which differs considerably from the native language of most Indonesian students, as well as limited opportunities to practice speaking in an authentic and supportive environment. In recent years, rapid advancements in artificial intelligence (AI) technology have introduced a new paradigm in language education(Naseha et al. , 2. AI offers innovative solutions to overcome traditional limitations in teaching speaking skills. One platform leveraging this technology is TalkPal AI, an AI-based application specifically designed to train speaking skills in various languages, including Arabic. This research sets out to analyze the effectiveness of TalkPal AI in enhancing students' fluency and pronunciation. To establish a comprehensive understanding, this theoretical foundation will elaborate on three main pillars: . the Concepts of Fluency and Pronunciation in Arabic Language Learning, . the Theory of Technology-Enhanced Language Learning (TELL), and . Artificial Intelligence Technology in Speaking Instruction, with a specific focus on TalkPal AI. Fluency in speaking a foreign language refers to the ability to produce speech that flows naturally, smoothly, and automatically without unnatural pauses or disruptive hesitation. According to Fillmore's theory . , fluency encompasses several dimensions: the ability to talk fluently for an extended period, the ability to express complex ideas coherently, the ability to use appropriate language in a variety of social contexts, and the ability to produce creative and original utterances. In the context of university students, fluency is measured through indicators such as speech rate . ords per minut. , mean length of run . verage number of words between pause. , and the frequency and duration of pauses. Pronunciation is the ability to produce a language's sounds . , word stress, sentence intonation, and rhythm in a manner that is native-like. The theory of accuracy in pronunciation, as proposed by Celce-Murcia et al. (Bahruddin et al. , 2. , emphasizes that intelligible pronunciation is the primary goal, rather than merely pursuing a perfect accent. Arabic possesses a unique and challenging phonological system, featuring pharyngeal consonants . uch as A Aand A)A, emphatic consonants . ike A A,A A,a, and A)A, and distinctions in vowel length . hort vowel vs. long vowe. that can alter the meaning of a word. Errors in pronouncing these sounds This is an open access article under the CCAeBY-SA license | 255 Analyzing The Improvement A (Naseha. Ghofur. Mufarrihah. Ahmada. Aziz. can lead to significant miscommunication. The relationship between fluency and pronunciation is a mutually beneficial symbiosis. Accurate pronunciation boosts a speaker's confidence, which in turn fosters greater speaking fluency. Conversely, good fluency allows the speaker to focus on refining other linguistic aspects, including pronunciation. Segalowitz's(Ulhaq & Lubis, 2. (Muslimin et al. , 2. Automaticity Theory posits that through consistent and repeated practice, speech production processes that initially require significant cognitive effort can become automatic. It is this automation process that is key to improving both fluency and The Technology-Enhanced Language Learning (TELL) approach represents an evolution from Computer-Assisted Language Learning (CALL). TELL is grounded in constructivist theory, pioneered by Piaget and Vygotsky, which emphasizes that learning is an active process where learners construct their own knowledge through interaction with their environment, including the digital environment(Aryani & Wahyuni, 2. (SyafeAoi, 2. Vygotsky's theory of the Zone of Proximal Development (ZPD) is highly relevant here, as technology can act as a "more knowledgeable other" providing scaffolding or support to help learners achieve ability levels just beyond their current grasp. In the context of speaking skills. TELL offers several theoretical advantages over the traditional classroom: Availability and Accessibility: Platforms like TalkPal AI are accessible anytime and anywhere, allowing students to practice beyond classroom hours, free from spatial and temporal constraints. A Low-Anxiety Practice Environment: Krashen's . Affective Filter Hypothesis states that anxiety can impede language acquisition. Speaking to an AI can reduce the fear of making mistakes, being embarrassed, or judged by peers or instructors, thereby lowering the affective filter and facilitating more effective learning. Immediate and Objective Feedback: This is a key principle of behaviorism. systems can analyze user speech and provide real-time correction of pronunciation errors. This immediate feedback reinforces correct responses and accelerates the learning Personalization: TELL can adapt materials and exercises based on the user's individual proficiency level and progress, applying the principles of personalized learning for optimal results. Artificial Intelligence Technology in Speaking Instruction: A Focus on TalkPal AI | 256 This is an open access article under the CCAeBY-SA license El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 TalkPal AI represents a practical embodiment of the convergence between language learning theory and advancements in AI technology, particularly in the fields of Natural Language Processing (NLP) and Speech Processing(Approach, n. )(Efendi Hidayatullah, 2. Automatic Speech Recognition (ASR): A core technology that enables TalkPal AI to understand and transcribe user speech. ASR works by breaking down audio signals into phonetic units, matching them against acoustic models trained on vast datasets of spoken language, and generating corresponding text. The accuracy of modern ASR is remarkably high, allowing it to detect even minor pronunciation deviations. In a research context. ASR serves as a diagnostic tool to identify specific errors in students' pronunciation. Text-to-Speech (TTS): This technology allows the AI to generate human-like speech. TalkPal AI utilizes TTS to provide users with correct and natural pronunciation models. Students can listen to accurate pronunciations of words or phrases before attempting them themselves, aligning with Bandura's social learning theory on modeling. Natural Language Processing (NLP) and Natural Language Generation (NLG): NLP enables the system to comprehend user input . oth text and speec. , while NLG allows it to generate contextually relevant and meaningful responses. This combination creates an interactive conversational simulation that closely mimics human dialogue. Such dialogic interaction offers an authentic context for students to practice fluency, moving beyond isolated word pronunciation. Speech Analysis and Feedback Provision: This is TalkPal AIAos distinguishing feature. The system not only recognizes speech but also analyzes it based on parameters such as: Phonetic Accuracy: Comparing the userAos voice frequency spectrum with native speaker models to evaluate the accuracy of each phonemeAos pronunciation. Fluency: Analyzing speech rate and pauses to provide suggestions for improving . Intonation and Pitch: Assessing prosodic features like rise and fall in tone within Based on this analysis. TalkPal AI offers scores and specific feedback. This process reflects the application of formative feedback theory, which is crucial for skill Based on the theoretical foundation previously outlined, the conceptual framework of this research is as follows: The utilization of TalkPal AI, which is equipped with ASR (Automatic Speech Recognitio. TTS (Text-to-Speec. , and NLP (Natural Language Processin. technologies, is hypothesized to create an effective, personalized, and low-risk learning environment for students(Sarif & AR, 2. Intensive conversational interaction with the AI and This is an open access article under the CCAeBY-SA license | 257 Analyzing The Improvement A (Naseha. Ghofur. Mufarrihah. Ahmada. Aziz. the receipt of immediate, objective feedback on pronunciation errors are believed to strengthen the process of speech production automaticity. This enhancement in automaticity is ultimately expected to manifest as a significant improvement in students' Arabic language fluencyAi characterized by reduced pauses and increased speech rateAiand pronunciation accuracyAi marked by a decrease in phonetic errors and an increased resemblance to native speaker models. METHOD This study employs a mixed methods approach, integrating both quantitative and qualitative analyses. It utilizes a convergent parallel design, which is aimed at achieving triangulation in the research. In this convergent design, the integration of both quantitative and qualitative results will provide a more comprehensive understanding than either method could yield independently(Ishtiaq, 2. The population of this study consists of students from the Arabic Language Education study program at UNHASY Tebuireng who are enrolled in a conversation practice course. The sample will be selected using purposive sampling, specifically choosing students at a certain proficiency level who are willing to participate in the intervention using TalkPal AI over a defined period. Quantitative data measurement is conducted to examine the effectiveness of TalkPal AI in improving fluency and pronunciation. The instrument used is an oral test administered as pre-test and post-test. The test is designed to include tasks such as reading short texts, describing images, and structured conversations. Fluency scores are measured based on indicators such as speech rate . ords per minut. , number of pauses, and length of utterance. Pronunciation scores are assessed based on the accuracy of articulating distinct Arabic phonemes . uch as /u/, /Ao/, /gh/), word stress, and intonation. To analyze the score improvement from pre-test to post-test, the NGain (Normalized Gai. formula will be applied. The N-Gain formula provides an overview of the intervention's effectiveness relative to the maximum possible achievement. The calculated NGain results will then be categorized as high, medium, or low. The relevant hypothesis testing technique for this quantitative data is the Paired Sample T-Test. This test is used to determine whether there is a significant difference between pre-test and post-test scores. The null hypothesis (H. states that there is no significant difference, while the alternative hypothesis (H. states that there is a significant difference. Significance is tested at the = 0. 05 level. If the significance value . -valu. is < 0. 05, then H0 is rejected, indicating that the use of TalkPal AI significantly improves fluency and pronunciation. Meanwhile, qualitative data is collected to explore students' perceptions, experiences, and challenges while using TalkPal AI. The primary instrument used is an open-ended questionnaire distributed after the post-test. The questionnaire questions are designed to explore aspects such | 258 This is an open access article under the CCAeBY-SA license El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 as the ease of using the application, the usefulness of the feedback feature for pronunciation, its impact on confidence, as well as any technical or non-technical constraints encountered. Data from this qualitative questionnaire are then analyzed using thematic analysis (Ismail et al. , 2. (Asrin. Handoyo. Hartati. Kuhu marlyn Maisje. Riyadi Sugeng. Wahyuningsih Dyah. Supadi. The process includes data transcription, repeated reading, coding, theme identification, and conclusion drawing. The findings from this qualitative analysis will provide context and explanation for the numerical results obtained from the N-Gain calculation and t-test. For example, if quantitative results show significant improvement, qualitative data may reveal the reasons behind this success, such as the interactive AI features. Conversely, if the improvement is relatively low, qualitative data can identify inhibiting factors, such as unstable internet connection or lack of understanding in using the application. By combining both approaches, this study is expected to provide robust and holistic evidence regarding the effectiveness of TalkPal AI as an aid in learning Arabic. DISCUSSION In the implementation of this TalkPal AI-based Arabic speaking learning, the researcher measured its effectiveness using the normalized gain or N-gain score. This metric is used to determine the effectiveness of a specific method or treatment in a one-group pretest-posttest design . n experimental or pre-experimental desig. or in research utilizing a control group . uasi-experimental or true experimental desig. (Wajdi et al. , 2. The N-gain score test is performed by calculating the difference between the pretest score . he test before the application of the specific method/treatmen. and the posttest score . he test after its applicatio. calculating this difference, or gain score, we can ascertain whether the application of a specific method can be deemed effective or not. The formula for calculating the N-gain score is: ycA yaycaycnycu = ycIycoycuyc ycEycuycycycyceycyc Oe ycIycoycuyc ycEycyceycyceycyc ycIycoycuyc yayccyceycayco Oe ycIycoycuyc ycEycyceycyceycyc The categorization of N-gain score results can be determined based on either the raw Ngain value or its percentage equivalent (%). The classification categories for the obtained N-gain are presented in the following table: Tabel 1 Classification categories for the obtained N-gain N-Gain Categories g>0,7 Tinggi 0,3OgO0,7 Sedang g<0,3 Rendah Sumber: Melzer dalam Syahfitri . 8, hlm. This is an open access article under the CCAeBY-SA license | 259 Analyzing The Improvement A (Naseha. Ghofur. Mufarrihah. Ahmada. Aziz. Meanwhile, the classification of N-gain score categories in percent (%) can be based on the table below: Tabel 2 Interpretation Categories of N-Gain Effectiveness Prosentase % Categories <40 Ineffective Less effective Sufficiently effective >76 Effective Sumber: Hake. R . The categorization used in this research is the classification of N-gain scores into percentage ranges, following the criteria presented in Table 2. This research aims to measure the effectiveness of the Automatic Speech Recognitionbased TalkPal AI application in improving students' speaking skills in learning Arabic The survey results yielded the following pretest and posttest scores. No. | 260 Tabel 3 Pre-test and Post-test Results Pre Test Post Test 65,00 76,00 68,75 79,50 56,00 72,00 65,00 75,25 69,75 79,50 70,50 81,50 69,25 76,50 70,75 76,75 70,00 76,75 70,00 77,25 68,75 79,75 69,00 77,50 72,25 82,00 70,00 82,25 61,25 77,00 65,25 78,25 69,75 77,25 64,25 77,00 67,75 77,50 68,50 82,50 67,50 77,50 63,75 79,00 71,50 78,25 72,25 78,75 68,25 77,25 Sumber: Hasil Surve diolah . This is an open access article under the CCAeBY-SA license El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 The survey results were then analyzed by calculating the N-gain score, yielding the following results: Tabel 4 N-Gain Score (%) No. N-Gain Score 39,47 40,44 41,18 47,86 50,42 38,17 28,69 47,46 45,97 49,56 49,62 47,12 40,16 37,76 40,48 44,07 40,82 40,50 43,44 47,46 39,83 54,08 27,61 63,73 49,56 Mean 43,8172 Min. 27,61 Max. 63,73 Sumber : Hasil olah data SPSS, . Based on the N-gain score test results above, the average N-gain score for UNHASY Tebuireng Jombang is 43. 8172 or 43. 8172%, which falls into the "less effective" category. The Ngain scores range from a minimum of 27. 61% to a maximum of 63. Thus, it can be concluded that the use of the Automatic Recognition-based TalkPal AI application is less effective in improving students' speaking skills in learning Arabic conversation among students at UNHASY Tebuireng Jombang. The next step to determine whether there is an influence of using the Automatic Recognition-based TalkPal AI application on improving students' speaking skills in Arabic conversation learning at UNHASY Tebuireng Jombang is to conduct a paired sample t-test as follows. This is an open access article under the CCAeBY-SA license | 261 Analyzing The Improvement A (Naseha. Ghofur. Mufarrihah. Ahmada. Aziz. Results of the Paired Sample T-Test at UNHASY Tebuireng Jombang Tabel 5 Paired Samples Statistics Pre_UNHASY Pair 1 Post_UNHASY Mean Std. Deviation Std. Error Mean Sumber : Hasil olah data SPSS . Based on the SPSS output above, it is shown that the Pre-Test scores have a mean of Meanwhile, the Post-Test scores obtained a higher mean of 82. The number of respondents or students used as the research sample was 25. The Standard Deviation value for the Pre-Test is 2. 90384 and for the Post-Test is 3. Finally, the Standard Error Mean for the Pre-Test is 0. 58077 and for the Post-Test is 0. Since the Pre-Test mean . is less than the Post-Test mean . , this descriptively indicates a difference in the average learning outcomes between the Pre-Test and the Post-Test. To further prove whether this difference is statistically significant or not, we must interpret the results of the paired sample t-test presented in the "Paired Samples Test" output table. Tabel 6 Paired Samples Correlations Pair 1 Pre_UNHASY & Post_UNHASY Correlation Sig. Sumber : Hasil olah data SPSS . The output above displays the results of a correlation test examining the relationship between the two datasets, specifically the relationship between the Pre-Test and Post-Test Based on the output, the correlation coefficient is 0. 860, with a significance value (Sig. Since the Sig. value of 0. 000 is less than the probability threshold of 0. 05, it can be concluded that there is a statistically significant relationship between the Pre-Test and Post-Test | 262 This is an open access article under the CCAeBY-SA license El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 Tabel 7 Paired Samples Test Mean Pair 1 Pre_UNHASY Post_UNHASY -13. Paired Differences 95% Confidence Interval of the Std. Difference Deviation Std. Error Mean Lower Upper Df Sig. -taile. Sumber : Hasil olah SPSS . Hypotheses: HCA (Null Hypothesi. : There is no significant difference between the mean scores of the pretest and posttest results, indicating that the use of the TalkPal AI application with Automatic Speech Recognition has no effect on improving students' Arabic speaking skills in conversation learning at UNHASY Tebuireng Jombang. HCa (Alternative Hypothesi. : There is a significant difference between the mean scores of the pretest and posttest results, indicating that the use of the TalkPal AI application with Automatic Speech Recognition has a significant effect on improving students' Arabic speaking skills in conversation learning at UNHASY Tebuireng Jombang. Decision Rule: If the Sig. -taile. value < 0. 05 or the t-statistic > t-critical value, then HCA is rejected and HCa is accepted. Conversely, if the Sig. -taile. value > 0. 05 or the t-statistic < t-critical value, then HCA is accepted and HCa is rejected. Based on the "Paired Samples Test" output table above, the Sig. -taile. value is 0. H0 is rejected and Ha is accepted. This leads to the conclusion that there is a significant difference between the mean scores of the Pre-Test and Post-Test learning outcomes. This indicates that the use of the TalkPal AI application, powered by Automatic Speech Recognition, has a significant effect on improving students' speaking skills in Arabic conversation learning at UNHASY Tebuireng Jombang. Based on the analysis, several strategic recommendations can be implemented to enhance the effectiveness of TalkPal AI in supporting Arabic language learning. These recommendations span technical, pedagogical, and user experience aspects. From a learning feature perspective, it is suggested that TalkPal AI provide conversation scripts relevant to various real-world contexts, including emergency situations, daily activities, and professional scenarios such as business negotiations and political discussions. Additionally, the development of exam simulations with authentic conditions and the provision This is an open access article under the CCAeBY-SA license | 263 Analyzing The Improvement A (Naseha. Ghofur. Mufarrihah. Ahmada. Aziz. of Arabic handwriting practice modules with instant visual feedback are considered essential for strengthening users' practical skills. In terms of enhancing interactive experiences. TalkPal AI is expected to provide fluency scores, daily progress reports, and clear metrics related to vocabulary mastery. The system should also offer difficulty levels, game modes, and leaderboards to make the learning process more The AI's ability to recognize accent errors, intonation issues, and even Tajweed errors in Quranic recitation is deemed important for enriching its learning functions. Furthermore, from a cultural and contextual approach perspective. TalkPal AI should provide explanations of the cultural backgrounds behind jokes, idioms, and social etiquette. Offering a wider selection of dialects . Sudanese. Iraqi, and Yemen. is also viewed as a strategy to enhance the authenticity of the learning experience. Support in the form of news summaries, business case studies, and authentic multimedia content . uch as social media posts or videos by native speaker. can further broaden users' perspectives. In terms of emotional engagement and personal adaptation. TalkPal AI is recommended to incorporate user sentiment analysis, tailor material recommendations, and consistently remember individual preferences or interests. To this end, adding features such as automated reminders, mood-based music recommendations, and daily creative writing prompts can help sustain longterm learning engagement. CLOSSING Overall, the provided suggestions emphasize the importance of developing TalkPal AI not merely as a technical tool, but also as a more adaptive, contextual, and user-experience-oriented learning partner. The implementation of these recommendations is expected to position TalkPal AI as a more comprehensive, interactive, and relevant Arabic language learning platform that is closely aligned with the real-world needs of its users. The author would like to express gratitude to all parties involved in the writing of this article. This article is the result of research and is funded by a grant from the BIMA Ministry of Education. Culture. Research, and Technology of the Republic of Indonesia for the year 2025. | 264 This is an open access article under the CCAeBY-SA license El-Tsaqafah: Jurnal Jurusan PBA. Vol. No. 2, 2025 BIBLIOGRAPHY