http://jurnal.
fkip-uwgm.
id/index.
php/Script P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
T h e Q u a l i t y of S c i e n ti fi c P a p e r s of Un d e r g r a d u a t e S t u d e n t s i n Re g i o n a l L a n g u a g e a n d Li t e r a tu r e S t u d i e s Le c t u r e s B e f o r e a n d Af t e r Us i n g Ch a t G P T a s a W r i ti n g To o l Ari Wulandari1.
Arif Budiman2.
Tri Ratna Herawati3.
Primasari Wahyuni4.
Diaz Danu Setyawan5 Universitas PGRI Yogyakarta.
Indonesia1,3,4,5 Universitas Islam Negeri Sunan Kalijaga.
Yogyakarta.
Indonesia 2 Email Correspondence: kinoysan@gmail.
Abstract Background:
This study addresses the growing use of ChatGPT in academic writing.
Grounded in constructivism, the study investigates changes in the quality of scientific papers produced by undergraduate students in the Regional Language and Literature Studies course at University AuAAy before and after the use of ChatGPT as a writing support Methodology:
A quasi-experimental pretestAeposttest design using document analysis was applied.
The data consisted of 30 student papers written before and after ChatGPT use.
Assessment covered five aspects: ideas, organization, grammar, diction, and spelling.
Findings:
The findings indicate substantial improvements in studentsAo scientific papers after the use of ChatGPT.
The mean writing score increased from 77.
9 before ChatGPT-assisted revision to 89.
0 after revision, and 27 of the 30 papers improved to the excellent category.
The strongest improvements were observed in organization, grammar, diction, and spelling, whereas gains in idea development were comparatively more modest.
Conclusion:
The findings suggest that ChatGPT-assisted revision can enhance the technical quality of studentsAo scientific However, lecturer guidance remains necessary to support critical thinking, independent analysis, and the responsible use of generative AI in academic writing.
Originality:
This study contributes empirical evidence on changes in the quality of studentsAo scientific writing associated with ChatGPT-assisted revision within the context an undergraduate language and literature course.
Keywords DOI ChatGPT.
Scientific Papers.
Undergraduate Students.
Regional Language and Literature Studies.
Quality of Academic Writing 24903/sj.
Received February 2026 Accepted July 2026 Published August 2026 Wulandari.
Budiman.
Herawati.
Wahyuni.
, & Setyawan.
The quality of scientific papers of undergraduate students in regional language and literature studies lectures before and after using ChatGPT as a writing tool.
Script Journal: Journal of Linguistics and English Teaching, 11.
, 340Ae417.
https://doi.
org/10.
24903/sj.
How to cite this article (APA) Copyright Notice Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.
International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan INTRODUCTION The rapid development of generative artificial intelligence (AI) has transformed academic writing practices in higher education.
Among the most widely adopted AI tools.
ChatGPT assists users in generating, revising, and refining academic texts with increasing linguistic accuracy and structural coherence.
Recent studies suggest that AI-assisted writing can enhance writing efficiency, language quality, and idea organization while simultaneously raising concerns regarding academic integrity, originality, and studentsAo critical engagement with the writing process (Dwivedi et al.
, 2023.
Kasneci et al.
, 2.
These opportunities and challenges have stimulated growing interest in understanding how generative AI influences the quality of studentsAo academic writing.
Academic writing is a fundamental competency in higher education because it requires students to construct logical arguments, synthesize evidence, and communicate scholarly ideas using appropriate academic conventions.
Developing high-quality academic writing is a complex process that extends beyond linguistic accuracy, requiring critical reading, information evaluation, and coherent argument construction (Hyland, 2019.
Lea & Street.
Recent research has shown that many university students continue to experience difficulties in formulating research problems, organizing ideas, integrating evidence, and maintaining coherence throughout scientific papers, particularly when engaging with discipline-specific academic genres (Wingate, 2012.
Shi et al.
, 2.
Furthermore, studentsAo academic writing quality is strongly associated with their academic literacy and information literacy, suggesting that effective writing depends not only on language proficiency but also on the ability to locate, evaluate, synthesize, and communicate knowledge critically (Shi et al.
Lim et al.
, 2.
Recent empirical research has increasingly examined the educational use of ChatGPT and other generative AI tools.
Existing evidence consistently indicates that AI-assisted writing can improve observable aspects of academic texts, including grammatical accuracy, lexical choice, organizational coherence, and linguistic fluency (Kasneci et al.
, 2023.
Lo, 2.
the same time, scholars caution that these benefits are accompanied by important pedagogical and ethical challenges.
Effective integration of generative AI requires appropriate lecturer supervision, transparent disclosure of AI assistance, and instructional strategies that encourage students to remain actively engaged in reasoning, source evaluation, and knowledge construction rather than relying excessively on AI-generated content (Dwivedi et al.
, 2023.
Tlili et al.
, 2.
Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
Despite the rapid growth of research on generative AI in education, several important gaps remain.
First, existing studies have predominantly examined studentsAo perceptions, acceptance, or general educational impacts of ChatGPT rather than evaluating changes in the quality of scientific writing using structured rubric-based assessments before and after AIassisted writing (Lo, 2023.
Zhai, 2.
Consequently, empirical evidence linking ChatGPT use to specific dimensions of writing quality remains limited.
Second, prior research has generally reported overall improvements in writing performance without distinguishing whether these gains occur primarily in lower-order writing features .
, grammar, diction, spelling, and organizatio.
or in higher-order dimensions such as idea development, argumentation, and critical analysis (Kasneci et al.
, 2023.
Farrokhnia et al.
, 2.
This distinction is pedagogically important because generative AI may be more effective in supporting linguistic refinement than in fostering deeper conceptual reasoning.
Addressing these gaps can provide a more nuanced understanding of which aspects of academic writing benefit most from ChatGPT-assisted revision and can inform evidence-based guidelines for integrating generative AI into higher education writing instruction.
Accordingly, this study addresses the following research question: How does ChatGPTassisted revision influence the quality of undergraduate studentsAo scientific papers across the dimensions of ideas, organization, grammar, diction, and spelling in a Regional Language and Literature Studies course at an Indonesian language teacher education program in Yogyakarta? This study contributes to the growing literature on AI-assisted academic writing in three Empirically, it provides rubric-based evidence of changes in studentsAo scientific writing quality before and after ChatGPT-assisted revision.
Conceptually, it distinguishes improvements in surface-level writing quality from those related to idea development, thereby offering a more nuanced understanding of AI-supported academic writing.
Practically, the findings provide evidence-based recommendations for integrating generative AI into Indonesian language teacher education while maintaining lecturer guidance, academic integrity, and studentsAo critical engagement throughout the writing process.
LITERATURE REVIEW
1 ChatGPT and Academic Writing Quality Generative artificial intelligence (AI), particularly ChatGPT, has rapidly emerged as a writing support tool in higher education.
Rather than functioning solely as a text generator.
ChatGPT assists students throughout multiple stages of the writing process, including planning, drafting, revising, and refining academic texts.
Recent empirical studies consistently The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan demonstrate that AI-assisted writing can improve grammatical accuracy, lexical choice, organizational coherence, and overall writing fluency, enabling students to produce more polished academic texts with greater efficiency (Dwivedi et al.
, 2023.
Kasneci et al.
, 2.
Nevertheless, these studies also emphasize that the educational value of ChatGPT depends on responsible pedagogical integration.
Excessive reliance on AI may reduce students' active engagement in reasoning, source evaluation, originality, and independent knowledge Consequently, current scholarship increasingly positions ChatGPT as a learning support tool that complements, rather than replaces, students' intellectual work and academic decision-making (Farrokhnia et al.
, 2024.
Tlili et al.
, 2.
In this study, the quality of academic writing is conceptualized as studentsAo ability to produce scientific papers that communicate ideas clearly, logically, and according to academic Consistent with commonly used analytical writing rubrics, writing quality is assessed through five dimensions: .
idea development, .
organization, .
grammar, .
diction, and .
These dimensions represent both higher-order writing skills, such as developing and organizing ideas, and lower-order language skills, including grammatical accuracy and language mechanics.
This distinction provides a more comprehensive evaluation of the aspects of writing that may benefit from ChatGPT-assisted revision.
2 Constructivist Perspective on AI-Assisted Writing This study is informed by constructivist learning theory, which views learning as an active process of knowledge construction through interaction, reflection, and problem solving (Vygotsky, 1978.
Bruner, 1.
Within this perspective.
ChatGPT functions as a form of digital scaffolding that supports students during the writing process by providing suggestions for language use, organization, and revision (Kasneci et al.
, 2023.
Tlili et al.
, 2.
Nevertheless, constructivism emphasizes that meaningful learning occurs only when students critically evaluate, adapt, and integrate these suggestions into their own reasoning (Wood et , 1976.
Reiser, 2.
Accordingly.
ChatGPT is positioned in this study as a cognitive support tool that facilitates writing development without replacing students' analytical and creative responsibilities (Dwivedi et al.
, 2.
3 Academic Literacy and AI-Assisted Writing Academic literacy encompasses the knowledge and skills required to understand, evaluate, and produce scholarly texts within academic communities (Lea & Street, 1998.
Wingate, 2.
Academic writing quality represents one important manifestation of academic Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
literacy because it reflects studentsAo ability to communicate ideas using accepted academic conventions (Hyland, 2.
However, the present study does not directly measure academic literacy as a multidimensional construct (Lea & Street, 1.
Instead, it evaluates changes in studentsAo academic writing quality through an analytic writing rubric (Nurgiyantoro, 2.
Therefore, academic literacy serves as the broader conceptual context, whereas writing quality constitutes the empirical focus of the study (Wingate, 2012.
Hyland, 2.
4 Previous Studies and State of the Art Recent research has consistently demonstrated that ChatGPT and other generative AI tools can support studentsAo academic writing in multiple ways.
Across different educational contexts.
AI-assisted writing has been associated with improvements in grammatical accuracy, lexical choice, text organization, and overall writing fluency (Dwivedi et al.
, 2023.
Kasneci et , 2023.
Lo, 2023.
Farrokhnia et al.
, 2.
These findings suggest that generative AI can reduce studentsAo cognitive load during drafting and revision, allowing them to produce more coherent, linguistically accurate, and well-organized academic texts (Tlili et al.
, 2023.
Farrokhnia et al.
, 2.
Despite these benefits, previous studies also emphasize important limitations.
While ChatGPT effectively supports surface-level aspects of academic writing, concerns remain regarding studentsAo overreliance on AI, reduced originality, diminished critical engagement, and potential threats to academic integrity (Dwivedi et al.
, 2023.
Bender et al.
, 2021.
Farrokhnia et al.
, 2.
Consequently, many scholars argue that ChatGPT should function as a learning support tool under appropriate lecturer supervision rather than as a substitute for studentsAo independent reasoning, knowledge construction, and academic decision-making (Kasneci et al.
, 2023.
Tlili et al.
, 2.
Another stream of research has examined factors influencing studentsAo academic writing beyond AI use.
Studies consistently show that the quality of academic writing is shaped by studentsAo academic literacy, information literacy, and disciplinary writing knowledge rather than linguistic proficiency alone (Lea & Street, 1998.
Wingate, 2012.
Lim et al.
, 2.
Research in multilingual contexts further demonstrates that studentsAo first-language (L.
resources influence rhetorical organization, discourse patterns, and linguistic choices in academic writing, particularly when composing in an additional language (Elosua & Lasagabaster, 2025.
Hyland, 2.
Collectively, these studies suggest that academic writing quality reflects the interaction of cognitive, linguistic, and educational factors that extend beyond technological assistance alone.
The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan Although previous studies generally agree that ChatGPT can improve the linguistic quality of studentsAo academic writing (Kasneci et al.
, 2023.
Lo, 2.
, important issues remain Existing research has predominantly evaluated studentsAo perceptions of AI, writing productivity, ethical considerations, or overall writing performance (Dwivedi et al.
, 2023.
Farrokhnia et al.
, 2.
Comparatively little attention has been given to identifying which specific dimensions of academic writing benefit most from ChatGPT-assisted revision.
particular, few studies distinguish improvements in higher-order writing skills, such as idea development, from improvements in lower-order writing skills, including grammar, diction, spelling, and organization, using an analytic writing rubric (Kasneci et al.
, 2023.
Tlili et al.
This unresolved issue provides an important direction for further empirical 5 Research Gap and Study Contribution The existing body of literature provides substantial evidence that ChatGPT can support studentsAo academic writing by improving linguistic accuracy, writing organization, and revision efficiency (Kasneci et al.
, 2023.
Lo, 2023.
Dwivedi et al.
, 2.
However, three important gaps remain.
First, most previous studies have examined studentsAo perceptions of ChatGPT, ethical issues, or overall writing performance rather than evaluating specific dimensions of writing quality through a systematic analytic rubric (Farrokhnia et al.
, 2024.
Tlili et al.
, 2.
Second, although improvements in grammar and language mechanics are widely reported, relatively little empirical evidence distinguishes the effects of ChatGPT on higherorder writing skills, such as idea development, from its effects on lower-order writing skills, including organization, diction, grammar, and spelling (Kasneci et al.
, 2023.
Lo, 2.
Third, evidence from Indonesian language teacher education, particularly within the context of Regional Language and Literature Studies, remains limited.
Addressing these conceptual, methodological, and contextual gaps is important for understanding not only whether ChatGPT improves studentsAo scientific writing, but also which aspects of writing benefit most from AI-assisted revision (Kasneci et al.
, 2023.
Farrokhnia et , 2.
Such evidence is needed to inform pedagogical decisions regarding the appropriate integration of generative AI into academic writing instruction while maintaining studentsAo independent reasoning and academic integrity (Dwivedi et al.
, 2023.
UNESCO, 2.
This study contributes to the literature in three ways.
Empirically, it provides rubric based evidence of changes in studentsAo scientific writing quality before and after ChatGPT assisted revision (Lim et al.
, 2024.
Zhai, 2.
Methodologically, it employs a before and after analytic Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
writing assessment that distinguishes idea development from lower-order writing features, including organization, grammar, diction, and spelling, thereby offering a more nuanced evaluation of AI-assisted writing than studies relying primarily on perceptions or overall writing performance (Crompton and Burke, 2024.
Cotton et al.
, 2.
Practically, the findings provide evidence-based recommendations for integrating generative AI into Indonesian language teacher education while preserving lecturer guidance, studentsAo active engagement in writing, and academic integrity (UNESCO, 2023.
Bearman and Ajjawi, 2.
METHODOLOGY
1 Research Design This study employed a one-group beforeAeafter design using document-based assessment and descriptive quantitative analysis (Creswell & Creswell, 2.
The design compared the quality of scientific papers produced by the same group of students before and after ChatGPTassisted revision.
Because all participants received the same instructional treatment and no control group was included, the findings describe changes in writing quality associated with ChatGPT use rather than establishing a causal relationship between ChatGPT and writing improvement (Shadish et al.
, 2.
The study examined changes in five dimensions of academic writing quality idea development, organization, grammar, diction, and spelling using an analytic writing rubric adapted from Nurgiyantoro .
To complement the quantitative findings, qualitative observations of recurring writing patterns, strengths, and weaknesses were conducted during rubric-based assessment.
These observations served only as supporting evidence for interpreting changes in writing quality and were not analyzed as an independent qualitative 2 Participants The study was conducted at AuUniversity A,Ay a private university in Indonesia.
This pseudonym protects institutional confidentiality and reduces the risk of indirect identification.
Participants were all 30 undergraduate students enrolled in one intact class within a language teacher education program.
They attended an undergraduate language and literature course during the 2025/2026 academic year.
A total sampling approach was employed because every eligible student receiving the instructional intervention participated in the study (Creswell & Creswell, 2.
Before the study, participants had completed an introductory academic writing course.
They were therefore familiar with basic scientific-writing conventions before completing the two assessed assignments.
Since the study involved one class at one institution.
The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan its findings remain context-dependent.
They should not be interpreted as representative of broader student populations.
3 Study Procedure and Data Collection The study was conducted from September 2025 to January 2026 using a pretestAeposttest design to examine changes in studentsAo scientific writing performance following ChatGPTassisted revision.
In the first stage (SeptemberAeOctober 2.
, 30 students produced scientific papers without AI assistance as baseline data.
After receiving instruction on the educational use of ChatGPT, students used the tool as a writing-support resource to improve language accuracy, organization, diction, and clarity while maintaining authorship and critically evaluating AI-generated suggestions.
This approach follows the principle of humanAeAI collaboration, in which AI functions as a support tool rather than a replacement for learnersAo cognitive engagement (Kasneci et al.
, 2023.
Tlili et al.
, 2.
In the second stage (NovemberAeDecember 2.
, students completed a second scientific paper using ChatGPT-assisted revision under lecturer supervision.
A total of 60 papers .
preintervention and 30 post-intervention manuscript.
were evaluated using the same analytical writing rubric.
The findings were interpreted cautiously because improvements may also have been influenced by studentsAo increased writing experience, lecturer feedback, and continuous classroom instruction, which are common considerations in educational intervention studies (Creswell & Creswell, 2.
4 Research Instrument The primary research instrument was an analytic rubric for assessing the quality of studentsAo scientific papers.
The rubric was adapted from Nurgiyantoro .
and aligned with the scientific writing guidelines of the institutionAos scientific writing guidelines.
To ensure consistency, the same rubric was applied to all papers produced before and after ChatGPTassisted writing.
The rubric evaluates five dimensions of academic writing: .
idea development, .
organization, .
grammar, .
diction, and .
Each dimension contributes to the overall score of 100 points, with the following weight distribution: idea development .
, organization .
, grammar .
, diction .
, and spelling .
The higher weighting assigned to idea development and organization reflects their importance in evaluating higher-order academic writing skills, whereas grammar, diction, and spelling represent lower-order language accuracy.
Table 1 presents the assessment rubric used in this study.
Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
Table 1.
Analytic Rubric for Assessing Scientific Papers Assessment Dimension Maximum Score Idea Development Organization Grammar Diction Spelling Total Assessment Focus Relevance, originality, clarity, and completeness of ideas Logical structure, coherence, and paragraph Grammatical accuracy and sentence construction Appropriateness and precision of vocabulary Compliance with Indonesian spelling conventions Source: Adapted from Nurgiyantoro .
Following assessment, each paper was classified into one of five quality categories adapted from Nurgiyantoro .
: Excellent .
Ae.
Good .
Ae.
Fair .
Ae.
Poor .
Ae .
, and Very Poor .
Ae.
These categories were used to describe changes in studentsAo writing quality before and after ChatGPT-assisted writing.
All scientific papers were evaluated by the researcher using the same analytic rubric.
Because the assessment was conducted by a single rater, inter-rater reliability could not be This constitutes a methodological limitation and should be considered when interpreting the findings.
Nevertheless, the use of a standardized rubric and identical assessment criteria for all papers was intended to maintain scoring consistency throughout the The complete analytic rubric, including assessment descriptors for each writing dimension, is provided in Table 2 to facilitate transparency and replication of the assessment The analytic rubric used in this study was adapted from Nurgiyantoro .
and aligned with the scientific University AuAAy writing guidelines.
The rubric consists of five assessment dimensions with a maximum total score of 100 points.
Table 2.
Analytic Rubric for Assessing Scientific Papers Assessment Dimension Excellent Good Fair Poor Idea Development .
Ideas are original, relevant, comprehensive, and supported by convincing Ideas are relevant and generally well developed but lack depth or Ideas are unclear, fragmented, or Organization .
Logical organization with clear introduction, body, conclusion, and strong coherence between Organization is generally logical with minor coherence problems.
Ideas are partially developed, with limited supporting Organization is inconsistent, and paragraphs are Grammar .
Grammar is consistently accurate with only negligible errors that do not affect comprehension.
Minor grammatical errors occur but do not interfere with Frequent affect clarity.
Persistent interfere with Diction .
Vocabulary is accurate, varied, and appropriate for academic writing.
Vocabulary is generally appropriate with occasional inaccuracies.
Vocabulary is repetitive or Vocabulary is limited and Organization is unclear, making the paper difficult to follow.
The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan Assessment Dimension Spelling .
Excellent Correct spelling and punctuation throughout the Good Very few spelling or punctuation errors.
Fair Several spelling or Poor inappropriate for academic writing.
Frequent spelling and punctuation errors reduce Source: Adapted from Nurgiyantoro .
5 Scoring Procedure Each scientific paper was assessed analytically by evaluating all five dimensions The scores from the five dimensions were summed to obtain a maximum score of 100 points.
The same rubric was consistently applied to all papers produced before and after ChatGPT-assisted writing.
6 Quality Classification The classification of scientific paper quality was established to provide a systematic framework for interpreting studentsAo writing performance.
The categories were determined based on the total scores obtained from the assessment rubric, representing different levels of writing quality.
Table 3 presents the criteria for scientific paper quality categories used in evaluating the participating students writing performance.
Table 3.
Scientific Paper Quality Categories Score Range 86Ae100 68Ae85 48Ae67 32Ae47 0Ae31 Category Excellent Good Fair Poor Very Poor Source: Adapted from Nurgiyantoro .
7 Data Analysis The data was analyzed using descriptive quantitative analysis supported by qualitative Quantitative analysis focused on comparing the mean scores of studentsAo scientific papers before and after ChatGPT-assisted writing.
Mean scores were calculated for the overall writing quality and for each of the five assessment dimensions: idea development, organization, grammar, diction, and spelling.
Descriptive statistics, including mean scores and frequency distributions, are commonly used in educational research to summarize patterns of learner performance and describe changes across measurement points (Creswell & Creswell.
Ary et al.
, 2.
The distribution of papers across the five quality categories (Excellent.
Good.
Fair.
Poor, and Very Poo.
was also calculated to describe changes in studentsAo writing performance after the use of ChatGPT.
Because the study employed a one-group beforeAeafter design without a control group, the analysis was limited to descriptive comparisons rather than inferential Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
statistical testing.
Such a design is appropriate for identifying patterns of change within a single however, the absence of a comparison group limits conclusions regarding causal effects (Creswell & Creswell, 2023.
Shadish et al.
, 2.
Therefore, the results indicate changes associated with ChatGPT-assisted writing and should not be interpreted as evidence of a causal To complement the quantitative findings, qualitative observations were conducted by repeatedly examining studentsAo papers to identify recurring patterns of improvement and persistent weaknesses across the five assessment dimensions.
This iterative document analysis followed the interactive qualitative data analysis approach proposed by Miles et al.
which involves data condensation, data display, and conclusion drawing.
Document analysis is widely recognized as a systematic method for examining written artifacts to identify patterns, meanings, and contextual information (Bowen, 2.
Finally, quantitative and qualitative findings were interpreted together to provide a comprehensive description of changes in studentsAo scientific writing quality before and after ChatGPT-assisted writing.
The integration of quantitative trends and qualitative interpretation enables researchers to develop a more complete understanding of educational phenomena by combining numerical evidence with contextual explanations (Creswell & Plano Clark, 2.
FINDINGS
1 Research Results This study compared the quality of scientific papers written by 30 student papers before and after ChatGPT assisted writing in the Regional Language and Literature Studies course.
The results are presented descriptively using overall scores, quality categories, aspect-level comparisons, and representative writing examples.
2 Overall Writing Quality Before and After ChatGPT Use This section presents the comparative analysis of studentsAo scientific writing quality before and after the integration of ChatGPT as a writing-support tool.
The analysis aims to identify changes in studentsAo overall writing performance based on the established assessment criteria for scientific papers.
The comparison focuses on the differences in writing quality across five dimensions: idea development, organization, grammar, diction, and spelling.
The assessment results provide an overview of how ChatGPT-assisted writing influenced the quality of studentsAo academic texts within the context of Indonesian language education.
Table The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan 4 presents the criteria used to evaluate the quality of scientific papers produced by participating students before using ChatGPT.
Table 4 Criteria for Assessing the Quality of Scientific Papers Participating Students Before Using ChatGPT No.
Sample Criteria for Assessing the Quality of Scientific Papers Quantity Value Average .
Source: Research Data .
Table Description:
Idea Organization Grammar Diction Spelling Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
The percentage distribution of studentsAo scientific paper assessment results before using ChatGPT provides an overview of their initial writing performance.
The analysis illustrates the proportion of papers classified into each quality category based on the predetermined assessment criteria.
Table 5 presents the percentage of writing quality levels among participating studentsAo scientific papers before the implementation of ChatGPT-assisted Table 5 Percentage of Assessment for Writing Scientific Papers for Students Before Using ChatGPT Category Value Quantity Percentage Excellent Good Enough Less Very less Source: Research Data .
Before using ChatGPT, the overall mean score of students' scientific papers was 77.
indicating that all papers were classified in the good category .
Ae.
None of the papers reached the Excellent category, while all 30 papers .
%) were categorized as Good.
Individual scores ranged from 72 to 82.
Following the implementation of ChatGPT-assisted writing, studentsAo scientific papers were reassessed using the same evaluation criteria to identify changes in writing quality.
The assessment focused on the five dimensions of scientific writing, including idea development, organization, grammar, diction, and spelling.
Applying consistent criteria before and after ChatGPT use enabled a direct comparison of studentsAo writing performance and the extent of improvement across different aspects of academic writing.
Table 6 presents the criteria used to assess the quality of scientific papers produced by students after using ChatGPT.
Table 6 Criteria for Assessing the Quality of Scientific Students paper After Using ChatGPT No Sample Quantity Criteria for Assessing the Quality of Scientific Papers .
The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan Value Average Source: Research Data .
Table Description:
Idea Organization Grammar Diction Spelling The evaluation results after the implementation of ChatGPT-assisted writing demonstrate the distribution of studentsAo scientific papers across different quality categories.
The percentage analysis was conducted to identify shifts in studentsAo writing performance and to illustrate the extent of improvement after using ChatGPT as a writing-support tool.
Table 7 presents the percentage distribution of scientific paper quality categories among students after using ChatGPT.
Table 7 Percentage of Scientific Paper Quality Category for Students After Using ChatGPT Category Value Quantity Percentage Excellent Good Enough Less Very less Source: Research Data .
Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
After ChatGPT-assisted writing, the overall mean score increased to 89.
0, placing the average performance in the Excellent category .
Ae.
Twenty-seven papers .
%) were classified as Excellent, while the remaining three papers .
%) remained in the Good category.
Individual scores ranged from 82 to 94.
3 Comparison of Writing Quality Before and After ChatGPT Use The comparison of mean scores across writing dimensions provides a more detailed view of changes in studentsAo scientific writing performance before and after ChatGPT-assisted The analysis examines the extent of improvement in each dimension, including idea development, organization, grammar, diction, and spelling.
Comparing the mean scores allows the researchers to identify which aspects of writing experienced the most significant improvement following the integration of ChatGPT as a writing-support tool.
Table 8 presents the comparison of studentsAo mean scores across the five assessed writing dimensions before and after using ChatGPT.
Table 8.
Comparison of Mean Scores Across Writing Dimensions No.
Overall Score Assessment Dimension Idea Development Organization Grammar Diction Spelling Before After Gain Percentage Gain Source: Research Data .
Table 8 shows improvements across all five assessment dimensions after ChatGPTassisted writing.
The largest relative improvements were observed in spelling .
1%), diction .
9%), and grammar .
9%).
Improvements in idea development .
2%) and organization .
3%) were comparatively smaller.
Overall, the mean writing score increased by 11.
1 points, representing a 14.
2% improvement over the initial average score.
4 Changes in Quality Categories The changes in scientific paper quality categories illustrate the overall shift in studentsAo writing performance after the implementation of ChatGPT assisted writing.
This analysis focuses on the movement of studentsAo papers across the five quality levels, namely Excellent.
Good.
Fair.
Poor, and Very Poor, based on the established assessment criteria.
The comparison of category distributions provides an overview of the extent to which ChatGPT assisted writing contributed to improvements in studentsAo scientific writing quality.
Table 9 presents the changes in scientific paper quality categories before and after the use of ChatGPT by University AuAAy students.
The Quality of Scientific Papers of Undergraduate Students in Regional Language and Literature Studies Lectures Before and After Using ChatGPT as a Writing Tool Ari Wulandari.
Arif Budiman.
Tri Ratna Herawati.
Primasari Wahyuni.
Diaz Danu Setyawan Table 9.
Changes in Scientific Paper Quality Categories No.
Quality Category Excellent Good Fair Poor Very Poor Before 0 .
%) 30 .
%) After 27 .
%) 3 .
%) Change Oe27 Source: Research Data .
As shown in Table 9, the quality distribution shifted substantially following ChatGPTassisted writing.
Before using ChatGPT, all students were classified in the good category.
After using ChatGPT, 27 students .
%) moved into the Excellent category, while only three students .
%) remained in the good category.
No papers were classified as Fair.
Poor, or Very Poor in either assessment.
5 Representative Example of Writing Improvement This section presents representative examples of studentsAo writing to illustrate qualitative changes that occurred after the implementation of ChatGPT assisted writing.
The examples were selected to demonstrate observable improvements in several writing dimensions, including idea development, organization, language accuracy, diction, and spelling.
comparing excerpts from studentsAo papers before and after ChatGPT use, this analysis provides contextual evidence supporting the quantitative findings presented in the previous sections.
Table 10 presents representative examples of writing improvements in studentsAo scientific papers before and after ChatGPT-assisted writing.
Table 10.
Representative Examples of Writing Improvements Before and After ChatGPT-Assisted Writing Writing Aspect Before ChatGPT Idea Development Regional languages are important because they are still used today.
Organization This paper discusses regional languages.
There are many The method is The results are explained Grammar Regional language have many function in society and still used by people.
After ChatGPT Regional languages play an essential role in preserving cultural identity because they function as a medium for transmitting local knowledge, traditions, and values across This paper begins by introducing the importance of regional language preservation, followed by a review of relevant theories, a description of the presentation of findings, and concluding Regional languages have many functions in society and are still widely used by local Improvement Observed Ideas became more elaborated, specific, and academically developed.
Paragraph organization and logical sequencing became clearer and more coherent.
Improved subjectAeverb agreement, sentence structure, and grammatical accuracy.
Script Journal: Journal of Linguistic and English Teaching P-ISSN: 2477-1880.
E-ISSN: 2502-6623
August 2026.
Vol.
11 No.
Diction The data is very good and shows many interesting things.
Spelling Based on the research,the students can write better.
The result also show The findings indicate several significant patterns regarding the preservation and use of regional languages within the studied Based on the research, the students were able to produce higherquality scientific papers.
The results also showed Informal vocabulary was replaced with more precise academic Correct punctuation, capitalization, spacing, and verb form enhanced Source: Representative excerpts adapted from students' scientific papers .
The representative excerpt illustrates improvements in grammatical accuracy, academic vocabulary, sentence structure, and overall readability following ChatGPT-assisted writing.
Similar improvements were consistently observed across many of the assessed papers.
Overall, the findings demonstrate consistent improvements in students' scientific writing after ChatGPT-assisted writing.
Improvements were observed across all five assessment dimensions, with the greatest gains occurring in spelling, diction, and grammar.
In addition, the distribution of writing quality shifted substantially, as 90% of students moved from the Good category to the Excellent category.
The interpretation of these findings in relation to constructivism, academic literacy, digital scaffolding, and previous studies is presented in the Discussion section.
DISCUSSION