COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
Text Similarity Analysis for Evaluating Alignment Between Lesson Plans and Teaching Reports Antonius Rachmat Chrismanto*1.
Willy Sudiarto Raharjo2.
Oscar Gilang Purnajati3 Prodi Informatika.
Fakultas Teknologi Informasi Universitas Kristen Duta Wacana.
Yogyakarta.
Indonesia Prodi Pendidikan Dokter.
Fakultas Kedokteran Universitas Kristen Duta Wacana.
Yogyakarta.
Indonesia e-mail: *1anton@ti.
id, 2willysr@ti.
id, 3oscargilang@staff.
Abstract RPS (Rencana Pembelajaran Semester, or called Lesson Plan.
is a class activity planning document in the higher education learning process that includes learning outcomes, methods, learning strategy, and evaluation criteria.
It is created by the lecturers in charge of the course and coordinated with the relevant department.
This document needs to be monitored throughout the semester for its conformity with the implementation document (Borang Pelaksanaan Perkuliahan (BPP)).
It was done manually through our eRPS system, but it requires a lot of effort and precision and is not time-efficient.
This research focused on evaluating the effectiveness of several content-based text similarity methods to detect RPS conformity compared with the BPP, or called Teaching Reports document.
The Boyer-Moore (B).
Rabin-Karp (R).
Jaccard (JC).
Jaro-Winkler (JW).
Smith-Waterman (SW).
Knuth-Morris-Pratt (K).
Levenehtein cosine similarity (C).
Dice (D).
Jaro (J), and Soundex (S) algorithms were evaluated in this paper.
In the vector-based similarity method.
TF-IDF was used.
The evaluation of 11 string-matching algorithms across four scenarios demonstrated clear performance trends.
Fuzzy algorithms (SW with accuracy 0,845Ae0,870, and JW with accuracy 0,840-0,.
achieved the highest accuracy in a single row of lecturer scenario, while exact/pattern-based algorithms (B.
K, and S with accuracy 0,8625Ae0,8.
on a combination of all rows of lectures with minimal variance (OO0,005Ae0,.
Pre-processing benefits fuzzy algorithms ( 2.
5%) but is neutral for exact/pattern-based algorithms.
The combined scenario improves the exact/phonetic algorithms ( 6Ae7%) but reduces the fuzzy performance algorithm (Oe10Ae14%).
The optimal thresholds were generally 40Ae50%, except for JW and J, which were 65%.
KeywordsAi Lesson Plans.
Teaching Reports.
Text Similarity algorithms.
Class evaluation INTRODUCTION RPS (Rencana Pembelajaran Semeste.
/ Lesson plan is a class activity planning document in the higher education learning process that includes learning outcomes, methods, learning strategy, and evaluation.
It is created by the lecturers in charge of the course and coordinated with the department .
as a plan before the course starts.
In reality, there is always a deviation between the RPS and its implementation during the course for various reasons.
These differences include content/schedule ordering, the content of the materials being taught, and the learning methodology.
Universitas Kristen Duta Wacana (UKDW) requires all lecturers to create RPS documents before the start of a new semester.
These documents are reviewed and evaluated by Lembaga Pengembangan Akademik dan Inovasi Pembelajaran (LPAIP) to ensure that all the plans in the RPS document are implemented accordingly.
LPAIP has a web-based eRPS system .
ttps://rps.
that lecturers can use to 415
ISSN: 1978-1520
create their RPS documents before a course begins, which must be validated by the head of the Currently, the eRPS is not equipped with a conformity detection module between the RPS and its implementation during a course.
All lecture activities are monitored and logged in BPP (Borang Pelaksanaan Perkuliaha.
/ teaching report document in a different system .
Clas.
This complicates the monitoring process because these two documents are separated across different systems.
Text similarity is a subfield of computer science called Natural Language Processing (NLP), which aims to identify the level of similarity between two or more texts.
Text similarity is commonly found in plagiarism .
, document similarity, news similarity, and other text-based content, such as social media .
, and others .
There are two main works on this project: .
developing a system that can detect conformity level between RPS and BPP which can be integrated into eRPS and .
measuring the accuracy and flexibility of several text similarity methods, including Boyer-Moore (B).
RabinKarp (R).
Jaccard (JC).
Jaro Winkler (JW).
Smith-Waterman (SW).
Knuth Morris Prat (K).
Lavensthein distance (L).
Dice (D).
Jaro (J) Soundex (S), and cosine similarity (C) algorithm.
The eRPS system development was completed by other work .
as part of an effort to help users improve their efficiency.
Text data is the most common form of data used in many applications.
It is also one of the easiest forms to process.
An example of an implementation that uses text data is a search Using search engine technology, this study developed text mining, text analysis, text processing, text similarity, and information retrieval technology.
There have been many applications that use these technologies, such as plagiarism detection, news similarity detection .
, spam detection .
, question duplications .
, question answering in the form of an essay .
, word similarity in class diagram generator applications in the software engineering field .
, and spam detection in social media platforms.
A higher level of text similarity is referred to as contextual text similarity.
Contextual text similarity is related to the proximity between texts that share the same meaning but have different structures, counts, numbers, positions, and lengths.
To solve this problem, a more comprehensive understanding of semantic similarity, which is above lexical similarity, is required.
Methods that go into the lexical similarity category are Cosine Similarity.
Jaccard Similarity.
SyrensenAeDice coefficient, and Levenshtein Distance.
Methods that go into semantic text similarity are word/sentence embeddings .
, contextual language models, machine learning .
, and deep learning using Transformers .
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
RESEARCH METHODS
The Methodology The prototyping method was used as the system development approach because it allowed us to showcase our work gradually over time to our users and revise it accordingly in each cycle.
This study was conducted in several steps as follows.
Data collection: collected RPS documents from the year 2023, which were manually validated by the LPAIP as the main reference for the system.
These documents are converted into an Excel document and then performed pre-processing .
, which includes tokenization, stemming .
, stopword removal .
, normalization .
, and cleaning steps.
The final step was to split the dataset into training, validation, and test data based on the scenario.
System development: a web-based module using PHP/Python was used to implement all the methods (Boyer-Moore .
Rabin-Karp .
Jaccard .
Jaro Winkler .
Smith-Waterman .
Khuth Morris Prat .
Lavensthein distance .
Dice Coefficient .
Jaro .
Soundex .
, and Cossine Similarity .
In the prototyping phase, the Jupyter Notebook was used as the development framework to assist us during testing.
ISSN: 1978-1520
Testing and evaluation: Accuracy and flexibility level evaluation methods were used in this study.
The accuracy evaluation method was implemented using the following accuracy metrics: precision, recall, and F1-score.
From these components, an accuracy evaluation performance benchmark of several text similarity methods used in the RPS study case.
The second evaluation was conducted to test the flexibility of text similarity methods against the possibility of typos in both RPS and BPP document content.
The main reference is the document validated by LPAIP UKDW in 2024 for the academic year 2023/2024 for all departments and for all odd and even semesters.
The detailed workflow of the development approach is presented in Table 1.
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
Table 1.
Development Workflow No.
Phase
Input
Process and Tools Output
Data Gathering RPS
BPP
document from 2023 Convert to Excel Excel documents Data Preprocessing and Cleaning Excel documents for RPS and BPP Clean dataset Scenario Generation Clean dataset Preprocessing .
okenisation, removal, normalizatio.
and Train.
Validation.
Test Split Methods Implementation Dataset split by train, validation, and test based on scenario Text similarity methods:
Boyer-Moore (B).
RobinKarp (R).
Jaccard (JC).
JaroWinkler (JW).
SmithWaterman (SW).
Knuth Morris Prat (K).
Levenshtein distance (L).
Dice (D).
Jaro (J).
Soundex (S), and cosine similarity (C).
Design and Prototyping Evaluation and Analysis Result Development program/module Python / PHP Evaluation: accuracy and flexibility metrics Dataset split by train, validation, and test based on scenario Program/module Comparison of the best methods for evaluation System Testing Plan Two evaluation methods are used: accuracy and flexibility evaluation methods.
The accuracy evaluation method is based on precision, recall, and F1-Score, while the flexibility evaluation method uses scenario-based testing against the dataset.
The details of the testing plan are presented in Table 2.
The baseline dataset is the RPS and BPP document without any preprocessing actions.
The dataset with pre-processing means that it goes through stemming, stopword removal, and normalization of duplicate content and typos.
Next, we compared the material content in each lecture with all lectures combined for one whole semester.
From Table 2, we can see the result for each algorithm performance used: Boyer-Moore (B).
Rabin-Karp (R).
Jaccard (JC).
Jaro Winkler (JW).
Smith-Waterman (SW).
Knuth Morris Prat (K).
Lavensthein distance (L).
Dice (D).
Jaro (J) Soundex (S), and cosine similarity (C).
Table 2.
System Testing Plan Scenario Accuracy All text similarity algorithms RPS/BPP Dataset without pre-processing .
ach lectur.
with thresholds 40, 50, 65, 85% A% RPS/BPP Dataset with pre-processing .
ingle row of lectur.
with thresholds 40, 50, 65, 85% COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
RPS/BPP Dataset without pre-processing .
ll rows of lectures combine.
with thresholds of 40, 50, 65, and 85% A% RPS/BPP Dataset with pre-processing .
ll rows of lectures combine.
with thresholds 40, 50, 65, 85% A% Average A% System Evaluation Several evaluation methods are used to determine the accuracy of the system.
In the text similarity detection system, the accuracy of the methods is tested against the dataset.
Test data were obtained from the dataset using K-Fold Validation .
, .
, .
, .
K-Fold Validation works by splitting the entire dataset into three parts: training, validation, and testing.
In the RPS implementation dataset, there was no training data.
therefore, we only experimented using the test The accuracy, precision, and F1-score are measured.
The flexibility of the method is tested using data that had not been tested during system development.
In the system testing, the conformity level between the content from the RPS is evaluated and its implementation in the BPP document using the confusion matrix in Table 3.
Table 3.
Confusion Matrix Prediction Class Real Class Negative Positive Negative True Negative (TN) False Negative (FN) Positive False Positive (FP) True Positive (TP) Based on the confusion matrix in Table 4, it is conducted further tests to obtain the accuracy, precision, recall, and F1-score using Equations 1 Ae 4 .
Accuracy = (TN TP) / (TN FP FN TP) .
Recall = TP / (FP TP) .
Precision = TP / (FN TP) .
F1-Score = 2 * TP / .
* TP FP FN) .
In addition to the evaluation metrics above, a scenario-based evaluation was implemented to determine the flexibility of the text similarity methods used.
The scenarios will use several combinations to determine the best combination, which can be used for the next prototype cycle.
RESULT AND DISCUSSION
Requirements gathering and Dataset Analysis In this phase, all the RPS and BPP documents are collected that would be used by the LPAIP and Puspindika units at UKDW.
The RPS document was retrieved from the online eRPS system .
ttps://rps.
id/).
The collected data were in Excel format, which had the following columns: materi_rps, sub_cpmk_rps, and a link to the finalized PDF.
From Puspindika, the BPP documents are collected after the courses were completed at the end of the semester.
The collected data consists of the following fields: kdsemester, kode, grup, prodi, nik, tatap_muka, tanggal_pertemuan, topik_pertemuan, keterangan_tambahan, metode_pembelajaran, bentuk_pembelajaran, media_pembelajaran, media_lain, dosen_pengampu, and jumlah_hadir.
An example of complete data from the accounting department is provided in Table 4.
Table 4.
Dataset example from the Accounting Department 418 COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
AK1113
AK1113
Prodi Akuntansi
Prodi Akuntansi
tatap_muka tanggal_pertemuan 2024-08-26
2024-09-02
14:30
14:30
topik_pertemuan Weygandt.
Kimmel, and Kieso .
Chapter 1 Pertemuan ke-1 Pertemuan ke-2 keterangan_tambahan gabungan_pertemuan Weygandt.
Kimmel, and Kieso .
Chapter 1 materi_rps Accounting Universe Weygandt.
Kimmel, and Kieso .
Chapter 1 sub_cpmk_rps Mahasiswa runtutan dunia akuntansi Mahasiswa mempraktikan, dan menyatakan aktivitas akuntansi, pengguna data akuntansi, serta mampu mengklasifikasikan data transaksi ke dalam persamaan dasar akuntansi.
Tidak metode_pembelajaran Kuliah/Transfer Knowledge (TCL).
Kuliah/Transfer Knowledge (TCL).
Small Group Discussion.
bentuk_pembelajaran Tatap Muka Tatap Muka media_pembelajaran eClass.
eClass.
media_lain dosen_pengampu Albertus Henri Listyanto Nugroho.
S, .
Albertus Henri Listyanto Nugroho.
S, .
jumlah_hadir https://rps.
id/archives/0_20241_A K1113_A.
https://rps.
id/archives/0_20241_A K1113_A.
The collected 25685 rows of data from the odd semester of 2024, the even semester of 2023, and the odd semester of 2023.
The details of the statistical data from the dataset profile are presented in Table 5.
Table 5.
Dataset profile Department name Total data count Yes Label No Label 885 data .
lank: 4 bpp 269 rp.
Number of Humaniora Akuntansi 2332 data .
lank: 46 bpp 327 rp.
Arsitektur 2587 data .
lank: 171 bpp 286 rp.
Biologi 1526 data .
lank: 29 bpp 394 rp.
Desain Produk 1143 data .
lank: 41 bpp 126 rp.
Filsafat Keilahian 2419 data .
lank: 22 bpp 218 rp.
419 COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
Informatika 4387 data .
lank 507 bpp 724 rp.
Manajemen 3784 data .
lank: 137 bpp 826 rp.
Pasca Teologi 91 data .
lank: 0 bpp 56 rp.
Pendidikan Bahasa Inggris Sistem Informasi 1645 data .
lank 37 bpp 275 rp.
1814 data .
lank: 11 bpp 375 rp.
Humanitas 1192 data .
lank 10 bpp 145 rp.
PPB 1640 data .
lank 161 bpp 1429 rp.
Puspindika 218 data .
lank 2 bpp 102 rp.
The results are summarized in Table 6.
It shows that there are many blank rows, mostly in the RPS documents.
This means that some lecturers did not fill the RPS document properly, with an average of 27,47%.
There were blank entries in the BPP document as well, but the percentage was lower .
21%).
Table 6.
EDA from the Dataset Department Humaniora Total data count RPS blank entries (%) 30,40 Yes Label (%) 885 data .
lank: 4 bpp 269 rp.
BPP blank
entries (%) 0,45 43,73 Label
(%)
54,01
Akuntansi 2332 data .
lank: 46 bpp 327 rp.
1,97 14,02 73,76 26,24 Arsitektur 2587 data .
lank: 171 bpp 286 rp.
6,61 11,06 80,25 19,75 Biologi 1526 data .
lank: 29 bpp 394 rp.
1,90 25,82 49,15 50,79 Desain Produk 1143 data .
lank: 41 bpp 126 rp.
3,59 11,02 71,13 28,87 Filsafat Keilahian 2419 data .
lank: 22 bpp 218 rp.
0,91 9,01 82,76 17,24 Informatika 4387 data .
lank 507 bpp 724 rp.
11,56 16,50 58,51 41,49 Manajemen 3784 data .
lank: 137 bpp 826 rp.
3,62 21,83 55,68 43,00 Pasca Teologi 91 data .
lank: 0 bpp 56 rp.
0,00 61,54 31,87 68,13 Pendidikan Bahasa Inggris Sistem Informasi 1645 data .
lank 37 bpp 275 rp.
2,25 16,72 88,21 11,79 1814 data .
lank: 11 bpp 375 rp.
0,61 20,67 71,44 28,56 Humanitas 1192 data .
lank 10 bpp 145 rp.
0,84 12,16 48,83 51,17 PPB 1640 data .
lank 161 bpp 1429 rp.
9,82 87,13 22,38 77,62 Puspindika 218 data .
lank 2 bpp 102 rp.
0,92 46,79 41,74 58,26 3,21786 27,4764 58,5314 41,2086 RATA-RATA Dataset Labeling After obtaining the dataset, the next step was to label it with two possible entries: YES and NO.
The Yes label indicates that the material contents matched between the RPS and BPP.
No label indicates that it does not match the criteria.
After one month of manual work in labeling those data, obtained the following results: the Yes label had an average of 58,5% from all datasets, 420
ISSN: 1978-1520
while the No label had an average of 41,3%.
From these results, it can conclude that the level of conformity between the RPS and BPP is approximately 58.
This result was then compared with the automation system using various algorithms described in the previous section.
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
Data Cleaning Data cleaning was performed by removing all extra white spaces and unknown characters and changing them into a single space.
It also merged three columns in the BPP materials into a single column, whereas in the RPS, merged two columns, namely, the material and sub-CPMK.
This is because in the BPP document, some lecturers often do not implement the course based on their planning document and overwrite it with new text that is manually inputted, while for RPS, some lecturers often use sub CPMK instead of the material content.
Once all the data were cleaned and verified, multiple algorithms were implemented.
Once the dataset was completed and labeled, it was saved in XLSX format and became the main dataset, which was processed in the next phase, that is, conformity detection, using the planned To process these files.
Python libraries are used, including OpenPyXL and Pandas.
Development Phase In the development phase, a program using Python is developed.
The system is divided into several modules.
Dataset processing This module uses OpenPyXL .
ttps://openpyxl.
io/en/stable/) and Pandas .
ttps://pandas.
org/) to read, parse, and load XLSX files into memory.
The final data were placed in a Pandas DataFrame format.
Pre-processing For preprocessing, several operations are performed: multiple space removal, case folding conversion, punctuation mark removal, word normalization, and repetitive Text similarity algorithm.
Several algorithms were implemented in the functions.
a Boyer-Moore (B).
Rabin-Karp (R).
Jaccard (JC).
Jaro-Winkler (JW).
SmithWaterman (SW).
Knuth Morris Prat (K).
Lavenstein distance (L).
Jaro (J).
Dice (D).
Soundex (S), and cosine similarity (C).
a A function was created to call all the algorithms using the input from the dataset.
The output of this function was saved in XLSX format so that it could be further processed for similarity percentage, along with its accuracy.
Metrics evaluation calculation for each algorithm, which measures the accuracy level for each data label, and saves the results.
Web-based publishing of the system.
Pipeline to upload to the GitHub repository for the source code management.
Streamlit repository setup for web access.
Desktop application as an alternative to a client application.
The front page of the system is presented in Figure 1.
Users can upload RPS and BPP in XLSX format, and they will be processed by the system, as shown in Figure 2.
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
Figure 1.
Front page of the system Figure 2.
The user uploaded an Excel document to the system.
Once it has been processed, the conformity level between the RPS and BPP is shown along with its accuracy, as shown in Figure 3.
Figure 3.
Conformity level from the XLS file and RPS/BPP accuracy Using the data analysis results, it gained insights into the last three semesters from the RPS and BPP documents.
Some lecturers did not make any RPS documents, or they were late in their submission.
Another possibility is that it is a new course, so it was entered after the submission date for the RPS had passed.
There are 3% blank rows in the BPP documents and 27.
4% in the RPS documents.
It is also found that there were 58,5% matched rows and 41,5% unmatched rows.
Therefore, 422
ISSN: 1978-1520
the conformity level using human manual verification for the RPS and BPP in the last three semesters was approximately 60%.
There are a lot of lecturers who did not write the RPS document properly:
Some did not enter the material contents Some only fill in the materials with reference materials The lecturers did not fill in the BPP form properly.
A lot of them did not fill in the materials contents There were many differences between RPS planning and its implementation in the BPP.
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
Evaluation The accuracy and conformity with a predefined threshold were measured.
The results were compared with those of the given labels.
Table 7 lists the parameters used for this evaluation.
Table 7.
Evaluation environment parameters Environment Parameter Reference data count Golden Dataset Scenario Value / Information Yes: 16250.
No: 9433 Without Preprocessing.
With Preprocessing.
Single row.
Combined row, using Threshold 40, 50, 65, 85%.
Number of Algorithms 11 algorithms (Boyer-Moore (B).
Rabin-Karp (R).
Jaccard (JC).
Jaro-Winkler (JW).
Smith-Waterman (SW).
Knuth Morris Prat (K).
Lavenstein distance (L).
Dice (D).
Jaro (J).
Soundex (S), and cosine similarity (C)) Based on the system developed, the conformity and accuracy outputs are based on several algorithms with and without pre-processing (Tables 8 and .
and with pre-processing (Tables 10 and 11, respectivel.
These algorithms were tested using a 50% threshold value.
These algorithms can be categorized as exact algorithms, such as Boyer-Moore (B).
Knuth-Morris-Pratt (K).
Rabin-Karp (R), and Soundex (S) and fuzzy algorithms such as Jaccard (JC).
Dice (D).
Cosine (C).
Jaro (J).
Jaro-Winkler (JW).
Smith-Waterman (SW), and Levenshtein (L).
The main reason to used a variety of algorithms was the non-existence of a single algorithm that works for short-to-mid text, especially for RPS and BPP documents, which also include a mix of both English and Indonesian.
The contents of the RPP and BPP documents vary greatly depending on each lecturerAos style.
therefore, an appropriate algorithm is needed for the text matching process.
By comparing multiple algorithms with different scenarios, we aim to obtain the best results from multiple algorithms.
Table 8.
Conformity percentage between RPS/BPP without pre-processing (Threshold 50%) Jaccard
Boyer
Moore Rabin
Karp
Jaro
Winkler
Smith
Waterman KMP
Levensht
Cosine Dice
Soundex
43,54
49,22
64,76
60,09
45,38
56,66
Jaccard
Boyer
Moore Rabin
Karp
Jaro
Winkler
Smith
Waterman KMP
Levensht
Cosine Dice
Soundex
Jaccard
Boyer
Moore Rabin
Karp
Jaro
Winkler
Smith
Waterman KMP
Levensht
Cosine Dice
44,37
56,07
49,08
65,27
60,45
55,08
45,66
46,45
Table 9.
Accuracy between RPS/BPP without pre-processing .
etween 0 - .
Table 10.
Conformity percentage between RPS/BPP with pre-processing (Threshold 50%) Table 11.
Accuracy between RPS/BPP with pre-processing .
etween 0 - .
Soundex
57,68
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
Jaccard Boyer Moore Rabin Karp Jaro Winkler Smith Waterman KMP Levensht Cosine Dice Soundex Furthermore, more test scenarios conducted without pre-processing with multiple thresholds of 40%, 50%, 65%, and 85%, using data from a single row (TS40.
TS50.
TS65.
TS.
, with pre-processing with the same threshold (PS40.
PS50.
PS65.
PS.
, and all combined data from multiple rows without pre-processing (TG40.
TG50.
TG65.
TG.
, and with pre-processing (PG40.
PG50.
PG65, and PG.
The results are presented in Table 12.
The highlights are as a TS scenario .
ithout pre-processing, single ro.
: Smith-Waterman has the highest accuracy, 0,845, followed by Jaro-Winkler .
and Jaro .
,8.
It can be seen that exact algorithms (Boyer-Moore.
Knuth-Morris-Pratt.
Rabin-Karp, and Sounde.
have stable accuracy but lower than that of the fuzzy algorithm.
a PS scenario .
ith pre-processing, single ro.
: Smith-Waterman has the highest accuracy, 0,870, followed by Jaro Winkler .
and Jaro .
,8.
Pre-processing had a significant effect on Smith-Watterman and Jaccard .
oth accuracy increased by 2,5%).
a TG .
ithout pre-processing, combine.
: Soudex has an accuracy of 0,865, similar to Boyer-Moore .
,8.
, similar to Knuth-Morris-Prat .
,8.
, followed by Rabin-Karp .
,8.
The combined scenario strengthens the exact/phonetic algorithms, whereas the fuzzy algorithm weakens them.
a PG .
ith pre-processing, combine.
: Boyer-Moore / Knuth-Morris-Prat / Soundex has 0,8725 accuracy .
he top three with the highest and most robust score.
, followed by Rabin Karp .
, and Cosine similarity .
Meanwhile, the impacts of the pre-processing and combined scenarios are as follows:
a Pre-processing and TS scenario (PSOeTS): Swith-WatermanAos accuracy is increased by 025.
Jaccard increased by 0.
Jaro / Jaro-Winkler/ Dice increased by 0.
Boyer Moore / Knuth-Morris-Prat / Rabin-Karp / Soundex are increased by -0,01.
Cosine -0,012.
Pre-processing is more useful for fuzzy algorithms than for precise algorithms.
a Combined scenario .
n TGOeTS): Boyer Moore 0.
Knuth-Morris-Prat 0.
Rabin-Karp 0.
Soundex 0.
060, but decreased for Smith-Waterman -0.
Jaro0.
Jaro Winkler -0.
097, and Jaccard -0.
The combined scenario was found to be beneficial in the exact/phonetic algorithm but not in the fuzzy algorithm.
a Pre-processing in the combined scenario (PGOeTG): improved in Jaro / Jaro Winkler / Dice / Jaccard ( 0.
043 - 0.
, but not significant in the exact algorithm ( 0.
From the comprehensive analysis of all algorithms used, several things: for a single-row scenario (TS/PS), it is recommended to use Smith-Waterman or Jaro-Winkler when there is a lot of noise or typos in the data, and it can also use pre-processing for consistency.
For the combined scenario (TG/PG).
Boyer-Moore.
Knuth-Morris-Pratt, or Soundex should be used for fast and consistent search results when the data are merged.
The best threshold is between 40% and 50%.
for the Jaro-Winkler or Jaro algorithm, it can consider increasing the threshold up to 65%.
The cosine (C) and Levenshtein (L) algorithms exhibited moderate performance.
however, they were sensitive to the threshold.
therefore, they were suitable for use as a baseline.
Table 12.
The accuracy results from all test scenarios Accuracy Algorith
Jaccard
Boyer
Rabin
TS5
TS6
TS8
PS4
PS5
PS6
PS8
0,71
0,73
0,64
0,82
0,78
0,74
0,65
0,78
0,71
0,46
0,82
0,77
0,67
0,49
0,87
0,77
0,83
0,81
0,78
0,71
0,87
0,87
0,87
0,84
0,87
0,88
0,88
0,86
0,85
0,77
0,66
0,84
0,81
0,76
0,67
0,84
0,85
0,85
0,85
0,84
0,86
0,85
0,85
COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
ISSN: 1978-1520
Jaro_wi Smith_w Kmp
Levensht
Cosine Dice
Jaro
Soundex
AVG
0,82
0,88
0,82
0,84
0,85
0,88
0,83
0,68
0,82
0,85
0,62
0,84
0,84
0,85
0,62
0,78
0,87
0,85
0,88
0,88
0,87
0,85
0,78
0,53
0,81
0,79
0,71
0,53
0,87
0,77
0,71
0,84
0,81
0,78
0,72
0,87
0,87
0,87
0,84
0,87
0,88
0,88
0,86
0,78
0,69
0,81
0,75
0,69
0,82
0,78
0,65
0,49
0,83
0,78
0,64
0,49
0,86
0,77
0,67
0,84
0,82
0,77
0,66
0,87
0,86
0,85
0,73
0,87
0,87
0,85
0,73
0,81
0,79
0,69
0,85
0,83
0,85
0,81
0,73
0,53
0,87
0,84
0,78
0,82
0,87
0,72
0,84
0,85
0,88
0,72
0,68
0,82
0,83
0,46
0,84
0,84
0,83
0,46
0,87
0,82
0,79
0,79
0,72
0,70
0,84
0,83
0,82
0,81
0,79
0,79
0,73
0,71
0,86
0,81
0,87
0,82
0,87
0,78
0,86
0,65
0,86
0,84
0,88
0,83
0,88
0,80
0,87
0,66
Based on the test results shown in Table 12, the algorithm performance is influenced by the characteristics of the RPS and BPP texts themselves.
Exact matching algorithms, such as Boyer-Moore (B) and Knuth-Morris-Pratt (K), demonstrated the highest and most stable accuracy in the combined text (TG/PG) scenario, as they could identify phrases and text that consistently appeared between lesson plans and implementations in long texts.
The Rabin-Karp (R) algorithm performed fairly well, but slightly lower due to its sensitivity to small changes in the text.
Conversely, token-based and character distance algorithms such as Jaccard (J).
Dice (D), and Levenshtein (L) produced lower accuracy because they were less able to represent similar meanings when there were text variations, additional words, or differences in sentence structure between the RPS and the BPP.
However, in the short text scenario for a single text without pre-processing, fuzzy similarity algorithms such as Jaro-Winkler (JW) and Smith-Waterman (SW) performed better because they were able to tolerate typos and text variations common in BPP.
The Soundex (S) and cosine similarity (C) algorithms provide fairly stable results, especially in handling spelling variations and general topic similarities, but they are not optimal without semantic pre-processing.
Based on the overall results.
Boyer Moore (B) and Knuth-Morris-Pratt (K) are recommended as algorithms used in the implementation of RPS systems on combined texts, while Jaro Winkler (JW) or Smith Waterman (SW) are more suitable for analysis in single short texts scenario, so that the combination approach is the most effective solution for learning evaluation needs according to the case study.
Also, based on the test results, the algorithm's sensitivity to the data length significantly impacted the accuracy level in the comparison between RPS and BPP.
Exact matching algorithms, such as Boyer-Moore (B) and Knuth-Morris-Pratt (K), demonstrated optimal performance as the text length increased.
In combined text scenarios (TG/PG), these two algorithms exploited recurring text occurrences and consistent text terms, resulting in increased accuracy and more stable results.
Conversely, for short texts and single texts, the performance of these algorithms was more sensitive to text variations, as even small character differences could immediately lead to a match failure.
Conversely, fuzzy-based algorithms such as Jaro-Winkler (JW) and Smith-Waterman (SW) demonstrated more stable performance on short texts, as they were designed to handle small differences, typos, and variations in character order.
For longer texts, the sensitivity of these algorithms decreases because of the increased computational complexity and diminishing influence of local similarity on the overall similarity score.
Token and vector-based algorithms such as Jaccard (J).
Dice (D), and cosine similarity (C) demonstrated intermediate sensitivity to text length.
their performance was relatively stable on long texts but declined on short texts owing to the limited number of tokens available for comparison.
This means that the choice of algorithm
must be adjusted to the length of the data, where exact matching algorithms are more effective 425
ISSN: 1978-1520
for long texts, whereas fuzzy-based algorithms are more suitable for short texts that contain text COGITO Smart Journal Ae Vol.
No.
December 2025.
P-ISSN: 2541-2221.
E-ISSN: 2477-8079
CONCLUSION
The text similarity methods are implemented to evaluate the consistency level between the RPS and BPP documents for each semester separately.
This system can help reduce the verification process in terms of time and increase overall efficiency.
It only takes approximately 15 min to evaluate all running classes in one semester.
In the TS/PS scenarios.
SW .
and JW .
840 - 0.
were top performers.
In the TG/PG scenarios.
B/K/S .
achieved the best results with excellent stability .
005Ae0.
, whereas preprocessing yields significant gains for SW and Jaccard (JC), whereas aggregation benefits exact/phonetic algorithms but harms fuzzy similarity metrics.
For production use.
PG B/K/S and PS SW/JW provided high and stable accuracies.
Finally, for future work, it is planned to expand the evaluation to use an ensemble algorithm .
rom exact and fuzzy algorithm.
and adaptive threshold tuning.
ACKNOWLEDGMENTS
This work was supported by the Fakultas Teknologi Informasi.
Universitas Kristen Duta Wacana.
Yogyakarta.
Indonesia, via the LPPM UKDW Research Scheme with Grant No.
121/D.
02/LPPM/202.
REFERENCES