ISSN Online: 2442-5605 Editorial Office: Faculty of Teacher Training and Education.
Universitas Islam Negeri Sultan Syarif Kasim Riau.
Indonesia Phone: 62781365280889 E-mail: potensia.
ftk@uin-suska.
Website: https://ejournal.
uin-suska.
id/index.
php/potensia Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model Herlini Puspika Saria .
Devi Arisantia .
Aldeva Ilhamia Faculty of Education and Teaching.
Universitas Islam Negeri Sultan Syarif Kasim Riau.
Indonesia http://dx.
org/10.
24014/potensia.
Abstract Keywords This study aims to validate an Artificial Intelligence (AI) literacy instrument for Islamic Education (PAI) students using Rasch analysis to evaluate its reliability, validity, item-fit statistics, and unidimensionality.
The instrument was developed to measure the extent of AI literacy among prospective Islamic Education teachers.
cross-sectional survey design was employed, involving 64 students from a university in Riau.
Indonesia.
Participants completed an AI literacy instrument consisting of 85 items.
The sampling technique used was quota sampling.
Data were analyzed using the Rasch model with the aid of Winsteps software version 3.
The results of the Rasch analysis indicated that the instrument demonstrated good reliability ( = 0.
, excellent item quality .
, and consistent respondent reliability .
In terms of validity, the three dimensions of AI literacy were confirmed to possess unidimensional properties.
The practical implication of this study lies in providing higher education institutions with a validated AI literacy instrument that can be used to assess the knowledge of future Islamic Education By employing Rasch analysis, this study contributes to enhancing the psychometric robustness of measurement tools, particularly in the context of assessing artificial intelligence literacy in Indonesia.
Validity.
Rasch model.
AI literacy.
islamic education.
A2025.
This is an Open Acces Research distributed under the term of the Creative Commons Attribution Licencee .
ttp://creativecommons.
org/licenses/by/4.
, which permits unrestricted use, distribution, and reproduction in any medium, provided the original works is properly cited.
Correspondence:
Herlini Puspika Sari.
Faculty of Education and Teaching.
Universitas Islam Negeri Sultan Syarif Kasim Riau.
Indonesia E-mail: herlini.
sari@uin-suska.
Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Introduction Artificial intelligence (AI) technology has developed rapidly, creating a significant impact in the field of education.
AI literacy has become an essential aspect that prospective educators must AI literacy not only includes understanding how AI systems operate but also involves applying them to support more effective and efficient learning processes.
As dependence on this technology increases, it becomes crucial for prospective Islamic Education (PAI) teachers to understand how AI can be integrated into PAI instruction.
The ability to master this technology has become an urgent necessity for future educators (Fadlelmula & Qadhi, 2.
Prospective PAI teachers are being prepared to become professional educators who are adaptive to technological changes (Wilton et al.
, 2.
Therefore, developing AI literacy skills among PAI pre-service teachers is of great importance.
AI literacy can be defined as the ability to understand and use AI intelligently and According to UNESCO .
AI literacy refers to the knowledge, skills, and values that instill an ethical understanding of human-built methods forming the foundation of AI AI literacy equips pre-service teachers to face the challenges emerging from technological advancements.
It also includes the ability to evaluate the social and ethical impacts that may arise from the use of such technology.
By mastering AI literacy, students are expected to utilize AI productively and think critically about its potential risks (Zootzky & Pfeiffer, 2.
he framework of artificial intelligence literacy (AI literac.
fosters a more comprehensive understanding of AI by integrating three main components: knowledge, skills, and attitudes (OECD, 2.
KThese components are interrelated and form the foundation for learners to interact with AI effectively and ethically.
Knowledge provides the theoretical basis of how AI functions, skills enable learners to utilize AI in various contexts, and attitudes encourage learners to apply AI responsibly and wisely.
AI technology offers various positive impacts that can facilitate teachersAo work.
In the educational context.
AI can be used to simplify administrative tasks, accelerate the assessment process, and deliver learning materials tailored to individual studentsAo needs.
AI can also improve educational accessibility by providing diverse resources that students worldwide can access.
This technology not only assists teachers in enhancing teaching efficiency but also enables them to focus on more strategic aspects of instruction, such as student interaction and social skills development (Faisal Tariq Hasan and Rafiq Rahman 2024.
Rahayu 2023.
Abbas 2.
However, the use of AI must be accompanied by a deep understanding of its potential social and ethical impacts A review of literature on the development of AI literacy instruments indicates that while significant progress has been made, there remains a lack of focus on prospective PAI teachers.
Kesuma & Fransen .
developed an AI literacy instrument for computer science students.
Biagini et al.
Grassini .
, and Shi .
designed multidimensional literacy questionnaires for higher education contexts, while Ng et al.
developed measurement instruments for secondary school students.
However, existing instruments have not fully captured the dimensions of AI literacy relevant to religious education, particularly in understanding fundamental AI concepts, developing critical and creative thinking skills, and fostering attitudes that promote ethical technology use in religious education.
This gap highlights the need for an AI literacy instrument specifically designed to assess AI literacy among prospective PAI Thus, developing such an instrument is a crucial endeavor.
On the other hand, the use of Rasch analysis in educational instrument development has proven to yield more accurate results compared to classical methods.
Rasch analysis can identify biased items and provide specific recommendations for instrument revision (Myller, 2020.
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Prasetya & Pratama, 2.
It enhances instrument quality by aligning item difficulty levels with the distribution of respondentsAo abilities (Metsymuuronen, 2.
This approach ensures that the instrument accurately differentiates respondentsAo competency levels and remains free from systematic bias.
The novelty of this study lies in the application of Rasch analysis to examine the validity and reliability of an AI literacy instrument.
The scope of the research includes item fit, unidimensionality, item and respondent reliability testing.
The findings may serve as a reference for educational research and practices that support the achievement of Sustainable Development Goals (SDG.
through improving the quality of PAI pre-service teachers.
This research is essential to support curriculum development for Islamic Religious Education in higher education.
With the increasing advancement of AI technology, universities must ensure that their curricula train PAI pre-service teachers to be adaptive to technological change (Sari et al.
, 2.
The curriculum must be aligned with contemporary needs to ensure that future teachers can effectively integrate technology into their teaching practices (Zhang & Zhang, 2.
This study is expected to make a significant contribution by developing an instrument that can be used to measure AI literacy among PAI pre-service teachers.
The purpose of this study is to measure the validity and reliability of an AI literacy instrument specifically designed for PAI students.
The instrument will be tested using Rasch analysis to ensure it possesses good quality in measuring AI literacy among PAI pre-service teachers.
Method This study employed a cross-sectional survey design, focusing on collecting information from the research sample through a questionnaire.
The instrument used was the AI literacy questionnaire, which followed the AI literacy framework (OECD, 2.
The instrument consisted of three dimensions knowledge, cognitive skills, and attitudes with a total of 85 items, all of which used a Likert scale.
The respondents comprised 64 students majoring in Islamic Religious Education (PAI) at universities across the Riau Province.
The instrument underwent content validation by three expert validators and was pilot tested to assess item readability, focusing on textual errors, typos, and image clarity.
Table 1.
AI Literacy Framework Dimension Knowledge Cognitive Skills Indicator The Nature of AI AI Reflects Human Choices and Perspectives AI Reshapes Human Work and Roles Capabilities and Limitations of AI The Role of AI for Society Critical Thinking Creative Thinking Self and social Awareness Collaboration Communication Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Items Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Problem solving Computational Thinking Responsibility Innovative Flexible Empathetic Curiosity Attitude The respondents in this study were PAI pre-service teachers.
A total of 64 students participated in this survey, and their demographic characteristics are presented in the table below.
This number is sufficient for instrument validation using the Rasch analysis approach.
Table 2.
Demographics of Respondents Aspect Gender Semester University Male Female UIN Sultan Syarif Kasim Riau IAIN Datuk Laksemana Bengkalis STAI Al Azhar pekanbaru IAI Rokan IAI diniyah pekanbaru STAI Nurul Falah Air Molek STAI Sulthan Syarif Hasyim Siak Sri Indrapura Riau Universitas Islam Kuantan Singingi Universitas Islam Riau IAI Dar Aswaja Institut keislaman tuah negeri Frequency Percentage Data were collected from student answer sheets and subsequently tabulated in Microsoft Excel.
The data were then coded and extracted into a raw data (.
format for analysis using Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
WINSTEPS software .
, which employs the Rasch measurement model.
The psychometric properties of the instrument were evaluated against six key parameters: validity, reliability, unidimensionality, item fit analysis, partial credit rating, and a variable map.
To ascertain the suitability of each item for measuring AI literacy, three fit statistics were examined: Outfit Mean Square (MNSQ).
Outfit Z-Standard (ZSTD), and Point-Measure Correlation (PTMEASURE Corr.
), as detailed in Table 2.
Any item that failed to meet the established criteria for two or more of these parameters was removed from the instrument.
The criteria for determining instrument reliability followed the rubric presented in Table 4.
Table 3.
Item fit Criteria Outfit mean square (MNSQ) Outfit Z-standard (ZSTD) Item Fit Value 0,5 < MNSQ < 1,5 -2,0 < ZSTD < 2,0 Point measure correlation 0,4< PTMeasure Corr.
<0,85
(Bond et al.
, 2.
Table 4.
Reliability Criteria
Score
<0,67
0,67-0,80
0,80-0,90
0,91-094
>0,94
Description Weak Sufficient High Very High Excellent (Sumintono & Widhiarso, 2.
Results and Discussion The data on the knowledge aspect of student AI literacy were analyzed using the Rasch model, a modern test theory approach.
This analysis yielded a comprehensive overview of the instrument's psychometric properties, including: .
item fit, based on Mean Square.
Z-Standard, and PointMeasure Correlation statistics.
item reliability.
person reliability.
partial credit rating scale function.
the distribution of respondent ability and item difficulty for the AI literacy knowledge construct, visualized on a Wright Map.
The detailed item statistics for Mean Square.
Z-Standard, and Point-Measure Correlation are presented in Table 2.
Item Fit Item fit analysis is conducted to ensure that each item functions appropriately within the context of the measurement objective and does not introduce confusion or misunderstanding for the The evaluation of item fit focuses on three key statistical values: Outfit Mean Square Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
(OUTFIT MNSQ).
Outfit Z-Standard (OUTFIT ZSTD), and Point-Measure Correlation (PTMEASURE CORR.
An item is determined to have a good fit if it aligns with the established criteria for these statistics (Sumintono & Widhiarso, 2.
The specific criteria applied to each of these three values are detailed in Table 5.
Table 5.
Item Fit
Statements MNS
ZSTD
AI operates using algorithms that combine step-by-step procedures and statistical inferences to process AI can provide recommendations based on available data, but the final decision still requires human AI is capable of processing data quickly and efficiently to produce useful information in education.
AI can replace the human role in deeply understanding social and cultural contexts.
AI has a high capability to understand religious values.
AI reflects human choices in system design and training, including decisions about the data AI can exhibit biases present in the training data, which affects how the system functions.
AI is designed by considering human perspectives.
AI can always perfectly reflect religious values.
Decisions made by AI systems are always in accordance with social or religious perspectives Ite
Informatio
PTMeasure (CORR) Invalid
Invalid
P10
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements MNS
ZSTD
AI can help automate administrative tasks in education.
The integration of AI in education can increase learning efficiency AI can help accelerate the learning process by providing customized AI can fully replace the role of a teacher in teaching the moral and ethical principles of Islam.
When using AI, teachers do not have to be involved in assessing and guiding students according to religious teachings.
AI is excellent at processing large data and automating routine tasks that require high efficiency AI can provide sophisticated analysis based on the data it is AI has the ability to process
information quickly and accurately AI cannot replace the moral or spiritual judgment required in the context of Islamic religious AI has no limitations in understanding knowledge, including Islamic religious teachings, because it already has valid and complete data sources AI is able to provide appropriate learning materials so that it has the potential to increase access to The use of AI in education can support the effectiveness of Islamic
religious learning Ite
Informatio
PTMeasure (CORR) P11
Invalid
P12
Invalid
P13
Invalid
P14
Invalid
P15
Invalid
P16
Invalid
P17
P18
Invalid
P19
Invalid
P20
P21
P22
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements MNS
ZSTD
AI can increase efficiency in the religious learning process because it provides fast results and according to needs The use of AI in Society must always consider its impact on social justice and diversity.
The use of AI in religious education can lead to misuse of materials that are not in accordance with religious principles.
I was able to analyze the results provided by the AI system and evaluate whether they fit the context of Islamic religious I was able to identify biases in the outputs generated by AI and examine their truth and relevance in the context of Islamic religious I have always used critical thinking to assess whether AI provides information that is in accordance with Islamic moral and ethical I feel there is no need to doublecheck the AI-generated material because it already has a valid data I find it difficult to evaluate the truth of the results of the AI system I can use AI to help develop creative ideas in teaching Islamic religious material to students.
I feel that AI can provide new
inspiration in designing more Ite
Informatio
PTMeasure (CORR) P23
Invalid
P24
P25
Invalid Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements interesting and relevant Islamic religious learning methods.
I can use AI to create innovative teaching materials, such as exam questions or presentations, that fit the learning material.
I don't feel the need to use AI in developing creative ideas for PAI teaching in the classroom, because creativity is better developed I often find using AI to generate PAI learning strategy ideas uninteresting and unnecessary.
I can identify problems in learning and can be solved using AI I am able to use logic and systematic step-by-step in solving problems related to learning using I can formulate the problem I want to solve so that it can be understood by the AI system to obtain a relevant solution .
commands/prompts in gpt cha.
I find it difficult to use AI to solve the problem I want to solve I'm not sure that AI can help solve the learning problems I'm finding I realized that AI is affecting my view of information related to Islam and education I am able to understand the social
impact of the use of AI in education, including how it can affect students MNS
ZSTD
PTMeasure (CORR) Ite
Informatio
K10
K11
Invalid
K12
K13
K14
K15
K16
K17
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements MNS
ZSTD
I am aware that the use of AI can reinforce or weaken certain social values in PAI learning because it depends a lot on how the technology is used.
I don't always consider the social impact of using AI in learning because AI is just a tool I find it difficult to consider the impact of using AI on students.
I can work with AI in designing Islamic teaching materials that suit
the needs and characteristics of I feel that the collaboration between me and AI technology can enrich
the learning process of Islam by providing faster and more effective I am able to give clear commands to the AI to produce appropriate learning materials I feel that collaboration with AI in PAI learning often does not make a meaningful contribution I don't feel the need to work with AI in developing Islamic teaching materials, because I prefer to do it I was able to explain how AI works in the context of teaching Islam to fellow educators I was able to communicate the benefits and limitations of AI in supporting PAI learning in a way
that was easy for others to Ite
Informatio
PTMeasure (CORR) K18
K19
Invalid
K20
K21
K22
K23
K24
K25
K26
K27
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements MNS
ZSTD
I feel confident in conveying the results or information generated by AI related to PAI learning materials to students clearly and precisely.
I find it difficult to explain AI working in the context of PAI due to a lack of understanding of how it I often feel that I don't need to explain how AI is used in PAI learning to students, because I feel like they already understand the I can use AI to help solve problems that arise in PAI learning such as creating HOTS skill-based exam I feel that AI can provide effective solutions in overcoming challenges in PAI learning I can design an AI-based solution to improve the quality of Islamic religious teaching in the classroom I feel that the use of AI cannot help in solving the problems I face in PAI learning I often feel that using AI for problem-solving in PAI learning is inefficient and ineffective.
I feel a responsibility to convey to students that the use of AI does not violate ethical principles I ensure that the output of AI in PAI learning always considers its impact on society and religious I recognize the importance of acting carefully when using AI
Ite
Informatio
PTMeasure (CORR) K28
Invalid
K29
K30
K31
K32
Invalid
K33
K34
Invalid
K35
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements MNS
ZSTD
I often assume that AI is a neutral technology so it doesn't need to be ethically considered in its use I feel like I don't need to worry too much about social and moral responsibility when using AI in PAI learning I have a high curiosity to learn more AI to use in PAI learning I am excited to explore different applications of AI to see how this technology can help me in preparing for more effective PAI I strive to understand how AI works in the context of PAI learning in depth.
I feel less interested in exploring further the use of AI I rarely feel interested in learning more about AI technology because I feel irrelevant to PAI learning I love finding new ways to integrate AI in PAI learning I believe that AI can be an innovative tool in supporting more interesting and relevant Islamic religious learning for today's I am ready to adapt AI technology in designing PAI learning methods that are more creative and in line with the times.
I feel like the use of AI in PAI
learning won't bring much significant change I feel there is no need to use AI in PAI learning because I believe Ite
Informatio
PTMeasure (CORR) Invalid
S10
S11
S12
S13
S14
S15
Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Statements more in the conventional approach
that already exists I find it easy to adapt to changes and technological advancements, including in the use of AI
I was able to quickly adjust to the use of AI which can help improve competence as a prospective I am ready to adapt the teaching methods in PAI by integrating AI
I often find it difficult to adapt to new technologies, including AI
I find it awkward to use new
technologies in PAI teaching including AI technology I strive to always consider the social and emotional impact of using AI in PAI
I realized the importance of understanding students' perspectives when using AI in PAI
learning so that this technology can be used for good
I strive to leverage AI that supports students' social and emotional The use of AI is not important to be feared to affect students socially and emotionally AI technology is already designed for good so there is no need to supervise students in using it MNS
ZSTD
PTMeasure (CORR) Ite
Informatio
S16
S17
S18
S19
S20
S21
S22
S23
S24
S25
Table 5 displays the measurement results for each item, including its Mean Square (MNSQ).
Z-Standard (ZSTD), and Point-Measure Correlation (Pt-Measure Corr.
) values.
The criteria for acceptable fit were defined by a value between -1.
5 and 1.
5, a Z-Standard score in the Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
range of -0.
2 < ZSTD < 2.
0, and a Point-Measure Correlation between 0.
4 and 0.
85 (Boone.
However, the analysis revealed that 18 items did not meet these criteria and were thus considered to have a poor fit .
and deemed unacceptable.
These items were: P1.
P8.
P11.
P12.
P13.
P14.
P15.
P16.
P18.
P19.
P23.
P25.
K11.
K19.
K28.
K32.
K34, and S4.
Conversely, the remaining 67 items satisfied the criteria and were retained for the final instrument.
The Rasch model explicitly focuses on the measurement of construct validity.
Each item is expected to make a meaningful contribution to the underlying construct or concept being Item fit statistics indicate the degree to which the observed response pattern for a particular item is consistent with the pattern predicted by the model, given the respondents' overall ability levels.
Through the analysis of item fit, errors that occur during the calibration phase of instrument development can be clearly detected (Myller, 2.
Instrument Reliability The reliability of an instrument is evaluated to verify that it can consistently produce stable and dependable measurements.
Within the Rasch modeling framework, reliability is assessed using three primary indices: person reliability, item reliability, and CronbachAos alpha (Sumintono & Widhiarso, 2.
Tabel 6.
Measurement of the Validity and Reliability of the Instrument
Aspect
Pearson Measurement Person reliability Separation MNSQ
ZSTD
Item Measurement Item reliability Cronbach alpha
Raw variance Unexplained variance 1st contrast
Score
2,53
0,89
As presented in Table 6, the item reliability index was 0.
97, with an item separation of This result indicates excellent item consistency and suggests that the items are sufficiently spread out along the measurement continuum to define a distinct hierarchy of difficulty.
Reliability is expressed as a numerical score, typically ranging from 0 to 1.
00, where a higher coefficient signifies greater reliability.
High reliability is associated with a small degree of measurement error.
consequently, the higher an instrumentAos reliability, the smaller its measurement error.
Person reliability is an index that reflects the consistency of studentsAo response patterns across the set of items.
For the AI literacy instrument, the person reliability was found to be 0.
This value indicates a good level of consistency in the studentsAo response patterns.
The overall instrument reliability reported in Table 6 is 0.
89, a value based on CronbachAos Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
alpha, which indicates a high degree of internal consistency.
This strong value signifies a robust interaction between the studentsAo abilities and the items.
It implies that the students responded with a high degree of consistency and that the items were effective in measuring a wide range of proficiency levels.
Therefore, the instrument can consistently measure the target construct and is expected to yield similar results upon repeated administration to the same subjects or to a sample with similar characteristics.
Unidimensionality The assumption of item unidimensionality is tested to confirm that an instrument measures a single, intended construct, thereby providing evidence for its validity.
Within the Rasch framework, this is evaluated using a Principal Component Analysis (PCA) of the residuals, which determines the proportion of variance explained by the primary construct (Ilhami & Hidayat.
An instrument is considered unidimensional if it meets two conditions: first, the raw variance explained by the measures must be at least 20%, and second, the unexplained variance in the first contrast must not exceed 15% (Sumintono & Widhiarso, 2.
The results showed that the raw variance explained by the measures was 45.
7%, indicating a strong primary Furthermore, the unexplained variance in the first contrast was 8.
5%, well below the 15% threshold.
These findings confirm that the instrument satisfies the criteria for unidimensionality, providing strong evidence that it effectively measures the intended construct.
The concept of unidimensionality within Rasch analysis posits that all items should relate to a single latent variable and measure it in a consistent direction.
While this property is often taken for granted in quantitative social science research, it is crucial to empirically test this This is particularly important in novel contexts where the precise nature of the latent variable is still under investigation (Bond et al.
, 2.
Rating Partial The quality of an instrument's rating scale, which comprises the set of alternative response options, can be evaluated using rating scale diagnostics.
The purpose of this analysis is to determine if the scale is functioning appropriately .
ble to discriminate between respondents of varying abilit.
, reliable .
onsistent in this discriminatio.
, and well-defined .
espondents understand the distinctions between response categorie.
(Boone et al.
, 2.
The AI literacy instrument employed a six-point Likert-type scale with the following response categories: 6 (Strongly Agre.
, 5 (Agre.
, 4 (Somewhat Agre.
, 3 (Somewhat Disagre.
, 2 (Disagre.
, and 1 (Strongly Disagre.
The diagnostic data for this rating scale analysis, generated from the partial credit model output, are presented in Table 7.
Table 7.
Diagnostic Rating Test Results Label Strongly Disagree Category Label Observed Andrich None Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
Somewhat Somewhat agree Agree Strongly agree For the response alternatives to be considered well-understood by respondents, both the observed average measures and the Andrich thresholds should increase monotonically with each successive rating scale category (Sumintono & Widhiarso, 2.
In this analysis, the observed average for respondentsAo answers showed a consistent increase.
The specific observed average value for response option 1 was -0.
for option 2, it was -0.
for option 3, it was -0.
option 4, it was 0.
for option 5, it was 1.
and for option 6, it was 1.
This result indicates that the AI literacy instrument for Islamic Religious Education (PAI) students functions appropriately for the sample and that the response categories were not confusing to the Figure 1.
Probability Category Curve Consecutive categories.
These thresholds are important for understanding the extent to which differences between categories are acceptable and distinguishable to respondents.
The Andrich threshold values monotonically progress from none toward negative logits and continue toward positive logits, indicating that the six response options provided are valid for the respondents (Sumintono & Widhiarso, 2.
Based on the analysis results, the threshold between the categories AuDisagreeAy and AuSlightly DisagreeAy was recorded at -2.
08, while the threshold between AuSlightly DisagreeAy and AuSlightly AgreeAy was considerably larger, at -6.
The substantial difference in thresholds between these categories indicates a significant gap in respondentsAo perceptions of the two categories.
Similarly, the threshold between AuSlightly AgreeAy and AuAgreeAy was recorded at -6.
2, also indicating a large difference.
However, the threshold Potensia: Jurnal Kependidikan Islam.
Vol.
No.
JuliAeDesember 2025 Herlini Puspika Sari.
Devi Arisanti.
Aldeva Ilhami: Validity and Reliability of the Artificial Intelligence Literacy for Prospective Islamic Education Teachers Using Rasch Model.
between AuAgreeAy and AuStrongly AgreeAy was recorded at 0.
55, suggesting that the difference between these categories is smaller, making the transition between them clearer for respondents.
Overall, this measurement instrument demonstrates sufficiently distinct average measurements across each category (Figure .
Nevertheless, the irregular threshold between the AuSlightly DisagreeAy and AuSlightly AgreeAy categories suggests that these categories may need to be revised to better differentiate respondentsAo levels of agreement.
Conclusion Based on the results of the study, the AI literacy instrument meets the necessary assumptions for the Rasch Model, including fit, unidimensionality, rating scale, and reliability.
The item reliability level was 0.
97, while person reliability was 0.
88, both of which fall into the good This instrument is reliable and demonstrates internal construct validity, as indicated by a raw variance explained measure of 45.
The AI literacy instrument can therefore be used to measure studentsAo AI literacy.
A total of 67 items were found to be valid, while 18 items should be considered for removal.
Acknowledgments The research team would like to express their gratitude to the Research and Community Service Institute (LPPM) of UIN Sultan Syarif Kasim Riau and the Directorate of Islamic Higher Education (Dikti.
of the Ministry of Religious Affairs of the Republic of Indonesia (Kemenag RI).
This program was carried out through a research grant program for the 2025 fiscal year.
References