Vol.
No.
1, 2025, pp.
DOI: https://doi.
org/10.
29210/1202526003
Contents lists available at Journal IICET Jurnal EDUCATIO (Jurnal Pendidikan Indonesi.
ISSN: 2476-9886 (Prin.
ISSN: 2477-0302 (Electroni.
Journal homepage: https://jurnal.
org/index.
php/jppi Development and validation of technological pedagogical content knowledge (TPACK) test instruments for prospective chemistry teachers Hasna Athaya Rifa*)1.
Nahadi1.
Sjaeful1
Universitas Pendidikan Indonesia
Article Info
ABSTRACT
Article history:
The rapid development of digital technology has significantly impacted education, requiring teachers to master not only pedagogical and content knowledge but also technological integration skills.
This study aims to develop and validate a digital-based assessment instrument to measure Technological Pedagogical Content Knowledge (TPACK) competence among prospective chemistry teachers.
Using the Development and Validation (D&V) model by Adams and Wieman .
, the research was conducted in three main stages:
needs analysis and formulation of development objectives, .
design of the test blueprint and item development based on the TPACK framework, and .
content and construct validation.
Content validation involved six expert validators, and the results were analyzed using Content Validity Ratio (CVR) and Content Validity Index (CVI), both showing strong content validity (CVR = 1.
S-CVI/Ave = 1.
Construct validation was performed through limited trials with 31 chemistry education students and analyzed using Rasch modeling.
Findings indicated high item reliability .
, with most items fitting the Rasch model well, although four items required revision to improve clarity and Overall, the study produced a content- and construct-valid TPACK test instrument that can serve as a standardized tool for evaluating and enhancing prospective chemistry teachersAo ability to effectively integrate technology, pedagogy, and chemistry content in future teaching practices.
Received Jun 13th, 2025 Revised Jul 20th, 2025 Accepted Aug 6th, 2025 Keyword:
TPACK
Assessment Test Instrument Development Prospective Chemistry Teachers Rasch analysis A 2025 The Authors.
Published by IICET.
This is an open access article under the CC BY-NC-SA license .
ttps://creativecommons.
org/licenses/by-nc-sa/4.
Corresponding Author:
Hasna Athaya Rifa.
Universitas Pendidikan Indonesia Email: hasnarifa@upi.
Introduction The rapid development of digital technology has driven transformation across various aspects of life, including Teachers, as the main agents of change, are required to possess competencies not only in pedagogy and content knowledge but also in the effective use of technology in teaching and learning (Dewi et al.
, 2024.
Maddukelleng et al.
, 2023.
Tiken et al.
, 2.
One widely used framework to describe the integration of these domains is the Technological Pedagogical and Content Knowledge (TPACK), which combines three core domainsAiContent Knowledge (CK).
Pedagogical Knowledge (PK), and Technological Knowledge (TK)Aias well as four integrated domains: Pedagogical Content Knowledge (PCK).
Technological Content Knowledge (TCK).
Technological Pedagogical Knowledge (TPK), and Technological Pedagogical Content Knowledge (TPACK).
Development and validation of technological pedagogical content knowledge (TPACK).
The application of TPACK is particularly important in chemistry education due to the abstract and complex nature of chemical concepts, which require appropriate pedagogical approaches and technological tools to facilitate studentsAo understanding (Abouelenein & Selim, 2024.
Purba et al.
, 2.
TPACK competence is crucial in chemistry teacher education as it enables future teachers to integrate technology, pedagogy, and chemistry content effectively.
This integration not only improves the quality of chemistry teaching but also prepares students to face future educational challenges (Deng et al.
, 2023.
Jammeh et al.
, 2024.
Mashuri et al.
To understand and analyze pre-service chemistry teachersAo ability to integrate technology, pedagogy, and chemistry content, it is essential to develop an assessment instrument that can measure such competence (Paristiowati et al.
, 2.
A TPACK assessment instrument can evaluate, develop, and validate pre-service teachersAo abilities to combine technological, pedagogical, and content knowledge, thereby enhancing professionalism and the effectiveness of 21st-century chemistry education (Deng et al.
, 2023.
Hairida et al.
, 2023.
Paristiowati et al.
, 2020.
Solikhin & Rohiat, 2.
In Indonesia, teacher competency tests have been implemented through the Teacher Professional Education Program Competency Test (UKpG), conducted at the end of the Teacher Professional Education Program (PPG) in accordance with Ministry of Education Regulation No.
19 of 2024.
The UKpG consists of a computer-based Written Exam lasting 150 minutes and a Performance Exam.
The Written Exam includes objective tests and subjective tests, with the objective tests comprising Pedagogical Content Knowledge (PCK) and a Situational Judgment Test (SJT), consisting of multiple-choice and complex multiple-choice questions (Directorate General of Teachers and Education Personnel, 2.
However, competency assessments in the PPG program predominantly focus on pedagogy and content, with limited emphasis on technology, despite its growing importance in the digital era.
Therefore, developing a standardized, digital-based test instrument is a strategic effort to equip pre-service teachers with measurable and valid TPACK competencies.
Previous studies have primarily assessed TPACK competence using self-assessment approaches.
However, this method is subjective, as it relies on participantsAo self-awareness, leading to potential uncertainty in evaluating their own competence (Meiliawati et al.
, 2.
Moreover, other studies highlight that self-assessment has limitations: it mainly captures lower-order cognitive levels (AurememberingAy and AuunderstandingA.
but does not adequately assess higher-order thinking skills (AuapplyingA.
, its effectiveness depends on prior knowledge, and it may not accurately measure individuals with lower TPACK competence.
Thus, alternative assessment tools are needed to complement self-assessment, which should also be tailored to the learning context (Max et al.
, 2.
Additionally, several studies have employed observational approaches to assess TPACK in pre-service This method is also subjective.
Morales et al.
found that while observation can capture pedagogical, technological, and content knowledge, it often reflects only instrumental mastery.
Even after coursework covering these knowledge domains, there was insufficient evidence that pre-service teachers could effectively integrate technology as a didactic resource in teaching practice.
This indicates that observational assessments alone may not comprehensively evaluate TPACK mastery and practical technology integration.
Similarly.
Njiku .
suggests that observational assessments may be suitable for measuring TPACK but must be adapted to specific content and technology contexts.
In this study, a test-based instrument is utilized to directly measure pre-service teachersAo knowledge mastery through test items (Arnan et al.
, 2.
Test-based assessments have advantages in more accurately evaluating cognitive competence, aligning with targeted cognitive levels and TPACK dimensions.
Furthermore, test instruments are easier to standardize, score consistently, and efficiently assess knowledge on a larger scale (Fitria, 2024.
Magdalena et al.
, 2.
TPACK represents teachersAo knowledge emerging from the dynamic integration of content knowledge (CK), pedagogical knowledge (PK), and technological knowledge (TK).
It is not merely a combination of these domains but rather a unified and evolving understanding of teaching with technology (Koehler et al.
, 2.
TPACK thus serves as a guide for teachers in determining appropriate technology use to deliver subject matter through suitable pedagogical approaches, taking into account studentsAo characteristics and learning contexts.
Based on this background, although previous research has developed various TPACK assessment instruments using self-assessment methods, the availability of empirically validated, digital-based test instruments remains very limited, particularly for pre-service chemistry teachers in Indonesia.
To address this gap, the present study is designed as an initial development phase focusing on the construction and preliminary validation of a TPACK test instrument.
This study aims to produce a content-valid instrument as a foundation for future development, where subsequent stages will examine its reliability and effectiveness through large-scale field testing.
Ultimately, this research is expected to provide an initial contribution to the availability of objective Journal homepage: https://jurnal.
org/index.
php/j-edu Rifa.
, et al Jurnal EDUCATIO (Jurnal Pendidikan Indonesi.
Vol.
No.
1, 2025, pp.
test-based assessments that support the enhancement of TPACK competence among pre-service chemistry Method This study employed the Development and Validation (D&V) model proposed by Adams and Wieman .
As an initial phase of development, the study comprised three main steps: .
identifying the need for a TPACK assessment and defining development objectives, .
designing a blueprint and developing digital-based test items aligned with the TPACK framework, and .
conducting content and construct validation.
Large-scale field testing and reliability analysis were not included at this stage.
therefore, the results of this research focus primarily on establishing validity as a foundation for future development.
In the first step, the researchers identified the need for a TPACK assessment by administering a questionnaire to pre-service chemistry teachers.
The results were used to define the objectives of the instrumentAos development.
Subsequently, a literature review was conducted covering test instruments, test development processes, teacher competency tests, and TPACK competencies.
Operational definitions and item indicators were established, followed by the development of a blueprint based on the TPACK framework.
In the final step, content validation was conducted by six experts .
ive chemistry education lecturers and one chemistry lecture.
Each validator was asked to evaluate all test items using a validation sheet comprising three criteria: .
the relevance of the content to the intended TPACK indicator, .
the appropriateness of the item with its answer key, and .
the consistency of the item with its scoring rubric.
Validators also provided qualitative feedback and suggestions for improving unclear or misaligned items.
Construct validation was then carried out through an empirical pilot test involving 31 pre-service chemistry teachers.
The study was conducted in the Chemistry Education program at a university in Bandung.
Indonesia.
Expert validation involved six experts with diverse backgrounds: the first was a senior lecturer in chemistry education across undergraduate to doctoral programs with extensive experience in assessment.
the second was an undergraduate chemistry education lecturer specializing in assessment research.
the third was a professor in the fourth was a lecturer focusing on chemistry instructional media development.
the fifth was a lecturer experienced in chemistry teaching and learning.
and the sixth was a junior lecturer focusing on assessment research.
For empirical validation, 31 pre-service chemistry teachers participated in a limited-scale pilot test.
Three instruments were used in this study: a TPACK needs questionnaire, an expert validation sheet, and the TPACK test instrument.
The overall development and validation process lasted approximately three months, covering item design, expert review, revisions based on feedback, and pilot testing.
To ensure objectivity during validation, all items were anonymized before distribution to the validators, and evaluations were performed Content validation data were analyzed quantitatively using the Content Validity Ratio (CVR) and the Content Validity Index (CVI).
The CVR was calculated to determine agreement among validators on the essentiality of each item, using the formula:
where ne is the number of validators rating the item as valid, and N is the total number of validators.
According to Lawshe .
, an item is considered valid if its CVR exceeds 0.
99 when evaluated by six validators .
ne-tailed test, = 0.
If any validator rated an item as invalid, the item was revised and re-discussed with the respective validator until consensus was reached.
Items that could not be revised were deemed invalid and For CVI analysis, the Item-Level Content Validity Index (I-CVI) was calculated by dividing the number of validators rating the item as relevant by the total number of validators:
Following Polit .
, items with I-CVI Ou 0.
78 were considered acceptable.
The Scale-Level Content Validity Index (S-CVI/Av.
was also computed to evaluate overall instrument validity, with a minimum Journal homepage: https://jurnal.
org/index.
php/j-edu Development and validation of technological pedagogical content knowledge (TPACK).
threshold of 0.
90 indicating excellent content validity.
Items not meeting these thresholds were revised or Construct validity from the pilot test data was analyzed using the Rasch model via Ministep software.
Item fit was evaluated using the Item Fit Order table.
Items were accepted if both infit and outfit mean-square (MNSQ) statistics fell within the 0.
50Ae1.
50 range.
Items outside this range were identified for revision or This study represents the initial stage of TPACK test instrument development, focusing on item construction and validation through expert review and limited pilot testing.
The aim of this phase was to produce an instrument that is both content-valid and empirically supported, serving as the foundation for subsequent research phases that will include large-scale reliability testing, effectiveness evaluation, and broader Results and Discussions This section presents the initial development results of the Technological Pedagogical Content Knowledge (TPACK) test instrument for pre-service chemistry teachers, consisting of three main stages: .
identification of TPACK assessment needs and formulation of development objectives, .
blueprint design and development of digital-based test items aligned with the TPACK framework, and .
content and construct validation of the The needs analysis was conducted to understand the current condition of TPACK competency assessment among pre-service chemistry teachers.
Data were collected through a questionnaire distributed to chemistry education students.
The analysis revealed that technology integration has already been embedded in several courses, including multimedia learning programming.
ICT literacy, microteaching, and school chemistry The use of technology such as virtual laboratory simulations, online learning resources, basic programming, educational video production, animations, and online quiz applications has provided students with experiences in delivering technology-integrated instruction.
Consequently, students reported confidence in their ability to combine technology, pedagogy, and chemistry content when planning lessons.
This finding aligns with Fitriani & Suryani .
, who reported that after completing fully online microteaching courses, pre-service chemistry teachers demonstrated high confidence in their TPACK.
However, responses were divided regarding the existence of assessments that comprehensively measure the integration of technology, pedagogy, and content knowledgeAihalf agreed such assessments were absent, while the other half disagreed, citing formative assessments conducted during microteaching through observation-based evaluation in single teaching simulations.
These mixed responses indicate a gap in formative assessment implementation within microteaching sessions.
Moreover, all respondents agreed on the need for a dedicated instrument to specifically measure TPACK competencies in chemistry teacher education, as such an instrument would provide valuable insights into preservice teachersAo readiness for the teaching profession.
This finding supports Purba .
, who emphasized that TPACK analysis can be used to evaluate the preparedness of future chemistry teachers.
Furthermore, 80% of respondents stated that there is currently no standardized test or lecturer-provided assessment specifically designed to measure TPACK competencies.
Therefore, this research aimed to develop a standardized TPACK test instrument capable of effectively assessing TPACK competencies in pre-service chemistry teachers.
The TPACK test instrument was developed based on the TPACK framework, which comprises seven core components: Content Knowledge (CK).
Pedagogical Knowledge (PK).
Technological Knowledge (TK).
Pedagogical Content Knowledge (PCK).
Technological Content Knowledge (TCK).
Technological Pedagogical Knowledge (TPK), and Technological Pedagogical Content Knowledge (TPACK).
From these components, 20 test indicators were derivedAithree indicators each for CK.
PK.
TK.
PCK.
TCK, and TPK, and two indicators for TPACK.
Each indicator was coded .
CK-1.
CK-2.
CK-.
and served as the foundation for constructing the TPACK test blueprint.
Test items were developed to support digital delivery, with various item types .
ultiple-choice, matching, true/false, complex multiple-choice, and essa.
targeting different cognitive levels (C2AeC.
The questions are accompanied by videos, images, and websites that can be accessed through a digital platform.
This allows for the measurement of macroscopic, microscopic, and symbolic aspects.
The instrument was then subjected to expert judgment for content validation.
Experts evaluated each item based on three criteria: .
alignment of the item with its intended TPACK indicator, .
appropriateness of the item with the answer key, and .
consistency of the item with the scoring rubric.
Journal homepage: https://jurnal.
org/index.
php/j-edu Jurnal EDUCATIO (Jurnal Pendidikan Indonesi.
Rifa.
, et al Vol.
No.
1, 2025, pp.
TPACK
Components Code
CK-1
CK-2
CK-3
PK-1
Complex
Multiple Choice Multiplechoice Multiplechoice Complex
Multiple Choice Complex
Multiple Choice Complex
Multiple Choice Essay
Multiplechoice Matching Essay
Multiplechoice Complex
Multiple Choice True/False Multiplechoice Complex
Multiple Choice True/False Matching Essay
TK-2
Demonstrating the effectiveness of digital media in learning TK-3
Designing digital-based learning Determining the reasons for using learning strategies for abstract
Relating pedagogical approaches to their reasons for explaining chemistry content
Designing learning based on content and student
Determining the appropriate digital media to explain specific chemistry concepts Analyzing the suitability of content and digital applications Demonstrating the application of digital-based chemistry experiment media Implementing ICT in learning Analyzing the suitability of digital media with a chemistry pedagogical approach
Determining technology-based active learning designs Analyzing technology-pedagogycontent-based learning scenarios and evaluating the appropriateness of ICT use Designing technology-pedagogycontent-based learning PCK-1
TCK-1
TCK-2
TCK-3
TPK-1
TPK-2
TPK-3
TPACK
Demonstrating the role of technology in learning PCK-3
TPK
Multiplechoice TK-1
PCK-2
TCK
Cognitive Level PK-3
PCK
Question
Type
Essay
Demonstrating appropriate learning strategies in real-world
Implementing learning methods appropriate to student
Critiquing learning designs PK-2
Table 1 The TPACK test blueprint
Question Item Indicators Number of Questions Explain the relationship between basic concepts in chemistry Analyzing students' misconceptions and chemistry Analyzing chemistry content in teaching materials TPACK-1
TPACK-2
Journal homepage: https://jurnal.
org/index.
php/j-edu Development and validation of technological pedagogical content knowledge (TPACK).
Table 2 Results of CVR analysis Number of validators who CVR value answered AuyesAy .
Ket
Ket
Ket
CK-1
*Valid *Valid *Valid CK-2
*Valid *Valid *Valid CK-3
Valid
Valid
*Valid PK-1
Valid
Valid
Valid
PK-2
Valid
Valid
Valid
PK-3
Valid
Valid
Valid
TK-1
Valid
Valid
Valid
TK-2
Valid
Valid
Valid
TK-3
Valid
Valid
*Valid PCK-1
Valid
Valid
Valid
PCK-2
Valid
Valid
Valid
PCK-3
Valid
Valid
Valid
TCK-1
Valid
Valid
Valid
TCK-2
Valid
Valid
Valid
TCK-3
Valid
Valid
Valid
TPK-1
Valid
Valid
Valid
TPK-2
Valid
Valid
Valid
TPK-3
*Valid *Valid *Valid TPACK-1
Valid
Valid
Valid
TPACK-2
*Valid *Valid *Valid Explanation .
The suitability of the item indicator with the item, .
The suitability of the item with the answer, .
The suitability of the item with the scoring rubric, *Valid = Valid if revised.
Item indicators TPACK Content validation results were analyzed using the Content Validity Ratio (CVR).
Initially, six items (CK-1.
CK-2.
CK-3.
TK-3.
TPK-3, and TPACK-.
required revisions due to issues with indicators, item wording, answer keys, or scoring rubrics.
After revisions and discussions with validators, all items were retained.
Postrevision, the CVR for all items reached 1.
00, surpassing the minimum threshold of 0.
99 for six validators, indicating strong content validity.
Table 3 Results of i-CVI analysis Validator Item
I-CVI
CK-1
1,00
CK-2
1,00
CK-3
1,00
PK-1
1,00
PK-2
1,00
PK-3
1,00
TK-1
1,00
TK-2
1,00
TK-3
1,00
PCK-1
1,00
PCK-2
1,00
PCK-3
1,00
TCK-1
1,00
TCK-2
1,00
TCK-3
1,00
TPK-1
1,00
TPK-2
1,00
TPK-3
1,00
TPACK-1
1,00
TPACK-2
1,00
S-CVI/Ave 1,00 Subsequently.
Content Validity Index (CVI) analysis was performed using a four-point relevance scale .
= highly relevant, 3 = quite relevant, 2 = somewhat relevant, 1 = not relevan.
According to Polit .
, items with I-CVI Ou 0.
78 and an overall scale-level CVI (S-CVI/Av.
Ou 0.
90 indicate strong content validity.
Results Journal homepage: https://jurnal.
org/index.
php/j-edu Rifa.
, et al Jurnal EDUCATIO (Jurnal Pendidikan Indonesi.
Vol.
No.
1, 2025, pp.
showed that all 20 items achieved I-CVI = 1.
00, and S-CVI/Ave = 1.
00, demonstrating excellent content This aligns with Paristiowati et al.
, who highlighted the critical role of expert validation in ensuring the content validity of TPACK assessment instruments.
Construct validity was assessed through a limited pilot test involving 31 chemistry education students.
Data were analyzed using the Rasch model.
The overall results indicated strong psychometric properties of the instrument, with mean infit MNSQ = 0.
99 and outfit MNSQ = 0.
98, values close to the ideal 1.
0, suggesting that participantsAo response patterns aligned well with the Rasch model expectations.
The item reliability was 91, indicating strong internal consistency and the instrumentAos ability to distinguish between different levels of TPACK competency among students.
However, four items (CK-2.
CK-1.
TK-2, and TCK-.
displayed potential misfit.
CK-2 had an outfit MNSQ 13 and a negative point-measure correlation (-0.
, suggesting poor alignment with the measured construct.
CK-1 and TK-2 had outfit MNSQ values approaching the upper threshold .
52 and 1.
and low correlations, indicating the need for improved wording and answer choices to reduce ambiguity.
TCK-3 showed an infit MNSQ of 1.
26, which, while within the acceptable range, suggested minor inconsistencies in participant The majority of other items had infit and outfit MNSQ values between 0.
70Ae1.
30 and positive pointmeasure correlations .
30Ae0.
, demonstrating their effectiveness in differentiating studentsAo TPACK competencies and supporting the unidimensionality assumption of the Rasch model.
These findings are consistent with previous research emphasizing the value of Rasch analysis in identifying non-functioning test items (Lestari et al.
, 2020.
Mustafa et al.
, 2.
Overall, the developed TPACK test instrument demonstrated strong construct validity, with minor revisions needed for four items to further enhance measurement accuracy and effectiveness.
Revisions will focus on refining item wording, improving answer options, and adding visual stimuli to improve clarity.
The findings of this study demonstrate that the development of a digital-based TPACK test instrument for prospective chemistry teachers successfully produced an instrument that is both content- and construct-valid.
The involvement of experts in the validation process played a crucial role in ensuring the relevance and quality of the test items, while Rasch analysis confirmed the consistency and the instrumentAos ability to differentiate levels of TPACK competence among students.
Although several items require revision, these results provide a solid foundation for future development, including broader-scale reliability and effectiveness testing.
Therefore, this research offers an initial contribution to providing a standardized assessment tool that can be used to measure and enhance the readiness of prospective chemistry teachers in effectively integrating technology, pedagogy, and chemistry content in 21st-century teaching practices.
Conclusions This study represents the initial stage of developing a digital-based test instrument designed to assess the Technological Pedagogical Content Knowledge (TPACK) competencies of pre-service chemistry teachers.
The instrument was developed through three main steps: needs analysis, blueprint design and item development, and content and construct validation.
The needs analysis revealed that although students are accustomed to integrating technology in their learning, there is still no standardized assessment that specifically measures the integration of TPACK The developed instrument consists of 20 items covering the seven TPACK components.
Content validation conducted by six expert validators resulted in a CVI score of 1.
00, indicating excellent content Construct validity testing using Rasch analysis with 31 pre-service teachers showed that the instrument demonstrates good measurement quality, with Infit MNSQ .
Outfit MNSQ .
, and item reliability of Four items were identified as suboptimal and require revision to improve measurement accuracy.
Overall, the instrument is valid in terms of content and construct and holds potential as an objective assessment tool for evaluating TPACK competencies in pre-service chemistry teachers.
Future studies should conduct large-scale field testing to further examine the reliability and effectiveness of the instrument.
Acknowledgments The author would like to express sincere gratitude to the Chemistry Education Study Program.
Faculty of Mathematics and Natural Sciences Education.
Universitas Pendidikan Indonesia, for the academic and institutional support provided throughout the course of this research.
The author also appreciates the guidance and feedback from the faculty members and all individuals who contributed to the successful completion of this Journal homepage: https://jurnal.
org/index.
php/j-edu Development and validation of technological pedagogical content knowledge (TPACK).
References