JURNAL PENDIDIKAN MATEMATIKA DAN IPA
Vol.
No.
http://jurnal.
id/index.
php/PMP ANALYSIS OF THE VALIDITY OF MULTIPLE-CHOICE
TEST ITEMS ON MASTERY OF TEMPERATURE AND
HEAT CONCEPTS FOR SENIOR HIGH SCHOOL STUDENTS
(SMA)
Jamiatul Ummah1.
Sutopo2.
Nandang Mufti3.
Sarah Lutfiah Ardilla4 Department of Physics.
Faculty of Mathematics and Natural Sciences.
Universitas Negeri Malang.
Jl.
Cakrawala No.
Sumbersari.
Lowokwaru District.
Malang City.
East Java.
Indonesia 65145 Email Corresponding: jamiatul.
2303218@students.
DOI: https://doi.
org/10.
26418/jpmipa.
Abstract This study aims to examine a test instrument to ensure its conformity with appropriate formats and procedures for measuring senior high school studentsAo mastery of temperature and heat concepts.
The instrument consists of 14 multiple-choice questions.
This study uses a quantitative descriptive method to evaluate and produce a highquality and valid test instrument.
The instrument underwent revision through consultation with expert lecturers and was then empirically tested on 95 students at SMAN 1 Pamekasan as respondents.
The test items were analyzed using SPSS version 30, covering four stages of empirical analysis: validity testing, reliability testing, item difficulty analysis, and item discrimination analysis, all conducted using classical test theory parameters.
Based on the results of the point biserial analysis, questions 1, 9, and 11 are below standard.
Reliability shows a Cronbach's alpha of 0.
708, which means it is The difficulty level test results show that out of 14 questions, 3 questions are categorized as difficult and 1 question is very easy.
In the discrimination test, 5 questions are categorized as poor, 3 questions are moderate, and 7 questions are good.
Overall, 11 questions were deemed acceptable and passed the empirical test, while 3 questions .
umbers 1, 9, and .
were disqualified.
The weakness of this instrument testing is that the results are highly dependent on the abilities of the participants.
For example, question number 9, which was adapted from PhysPort and simplified in language, still had to be disqualified because it did not meet the empirical criteria.
Keywords: Test Items.
Multiple Choice.
Concept Mastery.
Senior
High School
Received Revised Accepted : 2025-07-04
: 2025-09-02
: 2026-03-02
This work is licensed under a Creative Commons Attribution 4.
0 International License Jurnal Pendidikan Matematika dan IPA Vol.
No.
INTRODUCTION
Numerous studentsAo understanding of temperature and heat.
However, many findings in the field indicate persistent problems.
For example, among six physics topics investigated in a study .
namely kinematics, elasticity and HookeAos law, static fluids, optics, and temperature and hea-studentsAo understanding was found to be lowest in the concepts of temperature and heat.
Furthermore, students are often found to experience misconceptions regarding the concept of temperature and the effect of heat on thermal expansion related to changes in an objectAos dimensions .
This is caused by their failure to connect prior knowledge with new concepts and by possessing fragmented knowledge, such as merely memorizing facts without truly understanding the relationships among concepts .
, .
Students are also less skilled in analyzing physical quantities and in operating equations related to temperature and heat .
, .
addition, limited understanding arises from studentsAo inability to relate underlying scientific concepts .
These conditions have implications for studentsAo mastery of concepts.
StudentsAo mastery of concepts can be achieved when teaching strategiesAisuch as methods, learning models, approaches, and evaluation materials for each meetingAiare appropriately designed.
Evaluation serves as a benchmark for determining the extent to which learning objectives have been successfully achieved .
According to the Merdeka Curriculum, evaluation materials in current learning practices consist of three components:
formative assessment, summative instruments .
In practice, educators self-developed instruments without knowing the quality of those instruments .
Although assessment standards have been established, in reality there are still a number of schools that have not successfully achieved these assessment One contributing factor is the unavailability of standardized assessment instruments .
, .
The use of assessment instruments that do not meet established standards will reduce the accuracy of results or the achievement of learning objectives.
This condition makes it difficult for educators to distinguish between students who have achieved the required competencies and those who have not .
Consequently, when studentsAo concept mastery or learning outcomes fail to meet competency standards, the problem may lie in the test instrument itself.
The type of measurement commonly used to determine studentsAo mastery of concepts is multiple-choice testing, as employed in .
Multiplechoice items are test instruments that are widely used in formal assessments, for example in the administration of final examinations to measure studentsAo learning outcomes .
Ae.
However, the development of instruments to measure senior high school studentsAo concept mastery of temperature and heat in the form of multiple-choice items is still limited, resulting in a small variety of test items for each indicator.
Although there are validated and high-quality items available for the senior high school level, such as those provided by PhysPort, they are limited to thermal Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
and thermodynamics themes and do not specifically address instruments that examine the conceptual effects of heat on thermal expansion and BlackAos Other studies have also been conducted, such as .
, but the development of instruments in those studies focuses more on classical mechanics at higher education levels.
The implementation of final assessment in learning aims to evaluate the learning process and to identify deficiencies that need to be improved .
Therefore, the use of well-designed test instruments is essential for measuring studentsAo abilities.
This process is carried out to determine the characteristics of each test item and to assess whether it is suitable for use through systematic analysis .
, .
In addition, item analysis enables the evaluation of the extent to which the quality of test items measures the various aspects of the assessed competencies .
Consequently, item analysis constitutes an important step in producing assessment instruments that are valid, accurate, and capable of appropriately reflecting studentsAo Based on previous studies, this research aims to examine the validity, reliability, difficulty level, and discrimination index of the developed multiple-choice temperature and heat.
A valid instrument is able to measure the competencies that are intended to be assessed in learning .
, .
, .
The reliability of a test item is reflected in the consistency of studentsAo test results across different occasions .
, .
The level of difficulty should be balanced, meaning that items should be neither too easy nor too difficult .
, .
The use of appropriate, clear, and simple language that does not lead to multiple interpretations or confusion also greatly influences item quality .
, .
Therefore, it is important to conduct testing of the developed instrument to ensure that it meets the criteria of validity and reliability, as well as to determine the level of difficulty and the discrimination power of each item.
Instruments that have been proven to be valid and reliable will yield high-quality test items, measurement of studentsAo concept It is important to note that high-quality items do not function solely as tools for evaluating learning outcomes, but can also help strengthen studentsAo conceptual understanding.
Through items with proportional levels of difficulty and good discrimination power, students are encouraged to think more deeply, connect the concepts they have learned, and identify errors or misconceptions they may hold.
METHOD
This study is quantitative in nature and aims to provide an evaluation and produce a high-quality concept mastery test instrument suitable for general use.
The test consists of 14 multiple-choice items, which were administered to students at SMAN 1 Pamekasan who had completed the temperature and heat The study involved 95 respondents from three eleventh-grade classes, with each class comprising approximately 30 to 35 students.
sample size of around 30Ae100 respondents is considered sufficiently accurate for instrument testing .
Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
Prior to administration, the test items underwent validation by expert physics lecturers through consultation and several stages of revision.
The test items were designed to measure studentsAo concept mastery in the topic of temperature and heat.
A description of each test item is provided in Table 1.
Table 1.
Description of Test Items on Concept Mastery of Temperature and Heat Cognitive Level Topic Item Question Content Temperature Expansion Understanding the effect of low atmospheric pressure on the boiling point of water and its impact on making tea in mountainous areas Distinguishing between the expansion of a solid and a hollow sphere when subjected to the same temperature Explaining the process of opening a metal bottle cap through heating Explaining the relationship between the amount of heat energy absorbed by an object and the specific heat capacity of that object Determining which of two objects with different properties has a higher specific heat capacity based on the comparison of their temperature rise rates when heated simultaneously Explaining the effect of pressure on the boiling point of water based on the process of cooking with a pressure cooker Understanding the main idea of BlackAos principle .
wo objects with different temperatures mixed or in thermal contac.
Applying BlackAos principle to determine the temperature of two substances in thermal contact Analyzing the conservation of heat energy to determine the final temperature of a mixture of two substances with different masses and initial temperatures in an isolated system Explaining the structure of a thermos bottle to prevent rapid temperature loss Understanding the difference in the sensation of cold on objects made of different materials based on the concepts of thermal conductivity and heat transfer rate Explaining the rationale for placing an air conditioner at the top of a room based on the principle of heat transfer by convection Heat and Its Relationship Temperature and State Changes BlackAos Principle Heat Transfer Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
Topic Item Question Content Explaining the cause of temperature increase on asphalt surfaces exposed to sunlight based on the process of heat transfer by radiation Explaining the function of thick curtains in inhibiting heat transfer through conduction and radiation to maintain room temperature stability The empirical testing of the items was analyzed using classical test theory parameters with the assistance of SPSS version 30, including the point-biserial correlation coefficient .
, the discrimination index (D), and the difficulty index .
for each Validity testing was conducted using the point-biserial coefficient, examined through Pearson correlation and significance values to determine the level of validity.
Given the sample size of 95 students, the critical r-value used was 0.
202 at a 5% significance level .
Reliability testing was assessed using CronbachAos alpha to determine the accuracy and consistency of the test instrument in measuring the dependent variable.
Cognitive Level ensuring that it can be used repeatedly.
Item discrimination was evaluated using the Corrected ItemAeTotal Correlation for each item.
Item difficulty was assessed by dividing the mean score by the maximum score.
The final decision on whether an item is suitable for distribution as a measure of studentsAo concept mastery depends on whether it meets the parameter criteria, as presented in Table 2.
Table 2.
Test Eligibility Parameters Classical Test Parameters Point-Biserial Correlation Reliability Discrimination Index Difficulty Level Eligibility Criteria Ou 0,2 Ou 0,7 Ou 0,3 0,3 - 0,9 .
A ring and a solid ball are made of the same metal.
At first, the solid ball barely fits into the ring.
In order for the ball to fit into the ring, which alternative is the most effective? Image source:
https://youtu.
be/GgrnhHMqZDY?si=oQnZj_ 58ZYzU4OQJ The ring is heated, but the solid ball is left alone.
The solid ball is cooled, the ring is left alone.
The ring is heated and at the same time the solid ball is cooled.
The ring is cooled and at the same time the solid ball is heated.
Figure 1.
Example of a Concept Mastery Test Item on Temperature and Heat Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
RESULT AND DISCUSSION
The empirical testing of the concept mastery test instrument involved 14 multiple-choice items, which were administered to 95 eleventh-grade senior high school students who had completed the temperature and heat material.
Before presenting the analysis results, the researcher provides examples of several tested items in Figure 1, and access to the complete set of instruments is available at the https://drive.
com/file/d/1Y0bY BTD1iWDed4N16eupyELlC3TtaJBj/ view?usp=sharing.
The results of the classical test parameter analysis are presented in Table 3.
Table 3.
Results of Classical Test Parameter Analysis for the Concept Mastery Test on Temperature and Heat Item Point-Biserial Coefficient Test Based on Table 3, ycycyycaycn indicates the validity of each test item.
The analysis results are presented in Table Table 4.
Classification of ycycyycaycn for Test
Items Number
of items Item
2, 3, 4, 5, 6,
7, 8, 10, 12,
13, & 14
1,9, &11
Category Valid Not valid ycycyycaycn Tables 3 and 4 show that 11 items passed and were classified as valid, while 3 itemsAinumbers 1, 9, and 11Aiwere found to be invalid.
These three items cannot be used to measure studentsAo concept mastery because their Pearson correlation values fall into the low category, meaning the calculated r-values are below the critical r-value, indicating that the items are of low quality and therefore unsuitable for measuring the dependent variable in this study.
According to .
, .
, an instrument with low validity interpretation but a significance value below 0.
05 is not Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
suitable for use, as the minimum acceptable validity interpretation should be at least AuadequateAy for the instrument to be usable.
The invalid itemsAi1, 9, and 11Aiconfused students during completion .
This may be due to confusing item formats, such as improper wording, the presence of two correct answer options, or possibly studentsAo low ability, and in some cases, the items may have been too easy or too difficult.
Thus, multiple factors can contribute to an itemAos invalidity.
Based on these possible factors, the researcher predicts that all of them could have occurred, affecting the itemsAo validity, which is closely related to the difficulty level of the test items.
Meanwhile, the 11 valid items were mostly classified as having AuadequateAy quality, with one item rated as high This is because the item scores for each question were slightly lower than the total test score, resulting in the majority of valid items being rated as Auadequate.
Ay As noted by .
, .
, there is a relationship between the item score and total score regarding the height or significance of the Pearson correlation and the validity of the item.
If each item score aligns closely with the total score, the resulting validity is Reliability Test The conducting validity screening, is to ensure the reliability of the test The purpose is to confirm that the instrument produces consistent results when used, allowing for repeated measurements or use in the future and/or with different subjects .
Ae.
Figure 2.
Reliability Test Results for 14 Items Reliability Statistics Cronbach's Alpha N of Items Figure 3.
Reliability Test Results for 11 Items The reliability test was analyzed twice to examine the difference in reliability before and after item deletion, following the prior validity Figure 2 shows the results of the reliability test, indicating that the 14item version of the test was categorized as unreliable, with a CronbachAos Alpha In contrast, the 11 valid items, after reliability testing, were classified as moderate and acceptable, with a CronbachAos Alpha of 0.
This index value of 0.
708 meets the desired threshold of Ou0.
7, indicating that the 11 items generally measure the same conceptAitemperature and heat.
This category is referred to as moderate reliability because it lies on the borderline between reliable and The reliability index for the 11 items, classified as moderate, indicates a medium but adequate correlation among test items and consistency in measuring the concepts of temperature and heat .
, .
This demonstrates that the 11 items can be used to differentiate studentsAo levels of mastery in the concepts of temperature and heat.
Item Difficulty Test Based on Table 3, the analysis of the difficulty level for the 14 test items Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
shows that most items fall into the moderate category, meaning their difficulty is neither too high nor too The classification of item difficulty is presented in Table 4.
Table 4.
Difficulty Level of Each Test
Item
Number of items by Category item number
2, 3, 4, 5, 6, 7, 8, 12,
Moderate 13, & 14
9, 10, &11
Difficult Very easy The analysis results show that three itemsAinumbers 9, 10, and 11Ai fall into the difficult category, while one item is classified as very easy.
These findings align with student feedback in the field, indicating that most items were perceived as having moderate to high difficulty.
This is further supported by the studentsAo responses, where many did not provide an answer or gave two answers for the same item, suggesting confusion in selecting the correct option in the multiple-choice questions.
Nevertheless, item number 10 was retained because it is part of the 11 items with acceptable reliability.
Although its difficulty index is still 0.
it is conventionally considered adequate and acceptable .
Moreover, retaining test items with higher difficulty levels has been applied in the development of KCT instruments .
As long as the items are neither extremely difficult nor extremely easy, they can still be used to provide variation in item difficulty .
Therefore, the inclusion of difficult items in an instrument can be maintained as long as it aligns with the objectives of the instrumentAos development .
Item Discrimination Test Based on the data presented in Table 3, it can be concluded that the discrimination levels of the test items, in measuring concept mastery, range 009 to 0.
Analysis of the 14 items shows that they fall into three categories: poor, moderate, and good.
Among all items, five were classified as poor .
tems 1, 2, 6, 9, and .
, two items .
were classified as moderate, and seven items were classified as good.
These findings are in line with .
, who stated that items with very low discrimination should be removed or not reused.
This is further supported by .
, .
, who noted that the higher the discrimination value, the greater the proportion of students able to answer the item correctly.
a low discrimination value indicates that the item is more difficult and less effective in distinguishing studentsAo Based on this analysis, three items categorized as poor among the 14 test items were removed.
Meanwhile, items 2 and 6 were retained because they remain on the borderline between poor and moderate Another reason for retaining these items is that the instrument is intended to provide feedback to students and support improvements in the learning process, consistent with the findings of .
Therefore, it can be concluded that the test instrument suitable for use and meeting the discrimination criteria consists of 11 items, including those in the moderate category as well as items 2 and 6.
Jamiatul Ummah.
Sutopo.
Nandang Mufti.
Sarah Lutfiah Ardilla Analysis of The Validity of Multiple-Choice Test Items on Mastery of Temperature and Heat Concepts for Senior High School Students (SMA) Jurnal Pendidikan Matematika dan IPA Vol.
No.
This study is valuable for producing higher-quality assessment instruments on the topic of temperature and heat at the senior high school level.
The findings can serve as a reference for teachers and other researchers in designing test items that align with studentsAo characteristics, ensuring that the instruments used are not only valid and reliable but also accurately reflect concept mastery.
In the future, similar research can be extended to the same topics at different educational levels, providing comprehensive instruments tailored to specific learning needs.
However, a comprehensive analysis revealed a notable weakness in the testing of the instrument.
According to classical test theory, when examining item difficulty and discrimination characteristics, a significant limitation was observed in this study.
This weakness is related to the ability of the test takers, which aligns with the findings of .
Additional evidence of this limitation is seen in item number 9, originally taken from PhysPort and retested with language modifications to facilitate student comprehension.
practice, however, item 9 was found to be invalid, with its discrimination and difficulty indices rated as Auvery poor.
Ay This indicates that the issue was not with the quality or usability of the instrument itself, but rather depended on the studentsAo abilities and skills.
CONCLUSION
The analysis of the 14 test items used to measure concept mastery, administered to 95 eleventh-grade students at SMAN 1 Pamekasan, concluded that 11 items were valid and suitable for use.
The reliability analysis also showed a CronbachAos Alpha of 708, indicating that these 11 items are both valid and reliable.
Among the 14 items, students experienced difficulty answering items 9 and 11, which were classified as very difficult based on the difficulty test.
The item discrimination analysis categorized the 14 items into three groups: 5 items were poor, 3 items were moderate, and 7 items were good.
Based on this analysis, only 11 items met the empirical testing criteria according to classical test parameters.
The implication of this study is that the 11 items can be used to measure studentsAo concept mastery in future research and serve as assessment Additionally, the instrument can be further developed into other types of assessment instruments.
REFERENCES