Jurnal Ilmu Pendidikan (JIP) Vol.
Issue 1.
June 2023, pp.
ISSN: 0215-9643
e-ISSN: 2442-8655 A Cognitive Assessment Instrument for HOTS Development in Technical Drawing Course Utilizing Quizziz Learning Platform Widiyantia,1,*.
Roswina Dianawatib.
Futachia aUniversitas Negeri Malang.
Jl.
Semarang No.
Sumbersari.
Kota Malang, 65145 Indonesia bUniversitas Negeri Surabaya.
Jl.
Lidah Wetan.
Lidah Wetan.
Surabaya 60213.
Indonesia ft@um.
*corresponding author
ARTICLE INFO
Article history Received 11, 3, 2023 Revised 8, 10, 2023 Accepted 6, 01, 2023 Keywords Instrument Quizizz Higher Order Thinking Skills Technical Drawing ABSTRACT Assessment or evaluation is the activity of collecting student information or a program of activities related to certain dimensions or attributes, such as student learning outcomes in teaching a field of study or the effectiveness of studying a field of study organized by the teacher, to be compared.
Dimensional or characteristic excellence standards are associated with students or activity programs.
For example.
Quizizz, a digital platform that helps students master reading, is a multiplayer class activity that allows all students to practice reading skills using mobile devices, such as tablets, iPads, and Smartphones.
This research will use the Research and Development (R&D) method using the ADDIE research development model which is very applicable in the development of formal education.
Based on the results of the validation value analysis carried out by two validators, the average percentage of very feasible category assessment instruments was 90.
62 percent.
Based on the results of the validity test shown on the 32 multiple-choice HOTS questions used in the pilot study with 33 students, 26 items were determined to be valid.
Based on the results of the reliability test, the value of the test analysis is included in the "high" category.
The study of the difficulty level of the 32 questions yielded the following results: 15 simple questions, 14 moderate questions, and three challenging questions.
In addition, according to the analysis of discriminatory power, nine items fall into the proper category, 21 items fall into the good category, one item falls into the poor category, and one item falls into the very poor category.
This is an open-access article under the CCAeBY license.
Introduction According to Regulation No.
53 of the Minister of Education and Culture of the Republic of Indonesia for the Evaluation of Learning Outcomes by Educators and Education Units in Primary and Secondary Education (Mataputun, 2020.
Sherly et al.
, 2.
, educators evaluate learning outcomes in a planned and systematic manner regarding attitudes, knowledge, and skills to monitor processes and learning progress and to enhance the process of collecting information/data on student learning Technical drawing is a subject taught at Vocational High School (VHS) Turen, particularly in the Mechanical Engineering Department.
It is taught in vocational institutions to prepare students for entering employees, whether in industry or as entrepreneurs (Drastiawati et al.
, 2.
The mastery of drawing theory can influence the success of the practice in technical drawing.
In this course, students must comprehend the engineering drafting http://dx.
org/10.
17977/um048v29i1p1-6 In addition, students must understand various drawing parameters, including determining line thickness, pictograms, and projecting images.
As a result of the theory of technical drawing, students are anticipated to be more talented at drawing.
To comprehend the theory, an evaluation or assessment of student learning outcomes is Evaluation is one of the most critical activities in school education, from elementary to higher education (Kurniawaty et al.
, 2022.
Razi, 2.
In general, and in the context of school education, assessment is the activity of gathering information about students or about a program of activities related to certain dimensions or attributes, such as student learning outcomes in teaching a field of study or the effectiveness of learning in an area of study organized by the teacher (Zainal, 2.
This information is then compared with specific criteria to determine the quality of the dimensions or attributes being Jurnal Ilmu Pendidikan (JIP) Vol.
Issue 1.
June 2023, pp.
assessed is satisfactory (Setiawan & Tumardi, 2019.
Zainal, 2.
Further, the observations conducted at VHS Turen during teaching practice activities and interviews with instructors in the Mechanical Technical drawing subject indicate that students' understanding of Mechanical Technical drawing theory remains low.
Technical Drawing is introduced to the tenth graders, and theoretical learning is not conducted until the commencement of practical knowledge.
therefore, students are expected to know all drawing parameters quickly.
Students typically favor hands-on experience over learning theory first.
When students practice without drawing parameters, they are more likely to join their peers during practice and feel uncomfortable beginning first.
It is due to a lack of student proficiency with technical drawings, particularly mechanical ones.
In addition, the assessment or assessment process utilized at VHS Turen still employs paper-based assessment methods .
aper exam.
for measuring student learning outcomes.
The shortcoming of the paper-based or pencil-and-paper test is that the procurement of logistics in paper and the duplication of questions is expensive and inefficient.
In addition, the results of printing or photocopying are occasionally of poor quality, unappealing, blurry, and unclear writing due to poor photocopying quality or technical errors in duplicating queries.
It will undoubtedly make it more difficult for students to peruse the questions and take the time to correct and enhance them.
In contrast to the computerized test, the display of the questions is more aesthetically pleasing and visible.
It does not require the expense of printing or duplicating questions, making them more practical and cost-effective.
This investigation utilized a Higher Order Thinking Skills (HOTS)-type cognitive assessment instrument (Setiawan & Tumardi, 2.
HOTS are not limited to recalling, restating, or referring without processing .
(Hamzah et al.
, 2.
Thus.
HOTS helps students practice critical thinking by requiring them to solve problems using these skills.
It is consistent with the principle that higher-order thinking is higher than merely memorizing data or exactly repeating what was communicated (Ichsan.
Sigit, & Miarsyah, 2019.
Ichsan.
Sigit.
Miarsyah, et al.
, 2.
Higher cognitive levels are more elevated than memorizing in the dimensions of thought processes.
Those higher cognitive levels of Bloom's Taxonomy are C4 .
C5 .
, and C6 .
According to the researcher, the HOTS type of cognitive test instrument measures not only the ability to recall information .
(Hadi et al.
, 2018.
Husamah et , 2.
but also the ability to: .
transfer one concept to another, .
process and apply information, .
find links between different information, .
use the information to solve problems, and .
critically evaluate ideas and It does not follow that the HOTS questions ISSN: 0215-9643 e-ISSN: 2442-8655 are more complicated than the Lower Order Thinking Skills (LOTS) questions (Hadi et al.
, 2018.
Retnawati et , 2.
, which only measure C1 .
C2 .
, and C3 .
abilities based on the level of the HOTS instrument and the measured ability aspect (Hamzah et al.
, 2022.
Ichsan.
Sigit.
Miarsyah, et al.
This study will generate a cognitive research instrument of the HOTS type based on the game Quizizz (Candel et al.
, 2021.
JankoviN & LambiN, 2.
The objective is for students to have higher-order problemsolving abilities.
The assessment's HOTS instrument measures students' ability to transfer one concept to another, process and apply the obtained information, look for connections between various information, use data to solve problems, and critically evaluate ideas and The HOTS assessment instrument (Brookhart, 2.
has two types: multiple-choice and The objective of the multiple-choice assessment is to determine whether students comprehend a problem between two statements.
While the description instrument requires students to organize what they have learned by conveying ideas in their own words, the evaluation instrument requires students to manage what they have learned visually.
II.
Method Research and Development Models Utilizing research and development methodologies is the method of inquiry.
Research and Development (R&D) methods are techniques used to produce and evaluate the efficacy of specific products.
In the area of education, there are many different types of research and development The R&D applied the ADDIE development model (Branch, 2009.
Spatioti et al.
, 2.
comprising five phases described in Figure 1.
Fig.
Stages of Research Instruments for the Development of HOTS Questions Widiyanti et.
al (A Cognitive Assessment Instrument for HOTS Development in Technical Drawin.
ISSN: 0215-9643 e-ISSN: 2442-8655 Jurnal Ilmu Pendidikan (JIP) Vol.
Issue 1.
June 2023, pp.
Analyze Analyzing the requirements of the learners, the learning environment, and the intended learning outcomes is the first phase of the ADDIE model.
Next, we will examine the state of technical drawing instruction, pinpoint areas that need HOTS development, and decide how cognitive evaluation might fill these gaps.
Finally, to assist the growth of HOTS in technical drawing courses, we will also study the Quizziz learning platform.
Design After completing the analysis, the next stage is to create a blueprint for the learning experience.
It entails formulating learning objectives, choosing the most effective teaching strategies and media to accomplish those goals, and devising a thorough strategy for assessment and evaluation.
In this research study, we'll create a cognitive assessment tool that uses Quizziz to help students learn technical drawing concepts while supporting the development of HOTS.
We will establish the test's learning objectives, choose the ideal question kinds and degree of difficulty to encourage HOTS and develop a thorough plan for using and assessing the test.
Development In the development phase, the actual learning experience is created.
The creation of the instructional materials and the building of the assessment tool both occur at this stage.
Creating a database of questions, designing a user-friendly administration interface, and testing the instrument for validity and reliability are all steps in developing the cognitive assessment tool for HOTS development in technical drawing disciplines using Quizziz.
Implementation In the implementation phase, the learning experience is made available to learners.
The learners receive the educational materials during this phase, and the assessment tool is used.
We will use the Quizziz learning platform to implement the cognitive assessment tool in this research project.
We will collaborate with technical drawing teachers to deliver the test to students and gather information on their HOTS progress.
there were five 11th-grade classes, namely TPM 1.
TPM 2.
TPM 3.
TPM 4, and TPM 5 at VHS Turen.
The sampling approach employed stratified random sampling by defining the sample collected randomly depending on the levels included in the population.
The subject of this research and development work was conducted in the TPM 2 class.
The selection of TPM 2 was based on its superiority and high quality in terms of student Thirty-three students from the TPM 2 class participated in the product testing.
Product Trial Time In August 2021, two validators, consisting of a Mechanical Engineering lecturer from the State University of Malang and a technical drawing teacher from Turen Malang Vocational School, validated the assessment In September of 2021, during the Even Semester of the 2021/2022 Academic Year, assessment instruments of R&D for TPM 2 class VHS Turen were Data analysis technique Data analysis is the process of systematically searching and compiling the results of data obtained in the research process through data collection techniques such as documentation by selecting the most important things to be studied and drawing conclusions that are easily understood (Sugiyono, 2.
Descriptive quantitative data analysis techniques were utilized to develop the Quizizz-based HOTS cognitive assessment instrument for technical drawing subjects.
Development of HOTS Assessment Instruments Quantitative data validation techniques used in developing assessment instruments focused toward Quizizz-based HOTS for 10th-grade VHS students can be categorized into percentage calculation techniques and test and item quality analysis techniques.
Percentage calculation technique The first quantitative data analysis technique uses descriptive analysis using percentages according to the following formula:
Evaluation The effectiveness of the learning experience is evaluated at the end of the process.
At this stage, data on student performance is acquired, students' and teachers' input are received, and the usefulness of the assessment tool is examined.
In this research study, we will analyze student performance data, gather feedback from educators and learners, and assess the instrument for effectiveness to determine the efficacy of the cognitive assessment tool for HOTS development in technical drawing subjects using Quizziz.
Product Trial Subject The HOTS assessment instrument that the validator has validated is prepared for testing.
When in the field, ycE= Ocycu ycu100% ycu Description:
P = Percentage x = Total scoring score n = Ideal score .
ighest score for each aspect x number of validator.
Table 1.
Assessment instrument eligibility criteria Score Appropriateness 81% - 100% 61% - 80% Excellent Good Widiyanti et.
al (A Cognitive Assessment Instrument for HOTS Development in Technical Drawin.
Jurnal Ilmu Pendidikan (JIP) Vol.
Issue 1.
June 2023, pp.
Score Appropriateness No.
41% - 60%
21% - 40%
0% - 20 %
Average Poor Very Poor .
Test and item analysis techniques The item analysis technique was performed to find out information about the level of difficulty, discriminating power, and distracting analysis of the product of the i.
Results and Discussion Product Development Instrument This development study yielded a Quizizz-based cognitive assessment tool for Higher Order Thinking Skills (HOTS).
The product of the development of this assessment instrument consists of multiple components:
a grid of questions, .
questions, .
answer keys and discussion, .
an explanation of why the question is included in the HOTS question category, and .
a view of the Quizizz app.
Students' In addition, students' higherorder thinking skills are evaluated with the aid of valid queries that have been modified.
Analysis of Feasibility Test Assessment Instruments The following outcomes were obtained from the study of the product feasibility test of the Higher Order Thinking Skills (HOTS) type of cognitive assessment instrument based on Quizizz in the Technical drawing subject: .
product validation, .
item validity, .
item question reliability, .
item question difficulty level, and .
item differentiating power.
Product Validation of Assessment Instruments Two validators, one mechanical engineering lecturer from the State University of Malang and one technical drawing instructor from Turen Vocational High School, validated the assessment instrument.
Validation is performed to determine the assessment instrument's usability and capture qualitative data that can be used to enhance and refine the tool.
Expert validators assess numerous facets of the instrument product, such as its compatibility with the indicators, scientific knowledge, cognitive level, and language usage.
These facets are rated on a Likert scale, and remarks and suggestions for improving the development of the assessment instrument are provided.
Based on an analysis of the validation values offered by the two expert validators, the average percentage of the product's feasibility is 90.
62 percent, with a category of "very feasibleAy Auexcellent" category.
Ay Table 2.
Product validation results by two validators No.
Xmaks.
Description Very Worthy Very Worthy Worthy Very Worthy ISSN: 0215-9643 e-ISSN: 2442-8655 Xmaks.
Average Validation Value Description Very Worthy Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Worthy Very Worthy Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy Very Worthy 90,62% Item Validity Test Results According to the findings of the validity test on 32 multiple-choice HOTS questions utilized in a short trial with 33 students, 26 questions are valid since the computed r-value is larger than the r-table value.
Comparatively, six queries are invalid because the estimated r value is less than the r-table value .
-table = 344, significance = 0.
Table 3.
Item Validity Test Results No.
r Count
r Table Significance Category 0,428 0,481 0,007 0,321 0,384 0,403 0,528 0,645 0,500 0,390 0,430 0,403 0,340 0,429 0,390 0,446 0,434 0,340 0,508 0,436 0,486 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,344
0,013
0,005
0,968
0,069
0,027
0,020
0,002
0,000
0,003
0,025
0,012
0,020
0,053
0,013
0,025
0,009
0,012
0,053
0,003
0,011
0,004
Valid Valid Invalid Invalid Valid Valid Valid Valid Valid Valid Valid Valid Invalid Valid Valid Valid Valid Invalid Valid Valid Valid Widiyanti et.
al (A Cognitive Assessment Instrument for HOTS Development in Technical Drawin.
ISSN: 0215-9643 e-ISSN: 2442-8655 Jurnal Ilmu Pendidikan (JIP) Vol.
Issue 1.
June 2023, pp.
No.
r Count
r Table Significance Category 0,465 0,373 0,397 0,417 0,553 0,338 0,448 0,193 0,497 0,453 0,481 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,344 0,006 0,033 0,022 0,016 0,001 0,054 0,009 0,282
0,003
0,008
0,005
Valid Valid Valid Valid Valid Invalid Valid Invalid Valid Valid Valid Percentage of Item Difficulty Level Easy Currently Hard Fig.
Diagram of the Percentage Level of Difficulty of Items Results of the Differentiating Power of Questions Item Reliability Test Results Table 4.
Item Reliability Test Results Case Processing Summary Cases Valid Excludeda Total Listwise deletion based on all variables in the procedure.
Table 5.
Reliability Statistics Cronbach's Alpha N of Items Based on the reliability test results for the test of higher-order thinking abilities in technical drawing subjects using SPSS, the test reliability analysis value was determined to be 0.
843, placing it in the "high.
" The results of this analysis indicate that the test has a high level of confidence, suggesting that it can be applied repeatedly and on numerous occasions.
Test Results of Item Difficulty Level The calculation of the item's difficulty level represents a measurement of the item's degree of difficulty.
question is deemed excellent if its difficulty level is proportional to its value.
The test question should be simple enough.
Based on the results of the analysis of the level of difficulty of 32 questions, 15 questions fell into the category of easy questions with a proportion of 47%, 14 questions fell into the category of moderate questions with a ratio of 44%, and three questions fell into the category of difficult questions with a proportion of 9%.
Item Difficulty Level The calculation of discriminatory power measures the ability of an item discriminatory power calculation measures an item's ability to distinguish between students who can answer a question and students who cannot answer a question.
The coefficient of discriminating power of a question indicates the items' ability to distinguish between students who comprehend competence and those who do not.
The results of the analysis of the discriminatory power of the questions are as follows: 9 items in the good category with a 28% percentage, 21 items in the good category with a 66% percentage, 1 item in the bad category with a 3% percentage, and 1 item in the very bad category with a 3% percentage.
Power of Different Items 1 4 6 8 10 12 14 16 18 20 22 24 26 28 31 Power of Different Items Fig.
Diagram of the Differential Power of Questions Percentage of Differentiating Power of Items Very Good
Good
Enough Bad
Very Bad
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Item Difficulty Level Fig.
Diagram of the Percentage of Differential Power of Items Fig.
Diagram of Item Difficulty Level Widiyanti et.
al (A Cognitive Assessment Instrument for HOTS Development in Technical Drawin.
Jurnal Ilmu Pendidikan (JIP) Vol.
Issue 1.
June 2023, pp.
IV.
Conclusion Based on the discussion of prior research, it is possible to infer this research and development resulted in a Quizizz-based cognitive assessment tool of the Higher Order Thinking Skills (HOTS) variety.
The developed instrument's indicators of higher-order thinking skill.
a grid of questions, .
questions, .
answer keys and discussion, .
an explanation of why the question pertains to the HOTS question category, and .
the appearance of the Quizizz application is the components of the The devised instrument features 32 multiplechoice questions, 27 of which are multiple-choice and five complex multiple-choice questions with four answer However, the development of this instrument has several things that could be improved, including questions that were created based on the fundamental competencies (KD) and the fact that only 33 VHS Turen students served as test subjects.
The instrument for assessing higher-order thinking abilities was then validated by two validators, including a lecturer in mechanical engineering from the State University of Malang and a technical drawing teacher from Vocational High School Turen.
The average percentage of assessment instrument product feasibility is 62 percent.
These findings indicate that the developed instrument is a very appropriate tool for measuring students' higher order thinking abilities in technical drawing courses.
26 items out of 32 with query reliability 843% fall into the good category.
The reliability results indicate that the developed questions have a very high-reliability level, so if the test is repeated on the same sample at various times, the results will be virtually References