Indonesian Journal of Educational Development (IJED)
Volume 5, Issue 4, 2025, pp. 429-443
ISSN 2722-1059 (Online); ISSN 2722-3671 (Print)
DOI: https://doi.org/10.59672/ijed.v5i4.4324

DEVELOPMENT OF INSTRUMENT FOR UNDERSTANDING
THE CONCEPT OF WRITING SCIENTIFIC ARTICLES FOR
LECTURERS USING THE ADDIE MODEL: EMPIRICAL
VALIDITY AND RELIABILITY
Luthfi Ilham Ramdhani1, Yoga Dwi Arianda*)2, Awaluddin Tjalla3, Lussy Dwiutami
Wahyuni4
1Universitas Negeri Jakarta, Jakarta, Indonesia; luthfi.ilham@mhs.unj.ac.id
2Universitas Negeri Jakarta, Jakarta, Indonesia; yoga.dwi@mhs.unj.ac.id
3 Universitas Negeri Jakarta, Jakarta, Indonesia; awaluddin.tjalla@gmail.com
4Universitas Negeri Jakarta, Jakarta, Indonesia; lussysf@unj.ac.id

*)Corresponding author; Yoga Dwi Arianda: yoga.dwi@mhs.unj.ac.id

Abstract. This study aims to develop an instrument to assess
lecturers' conceptual understanding of scientific article
writing using the ADDIE model (Analyse, Design, Develop,
Article history:
Implement, Evaluate). The instrument is designed to evaluate
Received December 09, 2024
lecturers' comprehension of key aspects of scientific writing,
Revised December 20, 2024
including structure, methodology, publication ethics, and the
Accepted January 03, 2025
use of academic language. Adopting a research and
Available online February 20, 2025
development approach, the study involves expert validation
and empirical testing with 275 respondents from
variousregions across Indonesia. The findings indicate that
Keywords: ADDIE, conceptual
the instrument exhibits high content validity, with an average
understanding instrument,
Scale Content Validity Index (S-CVI/Ave) of 0.968, and
scientific article writing, validity,
robust internal reliability, as evidenced by a Cronbach’s
reliability
Alpha of 0.852. Item-total correlation analysis confirmes the
Copyright ©2024 by Author. Published by Lembaga
validity of all items, with Corrected Item-Total Correlation
Penelitian dan Pengabdian kepada Masyarakat
values exceeding 0.3044. The instrument functions both as
Universitas PGRI Mahadewa Indonesia
an evaluative tool for assessing lecturers' scientific writing
skills and as a learning resource to enhance their competencies. The study concludes that the
instrument is a valid and reliable measure of lecturers' conceptual understanding of scientific article
writing. Its implications highlight its potential as a vital resource in training and professional
development programs for lecturers in higher education institutions.
ARTICLE INFO

INTRODUCTION
Writing scientific articles is one of the essential skills that must be possessed by academics,
researchers, and students. Scientific articles are not only a medium for conveying research
results to the academic community, but also serve as an indicator of the author's academic
ability in understanding and applying relevant concepts. However, various studies show that
the ability to write scientific articles is still a major challenge among academics and students,
especially in developing countries, including Indonesia (Lubis et al., 2019; Udil, 2021). These
problems include limited understanding of the structure of scientific articles, lack of mastery

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

429

of research methodology, and minimal evidence-based argumentation skills. This condition
creates an urgent need for training and mentoring programs that can help academics improve
the quality of their writing, so that the resulting research can be more easily published and
have a positive impact on the development of science. These programs must be designed
comprehensively, covering theoretical and practical aspects, and involving experts in the field
of scientific writing to provide effective guidance (Yusuf et al., 2022).
The importance of writing scientific articles for university lecturers not only functions as an
indicator of academic productivity, but also as a form of responsibility in carrying out the
tridharma of higher education, namely education, research, and community service. Based
on Law Number 12 of 2012 concerning Higher Education, lecturers are required to conduct
research and disseminate the results in the form of scientific publications as a form of
developing science and technology. Furthermore, the Regulation of the Minister of
Education, Culture, Research, and Technology Number 39 of 2021 concerning Academic
Integrity emphasises that every lecturer must comply with ethical and integrity standards in
producing scientific works, including writing scientific articles. Thus, the ability to write
scientific articles is one of the competencies that cannot be ignored by lecturers to meet the
demands of professionalism and make a real contribution to the development of science
(Yasa et al., 2023).
The Guidelines for Scientific Journal Accreditation Dikti (2024) firmly places the assessment
of the substance of scientific articles as one of the main criteria. This emphasises that
lecturers, as the main pillars in the academic world, have a great responsibility to contribute
to the development of science through the publication of scientific works. By producing
quality articles, lecturers not only improve the reputation of higher education institutions,
but also encourage the birth of innovation and solutions to various problems faced by
society. In improving the ability of lecturers to write scientific articles, the Directorate of
Research, Technology, and Community Service of the Directorate General of Higher
Education, Research, and Technology of the Ministry of Higher Education, Research, and
Technology also plays a role in facilitating lecturers through organising scientific article
writing training held in 17 cities. In the 2024 Scientific Article Writing Training Guide
(DRTPM, 2024) it is stated that the program's target is to increase the ability of lecturers to
publish scientific articles from their research in accredited journals and reputable journals,
which can be used to fulfill the requirements for academic promotion and research output.
One important aspect in writing scientific articles is understanding the basic concepts
underlying the preparation of the article. This understanding includes the structure of
scientific articles, the use of academic language, and mastery of the rules for compiling
references and citations. According to Swales & Feak (2012), the structure of a scientific
article consists of several main components, such as introduction, methods, results, and
discussion (IMRAD), which each have their own role in conveying information logically and
systematically. Therefore, mastery of these concepts is a primary prerequisite for lecturers to
be able to produce scientific articles that meet quality standards. Day & Gastel (2006)
emphasise the importance of a systematic and logical writing structure to produce good
scientific articles. Day & Gastel's writing cycle theory provides a comprehensive framework
for scientific article writers. This model describes the writing process as a series of
interrelated stages, starting from planning, writing the initial draft, revision, to final
completion. In the context of writing scientific articles, each stage has a crucial role.

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

430

The conceptual theory of scientific article writing can be referred to the CARS (Create A
Research Space) model introduced by Swales (1990). This model emphasises the importance
of building a research context in a systematic manner through three main steps, namely (1)
establishing a research space by presenting relevant and important topics, (2) identifying gaps
in previous research, and (3) explaining the purpose or contribution of the research
conducted. This approach provides a clear conceptual framework for writers to compose an
effective introduction to a scientific article. In addition, the writing process theory of Flower
and Hayes (1981) is also relevant in understanding the dynamics of writing scientific articles.
This theory describes the writing process as a cognitive activity involving three main
components, namely the rhetorical situation, the writer's cognitive processes (writing
processes), and long-term memory. In the context of writing scientific articles, writers need
to combine the planning, translating, and revising processes iteratively to produce quality
writing. This approach emphasises the importance of reflection and evaluation during the
writing process, because writing is an iterative and non-linear process.
However, based on research conducted by Budhyani & Angendari (2021), many novice
writers have difficulty integrating various elements in writing scientific articles. This difficulty
is often caused by the lack of instruments that can be used to measure the extent of their
understanding of these concepts. Existing instruments often only focus on evaluating the
final results of the article without providing adequate guidance in the writing learning process
itself (Fajaruddin et al., 2021). The difficulty experienced by novice writers in writing
scientific articles is certainly the inability to understand the concept of writing scientific
articles Zaden & Meedya, (2024). Conceptual understanding is one of the initial knowledge
that must be possessed by lecturers because conceptual understanding is the basis for
formulating principles. Cognitive processes in the aspect of understanding include
interpreting, exemplifying, classifying, summarising, drawing inferences, comparing and
explaining (Anderson & Krathwohl, 2001).
The development of a concept understanding instrument in writing scientific articles is a
strategic step to overcome this problem. The instrument must be able to measure the author's
understanding of various important components in scientific articles, including structure,
substance, and presentation. A good instrument must have high validity and reliability in
order to measure what should be measured accurately and consistently (Nunnally &
Bernstein, 1994). The urgency of developing this instrument is increasing along with the
global demand for scientific publications. In recent decades, the number of scientific
publications from Indonesian authors has increased significantly. However, this increase in
quantity has not been fully balanced by an increase in quality (Arsyad & Adila, 2018). Many
articles are rejected by internationally reputable journals because they do not meet writing
standards, such as clarity of research objectives, appropriateness of methodology, and
accuracy of data presentation. Therefore, the development of an instrument that can help
authors understand and apply these concepts is very relevant.
In addition, the literature shows that the development of a concept understanding instrument
can also function as a learning tool. For example, research by Kolb (1984) on experiential
learning shows that learning based on experience and reflection can improve understanding
and skills. In the context of writing scientific articles, this instrument can be used as a medium
to provide constructive feedback to authors during the learning process (Widana & Ratnaya,
2021). Thus, this instrument not only functions as an evaluation tool, but also as a means to
strengthen the learning process.
Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

431

The novelty offered in the development of this instrument is a more comprehensive and
contextual approach. Most of the instruments that currently exist only focus on certain
aspects, such as mastery of grammar or article structure, without paying attention to the
integration between elements. In fact, in writing scientific articles, each element is interrelated
and affects the overall quality. For example, the clarity of the research objectives in the
introduction will affect how the author compiles the methodology and presents the results.
Therefore, the instrument developed must be able to measure the integration between these
elements.
On the other hand, instrument development must also be based on a data-based approach.
According to Brown (2014), the instrument development process must go through a series
of stages including needs analysis, design, testing, and validation. Each stage must be well
documented to ensure that the resulting instrument has a strong and reliable scientific basis
(Nunnally & Bernstein, 1994). This data-based approach allows researchers to systematically
collect and analyse empirical evidence that supports the validity and reliability of the
instrument (Messick, 1989). By considering these various aspects, this study aims to develop
a valid, reliable, and contextual instrument for understanding concepts in writing scientific
articles. This instrument is expected to be a practical solution for academics, researchers, and
students in improving their ability to write scientific articles. In addition, this study is also
expected to provide theoretical contributions in the field of instrument development and
higher education, especially in the context of Indonesia.
METHOD
This research is included in research and development which is a research method used to
produce certain products (creations), and test the effectiveness of the product Sugiyono,
(2017). The intended development product is an instrument for understanding the concept
of writing scientific articles designed with the aim of being a measuring tool for
understanding the concept of students, in this case lecturers as participants in scientific article
writing training.
This study adapts the ADDIE model research steps which consist of five stages, namely: a.
analyze, b. design, c. develop, d. implement, and e. evaluate Branch, (2009). In this study, it
is only carried out up to the fourth stage, namely the implementation stage. At the analysis
stage, the main focus is to identify the needs and problems that underlie the development of
the instrument. The steps taken by analysing and identifying the needs of academics,
lecturers, and students in understanding important concepts in writing scientific articles, as
well as collecting data through literature studies. Literature analysis reveals that training is a
commonly used intervention to improve scientific article writing skills. Previous studies, such
as those conducted by Rakhman, Surur, and Darmawati (2021), Nandiyanto et al. (2023), and
Budiwan & Suswandari (2021), have empirically proven that training can increase lecturers'
publication productivity. In addition, studies by Wardoyo et al (2022) highlight the
importance of a practical training approach in improving the technical skills of writing
scientific articles.
However, the development of standardised instruments to evaluate the understanding of the
concept of writing scientific articles in depth is still very rare. The focus of research so far
has been more directed at the final results in the form of publishable articles, without
measuring participants' conceptual understanding of the structure, scientific methodology,
Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

432

ethics, and use of academic language that underlies the writing. This is an important basis for
research that aims to develop evaluation instruments to strengthen basic competencies in
writing scientific articles. The results of this analysis are the basis for formulating
development objectives and determining the initial specifications of the instrument to be
developed.
The second stage is the design stage which aims to formulate the instrument design
conceptually and technically. The main activities at this stage include designing specific
instrument objectives that cover aspects of understanding the concept of writing scientific
articles consisting of seven dimensions, namely understanding the research topic, literature
analysis and synthesis skills, understanding research methodology, data analysis and
interpretation skills, ability to present research results and discussions, understanding
templates and scientific writing ethics, and language skills and data presentation. After that,
the instrument blueprint was prepared by developing the instrument framework from seven
dimensions into 21 indicators and 84 questions in a Likert-based questionnaire.
Table 1. Blueprint of Scientific Article Writing Concept Understanding Instrument and
Item Details Questions that Represent It
No. Dimension
1.

2.

3.

4.

Indicator

Understanding
of the research
topic

a. Ability to identify and understand
existing research gaps.
b. Understanding of basic theories
relevant to the topic.
c. Understanding of basic concepts
in the topic being researched.
Literature
a. Ability to identify relevant studies
analysis and
or literature.
synthesis skills
b. Ability to compile literature
reviews systematically and
critically.
c. Ability to present literature
synthesis to demonstrate research
relevance.
Understanding
a. Understanding the concept of
of research
appropriate research methods to
methodology
answer research problems.
b. Ability to explain research design,
population, sample, data collection
techniques, and selected analysis
tools.
c. Understanding the advantages and
limitations of the methods used.
Data analysis and a. Ability to analyse data using
result
appropriate techniques.
interpretation
b. Skills in presenting data analysis
skills
results clearly and concisely.

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

Number
Positive Negative
1.2
3.4
5.6

7.8

9.10

11.12

13.14

15.17

17.18

19.20

21.22

23.24

25.26

27.28

29.30

31.32

33.34

35.36

37.38

39.40

41.42

43.44

433

No. Dimension

5.

6.

7.

Indicator

c. Ability to interpret research results
according to the literature used.
Ability to present a. Ability to compare research results
research results
with previous literature.
and discuccion
b. Ability to build logical arguments
supported by data.
c. Ability to identify research
limitations and suggest further
research.
Understanding
a. Understanding of article templates
templates and
applicable to certain international
ethics of
journals.
scientific writing b. Understanding of publication
ethics, including plagiarism and
compliance with codes of ethics.
c. Ability to write abstracts,
introductions, methods, results
and discussions, and conclusions
according to standards.
Language Skills
a. Ability to write articles in good
and Data
and correct English (academic
Presentation
language).
b. The ability to construct clear,
concise and precise sentences.
c. Skills in presenting tables, graphs,
or data visualisations that are
informative and appropriate to the
research context.

Number
Positive Negative
45.46
47.48
49.50

51.52

53.54

55.56

57.58

59.60

61.62

63.64

65.66

67.68

69.70

71.72

73.74

75.76

77.78

79.80

81.82

83.84

The third stage is the development stage. In this study, the instrument for understanding the
concept of writing scientific articles is tested for validity and reliability. Content validation is
carried out by involving experts from universities to assess the relevance of each item to the
concept to be measured, as well as providing input on the dimensions, indicators, and
questions developed. Each question item is then tested through empirical validation to
ensure that the questions can measure conceptual understanding accurately. This process
aims to produce an instrument that is not only valid and reliable, but also representative of
important aspects in understanding the concept of writing scientific articles. An instrument
is said to be valid if it is truly able to measure what should be measured by the instrument
(Candiasa, 2010). Validity is related to the accuracy of a measuring instrument. Sugiyono
(2005) said that validity is an index that shows that the measuring instrument truly measures
what is intended to be measured. Validation from the Expert Team was analyzed using
Content Validity Ratio (CVR) analysis (Lawshe, 1975) with the following formula: CVR =
(2ne/n)–1
After identifying each item of the instrument question using CVR, the next stage is to
calculate the Content Validity Index (CVI). CVI is used to measure how valid the contents
Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

434

of an item are in measuring the construct to be measured. CVI is the average of the CVR
values for all items in the instrument. The CVI formula is as follows: CVI = (∑CVR)/n
(Lawshe, 1975).
After completing the content validity test, the instrument development process is continued
by revising the instrument based on the validation results and input provided by the expert
team. This revision aims to ensure that the developed instrument has good quality and is able
to measure the desired concept accurately. After the revision stage is complete, the next step
is the implementation stage which involves empirical validity testing. This test aims to test
the extent to which the instrument is able to produce valid and reliable data in the context
of its use. Empirical validation was carried out using Google Forms on participants in the
scientific article writing training from 17 training location cities, namely Bandar Lampung,
Bandung, Banyuwangi, Cirebon, Denpasar, Gorontalo, Jambi, Jakarta, Majene, Makassar,
Manado, Palembang, Palu, Pekanbaru, Solo, Tegal, and Ternate. This validation process
serves to ensure the validity and reliability of the questions. The validity of the questions is
analysed using the calculation of the correlation coefficient between the essay question score
and the total formulated essay questions. Test items are said to be valid if rᵢₜ > rₜ
After the validity test is carried out, the next stage of testing is to test the reliability of the
instrument. Reliability is calculated only for items that are not dropped (valid). In other
words, invalid items are not included in the test or calculation of reliability (Koyan, 2011).
The reliability of the instrument will be tested with the Cronbach's Alpha coefficient to
measure the internal consistency of the items in the instrument (Azwar, 2010; (Sekaran &
Bougie, 2016).
To determine the reliability of the instrument for understanding the concept of writing
scientific articles, Cronbach's Alpha is used with the following formula: α = (K / K-1) * (1 Σsi²/s²). Cronbach’s Alpha formula is used to measure the internal reliability of an
instrument, such as a questionnaire. Internal reliability measures how consistently the items
in an instrument measure the same construct.
RESULTS AND DISCUSSION
This research and development begin with determining the research objectives, namely
developing an instrument for understanding the concept of writing scientific articles and
determining the validity and reliability of the test. Based on an in-depth study of various
scientific writing theories, including the CARS model, the Day and Gastel writing cycle, and
the Flower and Hayes writing process theory, the authors develop an instrument designed to
measure participants' conceptual understanding of various aspects of writing scientific
articles. This instrument measures participants' ability to apply good writing principles, from
planning to final revision.
Content validation in this study is conducted through an expert judgment approach, where
a number of experts who have expertise and experience in related fields are asked to assess
the relevance of each item to the concept to be measured, as well as provide input on the
dimensions, indicators, and questions developed. Experts involved in this validation include
the following fields: educational evaluation, instrument measurement/construction, and
research methodology. This process aims to ensure that the instrument used can accurately
measure the desired variables and meet academic quality standards. The results of this
Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

435

validation are used to revise and improve the instrument, so as to improve the reliability and
validity of the measurements carried out in the study. The results of content validation are
shown in Table 2.
Table 2. Results of Calculation of Content Validity of Understanding the Concept of
Writing Scientific Articles
Panelist

No
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

ne

1

2

3

1
1
1
1
1
1
1
1
1
1
1
0
0
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
1
1
1
1
1
1
0
1
0
1
1

3
3
3
3
3
3
3
3
3
3
3
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
2
3
3
3
3
3
3
2
3
2
3
3

(2ne/n)–1

Min
Value
CVR

Description

1
1
1
1
1
1
1
1
1
1
1
0.333333
0.333333
1
1
1
1
1
1
1
1
1
1
1
1
1
0.333333
1
1
1
1
1
1
0.333333
1
0.333333
1
1

0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6

Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Not Valid
Not Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Not Valid
Valid
Valid
Valid
Valid
Valid
Valid
Not Valid
Valid
Not Valid
Valid
Valid

CVR

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

436

Panelist

No
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

ne

1

2

3

1
1
1
1
1
1
1
1
1
0
1
1
1
1
1
1
1
0
1
1
1
1
1
1
1
1
1
1
1
1
1
0
1
1
1
1
1
1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

3
3
3
3
3
3
3
3
3
2
3
3
3
3
3
3
3
2
3
3
3
3
3
3
3
3
3
3
3
3
3
2
3
3
3
3
3
3
3
3
3
3
3
3
3

(2ne/n)–1

Min
Value
CVR

Description

1
1
1
1
1
1
1
1
1
0.333333
1
1
1
1
1
1
1
0.333333
1
1
1
1
1
1
1
1
1
1
1
1
1
0.333333
1
1
1
1
1
1
1
1
1
1
1
1
1

0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6
0.6

Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Not Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Tidak Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Not Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid
Valid

CVR

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

437

Panelist

No
84

ne

1

2

3

1

1

1

3

(2ne/n)–1

Min
Value
CVR

Description

1

0.6

Valid

CVR

Each question item in this research instrument has been adjusted based on suggestions and
input from experts involved in the content validation process. Validation is carried out using
the Content Validity Ratio (CVR) method, in which experts assess the level of relevance of
each question item to the research objectives. The results of the CVR analysis show the
values used to determine the eligibility of each question item based on the critical CVR value
that corresponds to the number of experts. Question items that do not meet the content
validity criteria are revised or removed, while items that are considered essential are retained
to ensure that the instrument has adequate content validity.
Based on the content validity analysis, the calculation results show that the instrument has a
very good level of content validity. This is indicated by the average value of the level of
agreement of experts on each item in the instrument or S-CVI/Ave (Scale Content Validity
Index/Average) of 0.968 and is supported by the results of the proportion of items in the
instrument that have a level of agreement by all experts or S-CVI/UA (Scale Content Validity
Index/Universal Agreement) of 0.905. A good S-CVI/UA value indicates a high level of
agreement among experts on the relevance of the instrument items to the construct being
measured. Meanwhile, the overall index of the instrument's content validity or CVI value of
0.937 further strengthens this finding. These results provide strong evidence that the
instrument has met the content validity standards that are very adequate for use in research
or measurement.
Based on the results of content validation, revisions were made to the test items. Although
76 items were declared good based on content validation, this is not strong enough to
conclude that the items are valid and reliable to measure the understanding of the concept
of writing scientific articles. The content validation is only limited to the suitability of the
material with the understanding of the concept of writing scientific papers. The results of the
content validation could not yet show how the lecturers responded to the items. Therefore,
further validation is needed to determine the level of validity and reliability of the items that
have been developed and to determine the lecturers' responses to the items.
Empirical validation aims to ensure that the data obtained through the instrument is in
accordance with the concept being measured. The empirical validity test in this study was
conducted online by utilising the Google Forms platform which facilitates the distribution
of questionnaires and reaches respondents from various regions efficiently. This study
involved participants in scientific article writing training from 17 training location cities,
namely Bandar Lampung, Bandung, Banyuwangi, Cirebon, Denpasar, Gorontalo, Jambi,
Jakarta, Majene, Makassar, Manado, Palembang, Palu, Pekanbaru, Solo, Tegal, and Ternate.
The selection of participants as respondents is based on the diversity of their geographical
backgrounds and experiences as participants in scientific article writing training, so that it is
expected to provide a more comprehensive representation of the reliability and relevance of
the instruments being tested.

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

438

This validation process involves statistical analysis of the responses of 275 participants to
evaluate the overall quality of the instrument. A total of 84 questions developed in the initial
stage of this study are simplified into 42 questions after going through a content validation
process by a team of experts. This simplification was carried out to ensure that only questions
that have high relevance and the best quality were retained for further testing. Each selected
question item includes both positive and negative statements to ensure a balance of
perspectives in measurement and prevent bias in participant responses. Furthermore, the 42
questions are tested through empirical validity testing involving 275 respondents who
participate online.
Table 3. Empirical Validity Test Results
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

Corrected Item-Total
Correlation
.431
.447
.438
.530
.427
.509
.364
.471
463
.493
.432
.486
.374
.452
.412
.498
.418
.420
.475
.334
.379
.435
.384
.511
.436
.493
.428
.521
.415
.501
.433
.557
.365
.452
.391

Cronbach's Alpha if Item
Deleted
.859
.853
.859
.852
.859
.852
.859
.853
.859
.852
.860
.853
.860
.853
.859
.852
.859
.854
.858
.857
.860
.854
.860
.852
.859
.852
.859
.852
.860
.852
.860
.851
.860
.853
.861

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

Decsription
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
valid
439

Number
36
37
38
39
40
41
42

Corrected Item-Total
Correlation
.373
.359
.500
.446
.428
.337
.476

Cronbach's Alpha if Item
Deleted
.855
.860
.852
.859
.854
.860
.853

Decsription
valid
valid
valid
valid
valid
valid
valid

In the table above, see the Scale Corrected Item-Total Correlation value which is the Item
Validity value, while the Croncbach's Alpha if Item Deleted value is the Item Reliability value.
To assess whether the values above (Item Validity and Item Reliability) are valid and reliable,
compare them with the R Table at DF = N-2 and Probability 0.05.
The steps for item validity start from calculating the Corrected Item-Total Correlation for
each item to measure the relationship between the item score and the total score, which is
continued by evaluating the Cronbach's Alpha if Item Deleted value to determine the impact
of item deletion on the reliability of the instrument. Item validation is carried out to evaluate
the quality of each item in the instrument based on the analysis of the Corrected Item-Total
Correlation and Cronbach's Alpha if Item Deleted. The validity of the items was tested by
comparing the Corrected Item-Total Correlation value of each item with the R Table value
at df= 40 (n= 42, df= n-2) with a probability of 0.05. The R Table value used was 0.3044.
Based on the results of the calculation of the total item correlation for 42 questions, the
analysis results showed that all 42 items had a Corrected Item-Total Correlation value greater
than 0.3044, so all items were declared valid.
All items have a value of rᵢₜ > rₜ, indicating that each item has a significant correlation with
the total score. The item with the highest value is X32 (rᵢₜ= 0.557), indicating the greatest
contribution to the construct being measured. The item with the lowest value is X20 (rᵢₜ=
0.334), but it is still valid because it is greater than the critical value. This value indicates that
each item has a significant correlation with the total score of the instrument, so it can be said
to be able to measure aspects of the concept of writing scientific articles that are designed
accurately.
The reliability of the instrument is assessed using the Cronbach's Alpha method, which
measures the internal consistency of all items in the instrument. The steps start from
calculating the overall Cronbach's Alpha, then comparing the Cronbach's Alpha value with
the standard value of 0.7 as the minimum acceptable reliability limit.
Based on the results of the analysis, the Cronbach's Alpha Based on Standardised Items value
was 0.852, higher than the recommended minimum threshold of 0.70. This value indicates
that the instrument has very good internal consistency, so it can provide stable and reliable
measurement results in the same population. In other words, the items in the instrument
correlate well with each other in measuring the same concept, namely the understanding of
the concept of writing scientific articles so that the instrument has good reliability.

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

440

From the analysis results, the Cronbach's Alpha value of 0.852 indicates that the instrument
has high internal consistency. Each subdimension is also tested, and all had a Cronbach's
Alpha value above 0.7, indicating good reliability for each dimension.
CONCLUSION
This study successfully proves that the initial hypothesis regarding the importance of
developing valid and reliable instruments in writing scientific articles can be accepted.
Through the application of the ADDIE model, the developed instrument is able to accurately
measure lecturers' ability to understand the concept of writing scientific articles.
The argument that understanding the structure, methodology, and ethics of publication is an
essential element in writing scientific articles is strengthened by the results of empirical
validation. All 42 instrument items showed a significant correlation with the total score,
proving that each item supports the measurement of concepts consistently.
This study opens up new opportunities for the use of similar instruments in other writing
training contexts, including for students or novice researchers. It also suggests that training
based on valid and reliable instruments can significantly improve the quality of scientific
publications.
In general, the validity and reliability of this instrument make an important contribution to
academic evaluation. However, the generalization of the results needs to be further tested on
a wider population or with different settings. The implication of this study is the need for
policies that support the use of this instrument in academic training and evaluation to
improve the quality of scientific publications in Indonesia.
BIBLIOGRAPHY
Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for learning, teaching, and assessing.
Pearson.
Arsyad, S., & Adila, D. (2018). Using local style when writing in English: The citing behaviour
of Indonesian authors in English research article introductions. Asian Englishes, 20(2),
170–185. https://doi.org/10.1080/13488678.2017.1327835
Azwar, S. (2010). Reliabilitas dan validitas dalam penelitian pendidikan. Pustaka Pelajar.
Branch, R. M. (2009). Instructional design: The ADDIE approach. Springer.
Budhyani, I. D. A. M., & Angendari, M. D. (2021). Kesulitan dalam menulis karya
ilmiah. Mimbar Ilmu, 26(3), 400–407. https://doi.org/10.23887/mi.v26i3.40678
Budiwan, J., & Suswandari, M. (2020). Pelatihan menulis artikel ilmiah jurnal terakreditasi
Sinta dan terindeks Scopus dalam mengembangkan kompetensi profesional dosen.
Jurnal Pengabdian pada Masyarakat. https://doi.org/10.32585/educate.v1i1.1797
Brown, T. (2014). The development of measurement tools: Steps, validation, and reliability testing. SAGE
Publications.
Candiasa, I. M. (2010). Pengujian instrumen penelitian disertai aplikasi ITEMAN dan BIGSTEPS.
Undiksha Press.
Day, R. A., & Gastel, B. (2006). How to write and publish a scientific paper (6th ed.). Cambridge
University Press.
Direktorat Jenderal Pendidikan Tinggi, Riset, dan Teknologi (Diktiristek). (2024). Pedoman
akreditasi jurnal ilmiah. Dikti.
Direktorat Riset, Teknologi, dan Pengabdian kepada Masyarakat (DRTPM). (2024). Panduan
pelatihan penulisan artikel ilmiah tahun 2024. DRTPM.
Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

441

Fajaruddin, S., Retnawati, H., Wijaya, T., Ramadhan, S., & Prihatni, Y. (2021). Alhamdulillah,
butir pengembangan instrumen penilaian artikel jurnal ilmiah dikatakan valid oleh para
rater. Measurement
In
Educational
Research,
1(2),
89-96.
http://dx.doi.org/10.33292/meter.v1i2.156
Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College composition and
communication, 32(4), 365–387. https://doi.org/10.2307/356600
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development.
Pearson Education Inc.
Koyan, I. W. (2011). Assesment dalam pendidikan. Universitas Pendidikan Ganesha.
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4),
563–575. https://doi.org/10.1111/J.1744-6570.1975.TB01393.X
Lubis, M. S., Rahimah, A., & Lubis, I. S. (2019). Kesulitan-Kesulitan yang dihadapi oleh
mahasiswa yang mengampuh mata kuliah Bahasa Indonesia di Program Studi Bahasa
Indonesia IPTS dalam penulisan karya tulis ilmiah (KTI). Jurnal Education And
Development, 7(3), 193-199.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103).
American Council on Education/Macmillan
Nandiyanto, A. B. D., Hamidah, I., Haristiani, N., Muktiarni, M., & Rahayu, N. I. (2024).
Pelatihan penulisan artikel ilmiah hasil riset. Jurnal Abdimas Kartika Wijayakusuma, 5(2),
509–510. https://doi.org/10.26874/jakw.v5i2.536
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. McGraw-Hill
Brown, P., & Brown, C. (2014). Transformative Learning Theory in Gerontology: NonTraditional Students. Educational Gerontology, 40(5), 321-345
Peraturan Menteri Pendidikan, Kebudayaan, Riset, dan Teknologi Nomor 39 Tahun 2021
tentang Integritas Akademik.
Rakhman, F., Surur, M., & Darmawati, L. E. S. (2020). Peningkatan produktivitas publikasi
melalui pelatihan penulisan artikel ilmiah bagi dosen STIQ Wali Songo Situbondo.
Journal of Educational Development and Innovation
Riduwan. (2018). Dasar-Dasar Statistika. Alfabeta.
Sekaran, U., & Bougie, R. (2016). Research methods for business: A skill-building approach
(7th ed.). John Wiley & Sons.
Sugiyono. (2005). Metode penelitian kuantitatif, kualitatif, dan R&D. Alfabeta.
Sugiyono. (2017). Penelitian dan pengembangan: Teori dan implementasi. Alfabeta.
Swales, J. M. (1990). Genre analysis: English in academic and research settings. Cambridge University
Press.
Swales, J. M., & Feak, C. B. (2012). Academic writing for graduate students: Essential tasks and skills
(3rd ed.). University of Michigan Press.
Udil, P. A. (2021). Pelatihan penulisan artikel ilmiah penelitian tindakan kelas untuk publikasi
pada jurnal iIlmiah. Jurnal Nasional Pengabdian Masyarakat, 2(1), 21-27.
https://doi.org/10.47747/pengabdiankepadamasyarakat.v2i1.257
Undang-Undang Republik Indonesia Nomor 12 Tahun 2012 tentang Pendidikan Tinggi.
Wardoyo, C., Nuris, D. M., & Fauzan, S. (2022). Pelatihan penyusunan artikel ilmiah bagi
guru-guru akuntansi se-Kabupaten Malang. Jurnal Pendidikan dan Keilmuan.
https://doi.org/10.37729/abdimas.v6i1.1542
Widana, I. W. & Ratnaya, I. G. (2021). Relationship between divergent thinking and digital
literacy on teacher ability to develop HOTS assessment. Journal of Educational Research
and Evaluation, 5(4), 516-524. https://doi.org/10.23887/jere.v5i4.35128
Yasa, I. P. G., Widana, I. W., & Aisyah, S. (2023). The determination of the principal’s
leadership style, teachers’ work motivation, and mind-set of the performance of
Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

442

elementary school teachers. Edukasi: Jurnal Pendidikan dan Pengajaran, 10(1), 42-50.
https://doi.org/https://doi.org/10.19109/ejpp.v10i1.16790
Yusuf, M., Pahala, I., Ulupui, I. G. K. A., Muliasari, I., Hasanah, N., Jaya R, T. E., Sasmi, A.
A., & Zairin, G. (2022). Training basic skills of scientific writing and publication to
improve research competence. Jurnal Pemberdayaan Masyarakat Madani, 6(2), 375–382.
https://doi.org/10.21009/jpmm.006.2.12
Zaden, H., & Meedya, S. (2024). Writing for publication: Argument and evidence. Women and
Birth. https://doi.org/10.1016/j.wombi.2024.101595

Indonesian Journal of Educational Development (IJED), 5(4), pp. 429-443

443