JEELS
(Journal of English Education and Linguistics Studies)
P-ISSN: 2407-2575 E-ISSN: 2503-2194
https://jurnalfaktarbiyah.iainkediri.ac.id/index.php/jeels
LEARNERS ENGAGEMENT IN ONLINE TEACHER
PROFESSIONAL DEVELOPMENT:
SCALE DEVELOPMENT AND VALIDATION
*Johanes Leonardi Taloko1; Sri Rachmajanti2; Francisca Maria
Ivone3
1,2,3
English Department, Faculty of Letters, Universitas Negeri Malang,
Indonesia
1
English Education Study Program, Faculty of Teacher Education,
Widya Mandala Surabaya Catholic University, Indonesia
*jltaloko@ukwms.ac.id; sri.rachmajanti.fs@um.ac.id;
francisca.maria.fs@um.ac.id
(*) Corresponding Author

Abstract: Research has shown that effective online teacher
professional development (OTPD) requires teachers to learn
and engage professionally to become high-quality teachers.
Understanding teachers' engagement as learners in OTPD is
crucial as such engagement indicates active learning,
interaction, and collaboration with other teachers.
Nevertheless, there has not been any instrument measuring
how teachers behave (behavioral), feel (emotional), think
(cognitive), and socialize (social). Therefore, the goal of the
present study was to create a context-specific survey
instrument to measure the engagement of EFL teachers in
OTPD. This study, using the research and development
(R&D) method, involved 385 Indonesian in-service EFL
1

Citation in APA style:
Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.
DOI: 10.30762/jeels.v11i2.2966
Submission: May 2024, Revision: July 2024, Publication: September 2024

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

teachers in nine universities conducting OTPD in Indonesia.
The statistical results of explanatory (EFA) and
confirmatory factor analyses (CFA) confirmed the fourdimensional construct in the OTPD Learner Engagement
Instrument, mainly behavioral, emotional, cognitive, and
social dimensions of engagement. The study's policy and
practical implications addressing the policymakers,
institutions, and future research were also presented.
Keywords: dimensions of engagement, in-service EFL teacher,
learner engagement, online TPD

INTRODUCTION
Improving teacher quality as a means of enhancing the quality
of education has become a major concern for countries around the
globe (Harjanto et al., 2018; Keengwe & Kang, 2013). Teacher quality
determines student achievement and school quality based on what
and how teachers teach in the classroom (Keengwe & Kang, 2013). In
the current 21st century, it is pivotal for EFL teachers to be up to date
with current trends and master technology and implement it in the
classrooms (Rachmajanti et al., 2020). It requires teachers to learn and
develop themselves professionally to become high-quality teachers. In
that case, teacher professional development (TPD) for teachers is
crucial.
Desimone (2009) argues that effective TPD consists of five key
elements, mainly content focus, active learning, coherence, duration,
and collective participation. The last element is a powerful force of
teacher learning to stimulate potential interaction and engagement.
Engagement in TPD, referring to teachers’ pedagogical motivation and
efforts (Jiang et al., 2019), is important because it leads to the feeling of
satisfaction and success (Dyment et al., 2013), improves the quality of
teachers’ interactions (Williford et al., 2017), student outcome in
learning (Koh et al., 2017), teachers’ knowledge and instructional
practices (Avillanova & Kuswandono, 2019), and facilitates
collaboration (Beni et al., 2021). Studies on teacher engagement in TPD,
654

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

particularly in online mode (OTPD), have been investigated by some
scholars, such as Beni et al. (2021), Compen et al. (2020), and Picard &
Kutsyuruba (2017). Picard and Kutsyuruba (2017), for example,
investigated teacher interaction, whereas Beni et al. (2021) aimed at the
teachers’ collaboration with other participants in OTPD. Another
research shows that effective technology-focused professional
development occurs when teachers learn as learners (Curwood, 2011).
Depite this myriad of previous studies investigating teacher
engagement in OTPD, there has not been been any instrument to
measure how teachers engage themselves in OTPD as professional
learners (Czerniawski, 2013; Derrington & Kirk, 2017; Pietarinen et al.,
2016) using student engagement (Bowden et al., 2019; Fredricks et al.,
2004) as a major point of reference. On the other hand, the term ‘learner
engagement’ is used by Noe et al. (2010), Halverson & Graham (2019)
and Deng et al. (2020). In this study, the term ‘student engagement’ and
‘learner engagement’ are used interchangably as ‘student’ and ‘learner’
generally refer to a similar agent. Student engagement considers how
learners behave (behavioral), feel (emotional), think (cognitive), and
socialize (course/class) in educationally purposeful activities (Coates,
2006; Avcı & Ergün, 2019).
Research on student engagement concluded that it had a multidimensional construct mainly behavior, emotional, and cognitive
engagement (Bowden et al., 2019; Fredricks et al., 2004) and was
applied in investigating student engagement in various contexts by
Kuh (2009), Fredricks et al. (2011), Dobbins and Denton (2017), and
Raes et al. (2020). Behavior engagement refers to ‘school-related
conduct, involvement in learning, attendance, and participation in
school-related activities’ (Fredricks et al., 2019:1). The dimensions of
behavior engagement include academic participation, following
classroom norms (Gunuc & Kuzu, 2015), following rules (Gunuc &
Kuzu, 2015; Sun & Rueda, 2012), making efforts (Burch et al., 2015;
Dixson, 2015; Gunuc & Kuzu, 2015), paying attention, and asking
questions (Sun & Rueda, 2012).

655

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

Emotional engagement covers ‘students’ positive and negative
reactions to class and school, and relationships with teachers, peers,
and school belonging or connectedness’ (Fredricks et al., 2019:1). The
dimensions of this engagement consist of positive reactions to teachers,
classmates, academic staff and school (Burch et al., 2015; Deng et al.,
2020; Gunuc & Kuzu, 2015; Sun & Rueda, 2012), willingness to do the
work (Burch et al., 2015), student attitudes (Burch et al. 2015; Deng et
al. 2020; Gunuc & Kuzu 2015), belonging (a feeling of being important)
(Gunuc & Kuzu, 2015), value (an appreciation of success in schoolrelated outcomes) (Sun & Rueda, 2012), interest (enjoyment of the
activity) (Deng et al., 2020; Dixson, 2015; Sun & Rueda, 2012), and cost
(negative aspects of engaging in the task) (Sun & Rueda, 2012).
Cognitive engagement includes ‘self-regulated learning,
perceived relevance of schoolwork, use of deep learning strategies, and
exerting the necessary cognitive strategies for the comprehension of
complex ideas’ (Fredricks et al., 2019:2). Its dimension covers selfregulated (Sun & Rueda, 2012), willingness to use efforts to
comprehend & master knowledge & skills (Deng et al., 2020; Dixson,
2015; Sun & Rueda, 2012), a desire to go beyond the requirement,
flexibility in problem-solving (Sun & Rueda, 2012), preference for a
challenge (Dixson, 2015), various learning strategies: summarizing,
rehearsal, remembering, and so on (Deng et al., 2020), and investments
(Gunuc & Kuzu, 2015). The constructs in cognitive engagement are also
found in motivation literature, such as learning goals, valuing learning,
striving for knowledge and mastery, and self-regulated learning
(Fredricks et al., 2011). Scholars put their arguments that engagement
and motivation are related (Fredricks et al., 2016). Dunn & Kennedy
(2019) argued that motivation is the ignition of engagement.
Motivation is manifested into behavioral (e.g., participation in
classroom task), emotional/affective (e.g., interest and positive feelings
about task), and cognitive (e.g., self-regulated learning and deep
strategy use) engagement (Fredricks et al., 2016).
Later, a new dimension of engagement, social engagement, was
introduced by Chen et al. (2018), Bowden et al. (2019), and Deng et al.
656

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

(2020) as it encapsulated educational engagement through interactions
with instructors and peers. Social engagement considers the bonds of
identification and belongingness between students and their peers as
well as academic staff in their learning experience (Bowden et al., 2019).
More precisely, it focuses on learner-instructor and learner-learner
interactions (Deng et al., 2020). Though sometimes reported as a part
of behavioral engagement due to observable instructors-peers
interactions and a part of “ways of behaving” in educational activities,
social engagement is another key component of the student learning
experience and is treated separately from the other types of
engagement. The dimensions of this type of engagement include
participation (Burch et al., 2015; Gunuc & Kuzu, 2015), collaboration
(Dixson, 2015), active learning (Deng et al., 2020; Dixson, 2015), feeling
that belonging to a community (Dixson, 2015), online discussion (Deng
et al., 2020; Dixson, 2015), post arguments & response, interaction with
other students (Deng et al., 2020; Dixson, 2015), and feel as if they are
engaging with real people (Gunuc & Kuzu, 2015).
The present study aims to develop a validated survey
instrument for measuring teachers as professional learners’
engagement in online teacher professional development (OTPD)
consisting of the four dimensions above. Instruments measuring
learner engagement in OTPD remain scarce in literature. For example,
the existing instruments for measuring learners in online learning
contexts developed by Sun & Rueda (2012) was intended to measure
student engagement in online classes in a university context. The other
instrument created by Deng et al. (2020) was to ivestigate learner
engagement in MOOC (Massive Open Online Course). Developing a
valid survey instrument to determine engagement in OTPD is pivotal
to understand more comprehensively how teachers participate
themselves actively in such a formal learning context as it shows their
motivation and efforts to complete learning tasks based behavioral,
emotional, cognitive, and social dimensions of engagement. Also, it can
provide beneficial contribution to design more effective professional
development activities for teachers.
657

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

METHOD
Research Design
Aiming to develop and validate an educational product in the
form of a particular survey instrument, this study uses research and
development (R&D) approach (Gall et al., 2003). The steps proposed by
Deng et al. (2020) in developing and validating the survey instruments
were utilized. The steps are described in Figure 1 below.

Figure 1. Diagram of Survey Instrument Development

Step 1: Literature review and item pooling
Student engagement served as the underlying theory discussed
in the literature review. Extensive research mentioned that student
engagement consists of has multi-dimensional construct, mainly
behavioral, emotional, and cognitive dimensional engagement. Item
pooling for the initial step of constructing the survey instrument was
taken from the scales proposed by Sun and Rueda (2012), Deng et al.
(2020), and Gunuc and Kuzu (2015) with the total of 73 items
altogether.
658

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

Step 2: Pilot survey and interview
In this step, pilot survey as a draft survey was distributed to 21
non-targeted EFL in-service teachers. The results of the survey was
used as a basis for interviewing these participants to gain their insights
about the survey items for item purification.
Step 3: Expert review
After some items were added and omitted, three lecturers who
had expertise in ELT and TPD were chosen to review 58 survey items
for relevance, clarity, and simplicity of the survey.
Step 4: Data collection
According to Hair Jr et al. (2009), the number of participants
answering the survey must be at least five times higher than the
number of survey items in order to achieve statistical calculation
purposes. Hence, the survey in the form of online questionnaire were
distributed, targeting at least 300 participants.
Step 5: Construct validation
After collecting sufficient number of data, the final step was
assessing the construct validity and reliability of the survey through
Exploratory Factor Analysis (EFA), Cronbach’s Alpha (), and
Confirmatory Factor Analysis (CFA) (Hair Jr et al., 2009). The modified
survey instrument was later called OTPD Learner Engagement.
Research Context and Participants
The context of this study was an online teacher professional
development program (OTPD) initiated by the Indonesian Ministry of
Education called PPG (Pendidikan Profesi Guru, Indonesian) delivered
online for in-service teachers around the country. PPG is a kind of
teacher professional education to improve the quality of pre-service
and in-service teachers. This study focused on in-service teachers
teaching English subjects at schools. The participants of this study were
385 Indonesian in-service EFL teachers taking OTPD in nine different
universities from 2020 to 2023 after they completed the final
examination in the program. The English teachers joining that program
were mostly female (72.2%) living in various islands or regions in
659

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

Indonesia, most of which are Java (57.1%) and Sumatra (20.3%). Almost
all participants (90.4%) taught in secondary schools, both in junior and
senior high schools. Moreover, the length of their teaching experience
was mostly 6-10 years and 11-15 years covering up 81.1% of the
participants. Table 1 shows more specific information about the
participants.
Table 1.
The Demographic Information about the Participants
Variables
Gender
Schools Taught

Teaching Experience

Islands/regions

Descriptions
Male
Female
Elementary school
Junior high school
Senior high school
1-5 years
6-10 years
11-15 years
16-20 years
Above 20 years
Java
Sumatra
Kalimantan
Bali and Eastern
Timor
Riau Island and
Bangka Belitung
Sulawesi
Papua

Frequency
107
278
37
169
179
27
157
155
26
20
220
78
43
25

Percentage
27.8
72.2
9.6
43.9
46.5
7.0
40.8
40.3
6.8
5.2
57.1
20.3
11.2
6.5

7

1.8

6
6

1.6
1.6

FINDINGS
OTPD Learner Engagement Scale Validity and Reliability
The following is the process of EFA of each dimension, e.g.:
Behavioral, Emotional, Cognitive, and Social Dimensions.
EFA of Behavioral Dimension
The procedure and the results of calculating Exploratory Factor
Analysis of behavioral dimension are explained in brief. First of all,

660

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

statistical computation on Kaiser-Meyer-Olkin Measure of Sampling
Adequacy (.938) and Bartlett’s Test of Sphericity (Chi-Square= 1860.53;
df=66; p<.000) indicated that the data were highly suitable for factor
analysis because a KMO value higher than 0.50 can be continued to the
next statistical analysis (Gunuc & Kuzu, 2015). After that, EFA was
performed using Principal Component Analysis (PCA) with Varimax
rotation on 12 (twelve) questionnaire items. Items with factor loading
(FL) less than .40 and cross loading above .40, and communalities (h2)
below .40 were eliminated from the item pool (Hair Jr et al., 2009). Table
2 summarizes the results of analysis for each item in this dimension.
Table 2.
Construct validity, internal reliability, and descriptive data of Behavioral Engagement
̅̅̅̅)
̅ (𝐒𝐃
Item
FL
M (SD)
h2
𝐌

B00002

0.54

0.74

5.55 (0.59)

B00003

0.59

0.77

5.58 (0.57)

B00004

0.58

0.76

5.38 (0.70)

B00005

0.59

0.77

5.51 (0.64)

B00006

0.44

0.66

5.44 (0.71)

B00007

0.58

0.76

5.56 (0.58)

B00008

0.61

0.78

5.38 (0.63)

B00009

0.62

0.79

5.30 (0.66)

B00010

0.62

0.78

5.34 (0.68)

B00012

0.42

0.65

5.54 (0.65)

5.46 (0.64)

Note: h2= Communalities; FL= Factor Loading; M=Mean;
Deviation; = Cronbach’s Alpha

0.91

SD= Standard

Table 2 shows that there are 10 (ten) items remaining which
have communalities (h2) value > 0.40, ranging from 0.42 - 0.60, factor
loading value > 0.40, from 0.65 to 0.79, and Cronbach’s Alpha internal
consistency reliability coefficient is 0.91 which is higher than the
minimum of 0.70 (Spencer, 2013). The cut-off point for FL was .40
(Gunuc & Kuzu, 2015). In addition, the mean average is 5.46 and, the
standard deviation average is 0.64. The eigenvalues (5.58) yielded one
component which accounted for 55.80% of the variance. Having h2 and

661

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

FL above 0.40 and Cronbach’s Alpha value of 0.91 indicates that the
behavioral engagement items are worth processing for CFA.
EFA of Emotional Dimension
The procedure and the results of calculating Exploratory Factor
Analysis of emotional dimension are explained in brief. First of all,
statistical computation on Kaiser-Meyer-Olkin Measure of Sampling
Adequacy (.93) and Bartlett’s Test of Sphericity (Chi-Square= 3317.53;
df=136; p<.000) indicated that the data were highly suitable for factor
analysis. After that, EFA was performed using Principal Component
Analysis (PCA) with Varimax rotation on 17 (seventeen) questionnaire
items. Table 3 summarizes the results of analysis for each item in this
dimension.
Table 3.
Construct validity, internal reliability, and descriptive data of Emotional Engagement.
̅̅̅̅)
̅ (𝐒𝐃
Item
FL
M (SD)
h2
𝐌

E00001
0.46
0.68
5.36 (0.66)
5.56 (0.58)
0.92
E00002
0.56
0.75
5.66 (0.53)
E00003
0.52
0.72
5.73 (0.50)
E00004
0.47
0.68
5.70 (0.51)
E00005
0.41
0.64
5.51 (0.66)
E00007
0.48
0.69
5.81 (0.44)
E00008
0.53
0.73
5.46 (0.61)
E00009
0.50
0.71
5.56 (0.60)
E00010
0.52
0.72
5.40 (0.64)
E00011
0.56
0.75
5.51 (0.61)
E00012
0.52
0.72
5.58 (0.60)
E00013
0.56
0.75
5.51 (0.58)
E00014
0.54
0.73
5.60 (0.60)
E00015
0.53
0.73
5.49 (0.61)

Table 3 above shows that there are 14 (fourteen) items
remaining which have communalities (h2) value > 0.40, ranging from
0.41 - 0.56, factor loading value > 0.40, from 0.64 to 0.74, Cronbach’s
alpha is 0.92. In addition, the mean average is 5.56, and, the standard
deviation average is 0.58. The eigenvalues (7.14) yielded one
component which accounted for 51% of the variance. Having h2 and FL

662

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

above 0.40 and Cronbach’s alpha value 0.92 indicates that the
emotional engagement items are worth processing for CFA.
EFA for Cognitive Dimension
The procedure and the results of calculating the Exploratory
Factor Analysis of cognitive dimension are explained in brief. First of
all, statistical computation on Kaiser-Meyer-Olkin Measure of
Sampling Adequacy (.94) and Bartlett’s Test of Sphericity (Chi-Square=
2949.81; df=136; p<.000) indicated that the data were highly suitable for
factor analysis. After that, EFA was performed using Principal
Component Analysis (PCA) with Varimax rotation on 17 (seventeen)
questionnaire items. Table 4 below summarizes the results of analysis
for each item in this dimension.
Table 4.
Construct validity, internal reliability, and descriptive data of Cognitive Engagement
̅̅̅̅)
̅ (𝐒𝐃
Item
FL
M (SD)
h2
𝐌

C00001
0.56
0.75
5.35 (0.62)
5.41 (0.64)
0.93
C00003
0.55
0.74
5.36 (0.65)
C00004
0.54
0.73
5.30 (0.68)
C00005
0.52
0.72
5.37 (0.65)
C00006
0.61
0.78
5.38 (0.67)
C00007
0.43
0.66
5.28 (0.73)
C00008
0.51
0.71
5.44 (0.64)
C00009
0.58
0.76
5.58 (0.59)
C00010
0.51
0.72
5.45 (0.64)
C00011
0.52
0.72
5.27 (0.72)
C00014
0.45
0.67
5.36 (0.69)
C00015
0.55
0.74
5.45 (0.56)
C00016
0.47
0.69
5.65 (0.57)
C00017
0.45
0.67
5.48 (0.58)

Table 4 above shows that there are 14 (fourteen) items
remaining that have commonalities (h2) value > 0.40 ranging from 0.43
- 0.61, factor loading value > 0.40 from 0.67 to 0.75, a mean average is
5.41, and, standard deviation average is 0.64, and Cronbach’s alpha is
0.93. The eigenvalues (7.25) yielded one component which accounted
for 51.8% of the variance. Having h2 and FL above 0.40 and Cronbach’s

663

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

alpha value0.93 indicates that the cognitive engagement items are
worth processing for CFA.
EFA for Social Dimension
The procedure and the results of calculating the Exploratory
Factor Analysis of cognitive dimension are explained in brief. First of
all, statistical computation on Kaiser-Meyer-Olkin= 0.94; df= 91; p<.000)
indicated that the data were highly suitable for factor analysis. After
that, EFA was performed using Principal Component Analysis (PCA)
with Varimax rotation on 14 (fourteen) questionnaire items. Table 5
summarizes the results of analysis for each item in this dimension.
Table 5.
Construct validity, internal reliability, and descriptive data of Social Engagement
̅̅̅̅)
̅ (𝐒𝐃
Item
FL
M (SD)
h2
𝐌

S00003
0.41
0.64
5.39 (0.63)
5.36 (0.65)
0.92
S00006
0.46
0.68
5.12 (0.73)
S00007
0.55
0.75
5.19 (0.74)
S00008
0.60
0.78
5.32 (0.67)
S00009
0.65
0.80
5.30 (0.67)
S00010
0.74
0.86
5.35 (0.63)
S00011
0.75
0.86
5.43 (0.62)
S00012
0.63
0.80
5.52 (0.63)
S00013
0.62
0.78
5.51 (0.63)
S00014
0.60
0.77
5.52 (0.57)

Table 5 above shows that there are 10 (ten) items remaining
which have communalities (h2) value > 0.40 ranging from 0.41 - 0.75,
factor loading value > 0.40 from 0.64 to 0.86, the mean is 5.36 and,
standard deviation is 0.65. The eigenvalues (5.63) yielded one
component which accounted for 62.5% of the variance. Having h2 and
FL above 0.40 and Cronbach’s alpha value 0.92 indicates that the social
engagement items are worth-processing for CFA.
To sum up the statistical computation of the four dimensions of
OTPD Learner Engagement above, here are the items that are further
analyzed in Confirmatory Factor Analysis. Behavioral and Social
Engagement contribute 10 (ten) items each, while Emotional and
Cognitive Engagement 14 (fourteen) items each, Hence, there are 48
(forty-eight) items altogether.
664

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

CFA for All Dimensions of Learner Engagement
CFA calculation from the results of EFA process produces a
model or a structure. The hypothesized four-factor structure shows the
best model fit and reliability estimates among the four dimensions of
engagement. The standardized path diagram for the structure is
displayed in Figure 2.

Figure 2. The standardized path diagram of Learner Engagement

665

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

Multiple criteria were explored to evaluate the different aspects
of the goodness-of-fit model above. Ho (2006) mentions that there are
three types of indices or measures to assess goodness-of-fit measures,
mainly: (1) absolute fit measures, (2) incremental fit measures, and (3)
parsimonious fit measures. Table 6 shows the fitness between the fourfactor model and the sample data based on those three types of
measures.
Table 6.
Evaluation of the goodness-of-fit measures regarding CFA
Type of fit
measures
Absolute
fit
measures

Incremental fit
measures

Parsimonious
fit measures

Indices
X2/df

Sample
values
2.43

GFI

0.73

RMSEA

0.07

TLI

0.84

NFI

0.76

RFI

0.75

IFI

0.84

CFI

0.84

PNFI

0.72

AIC

0.05

Criteria
< 3.00
0.0 (poor) – 1.0
(perfect)
0.05
0.08
(acceptable)
0.08 to 0.10
(mediocre)
> 1.0 (poor)
0.0 (poor) – 1.0
(perfect)
0.0 (poor) – 1.0
(perfect)
0.0 (poor) – 1.0
(perfect)
0.0 (poor) – 1.0
(perfect)
0.0 (poor) – 1.0
(perfect)
0.05
–
0.09
(good)
Closer to 0.0

Source of
criteria
Kline
(2011)
Ho
(2006)
Ho
(2006)

Evaluation
outcomes
Good fit

Ho
(2006)
Ho
(2006)
Ho
(2006)
Ho
(2006)
Ho
(2006)
Ho
(2006)
Ho
(2006)

Good fit

Good fit
Acceptable

Good fit
Good fit
Good fit
Good fit
Good fit
Good fit

Note: Chi-square (X2)= 2609.018; df= 1074

Table 6 shows that all indices of the three kinds of goodness-offit measures of the four-factor model in Figure 2 are in good or
acceptable fit.
The last step in CFA is to measure the reliability and validity of
the model. Reliability is about the consistency of a measure. Cronbach’s
666

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

alpha () and Composite Reliability (CR) are the two main measures of
reliability. Validity is the accuracy of a measure. Convergent Validity
(measures of the same constructs) and Discriminant Validity (measures
of the distinct construct) are measured by Average Variant Extracted
(AVE).
The reliability of the four-factor model is indicated by the
Cronbach’s alpha () and Composite Reliability (CR) of each
engagement. Cronbach’s Alpha internal consistency reliability
coefficient value must be greater than 0.70. CR values between 0.6 and
0.7 are considered acceptable and if they are above 0.7 are considered
good (Spencer, 2013). Table 7 shows the values of  and CR.
Table 7.
Reliability based on  and CR values
Dimensions of Engagement
Behavior
Emotional
Cognitive
Social


0.91
0.92
0.93
0.92

CR
0.91
0.93
0.93
0.93

Table 7 indicates that values of all dimensions of Learner
Engagement are in the range of 0.91-0.93 which is higher than 0.70. The
same values are also found in CR in which its values are 0.91 and 0.93.
These CR values are above 0.70 which are considered good. Based on
the values of Cronbach’s alpha () and Composite Reliability (CR), it
can be concluded that the model has good reliability.
Validity is measured by calculating the convergent and
discriminant validities of the patterns of correlation among factors
(Tabachnick & Fidell, 2019). Convergent validity can be calculated by
inspecting the factor loading (FL). FL should be statistically higher than
0.5, but preferably higher than 0.7 (Malhotra, 2019). Another way to
measure convergent validity is by examining the average variance
extracted (AVE) which is recommended to have a value greater than
0.5 (Hair et al., 2009). Discriminant validity is measured by calculating
the square root of the AVE which should be greater than the correlation
coefficients. Appendix 1 displays the values of the validity measures

667

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

and depicts that FL values are between 0.59 – 0.86 which are
statistically higher than 0.50, though some items have values lower
than 0.7. AVE values also indicating convergent validity are between
0.47 – and 0.56 showing that two out of four factors or dimensions have
values greater than 0.50. These values indicate that the convergent
validity of this model is acceptable, even though it is not highly
satisfactory. To measure the discriminant validity of the model, the
values from Appendix 1 are presented in Table 8 together with the
correlation coefficient among factors.
Table 8.
Discriminant validity and correlation coefficient.
Behavioral
Emotional
Cognitive
Social

Behavioral
0.71
0.75
0.75
0.67

Emotional

Cognitive

Social

0.69
0.84
0.77

0.69
0.79

0.75

Table 8 shows that the value in each correlation of one
dimension compared to the other dimensions or factors is greater than
0.50. This means that the discriminant validity and the correlation
coefficient are good (Hair et al., 2009).
DISCUSSION
OTPD Learner Engagement Survey in Appendix 1 is suitable
with the multidimensional construct suggested by Fredrick et al.
(2004), Chen et al. (2018), and Bowden et al. (2019) consisting of
behavioral, emotional, cognitive, and social engagement.
Behavioral Dimension of Engagement
The behavioral dimension refers to ‘school-related conduct,
involvement in learning, attendance, and participation in schoolrelated activities’ (Fredricks et al., 2019:1). OTPD Learner Engagement
Survey also contained most of the sub-dimensions of the behavioral
dimensions, such as making efforts (Burch et al., 2015; Dixson, 2015;
Gunuc & Kuzu, 2015), paying attention (Sun & Rueda, 2012), following
668

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

classroom norms (Gunuc & Kuzu, 2015), and academic participation
(Gunuc & Kuzu, 2015). Nevertheless, the sub-dimension of asking
questions proposed by Sun and Rueda (20212) did not exist.
Since the participants of this study were in-service EFL teachers
in OTPD having teaching experience of 6-15 years (81.1%), they
reflected positive classroom behavior as what they expected their
students did in the classroom. It was indicated by three major subdimensions appearing in this dimension of engagement, mainly
regular attendance, being on-time, and attentive listeners.
First of all, following class rules in the forms of being on-time
and regular attendance indicates positive behavior engagement.
Nguyen et al. (2018) argued that classroom or school norms,
expectations, or rules had strong association with behavior
engagement. Another study by Mandernach (2015)concluded that
frequent attendance was one of the key indicators in behavioral
dimension of engagement.
Next, this study found out that the participants tended to
attentively listen to their teachers. It was confirmed by the participants
in the interviews that during the online meetings, they preferred
listening to the teachers’ explanations to asking questions to the
teachers due to their reluctance with other fellow participants.
Lastly, a survey conducted by Liu & Littlewood (1997) indicated
that the tendency of ‘listening to teachers’ was the most common
classroom experience in East Asian classes as the result of the students’
reluctance to take more active roles in classrooms. However, a more
recent study conducted by Loh & Teo (2017) argued that, despite of
over-generalizing that cultural attributes were allegedly causes of
Asian (particularly East Asian) students’ reticence and passivity (Shao
& Gao, 2016), the reasons were more situation-specific causes which
referred to dissimilarities on teaching methods and the lack of
sufficient foreign language proficiency. In this research context, it
might be true as the inequality of education level in Indonesia.

669

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

Emotional Dimension of Engagement
This dimension of engagement covers ‘students’ positive and
negative reactions to class and school, and relationships with teachers,
peers, and school belonging or connectedness’ (Fredricks et al., 2019:1).
This instrument consisted of several related sub-dimensions, such as
positive reactions to teachers and school staff school (Burch et al., 2015;
Deng et al., 2020; Gunuc & Kuzu, 2015; Sun & Rueda, 2012), student
attitudes (Burch et al. 2015; Deng et al. 2020; Gunuc & Kuzu 2015),
belonging (a feeling of being important) (Gunuc & Kuzu, 2015), valuing
success in the outcome (the feeling of being important) (Gunuc & Kuzu,
2015), and enjoying classroom activities (referred to as interests) (Deng
et al., 2020; Dixson, 2015; Sun & Rueda, 2012). However, the subdimensions of willingness to do the work (Burch et al., 2015) and
having negative aspects in doing tasks (referred to as cost) (Sun &
Rueda, 2012) did not appear in the scale.
The sub-dimensions in this kind of engagement indicated that
these in-service EFL teachers joining OTPD respected their teachers
due to their dedication and professionalism when teaching them. It
was also revealed that the participants had positive attitude not only
towards their teachers but also towards the program and the
administration staff. When these three parties did their roles well to
serve the participants during the program, it became the act of showing
respect that was felt by the participants and it was a crucial element of
the ethics of care and sustainable improvement (Thompson, 2018). He
suggested that maintaining organizational cultures and administrative
systems which developed the environment through which respect for
students was clearly demonstrated strengthened teacher-student
relationship and influenced students’ positive behaviors and academic
performance.
Cognitive Dimension of Engagement
This dimension contains all the sub-dimensions. It includes
‘self-regulated learning, perceived relevance of schoolwork, use of
deep learning strategies, and exerting the necessary cognitive
670

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

strategies for the comprehension of complex ideas’ (Fredricks et al.,
2019:2). In OTPD Learner Engagement instrument, these subdimensions were self-regulated, flexibility in problem-solving, and a
desire to go beyond the requirements (Sun & Rueda, 2012), willingness
to make necessary efforts to complete tasks and applying different
learning strategies (Deng et al., 2020; Dixson, 2015; Sun & Rueda, 2012),
preference to a challenge and putting investment (Gunuc & Kuzu,
2015).
The constructs in cognitive engagement are also found in
motivation literature, such as learning goals, valuing learning, striving
for knowledge and mastery, and self-regulated learning (Fredricks et
al., 2011). Motivation is the precursor factor of cognitive engagement
(Chi & Wylie, 2014). The in-service EFL teachers joining OTPD in this
study were highly motivated to take part in PPG as they believed that
it could transform them to be professional and more competent English
teachers. They would also feel ashamed to their fellow teachers in their
schools if they failed in this program. This motivation eventually led
them to try their best to accomplish their assignments and got good
scores in exams. This study supports evidence from previous research
by Yang et al. (2018) which found that cognitive engagement included
students’ investment in learning in order to succeed in academic
responsibilities, such as tasks, quizzes, and tests.
Social Dimension of Engagement
Social dimension is the one that considers bonds of
identification and belongingness between learners and their peers as
well as academic staff in their learning experience (Bowden et al., 2019)
and focuses on learner-instructor and learner-learner interactions
(Deng et al., 2020). The instrument in Appendix 1 contained related
sub-dimensions that focused on the social interactions among learners
in the online setting. The sub-dimensions were interaction with other
learners and posting arguments and discussions (Deng et al., 2020;
Dixson, 2015), as well as collaboration (Dixson, 2015). Since the
engagement scale development was measured in the online learning
671

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

context, some new items existed that made this scale unique, such as
online discussion (Deng et al., 2020; Dixson, 2015), posting arguments
and responses in LMS (learning management system), and feeling as if
being engaged with real people (not in an online mode) (Gunuc &
Kuzu, 2015).
Previous research by Gómez-Rey et al. (2016), Horzum (2015),
and Peacock & Cowan (2019) argued that social presence had a strong
correlation with social interaction. The more learners engaged
themselves personally and emotionally with other learners, the more
interactions they had with them. The difference between social
presence and social interaction was distinctive. Social interaction
emphasized on interactivity as a latent quality of communication that
might or might not be realized by individuals whereas social presence
occurred when they realized and noticed the interactivity (Cobb, 2009)
In this study, the in-service EFL teachers joining OTPD preferred
having interactions in the form of small group discussions with the
peers they had known before or were comfortable with. It could be
derived from this situation that, even though the participants knew
everyone in the program, they preferred to work together with the
people to whom they were personally or emotionally connected.
Hence, small group discussions were formed informally without
teachers’ or administrators’ interference. These sorts of discussions
improved learning effectively as participants’ created their own
learning community in which they learned from and with one another
via positive and collaborative online environment (Brindley et al.,
2009).
CONCLUSION
Understanding the engagement of teachers as learners in an
online TPD is crucial as such engagement indicates active learning as
well as interaction and collaboration with other teachers (Picard &
Kutsyuruba, 2017). Although online TPD has a shared goal to improve
teachers’ knowledge and competence in the classrooms, each program
has its uniqueness in terms of purposes, participants, geographies,
672

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

online platforms, internet access, and so on. This study creates a
context-specific instrument to measure the engagement of Indonesian
in-service EFL teachers in online teacher professional development.
The instrument is conceptualized as consisting of four key dimensions
of engagement, mainly behavioral, emotional, cognitive, and social
engagement. The results of explanatory and confirmatory factor
analyses confirm this hypothesis.
The present study can have policy and practical implications for
policymakers, universities hosting OTPD, and future research. The
OTPD Learner Engagement instrument is useful for the government to
identify features of online TPD contexts that facilitate behavioral,
emotional, cognitive, and social engagement. Also, the instrument can
be used by universities in Indonesia that conduct online TPD for inservice teachers as a diagnostic tool to provide general feedback to
instructors concerning the pattern of engagement in online TPD.
Future research into online TPD should focus on the LMS (learning
management system) of online platforms as it provides a rich, virtual
venue to investigate learners’ engagement, interaction, and
collaboration.
REFERENCES
Avcı, Ü., & Ergün, E. (2019). Online students’ LMS activities and their
effect on engagement, information literacy and academic
performance. Interactive Learning Environments, 0(0), 1–14.
https://doi.org/10.1080/10494820.2019.1636088
Avillanova, A. A., & Kuswandono, P. (2019). English teacher
professional development in Indonesia: the Challenges and
Opportunities. English Review: Journal of English Education, 8(1),
41. https://doi.org/10.25134/erjee.v8i1.1972
Beni, S., Fletcher, T., & Ní Chróinín, D. (2021). Teachers’ engagement
with professional development to support implementation of
meaningful physical education. Journal of Teaching in Physical
Education, 1–10. https://doi.org/10.1123/jtpe.2021-0137
Bowden, J. L. H., Tickle, L., & Naumann, K. (2019). The four pillars of
tertiary student engagement and success: a holistic measurement
approach. Studies in Higher Education, 0(0), 1–18.
673

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

https://doi.org/10.1080/03075079.2019.1672647
Brindley, J. E., Blaschke, L. M., & Walti, C. (2009). Creating effective
collaborative learning groups in an online environment creating
effective collaborative learning groups in an online environment
introduction : The challenge of creating. International Review of
Research in Open and Distributed Learning, 10(3), 1–18.
Burch, G. F., Heller, N. A., Burch, J. J., Freed, R., & Steed, S. A. (2015).
Student engagement: Developing a conceptual framework and
survey instrument. Journal of Education for Business, 90(4), 224–
229. https://doi.org/10.1080/08832323.2015.1019821
Chen, B., Chang, Y. H., Ouyang, F., & Zhou, W. (2018). Fostering
student engagement in online discussion through social learning
analytics. Internet and Higher Education, 37(2018), 21–30.
https://doi.org/10.1016/j.iheduc.2017.12.002
Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking
cognitive engagement to active learning outcomes. Educational
Psychologist,
49(4),
219–243.
https://doi.org/10.1080/00461520.2014.965823
Coates, H. (2006). Student engagement in campus-based and online
education: University connections. Student Engagement in
Campus-Based and Online Education: University Connections, 1–212.
https://doi.org/10.4324/9780203969465
Cobb, S. C. (2009). Social presence and online learning: A current view
from a research perspective. Journal of Interactive Online Learning,
8(3), 241–254.
Compen, B., De Witte, K., & Schelfhout, W. (2020). The impact of
teacher engagement in an interactive webinar series on the
effectiveness of financial literacy education. British Journal of
Educational
Technology,
0(0),
1–15.
https://doi.org/10.1111/bjet.13013
Curwood, J. S. (2011). Teachers as learners : What makes technologyfocused professional development effective? English in Australia,
46(3), 68–75.
Czerniawski, G. (2013). Professional development for professional
learners - teachers’ experiences in Norway, Germany and
England. Journal of Education for Teaching, 39(4), 383–399.
https://doi.org/https://doi.org/10.1080/02607476.2013.769339
Deng, R., Benckendorff, P., & Gannaway, D. (2020). Learner
engagement in MOOCs: Scale development and validation.
674

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

British Journal of Educational Technology, 51(1), 245–262.
https://doi.org/10.1111/bjet.12810
Derrington, M. L., & Kirk, J. (2017). Linking job-embedded professional
development and mandated teacher evaluation: Teacher as
learner. Professional Development in Education, 43(4), 630–644.
https://doi.org/10.1080/19415257.2016.1231707
Desimone, L. M. (2009). Improving impact studies of teachers’
professional development: Toward better conceptualizations and
measures.
Educational
Researcher,
38(3),
181–199.
https://doi.org/10.3102/0013189X08331140
Dixson, M. D. (2015). Measuring student engagement in the online
course: The Online Student Engagement scale (OSE).(Section II:
Faculty Attitudes and Student Engagement)(Report). Online
Learning
Journal
(OLJ),
19(4),
143.
https://doi.org/10.24059/olj.v19i4.561
Dobbins, C., & Denton, P. (2017). MyWallMate: An Investigation into
the use of mobile technology in enhancing student engagement.
TechTrends, 61(6), 541–549. https://doi.org/10.1007/s11528-0170188-y
Dyment, J., Downing, J., & Budd, Y. (2013). Framing teacher educator
engagement in an online environment. Australian Journal of
Teacher
Education,
38(1),
133–149.
https://doi.org/10.14221/ajte.2013v38n1.6
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004a). School
engagement: Potential of the concept, state of the evidence.
Review
of
Educational
Research,
74(1),
59–109.
https://doi.org/10.3102/00346543074001059
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004b). School
Engagement: Potential of the Concept, State of the Evidence.
Review of Educational Research, 74(1), 59–109.
Fredricks, J. A., Reschly, A. L., & Christenson, S. L. (2019). Interventions
for student engagement: Overview and state of the field. In
Handbook of Student Engagement Interventions: Working with
Disengaged Students. Elsevier Inc. https://doi.org/10.1016/B9780-12-813413-9.00001-2
Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., &
Mooney, K. (2011). Measuring student engagement in upper
elementary through high school: a description of 21 instruments.
Issues
and
Answers
Report,
098,
26–27.
675

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

http://ies.ed.gov/ncee/edlabs
Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational Research. Allyn
& Bacon.
Gómez-Rey, P., Barbera, E., & Fernández-Navarro, F. (2016).
Measuring teachers and learners’ perceptions of the quality of
their online learning experience. Distance Education, 37(2), 146–
163. https://doi.org/10.1080/01587919.2016.1184396
Gunuc, S., & Kuzu, A. (2015). Student engagement scale: Development,
reliability and validity. Assessment and Evaluation in Higher
Education,
40(4),
587–610.
https://doi.org/10.1080/02602938.2014.938019
Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009).
Multivariate Data Analysis (7th edition).
Halverson, L. R., & Graham, C. R. (2019). Learner engagement in
blended learning environments: A conceptual framework. Online
Learning
Journal,
23(2),
145–178.
https://doi.org/10.24059/olj.v23i2.1481
Harjanto, I., Lie, A., Wihardini, D., Pryor, L., & Wilson, M. (2018).
Community-based teacher professional development in remote
areas in Indonesia. Journal of Education for Teaching, 44(2), 212–231.
https://doi.org/10.1080/02607476.2017.1415515
Ho, R. (2006). Handbook of Univariate and Multivariate Data Analysis
and Interpretation with SPSS. Taylor & Francis Group.
https://doi.org/10.1198/000313008X332287
Horzum, M. B. (2015). Interaction, structure, social presence, and
satisfaction in online learning. Eurasia Journal of Mathematics,
Science
and
Technology
Education,
11(3),
505–512.
https://doi.org/10.12973/eurasia.2014.1324a
Jiang, L., Yu, S., & Zhao, Y. (2019). Teacher engagement with digital
multimodal composing in a Chinese tertiary EFL curriculum.
Language
Teaching
Research.
https://doi.org/10.1177/1362168819864975
Keengwe, J., & Kang, J. J. (2013). Preparing in-service language teachers
to design and implement technology-rich curriculum. Education
and
Information
Technologies,
18(4),
609–619.
https://doi.org/10.1007/s10639-012-9195-y
Kline, R. B. (2011). Convergence of structural equation modeling and
multilevel modeling. In M. William & W. P. Vogt (Eds.), The
SAGE Handbook of innovation in social research methods (1st ed., pp.
676

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

562–589). Sage Publication.
Koh, J. H. L., Chai, C. S., & Lim, W. Y. (2017). Teacher professional
development for TPACK-21CL: Effects on teacher ICT integration
and student outcomes. Journal of Educational Computing Research,
55(2), 172–196. https://doi.org/10.1177/0735633116656848
Kuh, G. D. (2009). The national survey of student engagement:
Conceptual and empirical foundations. New Directions for
Institutional
Research,
2009(141),
5–20.
https://doi.org/10.1002/ir.283
Liu, N. F., & Littlewood, W. (1997). Why do many students appear
reluctant to participate in classroom learning discourse? System,
25(3), 371–384. https://doi.org/10.1016/S0346-251X(97)00029-8
Loh, C. Y. R., & Teo, T. C. (2017). Understanding Asian students
learning styles, cultural influence and learning strategies. Journal
of Education & Social Policy, 7(1), 194–210.
Malhotra, N. K. (2019). Marketing research: An applied orientation. In
Pearson Education. Pearson Education.
Mandernach, B. (2015). Assessment of student engagement in higher
education: A synthesis of literature and assessment tools.
International Journal of Learning, Teaching and Educational Research,
12(2), 1–14.
Nguyen, T. D., Cannata, M., & Miller, J. (2018). Understanding student
behavioral engagement: Importance of student interaction with
peers and teachers. Journal of Educational Research, 111(2), 163–174.
https://doi.org/10.1080/00220671.2016.1220359
Noe, R. A., Tews, M. J., & Dachner, A. M. (2010). Learner engagement:
A new perspective for enhancing our understanding of learner
motivation and workplace learning. Academy of Management
Annals,
4(1),
279–315.
https://doi.org/10.1080/19416520.2010.493286
Peacock, S., & Cowan, J. (2019). Promoting sense of belonging in online
learning communities of inquiry in accredited courses. Online
Learning
Journal,
23(2),
67–81.
https://doi.org/10.24059/olj.v23i2.1488
Picard, K., & Kutsyuruba, B. (2017). Teachers’ engagement in
professional development: A collective case study. Journal of
Educational Policies and Current Practices, 2(2), 89–100.
https://doi.org/10.15340/21473501221011
Pietarinen, J., Pyhältö, K., & Soini, T. (2016). Teacher’s professional
677

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

agency–a relational approach to teacher learning. Learning:
Research
and
Practice,
2(2),
112–129.
https://doi.org/10.1080/23735082.2016.1181196
Rachmajanti, S., Sulistyo, G. H., Megawati, F., & Akbar, A. A. N. M.
(2020). Professional development as viewed by EFL teachers at
lower secondary schools. JEES (Journal of English Educators
Society), 5(2), 205–212. https://doi.org/10.21070/jees.v5i2.964
Raes, A., Vanneste, P., Pieters, M., Windey, I., Van Den Noortgate, W.,
& Depaepe, F. (2020). Learning and instruction in the hybrid
virtual classroom: An investigation of students’ engagement and
the effect of quizzes. Computers and Education, 143(April 2019), 1–
16. https://doi.org/10.1016/j.compedu.2019.103682
Shao, Q., & Gao, X. (Andy). (2016). Reticence and willingness to
communicate (WTC) of East Asian language learners. System, 63,
115–120. https://doi.org/10.1016/j.system.2016.10.001
Spencer, N. H. (2013). Essentials of multivariate data analysis. In
Essentials
of
Multivariate
Data
Analysis.
https://doi.org/10.1201/b16344
Sun, J. C. Y., & Rueda, R. (2012). Situational interest, computer selfefficacy and self-regulation: Their impact on student engagement
in distance education. British Journal of Educational Technology,
43(2), 191–204. https://doi.org/10.1111/j.1467-8535.2010.01157.x
Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics. In
Pearson
(7th
ed.).
Pearson.https://www.vlebooks.com/Vleweb/Product/Index/
437320?page=0
Thompson, C. S. (2018). The construct of ‘respect’ in teacher-student
relationships: Exploring dimensions of ethics of care and
sustainable. Journal of Leadership Education, 17(3), 42–60.
https://doi.org/10.12806/v17/i3/r3
Williford, A. P., Carter, L. M., Maier, M. F., Hamre, B. K., Cash, A.,
Pianta, R. C., & Downera, J. T. (2017). Teacher engagement in core
components of an effective, early childhood professional
development course: Links to changes in teacher–child
interactions. Journal of Early Childhood Teacher Education, 38(1),
102–118. https://doi.org/10.1080/10901027.2016.1269028
Yang, D., Lavonen, J. M., & Niemi, H. (2018). Online learning
engagement: Critical factors and research evidence from
literature. Themes in ELearning, 11(1), 1–18.
678

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

Appendix 1. The validity values of OTPD Learner Engagement
Survey and factors regarding the CFA findings.
Item

Behavioral Engagement
B-2: I followed the rules of the online
class.
B-3: I carefully listened to my
teachers in class.
B-4: I carefully listened to other
students in class.
B-5: I attended the class willingly.
B-6: I did my homework/tasks in
time.
B-7: I tried to do my best regarding
my responsibilities in a group work.
B-8: I made sure I studied on a
regular basis
B-9: I planned and organized my
study.
B-10: I was able to consistently pay
attention when I was taking the
online learning class.
B-12: I shared information with my
classmates.
Emotional Engagement
E-1: I found the online class
interesting.
E-2: I liked my teachers.
E-3: My teachers were competent in
their field.
E-4: The LMS administrator &
other academic staff were competent
in their field.
E-5: I had teachers that I could share
my problems with.
E-7: I respected my teachers.
E-8: I felt enthusiastic when I was in
this online class.
E-9: I strived as hard as I could to
complete assignments for this online
class.

Convergent
Validity
(Factor loading)

Convergent
Validity
(AVE)
0.51

Discriminant
Validity
(AVE)
0.71

0.47

0.69

0.70
0.73
0.72
0.74
0.61
0.74
0.75
0.77
0.75
0.62

0.66
0.70
0.67
0.63
0.61
0.66
0.71
0.69

679

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

E-10: I liked communicating with
my teachers dealing with the
materials.
E-11: I was proud of assignments I
completed in this online class.
E-12: My teachers respected me as
an individual.
E-13: My classmates respected my
thoughts/views.
E-14: I felt myself as a part/member
of a student group.
E-15: I felt excited by my work at the
online class.
Cognitive Engagement
C-1: I checked my work for mistakes.
C-3: When I read the course
materials, I asked myself questions
to make sure I understood what it
was about.
C-4: I took good notes over readings,
PowerPoints, or presentations.
C-5: When I had trouble
understanding a concept or an
example, I went over it again until I
understood it.
C-6: I applied the course material to
my teaching.
C-7: If I did not know about a
concept when learning in the online
class, I did something to figure it
out.
C-8: I discussed what I had learned
in class with my classmates out of
class.
C-9: I tried to do my homework in
the best way.
C-10: I spent enough time and made
enough effort to learn.
C-11: I studied the material from the
LMS before the class and discussed
it with my classmates
C-14: I enjoyed intellectual
difficulties I encountered while
learning.
C-15: I determined my own learning
goals.

0.71
0.74
0.70
0.73
0.71
0.71
0.48
0.72
0.72

0.71
0.68

0.76
0.63

0.69
0.75
0.68
0.69
0.64
0.73

680

0.69

Taloko, J.L., Rachmajanti, S., Ivone, F.M. (2024). Learners engagement in online teacher professional
development: Scale development and validation. JEELS, 11(2), 653-681.

C-16: I tried my best to get good
grades.
C-17: I did well on the teaching
practice activities and exam
Social Engagement
S-3: I helped fellow participants
during the online class and/or via
WhatsApp group.
S-6: I often responded to other
participants' questions.
S-7: I posted in the discussion forum
regularly.
S-8: I participated actively in smallgroup discussion forums.
S-9: I shared learning materials (e.g.
notes, multimedia, and links) with
other participants in the online class
and/or via WhatsApp group.
S-10: I helped fellow participants
during the class and/or through
WhatsApp group.
S-11: I engaged in conversations
online with my classmates (chat,
discussions, and email).
S-12: I felt comfortable interacting
with other participants.
S-13: I liked seeing my classmates in
the online class.
S-14: Getting to know other
participants in the online class gave
me a sense of belonging

0.65
0.65
0.56
0.59
0.63
0.70
0.74
0.78

0.85
0.86
0.77
0.76
0.75

681

0.75