Anima 2019.
Vol.
No.
3, 115-124
https://doi.
org/10.
24123/aipj.
P-ISSN: 0215-0158 E-ISSN: 2620-5963
Validation of the Emotion Regulation Questionnaire (ERQ):
Network Analysis as an Alternative of Confirmatory Factor Analysis (CFA) Christiany Suwartono Dewa Bintamur Faculty of Psychology Universitas Katolik Indonesia Atma Jaya Faculty of Psychology Universitas Indonesia Construct validity with a confirmatory factor analysis (CFA) method is often used by researchers to identify the structure of psychological measurement.
Although this CFA method has provided a complete evaluation of the structure of the test, sometimes there is no consensus reached regarding the number of factors.
With the CFA method, researchers determined and compared several models to choose the adequate scale structure, including the number of factors in a scale.
The network analysis can play a role to help researchers.
In this network analysis, the results of the analysis presented in a visual form, which can facilitate the researchers quickly identify the scale structure based on the relations, displayed in the form of lines .
hick-thi.
and colors .
reen-re.
Current research aims to test the reliability and construct validity.
also, shows that network analysis can be an alternative in presenting visualizations and interpretations compare with factor analysis methods, especially CFA.
We used The Emotion Regulation Questionnaire (ERQ) with 954 participants aged 16-57 years.
The results are either, with the CFA analysis method or network analysis.
it is evident that ERQ has two factors, namely suppression and reappraisal.
The results show that the network analysis method can be used as an alternative in identifying the structure of the psychological scale.
Keywords: centrality plot, confirmatory factor analysis, emotion regulation questionnaire, network psychometrics analysis, structural validity Validitas konstruk dengan metode analisis faktor konfirmatori (CFA) seringkali digunakan peneliti untuk mengidentifikasi struktur dari suatu alat ukur psikologis.
Pada metode CFA, peneliti melakukan komparasi beberapa model mengenai jumlah faktor yang dimiliki dalam suatu tes dan menentukan struktur skala yang paling adekuat.
Di sinilah, metode analisis jaringan dapat berperan.
Pada analisis jaringan ini, hasil analisis disajikan dalam bentuk visual, peneliti dapat dengan cepat menemukenali struktur skala berdasarkan keterikatan yang ditampilkan dalam bentuk garis .
ebal-tipi.
dan warna .
ijau-mera.
Ada pun tujuan penelitian ini adalah untuk pengujian reliabilitas dan validitas konstruk.
selain itu, menunjukkan bahwa model jaringan bisa menjadi alternatif dalam menyajikan visualisasi dan interpretasi selain pengujian validitas konstruk dengan metode faktor analisis, khususnya CFA.
Kami menggunakan The Emotion Regulation Questionnaire (ERQ) dengan 954 partisipan berusia 16-57 tahun.
Hasilnya, baik dengan metode analisis CFA juga jaringan, terbukti bahwa ERQ memiliki dua faktor, yaitu suppression dan reappraisal.
Hal ini menunjukkan bahwa metode analisis jaringan ini dapat dijadikan sebagai alternatif dalam identifikasi struktur dari suatu alat tes.
Kata kunci: centrality plot, analisis faktor konfirmatori, kuesioner regulasi emosi, analisis jaringan psikometrik, validitas struktural Individuals experience various emotions that need to be managed in their daily life in order to live with good mental health as well as accepted in their environment.
Emotional regulation defined as the emer- gence of thoughts or behaviors that affect emotions experienced by a person when the person experiences emotions, and how .
he strateg.
that a person experiences or expresses those emotions (Gross, 1.
Gross and John .
distinguish two types of emotion regulation strategies, namely antecedentfocused .
lso known as cognitive reappraisa.
and response-focused .
lso called expressive suppressio.
Correspondence concerning this article should be addressed to Christiany Suwartono.
Faculty of Psychology.
Atma Jaya Catholic University of Indonesia Jalan Jenderal Sudirman Kav.
Jakarta Selatan Indonesia.
E-mail: christiany.
suwartono@atmajaya.
SUWARTONO AND BINTAMUR
Reappraisal emotional regulation strategy is a strategy that processes emotions by cognitively reframing the events that cause emotions before the tendency of emotional response is entirely produced, making new interpretations of an event that .
he has experienced, and reinterpreting the experiences as a stimulus that cause emotions (Ochsner & Gross, 2.
Whereas the suppression emotion regulation strategy is a strategy that processes emotions by avoiding stimuli that can cause emotions (Richards & Gross, 2.
, accumulating negative emotions originate from the events experienced, and decreasing the expression of behavior both from negative and positive emotions (Gross & John, 2.
Gross and John .
have developed a psychological scale, namely The Emotion Regulation Questionnaire (ERQ) that measures to what extent an individual uses emotion suppression and reappraisal regulation strategies.
ERQ has been proven valid and reliable (Ali & Alea, 2018.
Balzarotti.
Stefania.
Gross, & John, 2010.
Cabello.
Rosario.
Salguero.
FernandezBerrocal, & Gross, 2013.
Gross & John, 2.
Although it has been proven valid and reliable, in this study, researchers conducted construct validation, specifically regarding the structure of the ERQ scale with a two-method approach.
First, assessing the construct validation by confirmatory factor analysis (CFA) method to confirm the structure of a psychological measurement, based on the structure published by the initial developer (Gross & John, 2.
Second, construct validation is examined by network analysis methods.
The CFA method has provided an evaluation of the structure of the scale, however, sometimes there is no consensus reached regarding the number of factors, because the results of goodness of fit indexes are equally adequate between one model structure and the other model structure (Suwartono & Moningka, 2017.
van Dijk.
Claassen.
Suwartono, van der Heijden, & Hendriks, 2.
In the CFA method, the researchers can still compare several models regarding the number of factors that structured the scale and determine the adequate model based on Akaike's Information Criterion (AIC.
Akaike, 1.
Determining the adequate model is a challenge to get a scale structure more efficiently.
Then, network analysis methods can play a role.
In this study, the researchers showed the possibility of using network analysis as an alternative method in mapping the structure of a psychological scale that can be useful as one alternative method of analysis for the construct validation .
an Dijk et al.
, 2.
Unlike the CFA, in this network analysis, the factors that exist on a psychological scale are determined by the extraction of probabilistic associations between Current literature focuses more on the application of undirected network models, better known as Markov Random Fields in psychological data (Golino & Epskamp, 2.
This is because the items on the psychological scale do not always meet the requirements of local independence, therefore at the CFA, the researcher carries out inter-residual correlations and on the reporting sometimes the researchers do not display the number of inter-residual correlations they did until the model is considered adequate (Epskamp.
Rhemtulla, & Borsboom, 2.
The area of network analysis .
etwork psychometric.
emerges as response to concerns from the estimation results regarding the number of factors that should be the structure of the psychological test .
(Golino & Epskamp, 2.
In network analysis, the psychological variables under study are the items of a scale represented by the term nodes.
The nodes are connected by an edge .
, which indicates the strength of the statistical relationship among them.
Edges can differ in strength of connections called weights (Epskamp.
Cramer.
Waldorp.
Schmittmann, & Borsboom, 2.
The weight indicates the strength of the relationship between nodes .
he stronger the association, the line will be thicke.
and the direction of the relationship between the modes .
ositive for green line and negative for red lin.
After estimating the network structure, graph visualization itself tells researchers about the structure that emerged from the existing data.
Also, this graph can be used to assess which node is the most important in the network .
he most central nod.
The formed network can be used to obtain overview and insights to researchers regarding the data structures, predictive patterns of data, or represent the structure of association from data (Epskamp et al.
, 2.
The advantages of this network analysis, readers can quickly identify existing nodes based on associations, displayed in the form of lines .
hick-thi.
and colors .
reen-re.
Moreover, compared to the factor analysis method, the network analysis through network plots and the centrality of nodes output can identify data structures very well even when the different factors have high associations (Dalege.
Borsboom, van Harreveld, & van der Maas.
Golino & Epskamp, 2.
In the current study, in addition to reliability testing, we did construct validation using two methods.
First, the researchers compared the scale structure of the ERQ based on the CFA.
Then, the researcher conducted a network analysis.
Researchers suspect that both methods of analyzing construct validity produce the same structure for the ERQ.
VALIDATION OF ERQ
Method Participants This study used convenience sampling technique.
A total of 954 people participated in this study, with the majority of were female participants .
and the rest were male participants .
7%).
Participants in this study were 16-57 years old (M = 20.
SD = 5.
The majority of participantsAo religion was Islam .
7%), followed by Catholic .
5%),
Protestant .
4%).
Buddhism .
8%).
Hinduism .
7%), indigenous religions .
3%), with the rest not answering .
6%).
Most of the participants .
6%) have a high school education background.
Then, 15.
6% of participants had completed a junior high school education, 3.
6% undergraduate education, 1% did not answer.
Instrument The Emotion Regulation Questionnaire (ERQ.
Gross & John, 2.
was first adapted by Suwartono .
and used in the research of Suwartono.
Prawasti, and Mullet .
This scale aims to measure the tendency of individuals to regulate their emotions through two aspects, namely reappraisal ("When I am faced with a stressful situation.
I make myself think about it in a way that helps me stay calm.
") and suppression ("I keep my emotions to myself.
").
ERQ consists of ten statements with six statements measuring reappraisal .
umbers 1, 3, 5, 7, 8 and .
and four measuring suppression statements .
umbers 2, 4, 6, and .
ERQ
used seven choices from 1 .
trongly disagre.
to 7 .
trongly agre.
Statistical Techniques First, we analyzed the ERQ reliability.
Reliability shows to the extent to which individual differences in test scores are caused by actual differences in the measured aspects and the extent to which they can be considered to be caused by random errors (Anastasi & Urbina, 1.
The calculation method used is the alpha coefficient (Murphy & Davidshofer, 2.
If the result of coefficient alpha is between .
60 - .
then the scale is deemed to be reliable (Hair.
Black.
Babin, & Anderson, 2.
The range is considered as the lower limit of acceptance that the scale is reliable.
After the reliability coefficient is obtained, the researcher looks for a standard error of measurement (SEM) co-efficient to estimate the range of scores that might be obtained by individual (Crocker & Algina.
Regarding the focus of this study, the researcher conducted a structure validation of the scale.
The researcher used the Confirmatory Factor Analysis (CFA), especially the measurement model.
Hu and Bentler .
suggest the Tucker-Lewis Index (TLI = NNFI) and Comparative Fit Index (CFI) equal or above .
to determine the model fit.
Meanwhile, determining the structure of the model adequately used the smallest Akaike's Information Criterion (AIC.
Akaike, 1.
value of the existing models.
Then.
Netemeyer.
Bearden, and Sharma .
suggested that a p-value from chisquare (N.
should be higher than .
Goodness of Fit Index (GFI) which is greater or equal to 0.
Root Mean Square Error of Approximation (RMSEA) is smaller or equal to .
Validation with the CFA method was carried out using the LISREL 8.
80 program (Jyreskog & Syrbom, 2.
The ERQ structure was also analyzed by using network analysis, in addition to CFA.
This analysis was carried out using Jeffreys' Amazing Statistics Program (JASP) version 0.
1 (JASP Team, 2.
JASP is an open-source statistical program that is userfriendly and supports the APA style of reporting.
Network analysis allows researchers to analyze the network structure of variables visually.
In the networks, the variables observed .
n this case were item.
are called nodes, and the estimated line of relations between nodes is called an edge (JASP Team, 2.
The estimator used is cor .
, in order to the input files in both data analysis software is the same, namely correlation.
Outputs requested to be analyzed network plots, weights matrix, centrality plots, and centrality tables.
Others default from the Network menu in JASP.
The centrality menu produces a network structure as a figure .
entrality plo.
and numbers .
entrality It provides three information, including betweenness, closeness, and strength or egree arrat art e lemy.
Pastor-Satorras, & Vespignani, 2004.
Dalege et al.
, 2.
Betweenness calculated the number of shortest paths given by a node.
Betweenness shows how strong a node can interfere with the flow of information in the network.
Closeness is the inverse of the sum of all the shortest paths between a node and all other nodes in the network.
Closeness represents how likely the information from a particular node "moves" through the entire network both directly and indirectly.
Whereas strength or degree is the absolute value of the edge weight connected to a node.
represents the direct influence of the node given to the network.
SUWARTONO AND BINTAMUR
Procedure The data collection held at the beginning until mid2018.
We distributed questionnaires to various regions in Jabodetabek (Jakarta-Bogor-Depok-Tangerang-Bekas.
93%) and Aceh .
07%).
We used the online method as well as paper and pencil.
The questionnaire link is distributed to various communities in Greater Jakarta and Aceh.
The researcher got 1,131 participants.
Then we did the completeness of each part of the participant's responses.
Also, we removed the outliers.
therefore, the final data processed was 954 participants.
Result Reliability Reliability analysis with the alpha coefficients of reliability for the reappraisal is .
80 (M = 31.
SD =
and for the suppression is .
60 (M = 18.
= 5.
Table 1 presents the reliability coefficients.
SEM, and the corrected item-total correlation coefficient per item statement in detail.
Validity In construct validation, we conducted a Confirmatory Factor Analysis (CFA).
First, we tested the uni- dimensionality of the two dimensions, which became the factor structure of the Emotion Regulation scale.
The results of the unidimensional analysis can be seen in Figure 1.
Based on the RMSEA, the CFI, and GFI, it was confirmed that the reappraisal and suppression both are unidimensional .
ee Table .
The next step, we conducted a first-order CFA test to ensure that the components underlying both reappraisal and suppression were The results can be seen in Figure 2.
Based on the RMSEA.
NNFI.
CFI, and GFI confirmed that the first level CFA test for this emotion regulation scale is adequately fit.
The results of the test validation with the CFA technique indicate that the two-factors as the first-order factor is adequate with field data.
More details regarding the goodness of fit indices can be seen in Table 2.
Comparing the three models tested with CFA, based on the AIC Model, the most adequate structure is Model 2 .
st orde.
The result of inter-dimensional correlation is r.
= .
19, p < .
001, r2=.
This result shows a weak correlation between the dimensions of reappraisal and suppression.
Moreover, the correlation between the reappraisal and age is r.
= - .
p < .
001, r2 = .
While the correlation between suppression and age is r.
= .
04, p = .
172, r2 = .
Regarding the network structure as a figure .
entrality plo.
, it can be seen visually that items 2, 4, 6, and 9 form separate groups than items 1, 3, 5, 7, 8, and Table 1 The Reliability of Emotional Regulation Questionnaire Dimensions/ Factors Suppression (M = 18.
SD = 5.
Items Reliability item-total RE_2 RE_4 RE_6 RE_9 RE_1 RE_3 Reappraisal (M = 31.
SD = 4.
Statements RE_5 RE_7 RE_8 RE_10 I keep my emotions to myself.
When I am feeling positive emotions.
I am careful not to express them.
I control my emotions by not expressing them.
When I am feeling negative emotions.
I make sure not to express them.
When I want to feel more positive emotion .
uch as joy or amusemen.
I c ange w at IAom t inking about.
When I want to feel less negative emotion .
uch as sadness or ange.
I c ange w at IAom t inking about.
W en IAom face wit a stressful situation I make myself t ink about it in a way that helps me stay calm.
When I want to feel more positive emotion.
I change the way IAom t inking about t e situation.
I control my emotions by changing the way I think about the situation IAom in.
When I want to feel less negative emotion.
I change the way IAom t inking about t e situation.
= .
SEM =
= .
SEM =
VALIDATION OF ERQ
Figure 1.
Unidimensionality test for the dimension of reappraisal .
and suppression .
Table 2
Goodness-of-Fit Indices from the Results of the Confirmatory Factor Analysis Emotion Regulation Scale
Dimensions/Factors Chi-square RMSEA
AIC Model
NNFI
CFI
GFI
Suppression Reappraisal Model 1: One factor model.
Model 2:
1st order.
Exclusive dimension.
Model 3:
2nd order.
Two-factors model.
More details can be seen in the results of the network structure in Figure 3.
From Figure 3, each item is connected with green lines with different thickness levels.
The green lines indicate that the weight of the relationship between the ten items on the ERQ scale is positive.
The magnitude of the weight can be seen in detail in Table 3 - the higher the weight between items, the thicker the green color.
The weight shows the strength .
of the relevant connections between items (Epskamp.
Borsboom, & Fried, 2.
The details could be seen in Table 3.
the three largest edge weights were between item 7 and 8 (.
, item 8 and 10 (.
, then item 6 and 9 (.
Figure 2.
Level one CFA test for emotional regulation scale.
SUWARTONO AND BINTAMUR
Table 3
Weight Matrix Between Items Variabel
RE_1
RE_2
RE_3
RE_4
RE_5
RE_6
RE_7
RE_8
RE_9
RE_10
RE_1
RE_2
RE_3
RE_4
RE_5
RE_6
RE_7
RE_8
RE_9
RE_10
Figure 3.
The network plot of the emotion regulation scale.
When reading at the contents of items with strong weights .
tem 7, 8, and .
, the similarity that connects them is the way individuals think about the situation they experience.
Whereas in item 6 and 9, the similarity is that individuals try not to express their emotions.
The results of the visual analysis of centrality can be seen in Figure 4.
The two nodes namely item 6 (AuControlling my emotions by not expressing them.
and 8 (AuControlling my emotions by changing my way of thinking about the situation I'm experiencing.
are the highest for betweenness (RE 6 = 1.
87 and RE 8 = 1.
Regarding the content of item, no.
ot expressing emotio.
really represents the suppression factor.
Also, the content of item 8 (Auchanging the way of thinkingA.
represents the reappraisal factor.
Therefore, it is not surprising that item .
6 and 8 have the highest centra- lity, which when viewed from the process of developing this scale by Gross and John .
, the two items are indeed derived from two different factors from ERQ.
Thus, item 6 and 8 do have more control among the items on the ERQ network, because more information will pass through the two items because the two items firmly have associations with other items on the emotion regulation scale network.
From Figure 4 and Table 4, information obtained that node item 6 (AuControlling my emotions by not expressing them.
is high for betweenness.
However, node item 8 is both the highest for closeness and degree.
The high strength value for item 8 .
indicates that controlling emotions by changing the way of thinking with a situation experienced by someone, is a way that tends to influence someone when .
he made the emotional regulation.
This result can be seen VALIDATION OF ERQ Figure 4.
Centrality plot.
Table 4
Centrality Plot Per Item
RE_1
Betweenness - 0.
Centrality Plot Closeness - 0.
Strength
RE_2
- 0.
- 1.
- 1.
RE_3
- 0.
RE_4
- 0.
- 1.
- 1.
RE_5
- 0.
RE_6
- 0.
RE_7
- 0.
RE_8
RE_9
- 0.
- 0.
RE_10
- 0.
Variable from the thickness of the edge .
reen lin.
in the visualization of ERQ network analysis in Figure 3 and the value of the centrality plot .
etweenness, closeness, and degre.
per item in Table 4.
Discussion Based on research by Permono and Kusristanti .
, the Emotion Regulation Questionnaire (ERQ) showed reliability for the reappraisal of .
73 and the suppression Whereas in the current study, the reliability for the reappraisal was .
80 and suppression was .
The similarities in the two studies include: the suppression dimension has a lower reliability coefficient compared to the reappraisal dimension.
The suppression dimension reliability coefficient from ERQ is still above .
So, based on Hair et al.
ERQ can be said to be a reliable psychological measure.
One of the purposes of this study is to show that the network analysis model offers more than interpretation and alternative analysis compare to factor analysis methods, especially CFA.
for the construct validation.
The results of the CFA based on the AIC model showed that the adequate model is a first-order model, which supports that each factor of ERQ is independent.
The same result was found with network analysis, finding that ERQ has two independent factor structures, namely suppression and reappraisal.
The findings with these two methods indicate that ERQ has the same factor structure.
Current research also supports the results of the ERQ scale development from Gross and John .
The similarity of these results also shows that this network analysis method can be used as an alternative in identifying the structure of a test.
Although network analysis is a relatively new method, network analysis can illustrate a model of interaction among items.
Different from the analysis factor, which tries to reduce the structure of variables by modeling latent variables .
, network analysis predicts the relationship between all variables directly.
Additionally, many inference methods from network analysis output can be used to assess which nodes .
n this case item.
are the most essential in the network (Epskamp.
Borsboom, & Fried, 2.
In this study, it is represented by item 8: "I control my emotions by SUWARTONO AND BINTAMUR changing the way I think about the situation I am experiencing.
" This is concluded by looking at the results of the centrality plot table .
etweenness, closeness, and degre.
for each item, that supported by the graph network of items produced by network analysis .
reen and thick lines from various items connected to item .
To conduct network analysis with continuous data, the sample size of 250 is sufficient for moderate networks (Dalege et al.
, 2017.
Epskamp, 2.
Therefore, the sample size is the strength of this research.
The sample size of this study was 954 participants.
However, this is also what the researchers suspect is a limitation of this study.
Research Limitations The current results show the similarity of the ERQ scale structure using the network analysis method and confirmatory factor analysis.
However, they showed less strength of the network analysis method as a better alternative than confirmatory factor analysis.
With a limited number of items .
and two factors, the visualization obtained does not yet show the advantages of the actual utility of network analysis.
This network analysis will significantly help researchers, especially if some items or dimensions are very high correlations so that it is difficult to determine whether they are from the same structure.
This is because network analysis presents network plots, weights matrix, centrality plot, and centrality table that can help researchers to make decisions.
The current study has a limitation regarding the sampleAos geograp ic location.
T e samples lack variation due to most of the data in this study was from Jabodetabek.
Therefore the sample is more representative of urban areas.
It should be considered for research in other areas, both in urban and rural areas.
In addition, some participants participated online in filling out the ERQ.
The online filing makes the participating participants likely only reach people who have supporting facilities, such as internet networks.
Generally, these supporting facilities are only in big Moreover, in terms of gender, the majority of participants were female .
3%) while there was an assumption that women should be able to control their emotions and it was natural for men to express their emotions (Branney & White, 2008.
Motro & Ellis, 2.
Conclusion and Implication The Emotion Regulation Questionnaire (ERQ) is a reliable psychological measure with adequate internal The result of ERQ analysis consistently showed that ERQ has two factors as its structure both using CFA analysis methods and network analysis.
Network analysis enables researchers to immediately recognize the structure of the scale by looking at the distribution of variables in the form of nodes in their interconnections .
Therefore, network analysis has an advantage compared to factor analysis.
It can detect factors that occur from several variables .
very well even when variables .
had a high correlation with each other.
The implication is the same structure with both methods, opens the opportunity for network analysis methods as an alternative in identifying the structure of a test.
Furthermore, this opens up an alternative to using open-source software for researchers for statistical analysis, namely JASP.
Recommendation The network analysis approach offers new opportunities for visualizing factor structures from a psychological test.
This network approach could analyze data for psychological measurements validation study, especially to determine the internal structure of psychological measurements.
In further research, network analysis can be applied on a multidimensional psychological scale.
Then, it can also be used on a scale that is still experiencing difficulty in determination of an item membership, should it will better to be a particular factor or dimension compared to other factor or dimension.
Further research needs to consider using a hardcopy of a questionnaire method for data collection to accommodate participants who have not received Internet Considering that data collection should not only be in urban areas but also in rural areas.
Further data collection should also take into account the geographical location so that regions other than Jabodetabek can be represented.
Also, the gender of participants, especially the representation of men in the sample.
References