IMR Press / JOMH / Volume 18 / Issue 5 / DOI: 10.31083/j.jomh1805110
Open Access Original Research
COVID-19 Traumatic Stress Scale and Preventive COVID-19 Infection Behaviors Scale: psychometric properties in Portuguese male adults
Show Less
1 Universidade Católica Portuguesa, 4710-362 Braga, Portugal
2 Research Center in Psychology (CIPsi), School of Psychology, University of Minho, 4710-057 Braga, Portugal
*Correspondence: gracep@psi.uminho.pt (M. Graça Pereira)
J. Mens. Health 2022 , 18(5), 110; https://doi.org/10.31083/j.jomh1805110
Submitted: 15 December 2021 | Revised: 11 January 2022 | Accepted: 4 February 2022 | Published: 12 May 2022
Copyright: © 2022 The Author(s). Published by IMR Press.
This is an open access article under the CC BY 4.0 license.
Abstract

Background: The coronavirus disease 2019 (COVID-19) pneumonia pandemic constitutes a serious public health emergency. Besides its detrimental social and economic implications, it has generated a negative psychological impact worldwide. Several studies have been carried out concerning psychological impact and mental health related to COVID-19, with the psychological constructs most studied being anxiety, fear, phobia, stress and depression. Other psychological constructs were less studied, namely post-traumatic stress related to COVID-19, as well as preventive behaviors towards COVID-19. Thus, the aims of this study were to validate the COVID-19 Traumatic Stress (C19TSS) Scale and the Preventive COVID-19 Infection Behaviors Scale (PCIBS) with a Portuguese male adult sample and to measure their invariance across age and education. Methods: Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to obtain the final factor structure of the Portuguese version of both scales. Configural, metric and scalar invariance were measured across age and education through multiple-group confirmatory factor analysis (MGCFA). Results: Results showed that both C19TSS and PIBS models fitted the data well. Configural, metric and scalar invariance across age as well as configural and metric invariance across education were found regarding C19TSS. Only configural invariance across education was found regarding PCIBS. Conclusions: The C19TSS and PCIBS are valid and reliable tools for researchers interested in examining post-traumatic stress related to COVID-19 and preventive behaviors towards COVID-19.

Keywords
COVID-19
COVID-19 Traumatic Stress Scale
male adults
Preventive COVID-19 Infection Behaviors Scale
1. Introduction

The coronavirus disease 2019 (COVID-19) pneumonia pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has spread globally and constitutes itself as a serious public health emergency [1], resulting in an unprecedented series of lockdowns worldwide [2]. The COVID-19 pandemic has generated worldwide social and economic upheaval [3], as well as a negative psychological impact [4].

Several studies have been carried out concerning psychological impact and mental health related to COVID-19. The psychological constructs most studied have been anxiety [5, 6, 7], fear [8, 9, 10], phobia [11, 12, 13], stress [14, 15, 16] and depression [17, 18, 19]. Most of these studies concluded that these constructs worsened in the pandemic context i.e., globally individuals felt more anxiety, fear, phobia, stress and depression. However, several systematic reviews and meta-analyses regarding the psychological impact of COVID-19 revealed that the psychological impact of COVID-19 lockdowns was not very large and differ greatly between subjects, with a large percentage of individuals being psychologically resilient to the effects of COVID-19 [2]. Furthermore, Luo et al. [20] showed that anxiety and depression were higher in patients with previous illnesses and COVID-19 infection but similar among health professionals and the general population. Kontoangelos et al. [21] found that children and older people were likely to experience worry, anxiety and fear feeling more vulnerable to COVID-19. Xiong et al. [22] observed high rates of anxiety, depression, and stress in China, Spain, Italy, Iran, United States, Turkey, Nepal and Denmark, in the general population, during the COVID-19 pandemic and concluded that being a female, belonging to a younger age group (less than 40 years old), being a student, having chronic and/or psychiatric illnesses, being unemployed, and permanently exposed to news about COVID-19 were risk factors for increased distress. Other psychological constructs were less studied, namely post-traumatic stress related to COVID-19, as well as preventive behaviors towards COVID-19. Kira et al. [14, 23] argued that COVID-19 is a new type of traumatic stress that has serious mental health effects [23]. Thus, Kira et al. [14] developed and validated a measure for COVID-19 as traumatic stress, with three dimensions: (1) threat/fear of infection and death, (2) economic hardship, and (3) disturbed routines/isolation. This instrument presented strong reliability, and structural, convergent, divergent, and predictive validity (total and subscales). Also, Tosun et al. [24] validated the COVID-19 Traumatic Stress Scale in Turkish and kept the three dimensions, but removed one item (item 9).

The COVID-19 pandemic is a threat to well-being due to social disruption such as financial insecurity, and confinement-related stress [25]. Brodeur et al. [26] based on google trends data, assessed changes in well-being, taking into consideration the topic search in the internet, and found a huge increase in searches on boredom, loneliness, worry and sadness. O’Connor et al. [27] carried out the UK COVID-19 mental health and wellbeing study and found that well-being had decreased with increasing suicidal thoughts among young adults.

To reduce the transmission rate of infectious disease, it seems that awareness and practice of preventive behaviors are key points [28], since individuals who perceived greater risks implemented more protective and preventive behaviors [29]. Also, knowledge about COVID-19 was associated with more preventive behaviors through risk perception [30]. According to Chang et al. [31], preventive COVID-19 infection behaviors were proposed by the World Health Organization (WHO) [32] and Chan et al. [31] conceived an instrument to assess those individual’s preventive behaviors.

In Portugal, although there are validated instruments to assess the psychological impact of COVID-19 with respect to anxiety (Coronavirus Anxiety Scale [33]), fear (Fear of COVID-19 Scale [33]), stress (COVID-19 Perceived Risk Scale [34]), phobia (COVID-19 Phobia Scale [34]) and depression (Negative Impact of COVID-19 [35]), no instrument assessing post-traumatic stress related to COVID-19, or preventive behaviors towards COVID-19 has been validated. Thus, the aim of this study was the validation for Portuguese male adults of the COVID-19 Traumatic Stress Scale and Preventive COVID-19 Infection Behaviors Scale.

In the Iranian validated version of PCIBS, distress assessed by the Hospital Anxiety and Depression Scale (HADS) was used to determine the adequacy of convergent validity. In the original C19TSS, well-being (assessed by WHO-5) was used to assess convergent validity. The present study is part of a bigger protocol that assessed the impact of Covid-19 on psychological wellbeing and posttraumatic growth in male adults [36] that included HADS and QGBEP-R to evaluate depression/anxiety and wellbeing, respectively. Therefore, in the present study, both instruments assessing distress and well-being were used to determine the adequacy of convergent validity of PCIBS and C19TSS.

2. Material and methods
2.1 Participants

The sample included 220 men, mostly Portuguese (93.2%), coming from an urban environment (75%), with an average age of 33.88 years (SD = 12.87), the majority being single (62.3%), not in a romantic relationship (68.6%) and having no children (71.4%). The vast majority of the sample was university educated (69.1%) and professionally active (79.1%). Only 14.1% of the sample had received a positive diagnosis of COVID-19, although not infected at the time of data collection, and only 10.4% lived with someone who had been previously infected with COVID-19.

2.2 Instruments
2.2.1 COVID-19 Traumatic Stress Scale (C19TSS)

The C19TSS [14] was developed to assess COVID-19 as traumatic stress. The scale consisted of 12 items answered in a 5-point Likert scale (0 = not at all; 4 = very much) and three dimensions: “threat/fear of infection and death”, “economic hardship”, and “disturbed routines/isolation” (e.g., thinking about the coronavirus (COVID-19) makes me feel threatened). A higher score indicates more traumatic stress related to COVID-19. Kira et al. [14] found that Cronbach’s alpha for COVID-19 traumatic stress scale was 0.88, for future infection/death subscale was 0.84, for economic trauma subscale was 0.75, and for routine disturbance subscale was 0.70.

2.2.2 Preventive COVID-19 Infection Behaviors Scale (PCIBS)

Based on the World Health Organization [32] preventive behaviors recommended for COVID-19, Chang et al. [31] developed the PCIBS to assess how individuals perform preventive COVID-19 infection behaviors (e.g., how often do you avoid touching eyes, nose, and mouth?) This instrument is unifactorial and includes five items answered in a 5-point Likert scale (1 = almost never; 5 = almost always). Higher scores indicate the use of more preventive behaviors. Chang et al. [31] found a single-factor structure with satisfactory fit indices and an internal consistency of 0.80.

2.2.3 Hospital Anxiety and Depression Scales (HADS)

Hospital Anxiety and Depression Scale [37] assesses psychological morbidity and includes two subscales: anxiety (e.g., I get a sort of frightened feeling as if something awful is about to happen) and depression (e.g., I still enjoy the things I used to enjoy) each with 7 items. The items are scored in a four-point Likert scale ranging from “never” (0) to “always” (3). A high score in each subscale indicates more anxiety and depression symptoms or distress considering the global score. Cronbach alpha for the full scale was 0.89 in the original version and 0.87 in the Portuguese version. In the present study, Cronbach alpha was 0.86 [38].

2.2.4 Psychological General Well-Being Index—short version (QGBEP-R)

The short version of the Psychological General Well-Being Index (PGWB-S) [39] assesses psychological general well-being and was adapted to the Portuguese population (QGBEP-R) by Soares et al. [40] and includes six items (e.g., I felt cheerful, lighthearted during the past month). Higher scores indicate higher levels of psychological well-being. In the original version Cronbach alpha was 0.73 and in the Portuguese version 0.86. In the present study, Cronbach alpha was 0.89.

2.3 Procedures

All procedures were in accordance with the Declaration of Helsinki [41]. The Ethics Committee of the University of Minho (Portugal) authorized the study and the permission, from the original authors, to translate and validate the instruments was also granted. The research protocol included an informed consent, in which the anonymity and confidentiality of the data were ensured; a sociodemographic questionnaire, the COVID-19 Traumatic Stress Scale (C19TSS), the Preventive COVID-19 Infection Behaviors Scale (PCIBS), the Hospital Anxiety and Depression Scales (HADS) and the Psychological General Well-Being Index - Short version (QGBEP-R). The protocol was made available online through a page allocated to a social network (Facebook) designed for that purpose.

The COVID-19 Traumatic Stress (C19TSS) Scale and the Preventive COVID-19 Infection Behaviors Scale (PCIBS) were translated from English into Portuguese by two bilingual translators; the two translations were compared by two psychologists and divergences were removed after reaching a consensus considering the initial translation. This version was back translated by other two translators and the versions were compared again by the psychologists. The divergences found were discussed and until a consensus was obtained.

2.4 Data analysis

The sociodemographic characteristics of men were described through descriptive analysis. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to obtain the final factor structure of the Portuguese version for both the C19TSS [14] and PCIBS [31]. The first statistical technique was used to confirm the possible structure of each scale.

Taking into account that the sample for CFA should be different than the sample used to establish EFA [42] and for a multi-group CFA, a general rule of thumb is 100 participants in each group [43, 44], in order to reduce probability of error variance, 100 random samples composed of 55 participants (the minimum to perform EFA - 5 subjects per item considering the total number (11) of items) were created, through SPSS version 27 (IBM Corp., Armonk, NY, USA).

The most frequent factorial models in the 100 random samples were chosen and their adequacy examined through CFA, using structural equation modeling (SEM). The following fit indices were considered: χ2/df (ratio of chi-square over the number of degrees of freedom, which should show a ratio of 3:1 or less to represent a good fit), RMSEA (root mean square error of approximation that should be below 0.08 to be considered acceptable), CFI and TLI (Comparative Fit Index and Tucker-Lewis Index that should both be 0.95 to be considered good fit indexes) and SRMR (standardized root mean square residual that should be above 0.05).

To compare the more frequent models for each scale, either the AIC (Akaike Information Criterion) and the BIC (Bayesian Information Criterion) were taken into consideration (should decrease compared to the original model in order to indicate a good fit) [45]. When CFA suggested modification indices, such as correlations between item errors or between factors and errors, this procedure was followed. That is why in all models, error 1 was correlated with error 4 to improve fit indexes, since all items showed significant standardized regression weights.

To examine the reliability of both the C19TSS scale and the PCIBS scale, Cronbach alphas were calculated considering that a coefficient 0.70 reflects a moderate reliability and very good when above 0.80 [46] and corrected item-total correlation were also taken into consideration (above 0.30 indicate good inter-item correlation) [47].

To analyze the convergent validity of each scale, Pearson correlations between C19TSS and QGBEP-R and between PCIBS and HADS were performed. Finally, configural, metric and scalar invariance across age and education were examined through multiple-group confirmatory factor analysis (MGCFA) by assessing the change of CFI values. Evidence for metric and scalar invariance is determined if CFI change is <0.010.

3 Results
3.1 Principal characteristics of the structural models

The EFA showed that both in C19TSS and in PCIBS 6 factorial models were found: model 1, 2, 3, 4, 5, 6 and 2 factorial models (model 1 and 2) were more frequent, respectively.

The adequacy of the sample for EFA was assessed by the Kayser-Meyer-Olkin measure (KMO) (0.778 on C19TSS scale and 0.627 in the PCIBS) and by the results of Bartlett’s test of sphericity (p < 0.000 in C19TSS scale and in the PCIBS scale) revealing that the study sample was adequate for both scales. A solution of 3 factors, like the original version [14] in the C19TSS scale was maintained, since the eigenvalues also showed results above 1.00 which represents a good indicator of latent factors (Table 1).

Table 1.Exploratory Factor Analysis (EFA) of the Portuguese version of the COVID-19 Traumatic Stress Scale (C19TSS) with model 1 (N = 220).
Factorial structure of the C19TSS
Factor 1 Factor 2 Factor 3 Mean SD
Item 1: I am afraid of the coronavirus (COVID-19). 0.853 0.445 0.166 5.89 4.01
Item 2: I am stressed around other people because I worry I’ll catch the coronavirus (COVID-19). 0.880 0.174 0.265 5.02 4.17
Item 3: Thinking about the coronavirus (COVID-19) makes me feel threatened. 0.756 0.358 0.401 36.11 1.08
Item 4: How concerned are you that you’ll be infected with the coronavirus? 0.873 0.141 0.212 4.84 2.96
Item 5: Over the past two weeks, I have felt nervous and fearful about the future because of the coronavirus. 0.864 0.275 0.257 3.64 2.97
Item 6: The Coronavirus (COVID-19) has impacted me negatively from a financial point of view. 0.289 0.833 0.339 4.58 3.95
Item 7: I have lost job-related income due to the Coronavirus (COVID-19). 0.187 0.901 0.207 3.75 4.38
Item 8: I have had a hard time getting needed resources (food, toilet paper) due to the Coronavirus (COVID-19). 0.265 0.677 0.247 1.40 1.42
Item 9: It has been difficult for me to get the things I need due to the Coronavirus (COVID-19). 0.212 0.271 0.856 6.22 4.25
Item 10: Over the past two weeks, I have felt socially isolated as a result of the coronavirus. 0.195 0.227 0.696 7.15 4.00
Item 11: Over the past two weeks, my life routines have been affected by the coronavirus situation. 0.305 0.262 0.854 6.15 4.24
Eigenvalues 4.427 1.855 1.371
% Variance 40.247 16.862 12.462
Total % Variance 69.571
The best factor loadings of each item are presented in bold. SD, standard deviation.

In the PCIBS, a solution of 1 factor, similar to the original scale [31] was found and the eigenvalue was also above 1.00 (Table 2).

Table 2.Exploratory Factor Analysis (EFA) of the Portuguese version of the Preventive COVID-19 Infection Behaviors Scale (PCIBS) with model 2 (N = 55).
Factorial Structure of the PCIBS
Factor 1 Mean SD
Item 1: How often do you regularly and thoroughly clean your hands with an alcohol-based hand rub or wash them with soap and water? 0.629 4.05 0.803
Item 2: How often do you avoid touching eyes, nose, and mouth? 0.885 3.47 10.069
Item 3: How often do you cover your mouth and nose with your bent elbow or tissue when you cough or sneeze? 0.655 4.38 0.991
Item 4: How often do you maintain at least 1-meter distance between yourself and others? 0.685 4.27 0.891
Item 5: How often do you stay home when you feel unwell? –0.099 5.07 0.790
Eigenvalues 2.051
Total % Variance 41.027
The best factor loadings of each item are presented in bold, and should be above 0.30. The item 5 was removed since the factor loading was below this criterion. SD, standard deviation.

The results of the CFA for C19TSS scale are presented in Table 3. In the analysis of the C19TSS, six factor models: model 1 and model 4 presented the best initial fit indices. The difference between the two models concerned factor 2 and factor 3 whose items were switched. The same was true for model 2 and model 5. To improve the adjustment of all models, according to the criteria previously defined, error 1 and error 4 were correlated, as suggested by the modification indices. Thus, the model with best adjustment were model 1 and model 4. Model 1 was selected since it revealed a factorial structure similar to the original version of the scale (Table 3 and Fig. 1).

Table 3.Factorial structure of each EFA model and fit indices from CFA in the Portuguese Version of C19TSS (N = 220).
χ2/df RMSEA CI 90% LL; HL p CFI TLI SRMR AIC BIC
Structure 1- Original model
Factor 1 (items 1, 2, 3, 4 and 5) 2.432 0.081 0.061; 0.101 0.007 0.935 0.912 0.068 149.715 234.556
Factor 2 (items 6, 7 and 8)
Factor 3 (items 9, 10 and 11)
Correlation between e1 and e4 1.968 0.066 0.044; 0.088 0.103 0.957 0.941 0.061 130.718 218.952
Structure 2
Factor 1 (items 1, 2, 3, 4, 5 and 8) 2.544 0.084 0.064; 0.104 0.003 0.930 0.906 0.070 154.300 239.141
Factor 2 (items 6 and 7)
Factor 3 (items 9, 10 and 11)
Correlation between e1 and e4 2.011 0.068 0.046; 0.089 0.084 0.955 0.938 0.064 132.446 220.68
Structure 3
Factor 1 (items 1, 2, 3 and 4) 3.744 0.112 0.093: 0.131 0.000 0.875 0.832 0.098 203.503 288.344
Factor 2 (items 6, 7 and 8)
Factor 3 (items 5, 9, 10 and 11)
Correlation between e1 and e4 3.348 0.104 00.84; 0.123 0.000 0.896 0.856 0.092 185.901 274.135
Structure 4
Factor 1 (items 1, 2, 3, 4 and 5) 2.432 0.081 0.061; 0.101 0.007 0.935 0.912 0.068 149.715 234.556
Factor 2 (items 9, 10 and 11)
Factor 3 (items 6, 7 and 8)
Correlation between e1 and e4 1.968 0.066 0.044; 0.088 0.103 0.957 0.941 0.061 130.718 218.952
Structure 5
Factor 1 (items 1, 2, 3, 4, 5 and 8) 2.544 0.084 0.064; 0.104 0.003 0.930 0.906 0.070 154.300 239.141
Factor 2 (items 9, 10 and 11)
Factor 3 (items 6 and 7)
Correlation between e1 and e4 2.011 0.068 0.046; 0.089 0.084 0.955 0.938 0.064 132.446 220.68
Structure 6
Factor 1 (items 1, 2, 3, 4 and 5) 2.628 0.086 0.067; 0.106 0.002 0.926 0.900 0.076 157.728 242.569
Factor 2 (items 6 and 7)
Factor 3 (items 8, 9, 10 and 11)
correlation between e1 and e4 2.143 0.072 0.051; 0.093 0.043 0.949 0.093 0.071 137.728 225.962
χ2, chi-square; def, degrees of freedom; RMSEA, root mean square error of approximation; CI, confidence interval; LL, low; HL, high: p, p-value; CFI, comparative fit index; TLI, Tucker-Lewis Index; SRMR, Standardized root mean square residual; AIC, Akaike Information Criterion; BIC, Bayesian Information Criterion.
Fig. 1.

CFA Results for Model 1 of the Portuguese version of C19TSS. Model fit indices: χ2/df: 1.968; RMSEA: 0.066; CI 90% LL; HL: 0.044; 0.088; p: 0.103; CFI: 0.957; TLI: 0.941; SRMR: 0.061.

Regarding the CFA analysis of the two most frequent factor models for PCBIS, after the removal of item 5 (due to the factor loading being below 0.30), and with the correlation between error 2 and error 3, model 2 showed the best adjustment to the data (Table 4 and Fig. 2).

Table 4.Factorial structure for EFA and model fit indices from the CFA of the Portuguese version of PCIBS (N = 220).
χ2/df RMSEA CI 90% LL; HL p CFI TLI SRMR AIC BIC
Structure 1- Original model
Factor 1 (items 1, 2, 3, 4 and 5) 2.309 0.077 0.014; 0.137 0.183 0.925 0.850 0.047 31.543 65.479
Structure 2
Factor 1 (items 1, 2, 3, and 4) 3.734 0.112 0.035; 0.202 0.083 0.933 0.799 0.044 23.468 50.617
correlation between e2 and e3 1.123 0.024 0.000; 0.183 0.410 0.998 0.991 0.019 19.123 49.666
χ2, chi-square; def, degrees of freedom; RMSEA, root mean square error of approximation; CI, confidence interval; LL, low; HL, high; p, p-value: CFI, comparative fit index; TLI, Tucker-Lewis Index; SRMR, Standardized root mean square residual; AIC, Akaike Information Criterion; BIC, Bayesian Information Criterion.
Fig. 2.

CFA results of Model 2 of Portuguese version of PCIBS. Model fit indices: χ2/df: 1.123; RMSEA: 0.024; CI 90% LL; HL: 0.000; 0.183; p: 0.410; CFI: 0.998; TLI: 0.991; SRMR: 0.019.

3.2 Reliability

The corrected item-total correlations showed reasonable results both for C19TSS (from 0.272 to 0.611) and in PCIBS (from 0.331 to 0.440). The C19TSS scale presented a Cronbach alpha of 0.80 for the total scale; 0.84 for factor 1; 0.74 for factor 2; 0.72 for factor 3. The Cronbach alpha of PCIBS was 0.59.

3.3 Convergent validity

The Pearson correlation between the Portuguese version of the C19TSS with QGBEP-R (well-being) scale showed good convergent validity, both the total scale and each factor: total scale (r = –0.532; p < 0.001), Factor 1 (r = –0.447; p < 0.001), Factor 2 (r = –0.274; p < 0.001) and Factor 3 (r = –0.401; p < 0.001). Also, the Pearson Correlation between the Portuguese version of the PCIBS with HADS (distress) presented good convergent validity: total scale (r = 0.388; p < 0.001).

3.4 Measurement invariance of the C19TSS and PCIBS across age and education

Table 5 presents the results for measurement invariance across age and education (both two levels) concerning C19TSS. The progressive age invariance showed that configural invariance model across age presented a good model fit. Also, the metric invariance test showed that the model fitted well the data. The change of CFI between configural and metric invariance was lower than 0.010 confirming the metric invariance of C19TSS across age. This change was also lower than 0.010 between metric and scalar variance, confirming the scalar variance of C19TSS across age. The same trend was found concerning education; however, scalar invariance for C19TSS across education was not found because the change of the CFI was greater than 0.010 (Table 5).

Table 5.Measurement invariance tests of the C19TSS across age and education.
χ2 df χ2/df p CFI RMSEA (90% CI) ΔCFI
Age
Configural invariance 150.635 80 1.883 0.000 0.924 0.064 (0.048–0.079)
Metric invariance 160.692 88 1.826 0.000 0.922 0.062 (0.046–0.076) 0.002
Scale invariance 171.046 94 1.820 0.000 0.917 0.061 (0.046–0.076) 0.005
Education
Configural invariance 131.482 80 1.644 0.000 0.944 0.054 (0.037–0.071)
Metric invariance 136.992 88 1.557 0.001 0.947 0.051 (0.033–0.066) –0.003
Scale invariance 155.693 94 1.656 0.000 0.933 0.055 (0.039–0.070) 0.014
*p < 0.001 for all indicators.
χ2, chi-square; def, degrees of freedom; CFI, comparative fit index; RMSEA, root mean square error of approximation; CI, confidence interval; ΔCFI, adjusted comparative fit index.

Table 6 shows the results of the measurement of invariance of the PCIBS across age and education. Concerning age, configural, metric and scalar invariance were found as there were no differences in CFI between them. Concerning education, only configural invariance was found since the differences in CFI were greater than 0.010.

Table 6.Measurement invariance tests of the PCIBS across age and education.
χ2 df χ2/df p CFI RMSEA (90% CI) ΔCFI
Age
Configural invariance 1.108 2 0.554 0.575 1.000 0.000 (0.000–0.113)
Metric invariance 3.648 5 0.730 0.601 1.000 0.000 (0.000–0.080) 0.000
Scale invariance 4.303 6 0.717 0.636 1.000 0.000 (0.000–0.072) 0.000
Education
Configural invariance 1.755 2 0.878 0.416 1.000 0.000 (0.000–0.113)
Metric invariance 6.011 5 1.202 0.305 0.987 0.051 (0.033–0.066) 0.013
Scale invariance 6.167 6 1.028 0.405 0.998 0.055 (0.039–0.070) 0.011
*p < 0.001 for all indicators.
χ2, chi-square; def, degrees of freedom; CFI, comparative fit index; RMSEA, root mean square error of approximation; CI, confidence interval; ΔCFI, adjusted comparative fit index.
4. Discussion

In Portugal, there are very few validated instruments to assess the psychological impact of COVID-19. Thus, the aims of this study were the validation of the COVID-19 Traumatic Stress (C19TSS) Scale and the Preventive COVID-19 Infection Behaviors Scale (PCIBS) in a Portuguese male adult sample including the measurement of their invariance across age and education. To achieve these goals, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to obtain the final factor structure of the Portuguese version of both the C19TSS [14] and PCIBS [31].

Several multiple-group confirmatory factor analyses (MGCFA) were also performed to assess configural, metric and scalar invariance. The literature confirms the adequacy of the statistical analysis to the study goals. In fact, exploratory factor analysis (EFA) is a multivariate statistical method that identifies the smallest number of hypothetical constructs that parsimoniously explain the cooperation among the measured variables [48]. CFA assesses the internal structure of an instrument after EFA to confirm the number of the latent variables and the patterns of relationships [49]. MGCFA is the most widely used method for testing, measurement invariance and includes simultaneous CFAs in two or more groups, using separate variance-covariance matrices for each group [49, 50].

The results of EFA showed that C19TSS kept a solution of 3 factors (as the original version of Kira et al. [14] and the Turkish version of Tosun, Dalgar and Altinoz [24]: (1) threat/fear of future infection/death, (2) economic stressors/traumas, (3) routine disturbance, isolation and related secondary traumas). The results of EFA also showed that PCIBS maintained a solution of 1 factor (as the original scale of Chang et al. [30] and the Spanish version of Caycho-Rodriguez et al. [51]).

The findings also showed that CFA confirmed the C19TSS structure found by EFA, like in the original version [14]; however, in the Turkish version, the item 9 was removed to find an adequate model fit. In the Portuguese version, the error of item 1 and the error of item 4 were correlated to improve the model fit of the final model, in order to preserve the original configuration of the scale [14]. The internal consistency of the total scale of the Portuguese version (α = 0. 80) was slightly smaller than the Turkish version (α = 0.85) [24] and the original version (α = 0.88) [14].

Regarding the internal consistency of the subscales in the Portuguese version and in the original version, Cronbach alphas were very similar, i.e., 0.84, 0.74, and 0.72 for factor 1, factor 2 and factor 3 in the Portuguese version and 0.84, 0.75, and 0.70 in the subscales fear from the future infection, economic impact and isolation, routine and disruption in the original version [14], respectively. Concerning PCBIS, item 5 was removed because the factor loading was below 0.30, but kept its unidimensional nature. The same was true regarding the Spanish version [51], adapted in 12 Latin American countries with a sample of 5183 participants, the PCBIS was composed only by four items, since item 5 showed a low factorial loading and was removed, which improved the model fit similar to the present study. Perhaps, common sociocultural reasons between Latin societies and the Portuguese society may explain the need to remove the item in both versions. One may hypothesize that the item that addresses the need to stay at home when not feeling well is not specific to the COVID-19 virus and was redundant, but more studies are needed to understand and pursue this hypothesis. The internal consistency of the Portuguese version of PCIBS was lower when compared either with the original version (α = 0.82) [30] or with the Spanish version that showed Cronbach’s alpha ranging between 0.71 and 0.86 [51].

The convergent validity of C19TSS (with well-being) and of PCIBS (with distress) was confirmed. As in the original version of C19TSS, Kira et al. [14] also found that this scale was negatively associated with well-being. Also, Tosun et al. [24], with an Iranian sample, found that the PCIBS showed a moderated and significant relationship with psychological distress, measured with HADS [37] as in the present validation.

Configural, metric and scalar invariance was found across age for C19TSS and PCIBS. These results are in accordance with Liu et al. [52] who found higher levels of post-traumatic stress disorder in younger people (men between 26 and 30 years old) during the pandemic, which is an age group near participants’ age in the present study, with similar relatively homogeneous results. However, these results are not in agreement with Kim and Crimmins [53] who found that different strategies were needed to motivate older and younger people to adopt behavioral modifications to reduce the further spread of COVID-19. In fact, younger adults showed coping appraisal (perceptions of the ability to perform protective behaviors from coronavirus and the recommended changes) linked to preventive behaviors while older adults’ perception of severity was linked to behavioral responses.

Configural and metric (but not scalar) invariance was also found across education for C19TSS. This result contradicts Ramírez et al. [54] who found no impact of education in posttraumatic stress related to COVID-19. Finally, configural (but not metric and scalar) invariance across education for PCIBS was found. This result is in line with Kim and Kim [55] who observed that education level played a moderating role in inducing preventive actions.

Despite its strengths, this study also presents some limitations such as the cross-sectional design limits the conclusions that can be drawn from the findings. Participants were recruited from the general population however, the sample was relatively homogeneous, also limiting the ability to generalize the findings. Therefore, future studies with more diverse samples should attempt to replicate these findings. COVID-19 Traumatic Stress and Preventive COVID-19 Infection Behaviors scales were assessed by self-report questionnaires, rather than a structured clinical interview conducted by a mental health professional. As such, the measure of traumatic stress (related to COVID-19) may not be fully accurate in capturing clinical levels of traumatic stress. Finally, future studies with larger samples should establish cutoff points for both instruments.

5. Conclusions

The results of this study show that the Traumatic Stress and Preventive COVID-19 Infection Behaviors are reliable instruments to measure COVID-19 in Portuguese men.

Author contributions

AL—Conceptualization; Data curation; Investigation; Methodology; Validation; Writing - review & editing; Final approval. MGP—Investigation; Methodology; Validation; and data analysis and interpretation. AL, MGP, ACA—Data curation; Review and editing; Final approval. AL—Project administration; Visualization; Roles/Writing - original draft; Writing - review and editing; Final approval. All authors participated in Conceptualization, Data Curation, Data Analysis and Interpretation, Writing, Review and Editing of the manuscript and approved the final version.

Ethics approval and consent to participate

This study was approved by the Ethics Committee for Research in Social and Human Sciences of the University of Minho, Braga, Portugal with the number: CEICSH 018/2021. All participants gave their informed consent for inclusion before they participated in the study.

Acknowledgment

The authors want to thank all participants for their contribution to the present study.

Funding

This research received no external funding.

Conflict of interest

The authors declare no conflict of interest. MGP is serving as one of the Editorial Board members of this journal. We declare that MGP was not involved in the peer review of this article and had no access to information regarding the peer review. Full responsibility for the editorial process for this article was delegated to AW.

References
[1]
World Health Organization. COVID-19 Public Health Emergency of International Concern (PHEIC) Global research and innovation forum. 22 February 2020. Available at: https://www.who.int/publications/m/item/covid-19-public-health-emergency-of-international-concern-(pheic)-global-research-and-innovation-forum (Accessed: 12 January 2022).
[2]
Greene C, Burleson S, Crosby C, Heimann M, Pigott D. Coronavirus disease 2019: International public health considerations. Journal of the American College of Emergency Physicians Open. 2020; 1: 70–77.
[3]
Buheji M, da Costa Cunha K, Beka G, Mavric B, De Souza Y, da Costa Silva S, et al. The extent of covid-19 pandemic socio-economic impact on global poverty. A global integrative multidisciplinary review. American Journal of Economics. 2020; 10: 213–224.
[4]
Pfefferbaum B, North C. Mental health and the Covid-19 pandemic. New England Journal of Medicine. 2020; 383: 510–512.
[5]
Elbay R, Kurtulmuş A, Arpacıoğlu S, Karadere E. Depression, anxiety, stress levels of physicians and associated factors in COVID-19 pandemics. Psychiatry Research. 2020; 290:113130.
[6]
Mazza M, De Lorenzo R, Conte C, Poletti S, Vai B, Bollettini I, et al. Anxiety and depression in COVID-19 survivors: Role of inflammatory and clinical predictors. Brain, Behavior, and Immunity. 2020; 89: 594–600.
[7]
Peteet J. COVID-19 Anxiety. Journal of Religion and Health. 2020; 59: 2203–2204.
[8]
Ahorsu DK, Lin C-Y, Imani V, Saffari M, Griffiths MD, Pakpour AH. The fear of COVID-19 scale: Development and initial validation. International Journal of Mental Health and Addiction. 2020; 1–9.
[9]
Fitzpatrick K, Harris C, Drawve G. Fear of COVID-19 and the mental health consequences in America. Psychological Trauma: Theory, Research, Practice, and Policy. 2020; 12: S17–S21.
[10]
Pakpour A, Griffiths M. The fear of COVID-19 and its role in preventive behaviours. Journal of Concurrent Disorders. 2020; 2: 58–63.
[11]
Arpaci I, Karataş K, Baloğlu M. The development and initial tests fr the psychometric properties of the COVID-19 Phobia Scale (C19P-S). Personality and Individual Differences. 2020; 164: 110108.
[12]
Dilbaz N, Noyan O, Alpar G, Kazan O. Development of the COVID-19 phobia scale: Validity and reliability study. The Journal of Neurobehavioral Sciences. 2020; 7:142.
[13]
Lindinger-Sternart S, Kaur V, Widyaningsih Y, Patel A. COVID-19 phobia across the world: Impact of resilience on COVID-19 phobia in different nations. Counselling and Psychotherapy Research. 2021; 21: 290–302.
[14]
Kira IA, Shuwiekh HAM, Rice KG, Ashby JS, Elwakeel SA, Sous MSF, et al. Measuring COVID-19 as Traumatic Stress: Initial Psychometrics and Validation. Journal of Loss and Trauma. 2021a; 26: 220–237.
[15]
Odriozola-González P, Planchuelo-Gómez Á, Irurtia MJ, de Luis-García R. Psychological symptoms of the outbreak of the COVID-19 confinement in Spain. Journal Health Psychology. 2020; 1359105320967086.
[16]
Park CL, Russell BS, Fendrich M, Finkelstein-Fox L, Hutchison M, Becker J. Americans’ COVID-19 Stress, Coping, and Adherence to CDC Guidelines. Journal of General Internal Medicine. 2020; 35: 2296–2303.
[17]
Bäuerle A, Teufel M, Musche V, Weismüller B, Kohler H, Hetkamp M, et al. Increased generalized anxiety, depression and distress during the COVID-19 pandemic: a cross-sectional study in Germany. Journal of Public Health. 2020; 42: 672–678.
[18]
Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence of Depression, Symptoms in US Adults Before and During the COVID-19 Pandemic. JAMA Network Open. 2020; 3: e2019686.
[19]
Shader RI. COVID-19 and Depression. Clinical Therapeutics. 2020; 42: 962–963.
[20]
Luo M, Guo L, Yu M, Jiang W, Wang H. The psychological and mental impact of coronavirus disease 2019 (COVID-19) on medical staff and general public – A systematic review and meta-analysis. Psychiatry Research. 2020; 291:113190.
[21]
Kontoangelos K, Economou M, Papageorgiou C. Mental Health Effects of COVID-19 Pandemia: A Review of Clinical and Psychological Traits. Psychiatry Investigation. 2020; 17: 491–505.
[22]
Xiong J, Lipsitz O, Nasri F, Lui LM, Gill H, Phan L, et al. Impact of COVID-19 pandemic on mental health in the general population: A systematic review. Journal of Affective Disorders. 2020; 277: 55–64.
[23]
Kira IA, Shuwiekh HA, Ashby JS, Elwakeel SA, Alhuwailah A, Sous MSF, et al. The Impact of COVID-19 Traumatic Stressors on Mental Health: Is COVID-19 a New Trauma Type. International Journal of Mental Health and Addiction. 2021; 1–20.
[24]
Tosun Altınöz Ş, Dalgar I, Altınöz AE. The Turkish adaptation and psychometric characteristics of the COVID-19 Traumatic Stress Scale. Death Studies. 2021; 1–7.
[25]
Prime H, Wade M, Browne DT. Risk and resilience in family well-being during the COVID-19 pandemic. American Psychologist. 2020; 75: 631–643.
[26]
Brodeur A, Clark AE, Fleche S, Powdthavee N. COVID-19, lockdowns and well-being: Evidence from Google Trends. Journal of Public Economics. 2021; 193: 104346.
[27]
O’Connor RC, Wetherall K, Cleare S, McClelland H, Melson AJ, Niedzwiedz CL, et al. Mental health and well-being during the COVID-19 pandemic: longitudinal analyses of adults in the UK COVID-19 Mental Health & Wellbeing study. The British Journal of Psychiatry. 2021; 218: 326–333.
[28]
Shaw R, Kim Y, Hua J. Governance, technology and citizen behavior in pandemic: Lessons from COVID-19 in East Asia. Progress in Disaster Science. 2020; 100090.
[29]
Bruine de Bruin, W, Bennett D. Relationships Between Initial COVID-19 Risk Perceptions and Protective Health Behaviors: A National Survey. American Journal of Preventive Medicine. 2020; 59: 157–167.
[30]
Iorfa SK, Ottu IFA, Oguntayo R, Ayandele O, Kolawole SO, Gandi JC, et al. COVID-19 Knowledge, Risk Perception, and Precautionary Behavior Among Nigerians: A Moderated Mediation Approach. Frontiers in Psychology. 2020; 11: 3292.
[31]
Chang K, Hou W, Pakpour AH, Lin C, Griffiths MD. Psychometric Testing of Three COVID-19-Related Scales among People with Mental Illness. International Journal of Mental Health and Addiction. 2020; 1–13.
[32]
World Health Organization Q & A on coronaviruses (COVID-19). 2021. Available at: https://www.who.int/news-room/q-a-detail/q-a-coronaviruses (Accessed: 20 September 2021).
[33]
Magano J, Vidal DG, Sousa HFPe, Dinis MAP, Leite Â. Validation and Psychometric Properties of the Portuguese Version of the Coronavirus Anxiety Scale (CAS) and Fear of COVID-19 Scale (FCV-19S) and Associations with Travel, Tourism and Hospitality. International Journal of Environmental Research and Public Health. 2021; 18: 427.
[34]
Leite Â, Vidal DG, Sousa HFPe, Dinis MAP, Magano J. Portuguese Version of COVID-19 Perceived Risk Scale and COVID-19 Phobia Scale: Psychometric Properties. European Journal of Investigation in Health, Psychology and Education. 2021; 11: 1044–1060.
[35]
Pereira H, Esgalhado G, Costa V, Monteiro S, Oliveira V. Propriedades psicométricas das escalas de medo e impacto negativo face À COVID-19. Psicologia da Saúde e Doenças. 2021; 22: 338–349.
[36]
Santos M, Machado AM, Bernardo AC, Leite A, Pereira MG. Men’s psychological well-being during COVID-19: the moderator role of posttraumatic growth. Journal of Men’s Health. 2021; 17: 62–71.
[37]
Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica. 1983; 67: 361–370.
[38]
Pays-Ribeiro J, Silva I, Ferreira T, Martins A, Meneses R, Baltar M. Validation study of a Portuguese version of the Hospital Anxiety and Depression Scale. Psychology, Health & Medicine. 2007; 12: 225–237.
[39]
Grossi E, Groth N, Mosconi P, Cerutti R, Pace F, Compare A, et al. Development and validation of the short version of the Psychological General Well-Being Index (PGWB-S). Health and Quality of Life Outcomes. 2006; 4: 88.
[40]
Soares M, Antunes M, da Costa, Correia, Brito I, dos Santos Pinto M. Adaptação e validação do Questionário Geral de Bem-Estar Psicológico: Análise fatorial confirmatória da versão reduzida. Revista de Enfermagem Referência. 2018; 4: 9–18.
[41]
Halonen JI, Erhola M, Furman E, Haahtela T, Jousilahti P, Barouki R, et al. The Helsinki Declaration 2020: Europe that protects. The Lancet Planetary Health. 2020; 4: e503–e505.
[42]
Pearson RH, Mundform DJ. Recommended Sample Size for Conducting Exploratory Factor Analysis on Dichotomous Data. Journal of Modern Applied Statistical Methods. 2010; 9: 359–368.
[43]
Kline RB. Principles and Practice of Structural Equation Modeling. 4th edn. Guilford Publications: New York. 2016.
[44]
Wang J, Wang X. Structural Equation Modeling. Applications using Mplus. Higher Education Press: New York. 2012.
[45]
Busemeyer JR, Diederich A. Estimation and testing of computational psychological models. Academic Press: Massachusetts. 2014.
[46]
Hair, J, Black, W, Babin, B. Multivariate Data Analysis: A Global Perspective. Pearson Education; 2009.
[47]
Nunnally, J, Bernstein I. Psychometric Theory. 3rd edn. MacGraw-Hill: New York. 1994.
[48]
Watkins MW. Exploratory Factor Analysis: A Guide to Best Practice. Journal of Black Psychology. 2018; 44: 219–246.
[49]
Brown TA. Confirmatory factor analysis for applied research. 2nd edn. Guilford: New York. 2015.
[50]
Joo S, Kim ES. Impact of error structure misspecification when testing, measurement invariance and latent-factor mean difference using MIMIC and multiple-group confirmatory factor analysis. Behavior Research Methods. 2019; 51: 2688–2699.
[51]
Caycho-Rodríguez T, Vilca LW, Valencia PD, Carbajal-León C, Vivanco-Vidal A, Saroli-Araníbar D, et al. Cross-cultural Validation of a New Version in Spanish of Four Items of the Preventive COVID-19 Infection Behaviors Scale (PCIBS) in Twelve Latin American Countries. Frontiers in Psychology. 2021; 12: 5088.
[52]
Liu C, Liu D, Huang N, Fu M, Ahmed JF, Zhang Y, et al. The Combined Impact of Gender and Age on Post-traumatic Stress Symptoms, Depression, and Insomnia During COVID-19 Outbreak in China. Frontiers in Public Health. 2021; 8.
[53]
Kim JK, Crimmins EM. Age differences in the relationship between threatening and coping mechanisms and preventive behaviors in the time of COVID-19 in the United States: Protection Motivation Theory. Research in Psychotherapy. 2021; 23: 485.
[54]
González Ramírez LP, Martínez Arriaga RJ, Hernández-Gonzalez M, De la Roca-Chiapas JM. Psychological distress and signs of post-traumatic stress in response to the COVID-19 health emergency in a Mexican sample. Psychology Research and Behavior Management, 2020; 13: 589–597.
[55]
Kim S, Kim S. Analysis of the impact of health beliefs and resource factors on preventive behaviors against the COVID-19 Pandemic. International Journal of Environmental Research and Public Health, 2020; 17: 8666.
Publisher’s Note: IMR Press stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Share
Back to top