SciELO - Scientific Electronic Library Online

 
vol.37 issue2-3How Does the Academic and Scientific Profile of European Health Ministers and State Secretaries Matter for Informed Decisions?Nuclear Imaging Exams Authorized by the Brazilian Health Care System and Nuclear Imaging Equipment Distribution in the State of Rio Grande do Sul author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

  • Have no similar articlesSimilars in SciELO

Share


Portuguese Journal of Public Health

Print version ISSN 2504-3137On-line version ISSN 2504-3145

Port J Public Health vol.37 no.2-3 Lisboa  2019

https://doi.org/10.1159/000504629 

RESEARCH ARTICLE

 

Study of the Psychometric Properties of the Diabetes Empowerment Scale Short Form (DES-SF)

 

Estudo das propriedades psicométricas da Escala do Empowerment na Diabetes Versão Reduzida (DES-SF)

 

Maria Rui Sousa,a Matilde Almeida,b Helena Loureiro,c Teresa Martinsa

a Escola Superior de Enfermagem do Porto (ESEP), Center for Research in Health Technologies and Services (CINTESIS), Porto, Portugal;

b Clínica da Glória Cuidados de Saúde, Porto, Portugal;

c Universidade de Aveiro, Escola Superior de Saúde, Porto, Portugal

 

ABSTRACT

Introduction: The control of diabetes mellitus type 2 implies that people are actively engaged in self-care behaviours and self-efficacy is one of the variables involved in this process. Based on the Model of Behavioural Change, the Diabetes Empowerment Scale assesses the psychosocial self-efficacy of people with this disease. The aim of this study is to analyse the psychometric properties of the Portuguese version of the Diabetes Empowerment Scale (DES-SF), namely its reliability and validity. - Method: A methodological nature study was conducted, with a convenience sample comprising 123 participants with diabetes mellitus type 2. In a primary health care context, the participants were asked to fill the DES-SF, the Diabetes Self-care Scale, and a sociodemographic and clinical questionnaire. Four weeks later and in order to evaluate stability over time, the self-efficacy scale was administered to a sub-sample comprised of 23 of these participants. - Results: The study showed good internal consistency (Cronbach’s alpha coefficient of 0.83) and stability over time ( r = 0.532, p = 0.009) of the DES-SF. The adjustment indices of the confirmatory factor analysis suggested the unidimensional nature of the scale. A positive association was found between self-efficacy and schooling ( r = 0.209, p = 0.020), and self-care behaviours, in particular dietary ( r = 0.221, p = 0.023) and physical exercise ( r = 0.296, p = 0.002). - Conclusion: Reliability and validity of the DES-SF was confirmed, which demonstrates its usefulness in the context of clinical practice or in research studies involving the Portuguese population.

Keywords: Diabetes mellitus Empowerment Psychometrics Self-efficacy Validation study Outcome measurement 

 

RESUMO

Introdução: O controlo da diabetes mellitus tipo 2 implica que as pessoas se envolvam ativamente em comportamentos de autocuidado, sendo a autoeficácia uma das variáveis que interfere nesse processo. Baseada no Modelo de Mudança Comportamental a Diabetes Empowerment Scale avalia a autoeficácia psicossocial em pessoas portadoras desta doença. Este estudo tem como objetivo analisar as propriedades psicométricas da versão portuguesa da Escala do Empowerment na Diabetes – versão reduzida (DES-SF), nomeadamente a sua fidelidade e validade. Método: Desenvolvemos um estudo de cariz metodológico, com uma amostra de conveniência de 123 participantes com diabetes mellitus tipo 2 que, em contexto dos cuidados de saúde primários, preencheram a DES-SF, a Escala do Autocuidado na Diabetes e um questionário sociodemográfico e clínico. Para avaliar a estabilidade temporal, a escala da autoeficácia foi novamente preenchida após 4 semanas, por uma subamostra de 23 desses participantes. Resultados: O estudo revelou boa consistência interna (alfa de Cronbach de 0.83) e estabilidade temporal (r = 0.532, p = 0.009) da DES-SF. Os índices de ajustamento, da análise fatorial confirmatória, sugerem a natureza unidimensional da escala. Verificamos uma associação positiva entre a autoeficácia e a escolaridade (r = 0.209, p = 0.020) e os comportamentos de autocuidado, nomeadamente a dieta (r = 0.221, p = 0.023) e o exercício físico (r = 0.296, p = 0.002). Conclusão: A DES-SF revelou ser uma medida fiável e válida, podendo ser utilizado em contexto da prática clínica ou em estudos de investigação, na população portuguesa.

Palavras Chave: Diabetes mellitus · Empowerment · Psicometria · Autoeficácia · Estudo de validação · Medida de resultado

 

Introduction

The prevalence of diabetes is an increasing phenomenon urging health organizations to find ways of controlling the disease, since it involves an effective loss of the quality of life of individuals and families and leads to high economic burden 1 ,2 . Current evidence shows that diabetes is considered a major public health problem at a European level 3.

This pathology is often associated with lifestyles and its control highly depends on each individual 4 - 6 . Although diabetes is a chronic disease, it is possible to control or delay its progression, provided that the person is engaged in a set of activities relating to the therapeutic management 6.

The existing programmes on self-management of diabetes aim to enhance empowerment and promote self-efficacy, enabling the identification and establishment of realistic goals, problem-solving, stress management, and the ability to mobilise adequate social resources. In this context, the philosophy of empowerment can frame the entire process of caring for the person with a chronic disease, since this concept refers to the ability to deal with individual health problems, with a positive impact on perceptions of self-efficacy and self-care behaviours 4 ,5 ,7 .

Anderson et al. 8 developed the Model of Empowerment in diabetes, underpinned by the fundamentals of Carl Rogers, which emphasise a person-centered care, and the Theory of Effectiveness of Bandura. Accountability for self-care in diabetes means that the individual should acknowledge self-values, needs, and goals and also have knowledge about the disease and its treatment 8. An educational approach in diabetes based on empowerment comprises a model that helps the person adopting behavioural changes. This process is divided into four main steps: (1) to help the person identifying problem areas; (2) to help the person exploring the emotions associated with these problems; (3) to help outline a set of goals as well as to develop a set of strategies to overcome possible obstacles which may undermine these goals; (4) to help the person be self-motivating to be able to comply with this transition plan 8.

The use of validated scales is likely to be a helpful resource allowing health professionals to effectively monitor some factors that interfere in the self-care behaviours in diabetes, such as self-efficacy. Importantly, health professionals should focus on this variable and its impact on self-care behaviours 9 ,10 . Some authors advocate the need to assess the readiness for change in people who will participate in support programmes for self-management, and the use of the Diabetes Empowerment Scale is often considered 4.

Based on the aforementioned four-step behavioural change model, the Diabetes Empowerment Scale was developed with the objective of assessing the psychosocial self-efficacy in people with diabetes 8. This construct denotes the willingness and ability of people to engage in various behavioural changes for the prevention and management of the disease 8. Although the original scale was composed of 37 items, distributed by eight dimensions (need to change behaviour; develop a care plan; overcoming obstacles; request for support; care of oneself; management of emotions; personal motivations; diabetes care-related decision-making), this instrument was reduced to 28 items divided into three subscales: (1) to manage the psychosocial aspects of diabetes; (2) to evaluate the dissatisfaction and readiness for change; (3) to establish and achieve goals in diabetes. Subsequently, the 28 items showing a higher correlation with the eight conceptual domains of the original version were used to create the reduced version of the scale, the Diabetes Empowerment Scale Short Form (DES-SF), composed of 8 items 11.

In 2014, due to the paucity of publications about the reduced version adapted to the Portuguese language and after been given authorization from the authors of the original scale, Sousa 12 proceeded with the translation and initial evaluation of the psychometric properties, the Diabetes Empowerment Scale Short Form, maintaining the initials (DES-SF) of the original version 12 ,13 .

The DES-SF has already been translated into different languages. There are three versions translated into Portuguese, two of those adapted to European Portuguese 12 ,14 and one to Portuguese from Brazil 15. A version of DES-SF adapted to rheumatic patients is also available 16.

Validation studies of DES-SF have focused on the criteria of reliability and/or exploratory factor analysis 12 ,14 ,16 . The objective of this study is to deepen the analysis of the psychometric properties of the DES-SF, contributing to its factorial validity, through confirmatory factor analysis (CFA).

 

Materials and Methods

Study Design and Sample

A methodological study of transversal type was conducted. A convenience sample of 123 people with diabetes mellitus type 2, diagnosed for more than 12 months, was invited to participate in the study. These participants attended consultations in family health units (Regional Health Administration in the North and Center of Portugal). The selected inclusion criteria were: being aged 18 years or more, having diabetes mellitus type 2 diagnosed for more than a year, and willing to participate in the study after signing a free informed consent form. Participants with cognitive changes causing impairment in the self-management of diabetes and/or unable to communicate effectively were excluded.

A sociodemographic and clinical questionnaire consisting of two sections was applied. The first section included the following variables: age, gender, marital status, and schooling. The second section included clinical variables such as the age of diagnosis, duration of the disease, glycosylated haemoglobin (HbA 1c ), and body mass index (BMI).

Instruments

The version of the DES-SF used in this study was translated and adapted by Sousa 12. This scale allows evaluating the perceived capacity (self-efficacy), to manage the psychosocial demands and challenges related to diabetes 11 - 13 . Similar to the original scale 11, this reduced version includes eight items, assessed through an ordinal scale of five possible answers (totally disagree, disagree, neither agree nor disagree, agree, and completely agree). The result of the scale is obtained by the average of the sum of all included items, in which higher values are related to higher perceptions of psychosocial self-efficacy 11. In the initial study, a coefficient of Cronbach’s alpha of 0.84 was obtained and the unidimensional nature of the scale was confirmed 12.

The diabetes self-care activities scale, translated and adapted to the Portuguese population, was also used 17. This is a self-report scale with 22 items, which allows measuring the self-care in diabetes and adherence indirectly. The scale assesses the frequency of activities of the therapeutic regimen, particularly in six dimensions: general dietary (three items), specific dietary (seven items), physical activity (two items), monitoring of blood glucose (two items), foot care (three items), medication (two items), and smoking habits (three items). The scale is parametrised in days per week, with each item score varying between 0 (the less desirable) and 7 (the most desirable). The items relating to the dimension of specific dietary are reversed. The level of adherence, by dimension, is obtained by the average of the sum of the items and the results are presented in days per week. This scale has shown good psychometric characteristics 17.

Ethical Aspects and Procedures

The study was approved by the ethics committees of the institutions involved and all the participants signed informed consent forms (CEARSC/16/2017; ARSN-SRSVR/2011). Anonymity and confidentiality of data were assured. The principles of the World Medical Association Declaration of Helsinki were upheld.

The participants were approached while waiting for consultation in family health units. Participants were free to participate in the study and were provided will full information on the objectives. The self-administered questionnaires were completed in the waiting room. In case of required assistance by the participant, the questionnaire was completed in a consultation room. To evaluate stability over time of the scale, a test-retest was conducted with a sub-sample comprised of 23 participants who agreed to fill out the questionnaire after 4 weeks.

Statistical Analysis

Data were computed using IBM SPSS version 24.0 and parametric statistics. In the first step, the data were explored through descriptive statistics. Pearson correlation measures were used to study the strength of association between continuous variables. In the comparison of means between independent samples, the Student t test was used, after checking criteria applicability. Internal consistency was determined by Cronbach’s alpha coefficient. Stability over time was assessed through test-retest using the intraclass correlation coefficient. Considering the homogeneity between studies that point to the unidimensionality of the DES-SF, the focus of this study was to use CFA to analyse the factorial validity through AMOS statistical software (IBM SPSS, version 24). The squared Mahalanobis distance was used to evaluate the presence of outliers, and the uni- and multivariate coefficients of skewness (Sk) and kurtosis (Ku) were applied to measure the normal distribution of variables. The covariance matrix was inputted, and the method of maximum likelihood estimates was used. The local fit was assessed by the factor loading and the items’ reliability. In addition to the modification indices, the fit of the model also included the inherent theoretical considerations and Kline recommendations 18. The fit indices included normalised chi-square (χ 2 /df), root mean square residual (RMR), the goodness of fit index (GFI) the comparative fit index (CFI), root mean square error of approximation (RMSEA), and the expected cross-validation index (ECVI). Statistical significance was obtained when p was below 0.05.

 

Results

The majority of the participants (53.1%) were female, with an average age of 66.8 years (standard deviation [SD] = 8.20) and an average schooling of 4.8 years (SD =2.50). 77.3% of participants were married or living in marital status. The average age of diabetes diagnosis was 54.4 years (SD = 9.77) and average time of disease 12.3 years (SD = 9.45). The BMI average was 30.3 (SD = 4.18). Available HbA 1c levels obtained from the medical records of 81 participants (63.3%) showed an average value of 7.6 mg/dL (SD = 1.96).

In self-care activities, participants showed higher levels of adherence to medication (mean [M] = 6.70; SD = 9.45) and foot care (M = 5.97; SD = 1.44), followed by dietary care: general dietary (M = 5.13; SD = 1.75) and specific dietary (M = 5.10; SD = 1.17). The physical activity showed the lowest levels of self-care behaviours (M = 3.10; SD = 2.49).

The mean for self-efficacy was 3.67 (SD = 0.59). The analysis of table 1 shows no ceiling and floor effect, which means that most of the answers have the highest scores of the DES-SF, although they are distributed across all response options. Item 5, relating to support, did not register any lowest value. It was also verified that more than 25% of the participants referred not knowing positive strategies in dealing with disease-related stress (item 4). Results also show a good perception of self-efficacy, since the participants tended to agree with the scale statements.

 

 

The exploratory analysis of DES-SF data shows a negative asymmetric distribution as well as a leptokurtic distribution for most items (table 2 ). Except for item 1, half of the participants reported self-efficacy values of 4 or 5. The scale reliability showed a Cronbach alpha coefficient of 0.83, indicative of good internal consistency. However, deleting item 1 would increase this value to 0.87. Test-retest was also calculated for 23 participants (table 3 ). A moderate correlation between the total score of the scale before and after ( r = 0.532; p = 0.009) was found. The mean values for self-efficacy were 3.79 (SD = 0.52) and 3.81 (SD = 0.48) for test-retest.

 

 

 

The intraclass correlation coefficients present a single item lower than 0.40 (item 2), confirming an acceptable stability, achieved mainly through items 6, 7, and 8. The DES-SF shows a weak yet positive and significant correlation with schooling ( r = 0.209; p = 0.02). No differences were found between DES-SF scores for men and women. Similarly, the final score of the DES-SF showed no correlation with age. In the DES-SF analysis with the dimensions of the self-care scale, a statistically significant and weak correlation with the general feeding dimension ( r = 0.221; p = 0.023) was found as well as a statistically significant association with physical activity ( r = 0.296; p = 0.002).

In order to study the factorial validity, the CFA of the DES-SF was used to test the unidimensional model proposed by the authors. The results of the tested model showed poor fit (χ 2 /df = 5.599; RMR = 0.055; GFI = 0.821; CFI = 0.790; RMSEA = 0.190; ECVI = 1.134). The model was then re-specified by the modification index suggested with the correlation of errors of items 2 with 3 and errors of items 4 with 6 (Fig. 1 ). Overall, the new model showed good goodness-of-fit indices. Only the RMSEA shows values out of range (χ 2 /df = 2.670; RMR = 0.042; GFI = 0.922; CFI = 0.931; RMSEA = 0.115; ECVI = 0.662).

 

 

Discussion

The sociodemographic and clinical characteristics overlap other studies carried out in Portugal, where participants are aged in average over 65 years, with low schooling and a considerable time of duration of the disease 2 , 14 . The BMI is indicative of a sample of overweight participants and the HbA 1C values are still within acceptable parameters. However, this marker should be interpreted with caution, since the individual therapeutic goals will depend on several factors, such as age and associated comorbidities 2 ,14 ,19 . In self-care behaviours, participants showed stronger constraints in the adherence to physical activity, which corroborates the findings of other studies 13.

The tendency of responses was placed above the mean score of the DES-SF scale, meaning that participants revealed a moderate to good perception of the psychosocial self-efficacy 15 ,16 . Contrarily to a previous study, which suggested a 3-item Likert scale 15, distribution across all items was verified. This has confirmed the need to maintain the five items, thus providing the participants with broader alternatives as to the level of agreement of the presented statements.

Moreover, health professionals should take into account the fact that over a quarter of the participants referred no knowledge of ways to deal with disease-related stress. Although educational programmes are now focused on supporting people in the psychosocial domain, it seems important to foster interventions that enhance better adjustment to diabetes 5 ,6 . It should be noted that diabetes is associated with the management of a complex therapeutic regimen 4, with a high prevalence of mood disorders such as anxiety and depression 5 ,6 .

The discriminative power of the scale was also verified since a positive correlation was found between self-efficacy and schooling, suggesting that the higher the literacy levels, the greater the perception of psychosocial skills, which corroborates available scientific evidence 16. The non-differentiation by gender, age, disease duration, and HbA 1c values overlaps the results of other scientific research 14 ,16 .

The scale showed good internal consistency, corroborated by other studies 13 ,14 . Similar to the research carried out by Chaves et al. 15 we verified that this value could improve with the elimination of item 1. However, because of its clinical utility, the item was preserved since participants scored on all possible options. In addition, this item may enable to identify potential dissatisfaction with some aspects of the treatment. A positive attitude towards treatment has been associated with good levels of adherence. Supporting users in identifying disease-related problem areas is one of the initial steps in the empowerment model 8.

The values of the intraclass correlation coefficient suggest moderate agreement, showing stability over time of the scale 20. The two measures confirm the good reliability of the instrument. Moreover, the CFA also confirms the unidimensional nature of the scale. Factor saturation was observed in the factor with significant factor loading. The adjustment indexes suggest a good fit to the theoretical model and only the RMSEA can be indicative of a problematic index of population discrepancy. Since the RMSEA value is influenced by the sample size, studies with a larger number of participants may contribute to this clarification.

Self-efficacy is considered as a good predictor of self-care behaviours 6 ,10 , 21 ,22 and in this study, significant yet low relationships were found in dietary and physical exercise. Thus, participants showing greater capacity in identifying their problems and emotions, setting goals, and drawing a functional plan controlling the obstacles (relating to the areas assessed by the scale) revealed greater adherence behaviours to a healthy eating pattern and practising of physical exercise. These areas of treatment are considered the most challenging and complex since they often demand a change in lifestyles. These results confirm the theoretical assumption of the relationship between confidence in the capacity to engage successfully in certain actions and implement these self-care activities, pointing to the convergent validity of this scale.

The sample size is the main limitation of this study. The use of larger samples is suggested, considering the applied statistical tests [18 ,23 . The model used in the CFA included 28 parameters. Considering the recommendation to have at least five observations per parameter, ideally 140 observations or participants should be included 23.

 

Conclusion

The DES-SF is an effective scale and can be used by health professionals to investigate the psychosocial self-efficacy of people with diabetes. It is also useful in monitoring the evolution of this variable before and after educational interventions, especially in programmes using the Empowerment Model approach.

The DES-SF has been used in different contexts and cultures and its adequacy in assessing psychosocial self-efficacy is confirmed. These study findings show the reliability and validity of the scale in the analysis of this construct. The measure shows good reliability and the CFA findings enhance the unidimensional nature of the scale. This study also attested the usefulness of the scale for research and clinical practice, since this instrument is sensitive to some variables such as schooling and self-care behaviours in diabetes.

 

References

1 International Diabetes Federation. Integrating diabetes evidence into practice: challenges and opportunities to bridge the gaps. Brussels: International Diabetes Federation; 2017. Available from https://www.idf.org/images/Integrating_Diabetes_IDF_1_e.pdf. (acessed April 15, 2019).         [ Links ]

2 Portugal. Ministério da Saúde. INSA. 1º Inquérito Nacional de Saúde com Exame Físico (INSEF 2015): Estado de Saúde. Lisboa: INSA; 2016. Available from http://hdl.handle.net/10400.18/4115 (acessed February 10, 2019).

3 Portugal. Ministério da Saúde. Direção-Geral da Saúde. Programa Nacional para a Diabetes 2017. Lisboa: Direção-Geral da Saúde; 2017. Available from http://www.dgs.pt. (acessed December 4, 2018).

4 Luczynski W, Glowinska-Olszewska B, Bossowski A. Empowerment in the treatment of diabetes and obesity. J Diabetes Res. 2016; 2016: 5671492.         [ Links ]

5 Marrero DG, Ard J, Delamater AM, Peragallo- Dittko V, Mayer-Davis EJ, Nwankwo R, et al. Twenty-first century behavioral medicine: a context for empowering clinicians and patients with diabetes: a consensus report. Diabetes Care. 2013 Feb; 36(2): 463–70.         [ Links ]

6 American Diabetes Association. Lifestyle management: standards of medical care in diabetes. Diabetes Care. 2018 Jan; 41 Suppl 1:S38–50.         [ Links ]

7 Tang TS, Funnell MM, Brown MB, Kurlander JE. Self-management support in “real-world” settings: an empowerment-based intervention. Patient Educ Couns. 2010 May; 79(2): 178–84.         [ Links ]

8 Anderson RM, Funnell MM, Fitzgerald JT, Marrero DG. The Diabetes Empowerment Scale: a measure of psychosocial self-efficacy. Diabetes Care. 2000 Jun; 23(6): 739–43.         [ Links ]

9 Cortez D, Santos J, Macedo M, Souza D, Reis I, Torres H. Effects of an educational program on self-care empowerment for the fulfillment of goals in diabetes. Cienc Enferm. 2018; 24(1): 23–32.         [ Links ]

10 Messina R, Rucci P, Sturt J, Mancini T, Fantini MP. Assessing self-efficacy in type 2 diabetes management: validation of the Italian version of the Diabetes Management Self-Efficacy Scale (IT-DMSES). Health Qual Life Outcomes. 2018 Apr; 16(1): 71.         [ Links ]

11 Anderson RM, Fitzgerald JT, Gruppen LD, Funnell MM, Oh MS. The Diabetes Empowerment Scale-Short Form (DES-SF). Diabetes Care. 2003 May; 26(5): 1641–2.         [ Links ]

12 Sousa M. Promover o autocuidado, apoiar a adesão e a gestão do regime terapêutico: programa de intervenção de enfermagem em pessoas com diabetes [Tese de Doutoramento]. Porto: Instituto de Ciências Biomédicas Abel Salazar. Universidade do Porto; 2014. https://hdl.handle.net/10216/80091 (acessed december 2, 2018).

13 Sousa M, Pereira F, Martins T, Rua I, Ribeiro I, Cerdeira C, et al. Impact of an educational programme in Portuguese people with diabetes. Action Res. 2017: https://doi.org/10.1177/1476750317736369.         [ Links ]

14 Aveiro M, Santiago LM, Ferreira PL, Simões JA. Fiability study of Diabetes Empowerment Scale: short version. Acta Med Port. 2015 Mar-Apr; 28(2): 177–81.         [ Links ]

15 Chaves FF, Reis IA, Pagano AS, Torres HC. Translation, cross-cultural adaptation and validation of the Diabetes Empowerment Scale: short form. Rev Saude Publica. 2017 Mar; 51(0): 16.         [ Links ]

16 Alves M, Cunha A, Lopes T, Abreu P, Vaz C. Measuring Empowerment in Patients with Chronic Disease: Psychometric Qualities of the Portuguese Rheumatic Disease Empowerment Scale. Port J Public Health. 2018; 36(2): 66–71.         [ Links ]

17 Bastos F, Severo M, Lopes C. Propriedades psicométricas da escala de autocuidado com a diabetes traduzida e adaptada. Acta Med Port. 2007 Jan-Feb; 20(1): 11–20.         [ Links ]

18 Kline RB. Principles and practice of structural equation modelling. 3rd ed. New York: Guilford; 2010.         [ Links ]

19 LeRoith D, Biessels GJ, Braithwaite SS, Casanueva FF, Draznin B, Halter JB, et al. Treatment of Diabetes in Older Adults: An Endocrine Society* Clinical Practice Guideline. J Clin Endocrinol Metab. 2019 May; 104(5): 1520–74.         [ Links ]

20 Rosner B. Fundamentals of biostatistics. 8th ed. Boston (MA): Cengage Learning; 2016.         [ Links ]

21 Fan X, Lv F. Psychosocial factors associated with self-efficacy for managing chronic disease in patients with chronic heart failure. Eur J Cardiovasc Nurs. 2016 Jun; 15(4): 255–61.         [ Links ]

22 Lee YJ, Shin SJ, Wang RH, Lin KD, Lee YL, Wang YH. Pathways of empowerment perceptions, health literacy, self-efficacy, and self-care behaviors to glycemic control in patients with type 2 diabetes mellitus. Patient Educ Couns. 2016 Feb; 99(2): 287–94.         [ Links ]

23 Marôco J. Análise de equações estruturais, fundamentos teóricos, software & aplicações. Lisboa: Editora Report Number; 2010.         [ Links ]

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License