Creating a healthy workplace to prevent risks and support the health of employees is a legal obligation in many European countries (based on the Framework Directive 89/391/EEC [1]) and therefore one of the major challenges in today’s organizational psychology. Next to preventing “common” risks such as having physical demanding work or working with harmful materials, other aspects of risks such as social, mental or psychological factors are becoming increasingly important for risk assessment at the workplace [2]. According to the “2014- 2015 Healthy Workplaces Campaign” (a campaign by the European Agency for Safety and Health at Work; EU-OSHA) psychosocial risks at the workplace (in the sense of critical working conditions or mental workload) must be assessed and managed for every workplace. In the European Framework for Psychosocial Risk Management (PRIMA-EF), an approach to conduct psychosocial risk assessment is presented that focuses on the processes to achieve the best possible outcome to support healthy workplaces [3]. The process of PRIMA-EF is similar to the requirements stated in the upcoming norm ISO 45001 [4], where healthy workplaces should be approached within the concept of the Plan-Do-Check-Act (PDCA) as a part of occupational health and safety management system. In these frameworks, the assessment of psychosocial risks as well as the development of specific interventions to address these risks is equally important.

The primary goal of creating healthy workplaces is to organize and change working conditions in such ways that health-supportive aspects of the job are increased and harmful aspects are decreased [5]. Changing working conditions (“organizational risk factors”) can achieve a much broader impact on employee health than solely focusing on reducing individual risk factors [6,7].

To reach the goal of healthy workplaces, high-quality instruments to assess psychosocial risks at the workplace must be developed in line with international standards (e.g., the ISO 10075-1 or the ISO 45001; [4,8]). In addition, psychosocial risks should be assessed as widely as possible to support deriving tailored interventions for organizations. In this sense, instruments that assess psychosocial risks at the workplace must fulfill the requirement to serve as a base for the development of specific interventions (in the line of the ISO 10075-2 [9]) but also national legal obligations. The instrument OrgFit has been especially developed to serve this purpose.

In a comprehensive risk assessment, all relevant risks at the workplace-and especially risks that might be linked to negative health outcomes-have to be assessed [10,11]. As workload, or specifically mental workload “has intuitive appeal, but remains surprisingly difficult to define” [12] it is important to use clear definitions. In the norm ISO 10075-1 [8], mental workload is used as an umbrella term that encompasses mental stress and mental strain. The view of mental workload as both characteristics of the workplace and effects on the individual is in line with past research in the field of ergonomics [12,13].

Mental stress is defined as follows [8]: Mental stress is “the total of all assessable influences impinging upon a human being from external sources and affecting it mentally”. Mental strain is the outcome of mental stress, more specifically, mental strain is “the immediate effect of mental stress within the individual (not the long-term effect) depending on his/ her individual habitual and actual preconditions including individual coping styles” [8]. Short-term effects of strain compass mental fatigue, monotony, satiation and stress sensations [14]. Long-term effects (e.g., burnout) result from repeated exposure to strain [15]. According to the ISO 10075-1 [8], stress is a neutral term without a negative or positive connotation.

Assessing stress as a total value is not the preferred goal in risk assessment as it is important to detect the different risk sources and then it is possible to develop the fields for improvement [16,17]. Hence different facets of stress for risk assessment have been proposed [18-20]. These authors highlight the importance of assessing different facets of stress simultaneously for a better understanding of the workplace and for a better development of interventions.

Facets which are used for the description of stress are often based on the ISO 10075-1 [8] and other concepts [10]. An instrument that is suitable for the usage of assessing stress should include these dimensions but should also comply to the requirements of the national laws for health and safety (as stated in the implementations of the Framework Directive 89/391/EEC [1]). These legal requirements are often stated in the recommendations (e.g., in Austria, Germany, or Switzerland [21-23]).

Development of the instrument OrgFit

The OrgFit was constructed to capture all relevant areas of stress according to the ISO 10075-1 [8] and the mentioned legal requirements. As in one recommendation [24] a finer partitioning of the dimensions was defined, these requirements were also included in the construction of the instrument. The breakdown of the dimensions was done with subtests.

A large pool of items was developed together with experts in the field of stress assessment (experts in the field of safety and health) to measure different areas of stress. In two unpublished studies, the reliability and validity of the items were tested and then items were removed or adapted if needed. In its current form, the OrgFit comprises 54 items, which can be categorized in 24 subtests and four higher-order dimensions. Each of the four dimensions consists between six and 17 items, each subtest has two or three items. Following dimensions are measured: a) Work activities and tasks, b) Organizational climate, c) Work environment, and d) Work flow and work organization. The subtests are presented in appendix A with examples of items. Based on the concepts of the ISO 10075 [8] and the recommendations [24] the subtests were assigned to the respective dimensions.

Work activities and tasks: The dimension work activities and tasks includes task requirements such as the cognitive or emotional demands of tasks (e.g., increased attention and concentration or working with clients/customers).

Organizational climate: The dimension organizational climate describes aspects of the organization or social contacts such as information and communication structures, participation possibilities or cooperation with leaders and co-workers.

Work environment: This dimension refers to all physical, biological and chemical conditions at work. This includes visual, acoustic and climate conditions as well as having enough work space and work equipment.

Work flow and work organization: This dimension measures aspects of the work flow and organizational processes such as the order of the work steps, interferences and interruptions as well as quantitative workload.

This paper presents the analysis of the psychometric properties of the OrgFit in two studies. The first study aimed at testing the factorial structure of the OrgFit with an exploratory factor analysis. The aim of the second study was to support the results found in study I by conducting a second factor analysis and to analyze the construct validity of the OrgFit. Construct validity was tested with the Recovery-Stress- Questionnaire for Work (RESTQ-Work), which measures work-related strain (convergent validity) and resources (discriminant validity) that have been restored in recovery processes [25,26].

Study I participants and procedure

Austrian workers were invited to participate in an online study in cooperation with a well-known German market research company by sending out e-mails. The participants had to fulfill the requirement of currently having a job; otherwise they were excluded at the beginning of the survey. In order to receive a representative sample for the Austrian labour market quotes were set for gender (female: 50%, male: 50%) and age (≤40 years: 50%, >40 years: 50%). All-in-all, 540 persons took part in the online study. After conducting a plausibility check, data sets with implausible response patterns were removed and a total sample of 514 could be achieved.

The participants in the study consisted of 50% men and 50% women. 18.5% were 30 years or younger, 31.3% were between 31 and 40 years, 31.7% were between 41 and 50 years and 18.5% were 51 years or older. 2.5% completed compulsory school, 48.8% completed an apprenticeship, 30.5% of the participants completed high school, and 18.1% had a university degree. 29.2% of the participants stated to be in a leadership position, 9.6% had their own company and the rest of the participants did not inherit a leadership position. The most frequently mentioned industrial sectors were general services (15.8%), commerce (15.5%), public administration (8.6%) and health care (8.4%).

Measures: The OrgFit has 54 items and every item can be assigned to each of the four dimensions (work activities and tasks, organizational climate, work environment, and work flow and organization) and the 24 underlying subtests. The 54 items are written as statements and refer to the last four weeks (“How many times have you experienced the following aspects in the last 4 weeks?”). The 6-point Likert scale ranges from 0 (never) till 6 (always). Example items are shown in appendix A.

Results: To prove the factorial structure of the OrgFit, a factor analysis using principal axis analysis with oblimin rotation was performed on basis of the subtests. The analysis revealed four factors with eigenvalues greater than 1. The four factors identified 58.7% of the total variance (KMO=0.91, χ2 (276)=5772.5, p<0.001). The categories physical tasks, qualification and competencies, internal interfaces, and breaks where assigned to another dimension instead of their recommended dimensions (Table 1).

Table 1: Factor loadings for the subtests of the OrgFit (study I).

The first factor consisted of three subtests of the dimension work activities and tasks. The second factor consisted of six of the seven subtests of the dimension organizational climate and the subtest qualification and competencies. The third factor consisted of all six subtests of the dimension work environment, the subtest physical tasks and the subtest breaks. The fourth factor consisted of five subtests of the dimension work flow and work organization as well as of the subtest internal interfaces. The subtest objectives and responsibilities had its highest factor loading on the second factor and the next factor loading on the fourth factor.

According to the results of the factor analysis, the subtests physical tasks, internal interfaces and breaks were assigned to their respective factors instead of their theoretical assumed dimensions. These new dimensions were recalculated and internal consistencies assessed by Cronbach’s Alpha of the newly constructed dimensions are depicted in Table 2. The four dimensions showed high Cronbach’s Alpha ranging from 0.79 to 0.93. The internal consistency of the whole questionnaire was 94.

Table 2: Cronbach Alpha (α) for the dimensions and subtests of the OrgFit for study I and study II.

Study II participants and procedure

The procedure was the same as in study I. Quotes were set for gender (female: 50%, male: 50%) and age (≤40 years: 50%, >40 years: 50%). After conducting a plausibility check, data sets with implausible response patterns were removed and a total sample of 1200 could be achieved.

The sample included 47.9% men and 52.1% women and the average age was 41 years (SD=10.6). 3.7% completed compulsory schooling, 41.5% completed an apprenticeship, and 29.3% of the participants completed high school and 25.5% had a university degree. 29.2% of the participants had a leadership position, 9.6% had their own company and the rest of the participants did not have a leadership position. The most frequently mentioned industrial sectors were general services (19.6%), commerce (12.2%), public administration (11.7%) and health care (9.6%).

Measures: The RESTQ-Work was specifically developed to assess recovery/resources and strain in the sense of the ISO 10075-1 [8]. The concept behind the RESTQ-Work uses the term “stress” to address the state of the organism, which results due to threats. Therefore, the RESTQ-Work measures strain according to definition of the ISO 10075. To avoid confusion, from now on the term “strain” is used when referring to the RESTQ-Work. The RESTQ-Work-55 [27] addresses different aspects of strain and recovery/resources in the past seven days/ nights with 55 items. The items can be categorized in seven different dimensions: Social emotional stress, performance (-related) stress, overall recovery, loss of meaning/burnout, leisure/breaks, psychosocial recovery, and work-related recovery. One example item for the dimension loss of meaning/burnout is “In the past 7 days/nights… I felt frustrated through my work” and for the dimension leisure/breaks “In the past 7 days/nights… I was able to relax during my breaks”. The items can be answered on a 7-point-Likert scale ranging from 0 (never) till 6 (always).

Similar to study I the factorial structure of the OrgFit was analyzed by conducting a factor analysis using principal axis analysis with oblimin rotation on basis of the subtests. The subtests physical tasks, qualification and competencies, internal interfaces, and breaks were rearranged according to the results obtained in study I. The analysis revealed four factors. The four factors identified 59.4% of the total variance (KMO=0.92, χ2 (276)=11739.23, p<0.001). The results showed that the factorial structure of study I was supported. The subtests physical tasks and breaks were assigned to the third factor, the subtest qualification and competencies was assigned to the second factor, and the subtest internal interfaces was assigned to the fourth factor (Table 3).

Table 3: Factor loadings for the subtests of the OrgFit (study II).

Reliabilities for the subtests and for the dimensions are based on the internal consistency assessed by Cronbach’s Alpha (Table 2). The Cronbach’s Alpha of the four dimensions ranged from 0.80 to 0.93. The internal consistency of the whole questionnaire was 0.94.

The construct validity was examined by analyzing the relationships of the dimensions of the OrgFit with the dimensions of the RESTQ-Work (Table 4). The correlations with the strain-related dimensions (social-emotional stress, performance (-related) stress, loss of meaning/ burnout) were in a range between 0.39 and 0.61. The correlations with the recovery/resources-related dimensions (overall recovery, leisure/breaks, psychosocial recovery, work-related recovery) varied between -0.25 and -0.72 for the dimensions organizational climate, work environment and work flow and organization. In contrast, the correlation coefficients between the dimension work activities and tasks (OrgFit) and the recovery-resources-related dimensions were lower (between -0.01 and -0.35).

Table 4: Correlations between the dimensions of the OrgFit and the RESTQ-Work (study II).

The OrgFit was developed as an instrument to assess stress at the workplace according to the requirements for risk assessment at the workplace. The results of the assessment can serve as a base for developing specific interventions on the organizational level [6,7,16].

The factorial structure, reliability and construct validity of the OrgFit were tested in two studies. The studies revealed that the factorial structure is similar to the assumed theoretical structure. The OrgFit shows four separate factors and the underlying subtests can be clearly located on these four factors. Twenty of the 24 subtests could be located on their theoretical assigned factors. The other four subtests (physical tasks, qualification and competencies, internal interfaces, and breaks) show high factor loadings on other factors. Therefore, these subtests were assigned to the dimensions as proposed in the factorial solution.

The results of the reliability analysis for the dimensions show high coefficients. The requirement of having a minimum alpha of 0.70 (for details see ISO 10075-3 [28]) for organizational level analyses was met. In study II, the internal consistencies of the dimension were even greater than 0.80 indicating very good reliability. Most of the underlying subtests achieved good internal consistencies greater than 0.70, but several subtests showed low values in both studies especially the subtests qualification and competencies, cooperation with leaders, visual conditions, internal interfaces, and objectives and responsibilities. Especially subtests that included reverse-scored items (e.g., qualification and competencies, cooperation with leaders) showed low Cronbach’s Alpha. Mixing positively and negatively phrased items can be one explanation when finding a very low Alpha. Further, the low internal consistencies of the subtests might be a result of the small number of items, as each subtest consists of only two or three items. With a low number of items, a high reliability is difficult to achieve. The results of the reliability analysis also have to be interpreted considering the background of the instrument conception. The instrument was developed to assess mental stress on an organizational level to consequently develop organizational interventions, and was not developed for an individual diagnosis of single workers. Therefore, we emphasize the point that interpreting the subtests can only be done for aggregated data on a group or organizational level where-on the other hand-these aggregated data can help to reduce response bias effects [29]. Using these subtests for group and organizational assessment and not for individual assessment, the reliability coefficients can be regarded as satisfactory.

Testing the construct validity, correlations with the RESTQ-Work- 55 were conducted that assess work-related recovery/resources and strain. The results showed moderate to high correlations between the dimensions in the OrgFit and the strain-related dimensions in the RESTQ-Work-55. These findings indicate that the dimensions in the OrgFit are capable of assessing stress that might lead to negative strain and therefore are an indicator for convergent validity.

The dimension organizational climate (OrgFit) is strongly related to the dimensions measuring recovery/resources (RESTQ-Work-55). Especially organizational climate shows a strong negative correlation with the dimension work-related recovery in the RESTQ-Work. Both dimensions depict aspects of participation possibilities at the workplace; therefore, this strong relationship was to be expected. Lower correlations with the recovery/resources-related dimensions result for the other three dimensions and especially for work activities and tasks. Thus, the dimensions in the OrgFit are not necessarily related to the resources at the workplace, indicating discriminant validity.

The OrgFit is suited for both scientific purposes and in the practical field. The OrgFit can be used during risk assessment in combination with the RESTQ-Work. Therefore, existing stressors at the workplace (OrgFit) can be measured together with possible negative strain at the workplace (RESTQ-Work). Furthermore, based on the findings obtained from the OrgFit organizations can develop specific interventions together witch health and safety specialists such as safety experts, work physicians and work psychologists. The results from the dimensions and subtests in the OrgFit can be primary used to develop process- and structure-oriented interventions. Therefore, the OrgFit is used to assess stress on the group or the organizational level and is never thought for individual diagnosis of the workers, which is also in line with the requirements of risk assessment.

Additionally it is important to emphasize, that questionnaire results should not be used alone. The typical models for assessing psychosocial risks at the workplace [7,30,31] include several levels of assessment like observation, interviews, questionnaires, document analysis etc. [32] and several steps in the process. A successful risk assessment process also needs the participation of all stakeholders. This can be done e.g., by using the results of the survey for an interactive workshop together with the employees of a special group to derive interventions. In addition, the practical results show the relevance of the different subtests for an in-depth implementation of interventions for a sustainable reduction of risks for mental health.

The authors would like to thank Christina Bramberger for her support in developing the first version of the OrgFit and also Zsofia Berkes, Michaela Höfer, Cornelia Schmon, Martin Wessel and other colleagues for the later versions of the instrument. The results obtained in study II were part of the study ”Working world in Austria” [33], conducted by the Department of Psychology from the University of Graz in cooperation with the psychological consulting company research-team GmbH. Study II was supported by the Styrian Medical Chamber, the Christian Trades Unionists, and Schuhfried GmbH. This publication was printed with the financial support of the University of Graz.