Reliability and validity of the workplace harassment questionnaire for Korean finance and service workers
© The Author(s). 2016
Received: 3 May 2016
Accepted: 2 September 2016
Published: 9 September 2016
Harassment means systemic and repeated unethical acts. Research on workplace harassment have been conducted widely and the NAQ-R has been widely used for the researches. But this tool, however the limitations in revealing differended in sub-factors depending on the culture and in reflecting that unique characteristics of the Koren society. So, The workplace harassment questionnaire for Korean finace and service workers has been developed to assess the level of personal harassment at work. This study aims to develop a tool to assess the level of personal harassment at work and to test its validity and reliability while examining specific characteristics of workplace harassment against finance and service workers in Korea.
The framework of survey was established based on literature review, focused-group interview for the Korean finance and service workers. To verify its reliability, Cronbach’s alpha coefficient was calculated; and to verify its validity, items and factors of the tool were analyzed. The correlation matrix analysis was examined to verify the tool’s convergent validity and discriminant validity. Structural validity was verified by checking statistical significance in relation to the BDI-K.
Cronbach’s alpha coefficient of this survey was 0.93, which indicates a quite high level of reliability. To verify the appropriateness of this survey tool, its construct validity was examined through factor analysis. As a result of the factor analysis, 3 factors were extracted, explaining 56.5 % of the total variance. The loading values and communalities of the 20 items were 0.85 to 0.48 and 0.71 to 0.46. The convergent validity and discriminant validity were analyzed and rate of item discriminant validity was 100 %. Finally, for the concurrent validity, We examined the relationship between the WHI-KFSW and pschosocial stress by examining the correlation with the BDI-K. The results of chi-square test and multiple logistic analysis indicated that the correlation with the BDI-K was satatisctically significant.
Workplace harassment in actual workplaces were investigated based on interviews, and the statistical analysis contributed to systematizing the types of actual workplace harassment. By statistical method, we developed the questionare, 20 items of 3 categories.
“Harassment” means systematic and repeated unethical acts that make someone feel insignificant as the victim cannot defend him/herself  and thus is assigned an inferior status . Particularly, workplace harassment may be defined as acts that are repeated and conducted systematically throughout a certain period of time such as humiliation, aggressive behavior, social exclusion, and interference with one’s duty . Unjust treatment can also be classified as a type of workplace harassment defined in a broad sense. Such workplace harassment has implications not only for the targeted worker but also for work productivity and economic efficiency. Furthermore, it affects the quality of the entire workforce, and thus ultimately is a deciding factor when it comes to a nation’s labor culture .
Research on workplace harassment have been conducted mostly in Western Europe, Austria, New Zealand, and the U.S [5, 6]. The research of Mikkelsen and Einarsen defines victims of workplace harassment as those who have experienced two or more types of negative behaviors of workmates or supervisors every week or every day for the last 6 months . According to this criterion, 6.2 % of workers in Norway, 26.9 % of workers at public agencies in Finland, and 25 % of workers in the U.S. are suffering from workplace harassment [8–10].
Usually, workplace harassment occurs more often and lasts for a longer period than mere interpersonal conflicts . Workplace harassment includes various types of negative experiences such as criticism from workmates (including supervisors and subordinates) regarding work, defamation of character, excessive monitoring of work, unbearable workload, unreasonable criticism, and indignation with no reason.
Workplace harassment is an important issue because exposure to such situations is known to be an immediate cause of health problems. Okechukwu et al. analyzed the effect of workplace injustice (discrimination, harassment, humiliation, etc.) on health inequality , and Willness et al. reported that sexual harassment was associated with PTSD, anxiety, and depressive symptom . According to one line of research, depressive symptoms sometimes leads to suicide . It has also been reported that merely witnessing injustice at work can cause harm to health .
For this reason, the need for workplace harassment assessment and intervention has been emphasized, and tools to assess workplace harassment have also been developed. One major example is the NAQ. This tool was originally developed in the Norwegian language by Einarsen and Raknes and then translated into English. Its reliability and validity were then verified . Thereafter, the English version of the NAQ-R (the Negative Acts Questionnaire – Revised) was developed through representative research conducted by Einarsen and Hoel among U.K. workers in 2001 , and this has been widely used for research on workplace harassment in foreign countries.
However, adopting this tool directly into this study involves several problems. First, to analyze workplace harassment, the socio-cultural background of the region needs to be taken into consideration , and in-depth interviews to grasp the organizational culture also need to be conducted for the tool to be applied . This tool(NAQ-R), however, has limitations in revealing differences in sub-factors depending on the culture and in reflecting that unique characteristics of the society . Second, workplace harassment has been recognized as personal harassment by those in higher positions at work, but in consideration of current domestic conditions, the relationship between an organization and an individual also needs to be analyzed. For example, harassment may be used to put pressure upon individuals not merely for ordinary labor management but for the sake of fulfilling organizational goals such as restructuring. In recent domestic conditions, workplace harassment between a corporate organization and an individual has become an issue especially in the business area of finance and service.
In the Korean finance and service industry, the problem of job insecurity is becoming worse due to the merger and abolition of branches and workforces, an excessively performance-based system, restructuring, etc. This is not a temporary phenomenon, but it is closely related to worldwide changes in financial industry sectors. Competition, performance-based systems, and job insecurity are rampant because of deregulation, intensified pressure due to competition, introduction of new technologies, etc. Such changes in industry conditions are a major factor that brings the issue of organizational harassment against individuals to the fore.
Accordingly, this study aims to develop a tool to assess the level of personal harassment at work and to test its validity and reliability while examining specific characteristics of workplace harassment against office and finance service workers in Korea.
Participants and data collection
A survey was conducted among members of ‘The Korean Finance and Service Workers’ Union’ for 4 weeks from June 3, 2015, via an online survey. For this survey, focused-group inteviews were implemented fot 2 weeks from April 27, 2015 and total 10 persons attended the meeting for 3 times. The attendants were consist of different union branches.
The personal URL of each union branch was established and distributed through personal emails or mobile applications. Each personal device (computer or mobile) could be used only once for the survey, and duplicate replies were not valid. Total responses were 3065, but 1314 responses in total were used for the analysis with responses involving missing values excluded.
Development of the harassment assessment tool
This is a methodological study to develop a tool to analyze types of workplace harassment among office and finance service workers in Korea and to verify its reliability and validity. The general framework of the survey was established based on a literature review on workplace harassment and the consultation of experts in areas of law, social studies, etc. A questionnaire was then developed specifically for members of ‘The Korean Finance and Service Workers’ Union’, and focused-group interviews were additionally conducted to examine specific harassment experiences. Interviewees included 11 individuals working at 11 subordinate unions (5 insurance agencies, 3 lending and deposit agencies, and 3 security agencies), and the interviews were conducted in a way that allowed participants to make free open-ended responses to the previously prepared questions. Based on the results, harassment was classified into 6 categories: humiliating words and deeds, harassment related to relationships, harassment related to work, supervision and control, physical harassment, and sexual harassment. A preliminary survey with the finalized items was conducted among union executive members and ordinary union members. After removal of improper terms and revision of item composition, 20 items on workplace harassment types were selected (Appendix).
The reliability and validity of the developed tool was analyzed by means of the trial version of SPSS20. To verify its reliability, Cronbach’s alpha coefficient was calculated; and to verify its validity, items and factors of the tool were analyzed. KMO and Bartlett’s test of sphericity were used to verify the appropriateness of the factor analysis, and the Varimax rotation method was utilized in the factor extraction. The correlation matrix analysis was examined to verify the tool’s convergent validity and discriminant validity.
Structural validity was verified by checking statistical significance in relation to the BDI-K, a measure of depressive symptoms.
Depressive symptoms are commonly observed in clinical areas of mental health. The criterion-related validity of the questionnaire was analyzed by means of the BDI-K(The Validation Study of Beck Depression Scale 2 in Korean Version), which is commonly used with regard to depressive symptoms. The BDI-K is self-recording type questionnaire but showed a reliable level of sensitivity  and widely used not only for selection and asessment of patients with depression but also for screening tests for epidemiological surveys and for common populations . According to existing research findings, harassment experiences have harmful effects on the workers’ mental health  and such situations have led even to suicide . Thus, this study also aimed to uncover the relationship between the questionnaire and psychosocial stress by examining the correlation with the BDI-K.
The deficnition of harassment victims is contracted the suggestion of Mikkelsen and Einarsen. The NAQ-R The NAQ-R defines harassment victims as those who have experienced at least one type of workplace harassment at least every week during the last 6 months . However, Mikkelsen and Einarsen suggestied a stricter condition:at least two two types of workplace harassment every week or every day .
General characteristics of study participants
General characteristics of study population
20 ~ 29
30 ~ 39
40 ~ 49
Last School Graduation
Office work/Supporting task
Section Chief/Deputy section Chief
0 ~ 4
5 ~ 9
Working time per week(hours)
41 ~ 50
Working day during holidays(days)
Reliability of workplace harassment questionnaire for Korean office and finance service workers
Rotated Factor Pattern Matrix and Communality of the Questionnaire
Proportion of variances (%)
KMO = 0.95; Bartlett test of sphericity = 12,785.687 (p < .0001)
Total Cronbach’s α = 0.93
Factor analysis of workplace harassment questionnaire for Korean office and finance service workers
The questionnaire used in this study adopts the format of the NAQ-R, but the content was prepared based on actual cases of office and finance service workers. Thus, this survey tool was used for the first time in this study. To verify the appropriateness of this survey tool, its construct validity was examined through factor analysis.
The KMO test was conducted to verify sample appropriateness, and the value was 0.95, which is higher than 0.50, indicating that the factor analysis is valid. As a result of Bartlett’s test of sphericity, the Chi-square statistic was 12785.687 (p < 0.001) and the correlation matrix was not the identity matrix. Thus, the factor analysis was valid. Factor extraction was conducted so that eigenvalues could be at least 1.0 and the accumulated percentage of the general variance could be at least 50 %. The factor loading values and communalities of each factor item were all 0.4 or higher. As a result of the factor analysis, 3 factors were extracted, explaining 56.5 % of the total variance. The loading values and communalities of the 20 items were 0.85 to 0.48 and 0.71 to 0.46, respectively, which were all 0.4 or higher and thus met the standards .
Regarding individual factors, factor 1 included 13 items in total which are about “excessive workload(Q11, Q12), unjust work-related threatening(Q04, Q14) and monitoring(Q15, Q16), injustice in work assignment(Q7, Q08 Q13), work-related defamation of character(Q01, Q03 Q10), exclusion from the work process(Q09),” etc. These items may be classified as “work-related harassment.” Factor 2 includes items about “Appearance-related statements(Q02) and monitoring(Q17), sexual humiliation(Q20), and negative rumors(Q06)” and these can be classified as “defamation of character.” Factor 3 includes items about “corporal punishment(Q19), physical threat(Q18) and exclusion from fellows(Q05),” and these were classified as “physical harassment.” (Appendix).
The explanation power of “work-related harassment (factor 1)” was 44.53 %, and that of “defamation of character (factor 2)” and “physical harassment (factor 3)” was 7.01 and 5.01 % (Table 2.)
Convergent validity and descriminent validity
Multi-trait/multi-item matrix of the questionnaire (Correlation matrix corrected for overlap)
This study also aimed to uncover the relationship between the questionnaire and psychosocial stress by examining the correlation with the BDI-K.
The questionnaire included 20 questions in total, each of which was differentiated depending on the frequency of harassment experiences during the last year (Appendix). Each answer to these questions was given points from 1 to 5, and the BDI-K was divided into 4 steps(minimal, mild, moderate, severe) to examine the statistical correlation through the Chi-square test. As a result, 19 items out of 20 (the 19th item being the exception, p = 0.3320) showed statistically significant correlations with depressive symptoms, whose level increased in proportion to the frequency of harassment experiences (p < 0.0001).
The association of distribution of Strength of harassment group by the questionnaire and depressive symptom group
The questionnaire distribution (p < 0.001)
0 ~ 25 %
350 (26.64 %)
260 (74.29 %)
44 (12.57 %)
29 (8.29 %)
17 (4.86 %)
25 ~ 50 %
321 (24.42 %)
203 (63.24 %)
62 (19.31 %)
45 (14.02 %)
11 (3.43 %)
50 ~ 75 %
348 (26.84 %)
146 (41.95 %)
80 (22.99 %)
95 (27.3 %)
27 (7.76 %)
75 ~ 100 %
295 (22.45 %)
80 (27.12 %)
63 (21.36 %)
93 (31.53 %)
59 (20 %)
1314 (100.00 %)
Strength of Harassment (p < 0.001)
1116 (84.94 %)
642 (57.53 %)
208 (18.64 %)
202 (18.1 %)
64 (5.73 %)
198 (15.06 %)
47 (23.74 %)
41 (20.71 %)
60 (30.30 %)
50 (25.25 %)
1314 (100.00 %)
The result of multiple logistic analysis of the association the questionnaire and the depressive symptoms
95 % CI
The questionnaire distribution
0 ~ 25 %
25 ~ 50 %
50 ~ 75 %
75 ~ 100 %
Strength of harassment
When it comes to research on the issue of workplace harassment, it is vital to consider both sociocultural backgrounds and peculiarities of assigned duties. Although the NAQ-R is certainly a verified harassment assessment tool that has long been used, it has limitations in reflecting the unique situation in Korea. Additionally, survey tools that have been developed in other lines of research are also limited to certain occupational groups such as nurses, and it may be inappropriate to apply them to office and finance service workers in Korea (members of the Korean Finance and Service Workers’ Union’). Since there was no specialized tool that can be used, an assessment tool specialized for office and finance service workers in Korea was developed in this study.
NAQ-R survey the frequency of harassment and bullying without the definition of workplace harassment and being victimized [7, 26]. That is toal 20 items, which is composed 3 sub categories (personal related bullying(12 items), intimidation related bullying(5 items), work-related bullying(5 items).
However, this assessment tool had 6 subcategories in the initial stage of design: “humiliating words and deeds,” “relationship-related harassment,” “work-related harassment,” “supervision and control,” “physical harassment,” and “sexual harassment.” This tool is more specific questionnaire for Korean finance and service workers, because the each categories are based on the in-depth interview form their experiences.
These 6 subgroups were reduced to 3 after factor analysis: “work-related harassment, “defamation of character,” and “physical harassment.” Twenty items satisfied the standards for loading and communality, explaining 56.6 % of the total variance. Cronbach’s alpha coefficient was 0.93, which indicates a high level of internal consistency.
The correlation between workplace harassment and individual depressive symptoms was examined by means of the BDI-K. As a result, it was shown that those who were exposed to two or more types of workplace harassment every week or every day showed an odds ratio as high as 4.02(CI = 3.01 ~ 5.37), which was higher than those who were not. In addition, scores from 1 to 5 points were given depending on the number of experiences, and the total score was converted to quartile values to examine the relationship with depressive symptoms: In the case of the 75th percentile or higher, the odds ratio was as high as 7.16 (CI = 5.16 ~ 9.94).
The questionnaire included some items that might lower the content consistency: 2 items of “physical harassment” (No. 18 and 19), 1 item of “sexual harassment” (No. 20), and 1 item of “work-related excessive demand” (No. 8). These items also represent types of workplace harassment. Although these too are serious problems at work, their frequency was not relatively high, and these were problems of personal character rather than duties at work. Thus, they might have lowered the consistency of the survey.
However, the rest of the items properly reflected kinds of workplace harassment that office and finance service workers in Korea might suffer during the work process, and these items take into consideration not only personal harassment but also harassment problems related to organizational cultures such as excessive workload upon changes in working conditions, understaffing, and excessive monitoring.
The relationship with the BDI-K was also clearly shown, verifying the statistical relationship between the level of exposure to workplace harassment and depressive symptoms. Among the relationships between the 3 factors and the BDI-K, “work-related harassment,” which was classified as factor 1, showed an especially significant positive correlation with depressive symptoms.
This survey tool, however, has several limitations: the survey was not conducted with other related surveys that could verify its external validity as part of research on the actual conditions of workplace harassment; even though the 6 categories in the questionnaire used different numbers of items along with the interviews, no corresponding weight was applied; and the participant ratio was not adjusted depending on the region. Additionally, the causal relationship between workplace harassment and depressive symptoms was not clarified because this was a cross-sectional study.
Nonetheless, types of workplace harassment in actual workplaces were investigated based on interviews, and the statistical analysis contributed to systematizing the types of actual workplace harassment. In addition, the survey was conducted not in the context of one certain workplace in Korea but over a more general set of office and finance service workers. With all these achievements, this study is of great significance. Furtheremore, there was a statistical relationship between harassment experience and depressive symptoms, and the existing study finding that experience of violence at work is related to depressive symptom was reconfirmed .
This study develop a tool to assess the workplace harassment, consised with 20 items. By the statistical method, the validity and reliability obtatined and the correlation with depressive symptom was significant. Based on the above study results, additional research needs to be conducted to determine appropriate cut-off points for proper selection of workers who need intervention. In addition, future research also needs to expand its scope to cover general office workers.
This study was supported by ’Korean Finance and Service Workers’ Union’ and a part of the ‘Research for the workplace harassment in Korean Finance and service workers’. Specially, Sanghyuk Im (Department of Occupational & Environmental Medicine, Wonjin Green Hospital), Myoungsook Choi(Sarangbang Group for Human Rights) and Jonghee Lee(Korean Lawyers for Public Interest and Human Rights) contributed to develop the questionnaire. Members of Korean Finance and Service Workers’ Union were engaged in survey.
This study was funded by ’Korean Finance and Service Workers’ Union’.
Availability of data and materials
Authors do not wish to share their data.
The data that support the findings of this study are available from ‘Korean Finance and Service Workers’ Union’ and a part of the ‘Research for the workplace harassment in Korean Finance and service workers’. The data were used under license for the current study from ‘Korean Finance and Service Workers’ Union’ and so are not publicly available.
MJL drafted this manuscript. MJL and HJK developed questionnaire and revised it. MJL, DHL and HJK conducted the survey and contributed to the data collection. MJL analyzed the data. SYL revised this manuscript critically. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The Institutional Review Board of Wonjin Institution for Occupational and Environmental Health(WIOEH) approved the IRB review exemption of this study(IRB No. 20160104-02).
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Einarsen S. The nature and causes of bullying at work. Int J Manpower. 1999;20(1/2):16–27.View ArticleGoogle Scholar
- Leymann H. Mobbing and psychological terror at workplaces. Violence Vict. 1990;5(2):119–26.PubMedGoogle Scholar
- Hoel H, Einarsen S, Keashly L, Zapf D, Cooper C. Bullying at work. The way forward. In: Bullying and emotional abuse in the workplace: International perspectives in research and practice. 2003. p. 412–6.Google Scholar
- Agervold M, Mikkelsen EG. Relationships between bullying, psychosocial work environment and individual stress reactions. Work Stress. 2004;18(4):336–51.View ArticleGoogle Scholar
- Farrell GA. From tall poppies to squashed weeds*: why don’t nurses pull together more? J Adv Nurs. 2001;35(1):26–33.View ArticlePubMedGoogle Scholar
- Quine L. Workplace bullying in NHS community trust: staff questionnaire survey. BMJ. 1999;318(7178):228–32.View ArticlePubMedPubMed CentralGoogle Scholar
- Mikkelsen EG, Einarsen S. Bullying in Danish work-life: prevalence and health correlates. Eur J Work Organ Psychol. 2001;10(4):393–413.View ArticleGoogle Scholar
- Nielsen MB, Skogstad A, Matthiesen SB, Glasø L, Aasland MS, Notelaers G, Einarsen S. Prevalence of workplace bullying in Norway: comparisons across time and estimation methods. Eur J Work Organ Psychol. 2009;18(1):81–101.View ArticleGoogle Scholar
- Agervold M. Bullying at work: a discussion of definitions and prevalence, based on an empirical study. Scand J Psychol. 2007;48(2):161–72.View ArticlePubMedGoogle Scholar
- Lutgen‐Sandvik P, Tracy SJ, Alberts JK. Burned by bullying in the American workplace: prevalence, perception, degree and impact*. J Manag Stud. 2007;44(6):837–62.View ArticleGoogle Scholar
- Leymann H. The content and development of mobbing at work. Eur J Work Organ Psychol. 1996;5(2):165–84.View ArticleGoogle Scholar
- Okechukwu CA, Souza K, Davis KD, de Castro AB. Discrimination, harassment, abuse, and bullying in the workplace: contribution of workplace injustice to occupational health disparities. Am J Ind Med. 2014;57(5):573–86.View ArticlePubMedGoogle Scholar
- Willness CR, Steel P, Lee K. A meta‐analysis of the antecedents and consequences of workplace sexual harassment. Pers Psychol. 2007;60(1):127–62.View ArticleGoogle Scholar
- Barling J. The prediction, experience, and consequences of workplace violence. In: Violence on the job: Identifying risks and developing solutions, vol. 2949. 1996.Google Scholar
- Saastamoinen P, Laaksonen M, Leino‐Arjas P, Lahelma E. Psychosocial risk factors of pain among employees. Eur J Pain. 2009;13(1):102–8.View ArticlePubMedGoogle Scholar
- Einarsen S, Raknes BI. Harassment in the workplace and the victimization of men. Violence Vict. 1997;12(3):247–63.PubMedGoogle Scholar
- Einarsen S. Hoel,(2001). The negative acts questionnaire, development, validation, and revision of a measure of bullying at work. In: 10th European Congress on Work and Organizational Psychology, Prague: 2001. 2001.Google Scholar
- Nam W, Kim JW, Kim YK, Koo JW, Park CY. The reliability and validity of the negative acts questionnaire-revised (NAQ-R) for nurses for the assessment of workplace bullying. Korean J Occup Environ Med. 2010;22(2):129–39.Google Scholar
- Lee Y, Lee M. Development and Validity of Workplace Bullying in Nursing-Type Inventory (WPBN-TI). J Korean Acad Nurs. 2014;44(2):209–18.Google Scholar
- Abe K, Henly SJ. Bullying (ijime) among Japanese hospital nurses: modeling responses to the revised Negative Acts Questionnaire. Nurs Res. 2010;59(2):110–8.View ArticlePubMedGoogle Scholar
- Beck AT, Steer R. Manual for the revised Beck depression inventory. San Antonio: Psychological Corporation; 1987.Google Scholar
- Archer RP, Maruish M, Imhof EA, Piotrowski C. Psychological test usage with adolescent clients: 1990 survey findings. Prof Psychol Res Pract. 1991;22(3):247.View ArticleGoogle Scholar
- Byun CB, Jung-Choi K, Cho Y, Paek D. Depressive symptoms of workplace violence exposed subjects in Korea. Korean J Occup Environ Med. 2009;21(4):314–23.Google Scholar
- Han S, Lee S. SPSS nursing and health statistical analysis. Seoul: Hyunmoon; 2006.Google Scholar
- Ware Jr J, Snow K, Kosinski M, Gandek B. SF-36 Health Survey Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center Hospitals. In.: Inc; 1993.Google Scholar
- Salin D. Prevalence and forms of bullying among business professionals: A comparison of two different strategies for measuring bullying. Eur J Work Organ Psychol. 2001;10(4):425–41.View ArticleGoogle Scholar