Advertisement

Translation and Psychometric Testing of the Norwegian Version of the “Patients’ Perspectives of Surgical Safety Questionnaire”

Open AccessPublished:December 06, 2022DOI:https://doi.org/10.1016/j.jopan.2022.08.013

      Abstract

      Purpose

      To translate the Patients’ Perspectives of Surgical Safety (PPSS) questionnaire into Norwegian and to test it for structural validity and internal consistency.

      Design

      This is a methodological study.

      Methods

      The original 20-item PPSS questionnaire was translated into Norwegian using a model of translation-back translation. We assessed content validity via a pretest with 20 surgical patients. A sample of 218 surgical patients in a university hospital in Norway completed the PPSS questionnaire. Psychometric analysis included item characteristics, and structural validity was evaluated by an exploratory factor analysis. Internal consistency was calculated using Cronbach's alpha.

      Findings

      We successfully translated and adapted the Norwegian PPSS questionnaire. Completion rate was 74%. Missing values were less than 5% and all 20 items had a high skewness (≥15 %) ranging from 52.8% to 95.9%. The exploratory factor analysis yielded two significant factors that explained 45.15% of variance. The Cronbach's alpha for Factor 1 “Team interaction safety” was 0.88 and for Factor 2 “Patient's ID safety”, 0.82. Overall, most patients reported a high sense of surgical safety.

      Conclusions

      The first Norwegian version of the PPSS measuring surgical patients’ perception shows promising psychometric properties regarding structural validity and internal consistency. However, future research on PPSS should provide an examination of construct validity, validation and testing in other populations of surgical patients. To improve safety of the surgical trajectory, it is necessary to pay more attention to patients’ perceptions of surgical safety.

      Keywords

      The World Health Organization (WHO) and the Joint Commission acknowledge patient involvement in patient safety.

      World Health Organization. Regional Office for Europe. Exploring patient participation in reducing health-care-related safety risks. World Health Organization; ‎2013. Assessed March 1, 2022. https://apps.who.int/iris/handle/10665/326442

      ,

      The Joint Commission. Busting the myths about engaging patients and families in patient safety. 2016. Assessed March 1, 2022. https://www.jointcommission.org/assets/1/18/PFAC_patient_family_and_safety_white_paper.pdf

      Moreover, the WHO Second Global Patient Safety Challenge, “Safe Surgery Saves Lives” recognizes the importance of considering the patient undergoing surgery as a member of the team.

      WHO Patient Safety & World Health Organization. WHO guidelines for safe surgery 2009: safe surgery saves lives. World Health Organization; 2009. Assessed March 1, 2022. https://apps.who.int/iris/handle/10665/44185

      While surgical procedures are intended to save lives, unsafe surgical procedures can cause substantial harm. Globally, more than every 10th patient experiences error during their hospital stay.
      WHO
      Patient Safety.
      In US health care, errors are the third leading cause of death, and numbers may exceed 250.000 per year.
      • Makary MA
      • Sexton JB
      • Freischlag JA
      • et al.
      Patient safety in surgery.
      In Europe, 23% of hospitalized patients are directly affected by errors, and 18% of patients experienced a serious error while in hospital.

      WHO. The World Health Organization Europe. Patient Safety, Data and Statistics. The World Health Organization Europe. Accessed March 1, 2022. https://www.euro.who.int/en/health-topics/Health-systems/patient-safety/dataand-statistics

      A large proportion of these errors are related to surgical care, most of which occur before or after surgery.
      • de Vries EN
      • Ramrattan MA
      • Smorenburg SM
      • Gouma DJ
      • Boermeester MA.
      The incidence and nature of in-hospital adverse events: a systematic review.
      A recent report shows that 13.1% of hospitalized patient are affected by errors in Norway. A large proportion of these errors are related to surgery.

      The Norwegian Directorate of Health. Pasientskader i Norge 2020. Målt med Global Trigger Tool [Patient harm in Norway 2020. Measured with Global Trigger Tool]. 2021. Assessed March 1, 2022. https://www.helsedirektoratet.no/rapporter/pasientskader-i-norge-2019-malt-med-global-trigger-tool

      Previous research demonstrates that unsafe surgical procedures can result in post-surgery deep vein thrombosis, catheter sepsis, and wound infection.
      • de Vries EN
      • Prins HA
      • Crolla RM
      • et al.
      Effect of a comprehensive surgical safety system on patient outcomes.
      • Howell AM
      • Panesar SS
      • Burns EM
      • Donaldson LJ
      • Darzi A.
      Reducing the burden of surgical harm: a systematic review of the interventions used to reduce adverse events in surgery.
      • Zegers M
      • de Bruijne MC
      • de Keizer B
      • et al.
      The incidence, root-causes, and outcomes of adverse events in surgical units: implication for potential prevention strategies.
      • Nilsson L
      • Risberg M
      • Montgomery A
      • Sjödahl R
      • Schildmeijer K
      • Rutberg H.
      Preventable adverse events in surgical care in Sweden. A nationwide review of patient notes.
      A culture of patient safety is a fundamental part of the approach to patient care in many departments of surgery. Several safety initiatives to improve patient safety in surgery have been introduced, for example a surgical safety checklist; however, few tools are available to measure the actual effect of interventions on outcome from the patient's perspective.
      • Makary MA
      • Sexton JB
      • Freischlag JA
      • et al.
      Patient safety in surgery.
      Surgical safety is composed of two concepts: surgical and safety. “Surgical” involves a routine sequence of events - preoperative evaluation of patients, surgical intervention, and postoperative care.

      WHO Patient Safety & World Health Organization. WHO guidelines for safe surgery 2009: safe surgery saves lives. World Health Organization; 2009. Assessed March 1, 2022. https://apps.who.int/iris/handle/10665/44185

      Patient “safety” is described as the avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of healthcare,
      • Vincent C.
      Patient Safety.
      in this case, the surgical trajectory. In accordance with WHO, elective surgery should give the patient the opportunity to improve the quality of care received by understanding treatment options and working with the surgical team to make surgery as safe as possible.

      World Health Organization. Regional Office for Europe. Exploring patient participation in reducing health-care-related safety risks. World Health Organization; ‎2013. Assessed March 1, 2022. https://apps.who.int/iris/handle/10665/326442

      Surgical patients feel safe when they receive good quality of care, for example, by being fully informed and having the ability to affect their own care.
      • Mako T
      • Svanäng P
      • Bjerså K.
      Patients’ perceptions of the meaning of good care in surgical care: a grounded theory study.
      To support patients in returning to normality after surgery, the surgical trajectory must be understood from the perspective of the patients who have experienced it.
      • Alvin R
      Postoperative Recovery. Development of a Multi-Dimentional Questionnaire for Assessment of Recovery.
      Patients also have the potential to reduce medical errors by being involved in the surgical trajectory.
      • Davis RE
      • Sevdalis N
      • Jacklin R
      • Vincent CA
      An examination of opportunities for the active patient in improving patient safety.
      For example, patient participation during the surgical trajectory reduced the incidence of wrong-site surgery.
      • Schwappach DL.
      Review: engaging patients as vigilant partners in safety: a systematic review.
      A review estimated wrong-site surgery at 1 event per 100 000, but the precision is uncertain.
      • Hempel S
      • Maggard-Gibbons M
      • Nguyen DK
      • et al.
      Wrong-site surgery, retained surgical items, and surgical fires: a systematic review of surgical never events.
      A frequently reported cause of wrong-site surgery was inadequate communication in the surgical team, including the patient. In the preoperative stage, the patient can tell the anesthesia provider about any medications currently taken and drug allergies; and in the postoperative stage, patients can alert members of the team if the intravenous catheter becomes loose or dislodged.
      • Davis RE
      • Sevdalis N
      • Jacklin R
      • Vincent CA
      An examination of opportunities for the active patient in improving patient safety.
      With a few exceptions,
      • Al-Abbadi HA
      • Basharaheel HA
      • Alharbi MR
      • Alharbi HA
      • Sindi D
      • Bamatraf M.
      patients' perspectives of surgical safety before and after their elective surgeries at King Abdulaziz University Hospital, Jeddah, Saudi Arabia.
      • Forsberg A
      • Vikman I
      • Wälivaara BM
      • Engström Å.
      Patients' perceptions of their postoperative recovery for one month.
      • Smiley K
      • Ofori L
      • Spangler C
      • et al.
      Safety culture and perioperative quality at the Volta River Authority Hospital in Akosombo, Ghana. Article.
      we have not found studies that have explored self-reported patient perception of surgical safety using a questionnaire. Al-Abbadi et al
      • Al-Abbadi HA
      • Basharaheel HA
      • Alharbi MR
      • Alharbi HA
      • Sindi D
      • Bamatraf M.
      patients' perspectives of surgical safety before and after their elective surgeries at King Abdulaziz University Hospital, Jeddah, Saudi Arabia.
      investigated patient perceptions of surgical safety with an emphasis on surgical team interaction throughout the phases of surgery. The results demonstrated that most patients valued surgeon-patient interaction as it was seen to reduce pre-surgery anxiety, helped in giving options, and improved the patient's overall understanding of the surgical procedure. Forsberg
      • Forsberg A
      • Vikman I
      • Wälivaara BM
      • Engström Å.
      Patients' perceptions of their postoperative recovery for one month.
      explored surgical patients' perceptions (orthopedic and general surgery patients) of their postoperative recovery one to four days after surgery (acute phase) and after one month. They found that a large number of the patients perceived severe or moderate pain, sleeping difficulties and problems with mobilization in the acute phase. Smiley
      • Smiley K
      • Ofori L
      • Spangler C
      • et al.
      Safety culture and perioperative quality at the Volta River Authority Hospital in Akosombo, Ghana. Article.
      evaluated patient and staff perceptions of safety and quality, as well as perioperative process variability. The results revealed that surgical inpatients’ perceptions of safety showed a median rating of 10 on a ten-point scale (with 1 as the worst and 10 as the best possible score), and that 90% gave maximal scores for pain management and 84.4% for nurse communication.
      In summary, few studies have sought the perceptions of patients on surgical safety. In particular, we do not know patients’ perceptions of the current surgical safety practice in a Norwegian context. To be able to do this, there was a need for a translated and validated questionnaire in Norwegian. In this context, the “Patients’ Perspectives of Surgical Safety Questionnaire” (PPSS)
      • Dixon JL
      • Tillman MM
      • Wehbe-Janek H
      • Song J
      • Papaconstantinou HT.
      Patients’ perspectives of surgical safety: do they feel safe?.
      developed in the United States was found to be suitable. To our knowledge, no previous reports have been published on surgical safety initiatives from the patient's perspective in Norway. Additionally, no psychometric testing of the original PPSS has been published in Norway.
      Studies evaluating measurement properties have to meet a high methodological quality. The COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN checklist) is a consensus-based checklist based on an international Delphi study to evaluate the methodological quality of studies on measurement properties of health status measurement instruments (Supplementary file 1).
      • Mokkink LB
      • Terwee CB
      • Patrick DL
      • et al.
      The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.
      The aim of this study was to translate the PPSS questionnaire into Norwegian and test the structural validity and internal consistency of the questionnaire.

      Material and Methods

      Reporting of this study was informed by the criteria of the COSMIN checklist.
      • Mokkink LB
      • Terwee CB
      • Patrick DL
      • et al.
      The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.

      Study Design

      We performed a cross-sectional study with two phases. In the first phase, the PPSS questionnaire was translated into the Norwegian language and adopted for use in a population of Norwegian-speaking surgical patients. In the second phase, the survey was used to assess the structural validity and internal consistency of the Norwegian PPSS in a sample of surgical patients in Norway.
      • Gerrish K
      • Lathlean J.
      Research Process in Nursing.
      ,
      • Polit DF
      • Beck CT.
      Essentials of Nursing Research: Appraising Evidence for Nursing Practice.

      The Original PPSS Questionnaire

      The original PPSS questionnaire was developed through an extensive process.
      • Dixon JL
      • Tillman MM
      • Wehbe-Janek H
      • Song J
      • Papaconstantinou HT.
      Patients’ perspectives of surgical safety: do they feel safe?.
      In the initial phase, a multidisciplinary surgical team who developed a 20-item questionnaire focused on patient perceptions of surgical safety practice. The questionnaire was reviewed and revised by multiple leaders of the surgical, anesthesia, and nursing teams. The PPSS questionnaire was categorized according to phase of care/team interaction subgroups: Preoperative Period (seven items), Recovery (one item), Human interactions (seven items) and others (five items). Questions addressed patient interactions with all team members, including nursing staff, surgeons, and anesthesia providers. Responses to the questions were tabulated using a seven-point Likert scale format with one representing strongly disagree and seven representing strongly agree. In the second phase, surveys were hand delivered to patients during their hospitalization on the first postoperative day and responses were returned using an envelope. In the third phase, qualitative feedback was obtained from seventeen patients during four separate moderated sessions. A moderator led group discussions on patients’ perceptions of the surgical experience that impacted their sense of safety. Thematic qualitative analysis was performed on the session content and common themes were added to the questionnaire.
      • Dixon JL
      • Tillman MM
      • Wehbe-Janek H
      • Song J
      • Papaconstantinou HT.
      Patients’ perspectives of surgical safety: do they feel safe?.
      The surgeon's personal interaction with the patient in the preoperative period was the most important factor influencing patient safety perceptions. Since the qualitative feedback was done after the data collection of the survey, no change of items in the PPSS questionnaire was done.

      Translation to Norwegian

      In the Norwegian version of the 20-item PPSS questionnaire, the scale was changed from seven response alternatives (1 representing strongly disagree and 7 representing strongly agree) in the original PPSS questionnaire to a 5-point Likert scale format from 1 (strongly disagree) to 5 (strongly agree). The reduction of response alternatives was done to reduce the burden on the respondents since patients often are in a particularly vulnerable position with unpleasant reactions to surgery that make them feel weak and incapable.
      • Sibbern T
      • Bull Sellevold V
      • Steindal SA
      • Dale C
      • Watt-Watson J
      • Dihle A.
      Patients’ experiences of enhanced recovery after surgery: a systematic review of qualitative studies.
      ,
      • Chen C
      • Lee S-Y
      • Stevenson HW.
      Response style and cross-cultural comparisons of rating scales among East Asian and North American Students.
      Permission to translate the PPSS questionnaire was obtained from the corresponding author, Matthew M. Tillman. We followed a standardized procedure described by Brislin
      • Brislin RW.
      The wording and translation of research instruments.
      to translate the American version of the PPSS questionnaire
      • Dixon JL
      • Tillman MM
      • Wehbe-Janek H
      • Song J
      • Papaconstantinou HT.
      Patients’ perspectives of surgical safety: do they feel safe?.
      into Norwegian. As the first step, a Norwegian bilingual academic in healthcare with Norwegian as the native language conducted a forward translation into the target language (Norwegian). In the second step, the research group reviewed the forward translation version and compared it with the original version for linguistic congruence and context relevance; some differences in meaning content were found and reviewed.

      Content Validity

      In the third step, the content validity of the reconciled Norwegian version of the PPSS questionnaire was evaluated by a panel consisting of four nurses with expert competencies in surgical care and one associated professor with expert competence in patient surgery research. For the constitution of the panel of experts, the criteria for the nurses were professional activity in the surgical unit of at least 5 years and work production within the scope of patient safety in the surgical unit (quality improvement and patient safety projects). The criteria for the associated professor were experience with similar research projects. The panel members were asked to review the items for relevance and clarity and to suggest possible changes in the Norwegian version of the PPSS questionnaire. After completion of three rounds in the panel, no suggestions or difficulties were manifested. In the fourth step, a bilingual professional translator, with English as native language, who was blinded to the English version, back translated the Norwegian version into English. The research group compared the back-translated version with the original version and found no differences in meaning.
      In the fifth step, a pretest was conducted in a convenience sample of surgical patients (n = 20) in five surgical units of a Norwegian hospital to evaluate the items in the translated PPSS questionnaire for relevance and clarity. Participants were asked four questions: (1) Were there any questions or statements that were difficult to understand? (2) Were there any questions or statements that were not relevant? (3) Were there any questions or statements that you felt were missing? and (4) Do you otherwise have any comments on the questionnaire (both positive and negative)? Feedback from the patients included a need to clarify the term “safety” in item 11, since most patients had anesthesia during surgery and were not able to perceive the communication during this period. Based on the patient's feedback, the term patient safety was explained in the introduction text of the questionnaire and item 11 was revised to "the surgical team effectively communicated before, during (if applicable) and after the surgical experience". In addition, some minor changes were made related to clarification of the language in three of the items.
      Background information such as demographic characteristics (gender, age, civil status, occupation, level of education, country of birth) and clinical characteristics (first surgery or not, and if a surgical complication occurred) was gathered at the end of the PPSS questionnaire.

      Recruitment

      We recruited participants from five surgical units (two gastrointestinal, one orthopedic, one urological, and one blood vessels/thorax) at a university hospital located in a rural area of the Southwestern part of Norway. A convenient sample strategy was used.
      • Polit DF
      • Beck CT.
      Essentials of Nursing Research: Appraising Evidence for Nursing Practice.
      The inclusion criteria were (1) patients ≥ 18 years old who had undergone any elective operation requiring hospitalization >24 hours, (2) patients understanding and writing Norwegian, and (3) patients whose mental and physical health made it possible to participate in the study.
      Ethics approval was provided by the Regional Committee for Medical Research Ethics South East Norway, Faculty of medicine, University of Oslo, Norway (No. 2018/151 – REC South East) and the hospital's research department approved the study. The study was conducted according to the Declaration of Helsinki and ethical guidelines for research.
      World Medical Association
      World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects.
      Eligible participants were recruited by a responsible nurse in each unit. The responsible nurses provided oral and written information about the study to each patient one day after surgery, referring to the principle of autonomy addressed by confidentiality and voluntariness. In the oral and written information that was provided to the participants, information was included about the aim of the study, confidentiality, and voluntary participation. The responsible nurse on each unit assessed whether the patient's mental and physical health made it ethically justifiable to ask him/her to participate. Patients who agreed to take part were instructed to return their completed questionnaire together with a written, informed consent form in a sealed envelope to the responsible nurse, who returned the envelope to the principal researcher.

      Data Collection

      Responsible nurses in each unit identified all patients who fulfilled the inclusion criteria and asked patients to participate in the study. Paper and pencil questionnaires were hand delivered to patients who agreed to take part during their hospitalization on postoperative day one during a 12-month period from November 2018 to October 2019. After completion, the questionnaire was returned in a sealed envelope to the nurse.

      Data Analysis

      All data analysis was performed using SPSS version 26 (IBM). Items were checked for missing values and we used
      • Polit DF
      • Beck CT.
      Essentials of Nursing Research: Appraising Evidence for Nursing Practice.
      the Markov Chain Monte Carlo method for imputation.
      • Levy R.
      The rise of markov chain monte carlo estimation for psychometric modeling.
      Because the significance value of Little's Missing Completely at Random test is less than 0.05 (P < .001) we can conclude that the data are not missing completely at random.
      • Little RJA
      • Rubin DB.
      Statistical analysis with missing data.
      This confirms the conclusion we drew from the descriptive statistics and tabulated patterns. Therefore, we used the Markov Chain Monte Carlo method for imputation.
      • Osborne JW.
      Best Practices in Data Cleaning : A Complete Guide to Everything You Need to do Before and After Collecting Your Data.
      ,
      • Pett MA
      • Lackey NR
      • Sullivan JJ.
      Making Sense of Factor Analysis.
      Descriptive statistics (numbers and percentages) were conducted to describe respondents’ demographic and clinical characteristics. For item analysis, we examined response distribution.
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      Floor and ceiling effects were assessed for each item and considered if ≥ 15 % of the responders scored the highest or the lowest PPSS score.
      • Terwee CB
      • Bot SD
      • de Boer MR
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      Additionally, frequencies for each response category, missing data, and skewness are presented.
      Structural validity was assessed using an explorative factor analysis (EFA).
      • Pallant J.
      SPSS Survival Manual : A Step by Step Guide to Data Analysis Using IBM SPSS.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      • Mokkink LB
      • Terwee CB
      • Patrick DL
      • et al.
      The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes.
      EFA was conducted to test the factor structure of the questionnaire because the objective was primarily to identify a meaningful underlying construct of the questionnaire.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      ,
      • Fabrigar LR
      • Wegener DT.
      Exploratory Factor Analysis.
      ,
      • Tabachnick B
      • Fidell L.
      Using Multivariate Statistics.
      Regarding sample size recommendations, we considered a sample of >200 participants to be large enough to perform an EFA.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      ,
      • Costello A
      • Osborne J.
      Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis.
      The ratio of our sample size to the number of items exceeded 20:1.
      To test assumptions of EFA, an inspection of the correlation matrix was performed to ensure correlation coefficients between 0.30 and 0.70.
      • Tabachnick B
      • Fidell L.
      Using Multivariate Statistics.
      A Kaiser-Meyer-Olkin (KMO) test was performed to measure the sample adequacy; a value of 0.60 and above is desired to conduct an EFA.
      • Tabachnick B
      • Fidell L.
      Using Multivariate Statistics.
      ,
      • Pett MA
      • Lackey NR
      • Sullivan JJ.
      Making Sense of Factor Analysis.
      Bartlett's test of sphericity was conducted to test the overall significant differences in the correlation matrix, with a value of P < .05 for EFA considered to be appropriate.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      Principal axis factoring (PAF) with direct oblimin rotation was used as the factor extracting method. We used PAF since we aimed to identify latent constructs that could explain the pattern of item-item correlations.
      • Fabrigar LR
      • Wegener DT.
      Exploratory Factor Analysis.
      The following criteria were used to determine the optimal number of factors to retain: eigenvalues ≥ 1,
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      percentage of variance, scree plot, parallel analysis, and by assessing the practicality of the factors.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      ,
      • Fabrigar LR
      • Wegener DT.
      Exploratory Factor Analysis.
      Loadings of the factors was conducted by examining the items that most strongly exemplify each factor, that is, that have the largest loadings on a particular factor and the underlying dimension corresponding to the factor was identified.
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      ,
      • DeVellis RF.
      Scale development : theory and applications.
      Different criteria have been proposed as minimum factor loadings. Tabachnick and Fidell
      • Tabachnick B
      • Fidell L.
      Using Multivariate Statistics.
      recommend an absolute value of 0.32 or greater, while Hair et al
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      assess factor loadings of 0.40 and above as significant according to a sample size >200. Pett et al
      • Pett MA
      • Lackey NR
      • Sullivan JJ.
      Making Sense of Factor Analysis.
      caution against arriving at a solution solely based on statistical criteria; it also needs to make theoretical sense.
      Regarding Identifying loadings and assessing the communalities, we considered and decided whether or not to keep the items with low communalities (less than 0.40), but with significant factor loadings.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      ,
      • Costello A
      • Osborne J.
      Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis.
      Variables with higher loadings are considered more important and have greater influence on the name or label selected to present a factor.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      Internal consistency of the questionnaire was calculated with Cronbach's alpha coefficient. Cronbach's alpha values > 0.70 were regarded as acceptable and > 0.80 as preferable.
      • Pett MA
      • Lackey NR
      • Sullivan JJ.
      Making Sense of Factor Analysis.
      ,
      • Cronbach LJ.
      Coefficient alpha and the internal structure of tests.
      Cronbach's alpha was computed for each final extracted factor.

      Results

      Item Analysis

      A total of 221 (74 %) surgical patients responded to the PPSS questionnaire. Three of the respondents were excluded from the analysis because of either missing data ≥50% or divergent responses, resulting in a final sample of 218 patients. Additionally, all items had less than 5% missing values (Table 1). Item 20 had most missing values with 10 respondents not completing it. Ten items had zero missing values. Missing data of 21 respondents for the PPSS were imputed (of whom 14 respondents had one missing item, three respondents had two missing items, three respondents had seven items and one respondent had nine missing items). Single-item mean score for the total sample ranged from 4.34 (SD 0.81) (item 20) to 4.94 (SD 0.33) (item 5). All 20 items had a high skewness (≥15 %) with a ‘Strongly agree’ response ranging from 52.8% (item 20) to 95.9% (item 5). Additionally, items 7, 15, and 20 had a high skewness (≥15 %) with 20.6%, 16.1%, or 30.3% providing a ‘Slightly agree’ response, respectively. See Table 2 for an overview of the background information of the sample (demographic and clinical characteristics).
      Table 1PPSS Items and Score Statistics (N = 218)
      ItemsMean ± SDStrongly Disagree(1)%Slightly Disagree(2)%Neither Agree or Disagree(3)%Slightly Agree(4)%Strongly Agree(5)%Missing%Skew-ness
      Preoperative period
      7. On the day of the surgery, measures taken to ensure my safety were explained to me by someone4.50 ± 0.892.80.97.820.667.93.2- 2.19
      12. On the day of the surgery, I felt safe because of the actions I saw taken by the surgical team4.90 ± 0.380.00.51.46.092.21.8- 4.49
      13. On the day of the surgery, I felt confident about my safety4.91 ± 0.360.00.50.96.092.71.4- 4.78
      16. On the day of the surgery, I felt safe when I was asked several times by the surgical team to repeat my name4.89 ± 0.370.00.01.87.390.80.0- 3.55
      17. On the day of the surgery, I felt safe when I was asked several times by the surgical team to repeat my birthday4.91 ± 0.350.00.01.85.592.70.0- 4.07
      18. I felt safe when I was asked by the surgical team about what kind of surgery was going to be performed that day4.50 ± 0.942.81.411.012.872.00.0- 2.05
      19. I felt safe when I was asked by the surgical team about my allergies4.93 ± 0.270.00.00.56.093.61.4- 4.11
      Recovery
      14. During recovery, the surgical team was concerned for my safety4.88 ± 0.390.00.02.37.390.42.3- 3.46
      Human iInteractions
      2. On the day of the surgery, all members of the surgical team introduced themselves and their roles4.76 ± 0.650.91.42.311.983.50.0- 3.44
      3. On the day of the surgery, the surgical team's top priority was my safety4.81 ± 0.520.00.05.58.386.22.3- 2.66
      4. I felt that my surgeon was concerned about the safety of my surgery4.86 ± 0.440.00.03.76.989.43.2- 3.22
      5. I felt that the anesthesiologist was concerned about the safety of my surgery4.94 ± 0.330.00.02.31.895.90.0- 5.29
      6. I felt that the nurses were concerned about the safety of my surgery4.93 ± 0.300.00.00.95.593.60.0- 4.33
      8. On the day of surgery, I felt safe because of what the surgical team asked me4.83 ± 0.540.50.53.26.989.00.0- 3.96
      11. The surgical team effectively communicated during my surgical experience4.80 ± 0.500.00.04.610.684.93.6- 2.55
      Other
      1. I felt safe the day of surgery4.83 ± 0.510.50.51.411.586.20.0- 3.97
      9. I understood what was involved in ensuring my safety for the surgery4.78 ± 0.570.50.92.312.883.50.0- 3.36
      10. Safety was the most important aspect of my surgery4.82 ± 0.440.00.02.313.883.90.0- 2.41
      15. I understood the risks associated with my surgery4.70 ± 0.640.50.55.516.177.51.4- 2.45
      20. Mistakes rarely happen during surgery4.34 ± 0.810.90.016.130.352.84.5- 1.12
      Table 2Background Information of the Respondents (N = 218)
      Demographic Characteristics Categoryn (%)
      Gender
       Female94 (43.1)
       Male124 (56.9)
      Age
       18-40 y28 (12.8)
       41-60 y60 (27.5)
       61-80 y114 (52.4)
       ≥ 81 y16 (7.3)
      Civil status
       Living with someone152 (69.7)
       Living alone66 (30.3)
      Occupation
       Working72 (33.0)
       Disability benefit26 (11.9)
       Retired105 (48.2)
       Other15 (6.9)
      Education level
       Primary school29 (13.3)
       High school or equivalent105 (48.2)
       3 y university college47 (21.5)
       University37 (17.0)
      Country of birth
       In Norway202 (92.7)
       In another country16 (7.3)
      Clinical characteristics
      First surgery
       Yes39 (17.9)
       No179 (82.1)
      Surgery due to complications
       Yes54 (24.8)
       No164 (75.2)

      Structural Validity of the PPSS Questionnaire

      Results from Bartlett's tests and KMO revealed that the sample met the criteria for conducting factor analyses for the PPSS questionnaire. The KMO measure showed 0.84 and the Bartlett's test of sphericity showed significance (χ2 =  2261.42, P <  .000).
      The initial factor analysis with an eigenvalue >1 extracted a potential five-factor solution explaining 64.58% of the variance prior to rotation (Table 3). An inspection of the screeplots revealed a clear break after the second component (Figure 1).
      • Fabrigar LR
      • Wegener DT.
      Exploratory Factor Analysis.
      In the five-factor solution, twelve items had a high loading (>.40) on Factor 1 (F1). Two of the items had cross loadings on F1 and on F3 (item 1) and F4 (item 7), resulting in either two or three items on F3 and F4. Factor 2 consisted of two items (items 16, 17) with high loading (>.40) and F5 had negative factor loadings on 18 items. These five factors showed low communalities (<.40) on four items (6, 7, 11, 20). The factor correlation matrix showed that only the correlation between F1 and F3 exceeded 0.30 (Table 4).
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      After oblimin rotation, no meaningful pattern in the loadings could be determined (see Supplementary file 3). Since there was no a priori theory, it was necessary to do multiple factor analysis, each with a different numbers of factors.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      Further, a fixed factor analysis with 4, 3, 2, or 1 factor by using PAF with direct oblimin was performed.
      Table 3Structure Coefficients, Communalities, Eigenvalue, and % of Variance Explained for 1-Factor, 2-Factor, 3-Factor, 4-Factor, and the Model With Initial Eigenvalues >1 Using EFA
      ItemsStructure Coefficients Eigenvalues > 1Structure Coefficients 4-Factor (Fixed)Structure Coefficients 3-Factor (Fixed)Structure Coefficients 2-Factor (Fixed)Structure Coefficients 1-Factor (Fixed)
      Factor 1Factor 2Factor 3Factor 4Factor 5h2Factor 1Factor 2Factor 3Factor 4h2Factor 1Factor 2Factor 3h2Factor 1 (Team interaction safety)Factor 2(Patient´s ID safety)h2Factor 1h2
      1.0.530.530.11-0.540.470.590.520.470.580.550.460.640.440.610.37
      2.0.610.300.37-0.660.520.680.350.490.680.330.410.480.690.480.690.48
      3.0.520.340.39-0.790.640.640.320.400.460.500.660.390.480.500.700.350.500.710.51
      4.0.390.34-0.750.570.520.370.370.340.540.430.340.580.340.580.34
      5.0.480.46-0.520.400.550.460.400.530.300.470.350.580.340.590.35
      6.0.480.30-0.410.290.510.300.500.300.260.500.260.510.26
      7.0.340.34-0.350.220.360.420.230.390.300.170.410.170.420.17
      8.0.840.270.38-0.500.770.810.310.700.760.350.580.730.530.720.52
      9.0.780.30-0.470.610.750.320.560.750.300.560.700.490.680.46
      10.0.68-0.510.490.690.370.490.700.300.490.660.440.660.44
      11.0.530.35-0.540.370.590.330.360.570.370.340.590.350.580.34
      12.0.380.85-0.470.750.450.840.330.750.460.320.860.750.610.380.620.38
      13.0.300.95-0.390.910.370.920.250.850.370.920.860.530.280.520.28
      14.0.310.360.580.430.300.320.570.370.360.380.190.410.170.420.17
      15.0.680.320.32-0.470.500.660.430.480.680.340.470.660.450.640.41
      16.0.970.960.950.910.980.960.310.980.970.400.16
      17.0.95-0.300.910.960.930.270.930.870.300.930.860.390.15
      18.0.530.43-0.480.450.540.540.460.570.360.520.280.530.28
      19.0.420.50-0.350.400.410.550.410.320.450.190.360.440.240.400.16
      20.0.58-0.340.370.540.340.340.560.330.510.280.480.23
      Eigenvalue7.081.961.591.291.027.081.961.581.297.071.961.587.081.967.08
      % of variance explained35.3745.1553.0759.5064.5835.3745.1553.0759.5035.3745.1553.0735.3745.1535.37
      Eigenvalue

      (Parallel analysis)
      1.581.461.381.311.25
      h2, commonalities.
      Figure 1
      Figure 1Screeplot of PPSS's factor structure.
      Table 4Oblimin Factor Correlation Matrix
      No. of FactorsFactor2345
      5-factor1.22.37.21-.61
      2.18.13-.33
      3.11-.44
      4-.23
      4-factor1.24.38.40
      2.18.17
      3.18
      3-factor1.34.47
      2.24
      2-factor1.33
      The fixed factor analysis with four factors extracted explained 59.50% of the variance (Table 3). The factor loadings showed that F1 consisted of 13 items with high loading (>.40), F2 and F3 consisted of two items (items 16, 17 and 12, 13, respectively). One item had cross loadings on F1 as on F4 (item 18), resulting in either twelve or 13 items on F1 and either three or four items on F4 with high loading (>.40). The four factors showed low communalities (<.40) on six items (items 4, 6, 7, 11, 14, 20). The factor correlation matrix showed that only correlations between F1 and F3 and between F1 and F4 exceeded 0.30 (Table 4).
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      The fixed factor analysis with three factors extracted explained 53.07% of the variance (Table 3). The factor loadings showed that F1 consisted of 14 items, F2 and F3 consisted of three items (items 16, 17, 19 and items 12, 13, 14, respectively) with modest to high loadings. Several items (item 1-5) had cross loadings on F1 and F3 with high loading (>.40). The three factors showed low communalities (<.40) on nine items (items 4-7, 11, 14, 18-20). The factor correlation matrix showed that only correlations between F1 and F2 and between F1 and F3 exceeded 0.30 (Table 4).
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      The fixed factor analysis with two factors extracted explained 45.15% of the variance (Table 3). The factor loadings showed that F1 consisted of 17 items, and F2 consisted of three items (items 16, 17, 19) with high loadings (>.40). The factor loading of F1 was from 0.41 to 0.73 and the factor loadings of F2 were from 0.44 to 0.98. No cross loadings were identified. The two factors showed low communalities (<.40) on eleven items (4-7, 11-14, 18-20) and acceptable communalities (0.97-0.44) on nine items. The factor correlation matrix showed that correlation between F1 and F2 exceeded 0.30 (Table 4).
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      The fixed factor analysis with one factor extracted explained 35.37% of the variance (Table 3). The factor loadings of all the items showed high loadings (>.40) except for item 17 (.39). The one factor showed low communalities (<.40) on fourteen items (1, 4-7, 11-14, 16-20).
      We decided to retain the two-factor solution because it led to the best interpretation and because factor loadings of 17 items (items 1-15, 18, and 20) were explained by one distinct factor with low or acceptable communalities and correlation exceeding 0.30 between F1 and F2.
      • Fabrigar LR
      • Wegener DT
      • MacCallum RC
      • Strahan EJ.
      Evaluating the use of exploratory factor analysis in psychological research.
      As seen in Table 3, the first factor consisted of 17 items. Four items were from the “Preoperative period”, one item was from “Recovery”, seven items were from “Human Interaction”, and five items were from “Other” in the original PPSS. Considering the three items with highest loading (items 3, 8, 9) in this category, we labelled it as “Team interaction safety”. The second factor contained three items (items 16, 17, 19) with high loadings (0.44-0.98), which were from the “Preoperative period” in the original PPSS. Based on the shared characteristics of each of them (name, birthday, and allergies), this factor was labelled “Patient's ID safety”.

      Internal Consistency of the PPSS Questionnaire

      After the research team decided how many factors to retain, Cronbach's alpha was calculated to assess the internal consistency of each of the two final extracted factors of the PPSS questionnaire. The internal consistency of F1 “Team interaction safety” was satisfactory with Cronbach's alpha of 0.88 for positive affect. The corrected item total correlations were all positive and ranged from 0.40 to 0.68, supporting the internal reliability of the factor. The internal consistency of F2 “Patient´s ID safety” was satisfactory with Cronbach's alpha of 0.82 for positive affect. The corrected item total correlations were all positive and ranged from 0.41 to 0.86, supporting the internal reliability of the factor.

      Discussion

      The aim of this study was to translate the PPSS questionnaire into Norwegian and test the structural validity and internal consistency of the questionnaire. Overall, the results indicate that PPSS is a psychometrically valid instrument, which can be used for most populations of Norwegian surgical patients ≥ 18 years undergoing any elective operation requiring hospitalization >24 hours. To our knowledge, this is the first psychometric evaluation of the questionnaire because Dixon et al
      • Dixon JL
      • Tillman MM
      • Wehbe-Janek H
      • Song J
      • Papaconstantinou HT.
      Patients’ perspectives of surgical safety: do they feel safe?.
      did not report psychometric testing, only development of the questionnaire and descriptive results based on surgical patients’ responses. The use of the questionnaire available in Norwegian may contribute to evaluate the patient's perspectives of safe surgery in Norway, to improve practice, or to evaluate the effects of interventions.

      Translation and Cross-cultural Adaption

      The original PPSS questionnaire consists of 20 items grouped into four phases of surgical care and team interaction (preoperative period, recovery, human interaction and other). As recommended by Davidov,
      • Davidov E
      • Meuleman B
      • Cieciuch J
      • Schmidt P
      • Billiet J.
      Measurement equivalence in cross-national research.
      the translation of the PPSS questionnaire followed a standardized procedure,
      • Brislin RW.
      The wording and translation of research instruments.
      except for in the first and the fourth steps where only one bilingual academic translator was used for the forward and one for the back-translation. This may be a limitation since a team of two independent translators for each step is recommended.
      • Borsa JC
      • Damásio BF
      • Bandeira DR.
      Cross-cultural adaptation and validation of psychological instruments: some considerations.
      The translation process was comprehensive, and both healthcare professionals and patients participated to assure the content and cross-cultural validity of the questionnaire.
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      The results of the translation process indicate that PPSS is already a questionnaire specific for patients undergoing surgery that appears to be transferable to different languages and cultures without much difficulty. Regarding the sample size, no consensus exists when it comes to the recommended size for EFA in psychometric studies. To perform a stable and reliable factor solution, the recommended sample size varies from a minimum ratio of 2:1 to 20:1.
      • Costello A
      • Osborne J.
      Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis.
      The sample size of 221 in the current study was thereby considered satisfactory with its ratio of 20:1. However, as pointed out by Costello and Osborne,
      • Costello A
      • Osborne J.
      Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis.
      adequate sample size is partly determined by the nature of the data, that is, strong data means uniformly high communalities without cross loadings and several variables loading strongly on each factor. Very few missing values (less than 5%) indicate that the study sample easily accepted the questionnaire and had few problems in understanding the translated items.
      • Schafer JL.
      Multiple imputation: a primer.
      Item 20 had most missing values, with 10 respondents (4.5%) not completing it.
      The results detected that the mean PPSS score was at a high level, indicating a strong ceiling effect in the Norwegian version of the questionnaire. The ceiling effect would normally be a sign to revise item wording to improve the ability of the tool to capture variance in patients’ perception of surgical safety or might preclude the ability of the instrument to distinguish patients with the lowest or highest end of the scale from each other. Furthermore, the responsiveness is limited because changes cannot be measured in these patients.
      • Terwee CB
      • Bot SD
      • de Boer MR
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      It seems that ceiling effects in patients’ perception measures in hospitals are not uncommon
      • Abbasi-Moghaddam MA
      • Zarei E
      • Bagherzadeh R
      • Dargahi H
      • Farrokhi P.
      Evaluation of service quality from patients’ viewpoint.
      as observed in patients who have undergone surgery,
      • Gebremedhn EG
      • Lemma GF.
      Patient satisfaction with the perioperative surgical services and associated factors at a University Referral and Teaching Hospital, 2014: a cross-sectional study.
      had knee,
      • Dinjens RN
      • Senden R
      • Heyligers IC
      • Grimm B.
      Clinimetric quality of the new 2011 Knee Society score: High validity, low completion rate.
      or hip replacement surgery.
      • Mahomed N
      • Gandhi R
      • Daltroy L
      • Katz JN.
      The self-administered patient satisfaction scale for primary hip and knee arthroplasty.
      Another plausible explanation for the ceiling effect might be the timing of the questionnaire completion. Patients may feel vulnerable during the 1st-day post-op or may feel "lucky" to have survived the surgery.
      Furthermore, an explanation for the ceiling effect mentioned previously might be that the item wording needs to be revised to improve the ability of the tool.
      • Terwee CB
      • Bot SD
      • de Boer MR
      • et al.
      Quality criteria were proposed for measurement properties of health status questionnaires.
      In the Norwegian version of the 20-item PPSS questionnaire, the scale was changed from a 7- point to 5-point scale (Likert scale). The change from seven to five response options can possibly result in decreased precision of the scores, but evidence is limited.
      • Fitzpatrick R
      • Davey C
      • Buxton M
      • Jones D.
      Evaluating patient-based outcome measures for use in clinical trials: a review.
      Furthermore, Streiner et al
      • Streiner D
      • Norman G
      • Cairney J.
      Health Measurement Scales : A Practical Guide to Their Development and Use.
      argue that the loss in reliability from 7 to 5 categories is quite small and that reducing the number of response options from seven to five will not result in significant loss of information. Likert scales ask individual to think along at least two different dimensions – content and intensity. By reducing the scale to five-points, we have reduced the affective dimensions this vulnerable patient group must decide on.
      • Hodge DR
      • Gillespie D.
      Phrase completions: an alternative to Likert scales.
      In the original PPSS article,
      • Dixon JL
      • Tillman MM
      • Wehbe-Janek H
      • Song J
      • Papaconstantinou HT.
      Patients’ perspectives of surgical safety: do they feel safe?.
      only the mean survey responses by question (between labels 6 and 7 on 19 items) were published. Therefore, comparison between the results in this study and the results in the original PPSS study is limited.

      Structural Validity of the Questionnaire

      The structural validity was evaluated by performing several EFAs before arriving at a model that produced meaningful solutions.
      • Fabrigar LR
      • Wegener DT.
      Exploratory Factor Analysis.
      There is no consensus as to what variance is explained by factor solution.
      • Peterson RA.
      A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis.
      In this study, the two-factor solution we decided to retain accounted for 45.15% of the variance, which is considered acceptable, but not preferable.
      • Peterson RA.
      A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis.
      Similarly, there is no consensus as to what constitutes a “high” and “low” factor loading.
      • Peterson RA.
      A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis.
      Compared to the results of the factor analysis with 1, 3, 4, and 5 factors, in the two-factor solution we retained, all factor loadings on both factors exceeded 0.40 and the majority were ≥ 0.50 (see Table 3), which Hair et al
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      have noted as significant for a sample size >200. This indicates that the items strongly influenced their respective factors.
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      The retained two-factor solution did not produce cross loadings as in the 3, 4 and 5-factor solutions, meaning that there is not very much instability in determining the retained factor structure.
      • Li Y
      • Wen Z
      • Hau K-T
      • Yuan K-H
      • Peng Y.
      Effects of cross-loadings on determining the number of factors to retain.
      Further, the retained two-factor solution was the only one that showed correlation >.30 between all factors.
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      Although no factor analysis had been performed in the original study in which the PPSS was presented, we had expected that the initial phases (Preoperative Period, Recovery, and Human interactions) would also be present in the adapted PPSS questionnaire. An explanation of this result could be that the distinction between phases and team members in the surgical trajectory is clearer for healthcare personnel than patients since patients rarely distinguish between the different healthcare professions or pre- and postoperative care.
      • Mako T
      • Svanäng P
      • Bjerså K.
      Patients’ perceptions of the meaning of good care in surgical care: a grounded theory study.
      ,
      • Hair JF
      • Babin BJ
      • Anderson RE.
      Multivariate Data Analysis.
      In our study, the EFA shows that PPSS consists of two factors. Factor 1 “Team interaction safety” is related to all phases of the surgical process, while factor 2 “Patient´s ID safety” is related to specific identity questions from the surgical team. The factors identified did not correspond to the phases of surgical care patients go through in the surgical trajectories, which was the basis for the developed PPSS questionnaire in the original study; however, the three items (items 16, 17, 19) that constitute Factor 2 belong to the “Preoperative period.” Thus, the structural validity needs to be further explored with different groups of surgical patients to confirm and refine the factors and items. A data collection measuring changes in item response from hospital to home will be useful. Future research should also include a confirmatory factor analysis conducted in a new sample.
      • Kentaro H
      • Yuan K.
      Exploratory factor analysis.

      Internal Consistency

      Cronbach's alpha has not been performed in any previous study of the PPSS questionnaire. Internal consistency assessed by Cronbach's alpha showed values of 0.88 on F1 and 0.82 on F2 which reflect satisfactory reliability of the two-factor model for this study sample.
      • DeVellis RF.
      Scale development : theory and applications.
      According to Streiner
      • Streiner D.
      Starting at the beginning: an introduction to coefficient alpha and internal consistency.
      the alpha value is dependent on the number of items in a factor. An especially low numbers of items (≤3) might result in low alpha, which was not the case in our study.
      • Streiner D.
      Starting at the beginning: an introduction to coefficient alpha and internal consistency.
      One possible explanation of the satisfactory alpha value in the current study might be that the length of the scale contains enough items (≥18) to obtain satisfactory alpha.
      • Streiner D.
      Starting at the beginning: an introduction to coefficient alpha and internal consistency.
      In future studies, a test – retest reliability of the PPSS questionnaire is recommended.
      • Mokkink LB
      • Terwee CB
      • Patrick DL
      • et al.
      The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.
      A test – retest is appropriate to test the “extent to which scores for participants who have not changed are the same when a measure is administered twice. It is an assessment of a measure's stability”.
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      , p. 333
      The Norwegian version of the PPSS questionnaire showed high Cronbach's alpha of 0.88 on F1 and 0.82 on F2. A questionnaire with Cronbach's alpha > 0.80 showed that the two-factor model composed of items from the PPSS had satisfactory internal consistency indicating a strong, positive relationship.
      • Polit DF
      • Yang FM.
      Measurement and the Measurement of Change : A Primer for the Health Professions.
      ,
      • Pett MA
      • Lackey NR
      • Sullivan JJ.
      Making Sense of Factor Analysis.

      Limitations of the Study

      Some limitations of the study must be stated. The response rate in the present study was acceptable, 74%. However, the participants may represent a selection of the healthiest patients one day after surgery who were probably more inclined to participate. A modest sample size was used for conducting the EFA. Therefore, the findings presented should be interpreted with caution and replicated in a larger sample. The sample of the study represented a convenience sample of surgical patients from one hospital in Norway. Therefore, we cannot ensure that the results are representative of the target population and that the questionnaire is performing as intended.
      • Polit DF
      • Beck CT.
      Essentials of Nursing Research: Appraising Evidence for Nursing Practice.
      A large ceiling effect was detected, which might represent a limitation. The study employed a cross-sectional design, and the findings are limited to the measurement and interpretation of the variables at a single point of time. Thus, it is not possible to know if these findings would be consistent over time with the same group of patients. The measurement of PPSS depends on the self-report of the respondents and may be prone to social biases. Since the PPSS questionnaire previously has not been tested psychometrically, a limitation might be that we cannot compare measurement properties between the original and the translated version.

      Conclusion

      This study has provided promising structural validity and internal consistency of the 20-item PPSS questionnaire for the assessment of patients’ perspectives on surgical safety in a Norwegian context. PPSS is easy to administer and well understood by patients. However, future research on PPSS should provide an examination of construct validity, validation and testing in other populations of surgical patients.
      Future improvement initiatives should include the response option “do not know”. To be of optimal use, the questionnaire needs some adjustment to capture variability in scores. The use of the questionnaire available in Norwegian may contribute to evaluate the patient's perspectives of safe surgery in Norway, to improve practice, or to evaluate the effects of interventions.

      Data availability

      Data are available from the corresponding author on reasonable request.

      Acknowledgments

      The authors want to thank the patients who took time to participate in the questionnaire, the nurses in the expert team and the nurses involved in the data collection.

      Appendix. Supplementary materials

      References

      1. World Health Organization. Regional Office for Europe. Exploring patient participation in reducing health-care-related safety risks. World Health Organization; ‎2013. Assessed March 1, 2022. https://apps.who.int/iris/handle/10665/326442

      2. The Joint Commission. Busting the myths about engaging patients and families in patient safety. 2016. Assessed March 1, 2022. https://www.jointcommission.org/assets/1/18/PFAC_patient_family_and_safety_white_paper.pdf

      3. WHO Patient Safety & World Health Organization. WHO guidelines for safe surgery 2009: safe surgery saves lives. World Health Organization; 2009. Assessed March 1, 2022. https://apps.who.int/iris/handle/10665/44185

        • WHO
        Patient Safety.
        World Health Organization, 2022 (Accessed March 1, 2022)
        • Makary MA
        • Sexton JB
        • Freischlag JA
        • et al.
        Patient safety in surgery.
        Ann Surg. 2006; 243: 628-635https://doi.org/10.1097/01.sla.0000216410.74062.0f
      4. WHO. The World Health Organization Europe. Patient Safety, Data and Statistics. The World Health Organization Europe. Accessed March 1, 2022. https://www.euro.who.int/en/health-topics/Health-systems/patient-safety/dataand-statistics

        • de Vries EN
        • Ramrattan MA
        • Smorenburg SM
        • Gouma DJ
        • Boermeester MA.
        The incidence and nature of in-hospital adverse events: a systematic review.
        Qual Saf Health Care. 2008; 17: 216-223https://doi.org/10.1136/qshc.2007.023622
      5. The Norwegian Directorate of Health. Pasientskader i Norge 2020. Målt med Global Trigger Tool [Patient harm in Norway 2020. Measured with Global Trigger Tool]. 2021. Assessed March 1, 2022. https://www.helsedirektoratet.no/rapporter/pasientskader-i-norge-2019-malt-med-global-trigger-tool

        • de Vries EN
        • Prins HA
        • Crolla RM
        • et al.
        Effect of a comprehensive surgical safety system on patient outcomes.
        N Engl J Med. 2010; 363: 1928-1937https://doi.org/10.1056/NEJMsa0911535
        • Howell AM
        • Panesar SS
        • Burns EM
        • Donaldson LJ
        • Darzi A.
        Reducing the burden of surgical harm: a systematic review of the interventions used to reduce adverse events in surgery.
        Ann Surg. 2014; 259: 630-641https://doi.org/10.1097/sla.0000000000000371
        • Zegers M
        • de Bruijne MC
        • de Keizer B
        • et al.
        The incidence, root-causes, and outcomes of adverse events in surgical units: implication for potential prevention strategies.
        Patient Saf Surg. 2011; 5: 13https://doi.org/10.1186/1754-9493-5-13
        • Nilsson L
        • Risberg M
        • Montgomery A
        • Sjödahl R
        • Schildmeijer K
        • Rutberg H.
        Preventable adverse events in surgical care in Sweden. A nationwide review of patient notes.
        Medicine. 2016; 95: e3047
        • Vincent C.
        Patient Safety.
        Wiley-Blackwell, 2010
        • Mako T
        • Svanäng P
        • Bjerså K.
        Patients’ perceptions of the meaning of good care in surgical care: a grounded theory study.
        BMC Nurs. 2016; 15 (2016/08/03): 47https://doi.org/10.1186/s12912-016-0168-0
        • Alvin R
        Postoperative Recovery. Development of a Multi-Dimentional Questionnaire for Assessment of Recovery.
        The University of Örebro, Örebro2009
        • Davis RE
        • Sevdalis N
        • Jacklin R
        • Vincent CA
        An examination of opportunities for the active patient in improving patient safety.
        J Patient Saf. 2012; 8: 36-43https://doi.org/10.1097/pts.0b013e31823cba94
        • Schwappach DL.
        Review: engaging patients as vigilant partners in safety: a systematic review.
        Med Care Res Rev. 2010; 67: 119-148https://doi.org/10.1177/1077558709342254
        • Hempel S
        • Maggard-Gibbons M
        • Nguyen DK
        • et al.
        Wrong-site surgery, retained surgical items, and surgical fires: a systematic review of surgical never events.
        JAMA Surg. 2015; 150: 796-805https://doi.org/10.1001/jamasurg.2015.0301
        • Al-Abbadi HA
        • Basharaheel HA
        • Alharbi MR
        • Alharbi HA
        • Sindi D
        • Bamatraf M.
        patients' perspectives of surgical safety before and after their elective surgeries at King Abdulaziz University Hospital, Jeddah, Saudi Arabia.
        Cureus. 2019; 11: e6171https://doi.org/10.7759/cureus.6171
        • Forsberg A
        • Vikman I
        • Wälivaara BM
        • Engström Å.
        Patients' perceptions of their postoperative recovery for one month.
        J Clin Nurs. 2015; 24: 1825-1836https://doi.org/10.1111/jocn.12793
        • Smiley K
        • Ofori L
        • Spangler C
        • et al.
        Safety culture and perioperative quality at the Volta River Authority Hospital in Akosombo, Ghana. Article.
        World J Surg. 2019; 43: 16-23https://doi.org/10.1007/s00268-018-4763-y
        • Dixon JL
        • Tillman MM
        • Wehbe-Janek H
        • Song J
        • Papaconstantinou HT.
        Patients’ perspectives of surgical safety: do they feel safe?.
        Ochsner J. 2015; 15 (Summer): 143-148
        • Mokkink LB
        • Terwee CB
        • Patrick DL
        • et al.
        The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.
        Qual Life Res. 2010; 19: 539-549https://doi.org/10.1007/s11136-010-9606-8
        • Gerrish K
        • Lathlean J.
        Research Process in Nursing.
        7th ed. Wiley Blackwell, 2015
        • Polit DF
        • Beck CT.
        Essentials of Nursing Research: Appraising Evidence for Nursing Practice.
        9th ed. Wolters Kluwer/Lippincott Williams & Wilkins, 2018 (XIV, 442 s)
        • Sibbern T
        • Bull Sellevold V
        • Steindal SA
        • Dale C
        • Watt-Watson J
        • Dihle A.
        Patients’ experiences of enhanced recovery after surgery: a systematic review of qualitative studies.
        J Clin Nurs. 2017; 26: 1172-1188https://doi.org/10.1111/jocn.13456
        • Chen C
        • Lee S-Y
        • Stevenson HW.
        Response style and cross-cultural comparisons of rating scales among East Asian and North American Students.
        Psycholog Sci. 1995; 6: 170-175
        • Brislin RW.
        The wording and translation of research instruments.
        Field Methods in Cross-Cultural Research. Sage Publications, Inc, 1986: 137-164 (Cross-cultural research and methodology series, Vol. 8)
        • World Medical Association
        World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects.
        Bullet World Health Organ. 2001; 79: 373
        • Levy R.
        The rise of markov chain monte carlo estimation for psychometric modeling.
        J Probab Stat. 2009; (2009/12/30 2009)537139https://doi.org/10.1155/2009/537139
        • Little RJA
        • Rubin DB.
        Statistical analysis with missing data.
        (Third edition)Wiley Series in Probability and Statistics. Wiley, 2020
        • Osborne JW.
        Best Practices in Data Cleaning : A Complete Guide to Everything You Need to do Before and After Collecting Your Data.
        SAGE, 2013
        • Pett MA
        • Lackey NR
        • Sullivan JJ.
        Making Sense of Factor Analysis.
        SAGE Publications, 2003
        • Polit DF
        • Yang FM.
        Measurement and the Measurement of Change : A Primer for the Health Professions.
        Wolters Kluwer, 2016
        • Terwee CB
        • Bot SD
        • de Boer MR
        • et al.
        Quality criteria were proposed for measurement properties of health status questionnaires.
        J Clin Epidemiol. 2007; 60: 34-42https://doi.org/10.1016/j.jclinepi.2006.03.012
        • Pallant J.
        SPSS Survival Manual : A Step by Step Guide to Data Analysis Using IBM SPSS.
        7th edition. Open University Press, 2020
        • Hair JF
        • Babin BJ
        • Anderson RE.
        Multivariate Data Analysis.
        Cengage, Mason, OH2018
        • Mokkink LB
        • Terwee CB
        • Patrick DL
        • et al.
        The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes.
        J Clin Epidemiol. 2010; 63: 737-745https://doi.org/10.1016/j.jclinepi.2010.02.006
        • Fabrigar LR
        • Wegener DT.
        Exploratory Factor Analysis.
        Oxford University Press, 2012
        • Tabachnick B
        • Fidell L.
        Using Multivariate Statistics.
        Sage Publications, 2020 (7th international edition (cover))
        • Costello A
        • Osborne J.
        Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis.
        Pract Assess Res Evaluation. 2005; 10https://doi.org/10.7275/jyj1-4868
        • DeVellis RF.
        Scale development : theory and applications.
        (2017)Fourth edition. ed. Applied Social Research Methods Series. 26. SAGE, 2017
        • Cronbach LJ.
        Coefficient alpha and the internal structure of tests.
        Psychometrika. 1951; 16 (1951/09/01): 297-334https://doi.org/10.1007/BF02310555
        • Fabrigar LR
        • Wegener DT
        • MacCallum RC
        • Strahan EJ.
        Evaluating the use of exploratory factor analysis in psychological research.
        Psychol Methods. 1999; 4: 272-299https://doi.org/10.1037/1082-989X.4.3.272
        • Davidov E
        • Meuleman B
        • Cieciuch J
        • Schmidt P
        • Billiet J.
        Measurement equivalence in cross-national research.
        Annual Rev Sociol. 2014; 40: 55-75
        • Borsa JC
        • Damásio BF
        • Bandeira DR.
        Cross-cultural adaptation and validation of psychological instruments: some considerations.
        Paidéia (Ribeirão Preto). 2012; 22: 423-432
        • Schafer JL.
        Multiple imputation: a primer.
        Stat Methods Med Res. 1999; 8: 3-15https://doi.org/10.1177/096228029900800102
        • Abbasi-Moghaddam MA
        • Zarei E
        • Bagherzadeh R
        • Dargahi H
        • Farrokhi P.
        Evaluation of service quality from patients’ viewpoint.
        BMC Health Serv Res. 2019; 19 (2019/03/15): 170https://doi.org/10.1186/s12913-019-3998-0
        • Gebremedhn EG
        • Lemma GF.
        Patient satisfaction with the perioperative surgical services and associated factors at a University Referral and Teaching Hospital, 2014: a cross-sectional study.
        Pan Afr Med J. 2017; 27: 176https://doi.org/10.11604/pamj.2017.27.176.10671
        • Dinjens RN
        • Senden R
        • Heyligers IC
        • Grimm B.
        Clinimetric quality of the new 2011 Knee Society score: High validity, low completion rate.
        Knee. 2014; 21: 647-654https://doi.org/10.1016/j.knee.2014.02.004
        • Mahomed N
        • Gandhi R
        • Daltroy L
        • Katz JN.
        The self-administered patient satisfaction scale for primary hip and knee arthroplasty.
        Arthritis. 2011; (2011/01/10 2011)591253https://doi.org/10.1155/2011/591253
        • Fitzpatrick R
        • Davey C
        • Buxton M
        • Jones D.
        Evaluating patient-based outcome measures for use in clinical trials: a review.
        Health Technol Assess. 1998; 2 (i-iv): 1-74
        • Streiner D
        • Norman G
        • Cairney J.
        Health Measurement Scales : A Practical Guide to Their Development and Use.
        5th ed. Oxford University Press, 2015
        • Hodge DR
        • Gillespie D.
        Phrase completions: an alternative to Likert scales.
        Soc Work Res. 2003; 27: 45-55https://doi.org/10.1093/swr/27.1.45
        • Peterson RA.
        A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis.
        Market Lett. 2000; 11 (2000/08/01): 261-275https://doi.org/10.1023/A:1008191211004
        • Li Y
        • Wen Z
        • Hau K-T
        • Yuan K-H
        • Peng Y.
        Effects of cross-loadings on determining the number of factors to retain.
        Struct Equ Model. 2020; 27 (2020/11/01): 841-863https://doi.org/10.1080/10705511.2020.1745075
        • Kentaro H
        • Yuan K.
        Exploratory factor analysis.
        in: Salkind NJ Encyclopedia of Research Design. SAGE Publications, Thousand Oaks2010
        • Streiner D.
        Starting at the beginning: an introduction to coefficient alpha and internal consistency.
        J Pers Assess. 2003; 80: 99-103https://doi.org/10.1207/s15327752jpa8001_1