Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses

All data analyses were carried out using SPSS v. 14.0. Reliability, in the form of internal consistency, was assessed via Cronbach a coefficients (and we also report the lowest item–total correlation obtained for each item). Cronbach a values range between 0 and 1 with values >0.70 typically considered adequate in research contexts. Content validity was assessed via Spearman r correlation coefficients between different core skills as rated by the trainer nurse. Non-parametric coefficients were computed due to small sample size. Medium-sized correlations between skills will provide evidence for adequate content validity. Correlations between trainer and trainees’ ratings across skills were reported as well.

The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1–6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room.

Feasibility: ICATS-N_ should be reasonably easy to use in real ORs and also in the context of simulation-based training.a. The tool is observational, and thus non-intrusive in its useb. The tool consists of a relatively small number of itemsc. The tool consists of observable behaviours (i.e., the amount of inference involved from the part of the rater/trainer is minimised) Feasibility was evaluated qualitatively: did the trainer OR nurse of our study find the tool usable?Reliability: ICATS-N_ should capture OR nurses’ technical skills consistently.The tool consists of observable behaviours (i.e., the amount of inference involved from the part of the rater/trainer is minimised) Reliability was evaluated quantitatively in the form of internal consistency (Cronbach a coefficients)Validity: ICATS-N_ should capture what it purports to capture (i.e., OR nurses’ technical skills). Face validity was evaluated qualitatively, via expert opinion in the development of the tool. Content validity was evaluated (i) qualitatively, via expert opinion in the development of the tool and (ii) quantitatively, via correlational analyses of core skills as defined in ICATS-N.

Twenty trainee OR nurses and two senior OR trainer nurses took part in the study. ICATS-N was tested in the context of surgical crisis simulations. The simulations were carried out in a Simulated Operating Room (SOR) in the hospital. The SOR is a fully equipped OR with a control room, through which trainers can direct simulation exercises and observe the trainees via one-way mirror.

Through the control room, one of two trainer nurses (JH, ES) rated trainees’ technical skill using ICATS-N. (For practical reasons (ongoing clinical commitment), it was not possible to have both trainers rating the trainees simultaneously, which precludes any analysis of interrater reliability.) On completion of the procedure, trainees also completed ICATS-N to rate their own performance and received feedback on their performance from the trainer.

Assessment of OR nurses’ technical skill is becoming a training priority. The present evidence suggests that the ICATS-N_ could be considered for use as an assessment/training tool for junior OR nurses.In future empirical work, it is essential to replicate the present findings with larger samples that should ideally be drawn from different institutions.

The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach a) was obtained across trainers’ and trainees’ scores (analyzed jointly and separately). Moreover, trainer nurse’s ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer’s and trainees’ ratings did not correlate.

Ethical approval was obtained for the study and informed consent provided by all participants.

To develop an observational tool that captures operating room (OR) nurses’ technical skill and can be used for assessment and training.