Validity evidence of non-technical skills assessment instruments in simulated anaesthesia crisis management.
We sought to evaluate the validity of two non-technical skills evaluation instruments, the Anaesthetists' Non-Technical Skills (ANTS) behavioural marker system and the Ottawa Global Rating Scale (GRS), to apply them to anaesthesia training. The content validity, response process, internal structure, relations with other variables and consequences were described for validity evidence. Simulated crisis management sessions were initiated during which two trained raters evaluated the performance of postgraduate first-, second- and third-year (PGY-1, PGY-2 and PGY-3) anaesthesia residents. The study included 70 participants, composed of 24 PGY-1, 24 PGY-2 and 22 PGY-3 residents. Both instruments differentiated the non-technical skills of PGY-1 from PGY-3 residents (P <0.05). Inter-rater agreement was measured using the intraclass correlation coefficient (ICC). For the ANTS instrument, the intraclass correlation coefficients for task management, team-working, situation awareness and decision-making were 0.79, 0.34, 0.81 and 0.70, respectively. For the Ottawa GRS, the intraclass correlation coefficients for overall performance, leadership, problem-solving, situation awareness, resource utilisation and communication skills were 0.86, 0.83, 0.84, 0.87, 0.80 and 0.86, respectively. The Cronbach's alpha for internal consistency of the ANTS instrument was 0.93, and was 0.96 for the Ottawa GRS. There was a high correlation between the ANTS and Ottawa GRS. The raters reported the ease of use of the Ottawa GRS compared to the ANTS. We found sufficient evidence of validity in the ANTS instrument and the Ottawa GRS for the evaluation of non-technical skills in a simulated anaesthesia setting, but the Ottawa GRS was more practical and had higher reliability. [ABSTRACT FROM AUTHOR]/nCopyright of Anaesthesia & Intensive Care is the property of Australian Society of Anaesthetists and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Statistical analysis was conducted using PASW Statistics.Cronbach’s alpha was used to assess the internal consistency of the test. The scores obtained using the ANTS and Ottawa GRS instruments were analysed for all three participant groups, using one-way analysis of variance, the F-test and Bonferroni multiple comparisons
Simulation
LeadershipProblem-solvingSituation awarenessResource utilizationCommunication
The simulation laboratory was configured as an operating room with a full-body manikin patient simulator (SimMan 3G; Laerdal Medical, Stavanger, Norway).Participants were briefed about the and the equipment before thestart of the session. The participant was assigned the role of the anaesthetist in charge of a 45-year-old woman (American Society of Anesthesiologists physical status classification 1) undergoing laparoscopic myomectomy.
The study included 70 participants, composed of 24 PGY-1, 24 PGY-2 and 22 PGY-3 residents
Evaluate the validity of two non-technical skills instruments, the Anaesthetists’ Non-Technical Skills (ANTS) behavioural marker system and the Ottawa Global Rating Scale (GRS), to apply them to anaesthesia .
Although the use of simulation is beneficial in non-technical skills training25 , there is limited evidence for its role in . Care should be taken with the design, configuration of the simulation session and the process.
The ANTS and Ottawa GRS instruments shared various aspects of validity evidence and correlated well with each other in anaesthesia in the simulation.
Study protocol was approved by the Ethical Committee of Siriraj Institutional Review Board (SI170/2015)
Simulated management sessions were initiated during which two trained raters evaluated the performance of postgraduate first-, second- and third-year (PGY-1, PGY-2 and PGY-3) anaesthesia residents.A was developed to test the residents’ nontechnicalskills. An unexpected cardiac arrest during emergence from anaesthesia was chosen, as cardiac arrest management is performed according to standard guidelines that anaesthesia residents are expected to know.
 |
Portfolio of Solutions web site has been initially developed in the scope of DRIVER+ project. Today, the service is managed by AIT Austrian Institute of Technology GmbH., for the benefit of the European Management. PoS is endorsed and supported by the Disaster Competence Network Austria (DCNA) as well as by the STAMINA and TeamAware H2020 projects. |