Keywords
Surgical training has been greatly affected by the COVID-19 pandemic with limited opportunities and learning experiences for trainees in the operating theatre.
1
During recent years, there has also been a shift from time-based and numbers-based training to competency-based training, as demonstrated by the new ISCP curriculum.2
As such, more than ever, there is pressure for surgical trainers and trainees alike to maximise every learning experience.ISCP surgical curriculum.
2021
https://www.iscp.ac.uk/iscp/curriculum-2021/
Date accessed: September 19, 2022
Simulation training is an effective training opportunity and has been increasingly used to fill the void created by the pandemic.
3
With the increased demand for simulation training from trainees, comes an increased need for confident and competent simulation trainers. The Non-Technical Skills of a Technical Skills Trainer (NTS-TeST) is a tool developed to aid in developing trainers’ operative teaching skills. It is an 18-point assessment tool with 4 subcategories: Pre-skill training, during skill training, post skill training and a global rating. These are scored out of 25, 40, 10 and 15 points, respectively, combined to give an overall score of 90. It is designed to evaluate trainers in a simulation context and allows trainers to reflect upon and improve their teaching skills.4
As such, this tool could be of significant utility to trainers and trainees alike if shown to .Cognitive Mental Load Theory is a framework within educational research that is used to evaluate learning environments.
5
It is based on the principle that when dealing with new information, working memory has a limited capacity and when this capacity is surpassed, learning is impaired. Recently, validated measurement tools have been developed to allow the measurement of cognitive load during simulation training. One such tool developed by Klepsch et al. is an 8-point assessment tool, scored out of 80.6
Here we assess the utility of NTS-TeST in the context of a formal surgical skills simulation session, to improve teaching quality for trainers and trainees alike and reduce the cognitive load on trainees.The study was conducted during the 6th Urology Simulation Boot Camp in Leeds, UK. It is a 5-day introductory hands-on course for the new urology registrars.
7
,8
Twenty-four urology residents took part in a technical skill simulation session comprised of different stations supervised by 12 consultants (trainers) in the study. The first trainee at each station was asked to complete the NTS-TeST score and a Cognitive Mental Load score after completing the technical skills training session following the first encounter with a trainer. Trainers were not aware of this part. Each trainer also completed the NTS-TeST sheet (Supplementary File) and provided a self-rating of their first training episode. The second trainee at the same station with the same trainer again completed the NTS-TeST and a Cognitive Mental Load score after the training episode. Some of the teaching episodes were observed by expert trainers, and tips were shared. Trainers completed a self-rating of their teaching using the NTS-TeST as a checklist to improve teaching quality on their next station run. NTS-TeST scores for trainees and trainers pre- and post-checklist implementation were compared, and statistical significance was assessed using the unpaired t-test. The same was done for trainees’ Cognitive Mental Load scores.Pre- and post-checklist implementation trainers gave mean NTS-TeST self-ratings of 69.75 ± 2.54 and 77.50 ± 2.57 respectively (p = 0.0431). Pre- and post-checklist implementation Trainees gave mean NTS-TeST scores of 86.55 ± 0.87 and 86.58 ± 1.17 respectively (p = 0.9798). Pre- and post-checklist implementation trainees gave mean cognitive mental load scores of 42.18 ± 3.56 and 38.40 ± 2.42 respectively (p = 0.4002). These results are summarised in Table 1 below. Subcategory scores (Pre, During, Post, Global) for trainers' pre and post-checklist implementation and subcategory scores (Pre, During, Post, Global) for trainees’ pre and post-checklist implementation are summarised in Table 2.
Table 1Trainers and trainees' NTS-TeST rating and trainees' cognitive mental load scores pre- and post-checklist implementation.
| Trainer NTS-TeST, Mean ( ± SE) | Trainee NTS-TeST, Mean ( ± SE) | Trainee Cognitive Mental Load, Mean ( ± SE) | |
|---|---|---|---|
| Pre-checklist | 69.75 (2.54) | 86.55 (0.87) | 42.18 (3.56) |
| Post-checklist | 77.50 (2.57) | 86.58 (1.17) | 38.40 (2.42) |
| Unpaired t- test | p = 0.0431 | p = 0.9798 | P = 0.4002 |
Table 2Trainers’ and trainees, subcategory scores (Pre, During, Post, Global) pre- and post-checklist implementation.
| Trainers | NTS-TeST Pre-skills training, Mean ( ± SE) | NTS-TeST During skills training, Mean ( ± SE) | NTS-TeST Post-skills training, Mean ( ± SE) | NTS-TeST Global Rating, Mean ( ± SE) |
|---|---|---|---|---|
| Pre-checklist | 17.00 (1.00) | 32.00 (1.09) | 8.08 (0.47) | 12.58 (0.56) |
| Post-checklist | 20.67 (1.08) | 34.42 (1.26) | 9.08 (0.38) | 13.08(0.56) |
| Unpaired t- test | p = 0.0205 | p = 0.1613 | p = 0.1109 | p = 0.5320 |
| Trainees | NTS-TeST Pre- skills training ( ± SE) | NTS-TeST During- skills training ( ± SE) | NTS-TeST Post- skills training ( ± SE) | NTS-TeST Global Rating ( ± SE) |
| Pre-Checklist | 23.91 (0.58) | 39.00 (0.73) | 9.91 (0.09) | 13.36 (0.58) |
| Post-Checklist | 23.67 (0.57) | 38.75 (0.49) | 9.75 (0.18) | 14.42 (0.26) |
| Unpaired t- test | p = 0.7684 | p = 0.7755 | p = 0.4505 | p = 0.1012 |
Trainers noted an increase in the perceived quality of their teaching following the implementation of the checklist, as demonstrated by the increase in their NTS-TeST self-rating scores by 7.75. This represented an increase of 11.1% and was found to be statistically significant. When looking at the subcategories, the main area trainers felt they had improved was in the pre-skills training domain. As such, implementing the NTS-TeST as a checklist for trainers appears to be effective in increasing the trainer's perception of the quality of their teaching in the simulation setting, particularly in the pre-skill stage. This is clearly of use, as improving trainer confidence in simulation teaching will become increasingly important given the increased utilisation of simulation training within surgical teaching.
4
,6
However, this was not the case for trainees, with their perceived quality of teaching having no statistically significant difference despite the implementation of the checklist regardless of the NTS-TeST subcategory. As a result, whilst NTS-TeST seems to be a useful tool for trainers to reflect on their performance and make changes that they feel have improved it, trainees did not have the same perception. Interestingly, trainees gave very high overall NTS-TeST feedback scores before the checklist's implementation. This left little room for improvement with such a high initial score and demonstrated that trainees felt that the overall quality of the teaching was high both before and after checklist implementation. As to why trainees scored the teaching much higher than trainers in both pre-and post-checklist implementation, it may be that trainees were less objective about the teaching quality as they were pleased to be having the opportunity at all given the reduction in training experiences during the pandemic.
7
As a result, they rated the teaching highly regardless of its quality, while trainers who were more experienced were able to be more objective and therefore more critical of their teaching. It seems that, whilst utilising NTS-TeST as a checklist for trainers may be promising in reducing trainee cognitive mental load, this study has not definitively proven it.We acknowledge that only endoscopic and laparoscopic simulations were used to examine the two types of surgical skills. In addition, it is too early to assess whether the improvement noted by trainers was educationally significant. It would be beneficial to check in the future to determine if open skills simulation settings may yield the same encouraging results. It would also be valuable to see if this can be extended beyond the simulation setting and applied to a clinical setting, such as a training list in the operating theatre.
Appendix A. Supplementary data
The following is the Supplementary data to this article:
- Multimedia component 1
References
- Disruption to surgical training during covid-19 in the United States, United Kingdom, Canada, and australasia: a rapid review of impact and mitigation efforts.J Surg Educ. 2021; 78: 308-314
- ISCP surgical curriculum.2021https://www.iscp.ac.uk/iscp/curriculum-2021/Date accessed: September 19, 2022
- Doing more with less: surgical training in the COVID-19 era.J Invest Surg. 2022; 35: 171-179
- Capturing the non-technical skills of a technical skills trainer (NTS-TeST) during simulation.Scot Med J. 2021; 66: 124-133
- Evidence for cognitive load theory.Cognit InStruct. 1991; 8: 351-362https://doi.org/10.1207/s1532690xci0804_5
- Going the (social) distance: comparing the effectiveness of online versus in-person internal jugular central venous catheterization procedural training.Am J Surg. 2022 Sep; 224: 903-907
- The success and evolution of a urological "Boot camp" for newly appointed UK urology registrars: incorporating simulation, nontechnical skills and assessment.J Surg Educ. 2019 Sep-Oct; 76: 1425-1432
- A systematic review of methodological principles and delivery of surgical simulation bootcamps.Am J Surg. 2022 Jun; 223: 1079-1087
Article info
Publication history
Published online: February 10, 2023
Accepted:
February 9,
2023
Received:
February 9,
2023
Publication stage
In Press Journal Pre-ProofIdentification
Copyright
© 2023 Published by Elsevier Inc.
