The development and measurement properties of the trauma NOn-TECHnical skills (T-NOTECHS) scale: A scoping review

  • Christina Stevenson
    Corresponding author. Division of Emergency Medicine, Department of Medicine, University of Toronto, 1 King's College Cir, Toronto, ON, M5S 1A8, Canada.
    Division of Emergency Medicine, Department of Medicine, University of Toronto, 1 King's College Cir, Toronto, ON, M5S 1A8, Canada
    Search for articles by this author
  • Avneesh Bhangu
    School of Medicine, Faculty of Health Sciences, Queen's University, 15 Arch St, Kingston, ON, K7L 3L4, Canada
    Search for articles by this author
  • James J. Jung
    Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria St, Toronto, ON, M5B 1T8, Canada

    Department of Surgery, University of Toronto, 1 King's College Cir, Toronto, ON, M5S 1A8, Canada

    Codman Center of Clinical Effectiveness in Surgery, Massachusetts General Hospital, 55 Fruit St, Boston, MA, 02114, United States
    Search for articles by this author
  • Aidan MacDonald
    Department of Biological Sciences, Bishop's University, 2600 College St, Sherbrooke, Quebec, J1M 1Z7, Canada
    Search for articles by this author
  • Brodie Nolan
    Division of Emergency Medicine, Department of Medicine, University of Toronto, 1 King's College Cir, Toronto, ON, M5S 1A8, Canada

    Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria St, Toronto, ON, M5B 1T8, Canada

    Department of Emergency Medicine, St. Michael's Hospital, 36 Queen St E, Toronto, ON, M5B 1W8, Canada
    Search for articles by this author


      • T-NOTECHS was developed with an acceptable robust methodology.
      • The tool has good sensibility with adequate content, face validity, & feasibility.
      • Synthesis of evidence supports T-NOTECHS′ use as a discriminative & evaluative tool.
      • Ideal rater has clinical expertise in trauma care & simulation-based teaching.
      • Ideal setting to utilize T-NOTECHS is with trauma video review.



      Conduct a scoping review to critically appraise the development and summarize the evidence on the measurement properties of T-NOTECHS including sensibility, reliability, and validity.


      A literature search was performed using Pubmed and Ovid databases. Studies that described the development process of T-NOTECHS and primary studies that presented evidence of reliability and validity were identified and included. Measurement properties of T-NOTECHS was assessed and summarized under the following: scale development, sensibility, reliability, and validity.


      The literature search yielded 245 articles with 24 studies meeting inclusion criteria. The T-NOTECHS was developed with an acceptable robust methodology. It has good sensibility with adequate content, face validity, and feasibility. It is a reliable measure of non-technical skills in the setting of trauma video review, which improves with expert raters or extensive training. The T-NOTECHS is a valid discriminative and evaluative instrument that measures non-technical skills of multidisciplinary trauma teams.


      T-NOTECHS provides reliable and valid measurements of non-technical skills of trauma teams, particularly when assessing trauma video review and non-technical skills training interventions by expert raters.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The American Journal of Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Flin R.
        • O'Connor P.
        • Crichton M.
        Safety at the Sharp End: A Guide to Non-technical Skills.
        first ed. CRC Press, 2008
        • Hull L.
        • Arora S.
        • Aggarwal R.
        • Darzi A.
        • Vincent C.
        • Sevdalis N.
        The impact of nontechnical skills on technical performance in surgery: a systematic review.
        J Am Coll Surg. 2012; 214: 214-230
        • Gjeraa K.
        • Møller T.P.
        • Østergaard D.
        Efficacy of simulation-based trauma team training of non-technical skills. A systematic review.
        Acta Anaesthesiol Scand. 2014; 58: 775-787
        • Murphy M.
        • McCloughen A.
        • Curtis K.
        The impact of simulated multidisciplinary Trauma Team Training on team performance: a qualitative study.
        Australas Emerg Care. 2019; 22: 1-7
        • Nolan B.
        • Hicks C.M.
        • Petrosoniak A.
        • Jung J.
        • Grantcharov T.
        Pushing boundaries of video review in trauma: using comprehensive data to improve the safety of trauma care.
        Trauma Surg Acute Care Open. 2020; 5e000510
        • Capella J.
        • Smith S.
        • Philp A.
        • et al.
        Teamwork training improves the clinical care of trauma patients.
        J Surg Educ. 2010; 67: 439-443
        • Westli H.K.
        • Johnsen B.H.
        • Eid J.
        • Rasten I.
        • Brattebø G.
        Teamwork skills, shared mental models, and performance in simulated trauma teams: an independent group design.
        Scand J Trauma Resuscitation Emerg Med. 2010; 18: 47
        • Steinemann S.
        • Berg B.
        • DiTullio A.
        • et al.
        Assessing teamwork in the trauma bay: introduction of a modified “NOTECHS” scale for trauma.
        Am J Surg. 2012; 203: 69-75
        • Flin R.
        • Martin L.
        • Goeters K.M.
        • et al.
        Development of the NOTECHS (non-technical skills) system for assessing pilots' CRM skills.
        Hum Factors Aero Saf. 2003; 3: 97-119
        • Sevdalis N.
        • Davis R.
        • Koutantji M.
        • Undre S.
        • Darzi A.
        • Vincent C.A.
        Reliability of a revised NOTECHS scale for use in surgical teams.
        Am J Surg. 2008; 196: 184-190
        • Yule S.
        • Flin R.
        • Paterson-Brown S.
        • Maran N.
        • Rowley D.
        Development of a rating system for surgeons' non-technical skills.
        Med Educ. 2006; 40: 1098-1104
        • Bhangu A.
        • Stevenson C.
        • Szulewski A.
        • MacDonald A.
        • Nolan B.
        A scoping review of nontechnical skill assessment tools to evaluate trauma team performance.
        J Trauma Acute Care Surg. Published online December. 2021; 14
        • Pucher P.H.
        • Aggarwal R.
        • Batrick N.
        • Jenkins M.
        • Darzi A.
        Nontechnical skills performance and care processes in the management of the acute trauma patient.
        Surgery. 2014; 155: 902-909
        • Dumas R.P.
        • Vella M.A.
        • Chreiman K.C.
        • et al.
        Team assessment and decision making is associated with outcomes: a trauma video review analysis.
        J Surg Res. 2020; 246: 544-549
        • Briggs A.
        • Raja A.S.
        • Joyce M.F.
        • et al.
        The role of nontechnical skills in simulated trauma resuscitation.
        J Surg Educ. 2015; 72: 732-739
        • Sullivan S.
        • Campbell K.
        • Ross J.C.
        • et al.
        Identifying nontechnical skill deficits in trainees through interdisciplinary trauma simulation.
        J Surg Educ. 2018; 75: 978-983
        • Rosqvist E.
        • Lauritsalo S.
        • Paloneva J.
        Short 2-H in situ trauma team simulation training effectively improves nontechnical skills of hospital trauma teams.
        Scand J Surg SJS Off Organ Finn Surg Soc Scand Surg Soc. 2019; 108: 117-123
        • Tricco A.C.
        • Lillie E.
        • Zarin W.
        • et al.
        PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation.
        Ann Intern Med. 2018; 169: 467-473
        • Jung J.J.
        • Borkhoff C.M.
        • Jüni P.
        • Grantcharov T.P.
        Non-Technical Skills for Surgeons (NOTSS): critical appraisal of its measurement properties.
        Am J Surg. 2018; 216: 990-997
        • Fletcher G.
        • Flin R.
        • McGeorge P.
        • Glavin R.
        • Maran N.
        • Patey R.
        Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system.
        Br J Anaesth. 2003; 90: 580-588
        • Feinstein A.R.
        Clinimetric perspectives.
        J Chron Dis. 1987; 40: 635-640
        • Quirion A.
        • Nikouline A.
        • Jung J.
        • Nolan B.
        Contemporary uses of trauma video review: a scoping review.
        Can J Emerg Med. Published online August. 2021; 28
        • Mokkink L.B.
        • Terwee C.B.
        • Knol D.L.
        • et al.
        The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content.
        BMC Med Res Methodol. 2010; 10: 22
      1. Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. Oxford University Press Accessed August 15, 2021.

        • Cicchetti D.
        • Bronen R.
        • Spencer S.
        • et al.
        Rating scales, scales of measurement, issues of reliability: resolving some critical issues for clinicians and researchers.
        J Nerv Ment Dis. 2006; 194: 557-564
        • Huffman E.M.
        • Anton N.E.
        • Athanasiadis D.I.
        • et al.
        Multidisciplinary simulation-based trauma team training with an emphasis on crisis resource management improves residents' non-technical skills.
        Surgery. 2021; (Published online April 13) (S0039-6060(21)00207-5)
        • Lim Y.S.
        • Steinemann S.
        • Berg B.W.
        Team size impact on assessment of teamwork in simulation-based trauma team training.
        Hawaii J Med Public Health J Asia Pac Med Public Health. 2014; 73: 358-361
        • Repo J.P.
        • Rosqvist E.
        • Lauritsalo S.
        • Paloneva J.
        Translatability and validation of non-technical skills scale for trauma (T-NOTECHS) for assessing simulated multi-professional trauma team resuscitations.
        BMC Med Educ. 2019; 19: 40
        • Rosqvist E.
        • Ylönen M.
        • Torkki P.
        • Repo J.P.
        • Paloneva J.
        Costs of hospital trauma team simulation training: a prospective cohort study.
        BMJ Open. 2021; 11e046845
        • Sullivan S.
        • Warner-Hillard C.
        • Eagan B.
        • et al.
        Using epistemic network analysis to identify targets for educational interventions in trauma team communication.
        Surgery. 2018; 163: 938-943
        • Wieck M.M.
        • McLaughlin C.
        • Chang T.P.
        • et al.
        Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment.
        Am J Surg. 2018; 216: 630-635
        • Taylor S.
        • Avrith N.
        • Loo G.
        • et al.
        Impact of a focused trauma course on retention of provider skills, knowledge and confidence at a regional hospital in the Dominican Republic.
        Injury. Published online June. 2021; 17 (S0020-1383(21)00499-X)
        • Steinemann S.
        • Kurosawa G.
        • Wei A.
        • et al.
        Role confusion and self-assessment in interprofessional trauma teams.
        Am J Surg. 2016; 211: 482-488
        • van Maarseveen O.E.C.
        • Ham W.H.W.
        • Huijsmans R.L.N.
        • Dolmans R.G.F.
        • Leenen L.P.H.
        Reliability of the assessment of non-technical skills by using video-recorded trauma resuscitations.
        Eur J Trauma Emerg Surg Off Publ Eur Trauma Soc. 2020; 2 (Published online July)
        • Cohen D.
        • Sevdalis N.
        • Patel V.
        • et al.
        Tactical and operational response to major incidents: feasibility and reliability of skills assessment using novel virtual environments.
        Resuscitation. 2013; 84: 992-998
        • Pucher P.H.
        • Batrick N.
        • Taylor D.
        • Chaudery M.
        • Cohen D.
        • Darzi A.
        Virtual-world hospital simulation for real-world disaster response: design and validation of a virtual reality simulator for mass casualty incident management.
        J Trauma Acute Care Surg. 2014; 77: 315-321
        • Kirshner B.
        • Guyatt G.
        A methodological framework for assessing health indices.
        J Chron Dis. 1985; 38: 27-36
        • Jung J.J.
        • Yule S.
        • Boet S.
        • Szasz P.
        • Schulthess P.
        • Grantcharov T.
        Nontechnical skill assessment of the collective surgical team using the Non-Technical Skills for Surgeons (NOTSS) system.
        Ann Surg. 2020; 272: 1158-1163
        • Steinemann S.
        • Berg B.
        • Skinner A.
        • et al.
        In situ, multidisciplinary, simulation-based teamwork training improves early trauma care.
        J Surg Educ. 2011; 68: 472-477