The American Journal of Surgery
Volume 199, Issue 1 , Pages 110-114 , January 2010

Reliability of a procedural checklist as a high-stakes measurement of advanced technical skill

  • Sarah E. Peyré, Ed.D.

      Affiliations

    • Department of Surgery, Brigham and Women's Hospital, 75 Francis Street, Neville House, Boston, MA 02115
    • Corresponding Author InformationCorresponding author: Tel.: +1-617-595-9588; fax: +1-617-264-6333
    • ,
  • Christian G. Peyré, M.D.

      Affiliations

    • Department of Surgery, Brigham and Women's Hospital, 75 Francis Street, Neville House, Boston, MA 02115
    • ,
  • Jeffrey A. Hagen, M.D.

      Affiliations

    • Department of Surgery, Keck School of Medicine, At the University of Southern California, Los Angeles, CA, USA
    • ,
  • Maura E. Sullivan, Ph.D.

      Affiliations

    • Department of Surgery, Keck School of Medicine, At the University of Southern California, Los Angeles, CA, USA

Received 4 June 2009 ,Revised 10 August 2009

References 

  1. Bhatti NI, Cummings CW. Viewpoint: competency in surgical residency training: defining and raising the Bar. Acad Med. 2007;82:569–573
  2. Gallagher AG, Ritter EM, Satava RM. Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc. 2003;17:1525–1529
  3. Salkind N. Tests and Measurement for People Who (Think They) Hate Tests and Measurement. Thousand Oaks, CA: Sage Publications; 2006;
  4. APA/AREA/NCME. Standards for Educational and Psychological Testing. Washington, DC: APA; 1999;
  5. Downing SM. Reliability: on the reproducibility of assessment data. Med Educ. 2004;38:1006–1012
  6. Peyre S, Peyre C, Hagen J, et al. Laparoscopic Nissen fundoplication assessment: task analysis as a model for the development of a procedural checklist. Surg Endosc. 2009;23:1227–1232
  7. Fleiss JL. Measuring nominal scale agreement among many raters. Psychological Bulletin. 1971;76:378–382
  8. Brasel KJ, Bragg D, Simpson DE, et al. Meeting the Accreditation Council for Graduate Medical Education competencies using established residency training program assessment tools. Am J Surg. 2004;188:9–12
  9. Scott D, Valentine R, Bergen P, et al. Evaluating surgical competency with the American Board of Surgery In-training examination, skill testing, and intraoperative assessment. Surgery. 2000;128:613–622
  10. Tavakol M, Mohagheghi MA, Dennick R. Assessing the skills of surgical residents using simulation. J Surg Educ. 2008;65:77–83
  11. Reznick R, Regehr G, MacRae H, et al. Testing technical skill via an innovative “bench station” examination. Am J Surg. 1997;173:226–230
  12. Ault G, Reznick R, MacRae H, et al. Exporting a technical skills evaluation technology to other sites. Am J Surg. 2001;182:254–256
  13. Willems MC, van der Vliet JA, Williams V, et al. Assessing endovascular skills using the simulator for testing and rating endovascular skills (STRESS) machine. Eur J Vasc Endovasc Surg. 2009;37:431–436
  14. Bann S, Khan M, Datta V, et al. Surgical skill is predicted by the ability to detect errors. Am J Surg. 2005;189:412–415
  15. Beard JD, Choksy S, Khan S. Assessment of operative competence during carotid endarterectomy. Br J Surg. 2007;94:726–730
  16. Larson JL, Williams RG, Ketchum J, et al. Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents. Surgery. 2005;138:640–649
  17. Aggarwal R, Grantcharov T, Moorthy K, et al. Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room. Ann Surg. 2008;247:372–379

PII: S0002-9610(09)00621-7

doi: 10.1016/j.amjsurg.2009.10.003

The American Journal of Surgery
Volume 199, Issue 1 , Pages 110-114 , January 2010

Article Tools