The American Journal of Surgery
Volume 199, Issue 1 , Pages 110-114, January 2010

Reliability of a procedural checklist as a high-stakes measurement of advanced technical skill

  • Sarah E. Peyré, Ed.D.

      Affiliations

    • Department of Surgery, Brigham and Women's Hospital, 75 Francis Street, Neville House, Boston, MA 02115
    • Corresponding Author InformationCorresponding author: Tel.: +1-617-595-9588; fax: +1-617-264-6333
    • ,
  • Christian G. Peyré, M.D.

      Affiliations

    • Department of Surgery, Brigham and Women's Hospital, 75 Francis Street, Neville House, Boston, MA 02115
    • ,
  • Jeffrey A. Hagen, M.D.

      Affiliations

    • Department of Surgery, Keck School of Medicine, At the University of Southern California, Los Angeles, CA, USA
    • ,
  • Maura E. Sullivan, Ph.D.

      Affiliations

    • Department of Surgery, Keck School of Medicine, At the University of Southern California, Los Angeles, CA, USA

Received 4 June 2009; received in revised form 10 August 2009

Abstract 

Background

The purpose of this study was to determine the reliability of a previously validated laparoscopic Nissen fundoplication procedural checklist as a measurement of advanced technical skill.

Methods

Five surgeons, using a 65-step procedural checklist, independently evaluated 2 video recordings of expert surgeon operative performances of a laparoscopic Nissen fundoplication. Results were analyzed for percent agreement and Fleiss kappa correlation for each operation independently and combined as a whole.

Results

Sixty-four of the 65 steps had 80% or higher percent agreement for both operative case A and B independently and when considered overall. The Fleiss kappa coefficients for operative case A (kappa = .95) and operative case B (kappa = .96) and overall (operative case A + B) (kappa = .95).

Conclusion

The percentage agreement and kappa coefficients for all 3 analyses indicate a high degree of reliability (>.80) that would support the use of this instrument for high-stakes assessment of a laparoscopic Nissen fundoplication.

Keywords: Reliability, Assessment, Technical skill, Procedural checklist

To access this article, please choose from the options below

Login to an existing account or Register a new account.

  • Purchase this article for 31.50 USD (You must login/register to purchase this article)

    Online access for 24 hours. The PDF version can be downloaded as your permanent record.

  • Subscribe to this title

    Get unlimited online access to this article and all other articles in this title 24/7 for one year.

  • Claim access now

    For current subscribers with Society Membership or Account Number.

  • Visit SciVerse ScienceDirect to see if you have access via your institution.
 

PII: S0002-9610(09)00621-7

doi:10.1016/j.amjsurg.2009.10.003

The American Journal of Surgery
Volume 199, Issue 1 , Pages 110-114, January 2010

Article Tools