Comparison of resident self-assessments with trained faculty and standardized patient assessments of clinical and technical skills in a structured educational module

Brewster, Luke P.; Risucci, Donald A.; Joehl, Raymond J.; Littooy, Fred N.; Temeck, Barbara K.; Blair, Patrice Gabler; Sachdeva, Ajit K.

doi:10.1016/j.amjsurg.2007.08.048

« Previous Next »The American Journal of Surgery
Volume 195, Issue 1 , Pages 1-4, January 2008

Comparison of resident self-assessments with trained faculty and standardized patient assessments of clinical and technical skills in a structured educational module

Presented at the Annual Meeting of the Association for Surgical Education, Washington, DC, April 11–14, 2007

Luke P. Brewster, M.D., Ph.D., M.A.
- Loyola University Medical Center, Department of Surgery, Maywood, IL, USA
- Edward Hines, Jr. VA Hospital, Surgical Services, Hines, IL, USA
Donald A. Risucci, Ph.D.
- New York Medical College, Department of Surgery, Valhalla, NY, USA
Raymond J. Joehl, M.D., F.A.C.S.
- Loyola University Medical Center, Department of Surgery, Maywood, IL, USA
- Edward Hines, Jr. VA Hospital, Surgical Services, Hines, IL, USA
Fred N. Littooy, M.D., F.A.C.S.
- Loyola University Medical Center, Department of Surgery, Maywood, IL, USA
- Edward Hines, Jr. VA Hospital, Surgical Services, Hines, IL, USA
Barbara K. Temeck, M.D., F.A.C.S.
- Edward Hines, Jr. VA Hospital, Surgical Services, Hines, IL, USA
Patrice Gabler Blair, M.P.H.
- Division of Education, American College of Surgeons, Chicago, IL, USA
Ajit K. Sachdeva, M.D., F.R.C.S.C., F.A.C.S.
- Edward Hines, Jr. VA Hospital, Surgical Services, Hines, IL, USA
- Division of Education, American College of Surgeons, Chicago, IL, USA
- Corresponding author. Tel.: +1-312-202-5405; fax: +1-312-202-5011.

Received 24 April 2007; received in revised form 24 August 2007

Abstract

Background

This study assessed the reliability of surgical resident self-assessment in comparison with faculty and standardized patient (SP) assessments during a structured educational module focused on perioperative management of a simulated adverse event.

Methods

Seven general surgery residents participated in this module. Residents were assessed during videotaped preoperative and postoperative SP encounters and when dissecting a tumor off of a standardized inanimate vena cava model in a simulated operating room.

Results

Preoperative and postoperative assessments by SPs correlated significantly (P < .05) with faculty assessments (r = .75 and r = .79, respectively), but not resident self-assessments. Coefficient alpha was greater than .70 for all assessments except resident preoperative self-assessments.

Conclusions

Faculty and SP assessments can provide reliable data useful for formative feedback. Although resident self-assessment may be useful for the formative assessment of technical skills, results suggest that in the absence of training, residents are not reliable self-assessors of preoperative and postoperative interactions with SPs.

Keywords: Simulation, Surgical education, Breaking bad news, Reliability, Inter-rater agreement