Advertisement

Surgical skills assessment: A blinded examination of obstetrics and gynecology residents

      Abstract

      Objective: We have previously shown that objective structured assessment of technical skills (OSATS) is an innovative, reliable, and valid method of assessing surgical skills. Our goal was to establish the feasibility, reliability, and validity of our surgical skills assessment instrument when administered in a blinded fashion. Study design: A 7-station OSATS was administered to 16 obstetric and gynecology residents from Madigan Army Medical Center. The test included laparoscopic (salpingostomy, intracorporeal knot, and ligation of vessels with clips) and open abdominal procedures (subcuticular closure, bladder neck suspension, enterotomy repair, and abdominal wall closure). All tasks were performed with lifelike surgical models. Residents were timed and assessed at each station with 3 methods of scoring: task-specific checklist, global rating scale, and pass/fail grade. Each resident was evaluated by one examiner blinded as to the postgraduate year level and one examiner who had previously worked with the resident. Results: Assessment of construct validity (the ability to distinguish between resident levels) found significant differences on the checklist, global rating scale, and pass/fail grade by residency level for both blinded and unblinded examiners. Reliability indices calculated with Cronbach's α were .82 for the checklists and .93 for the global rating scale. Overall interrater reliability between blinded and unblinded examiners was 0.95 for global rating scale and ranged from 0.74 to 0.97 for the checklists. The cost to administer the exam for the 16 residents was approximately $1000. Conclusions: OSATS administered in either a blinded or unblinded fashion can assess residents' surgical skills with a high degree of reliability and validity. This study provides further evidence that OSATS can be used to establish surgical competence. (Am J Obstet Gynecol 2002;186:613-7.)

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to American Journal of Obstetrics & Gynecology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Reznick RK
        Teaching and testing technical skills.
        Am J Surg. 1993; 165: 358-361
        • Mandel LS
        • Lentz GM
        • Goff BA
        Teaching and evaluating surgical skills.
        Obstet Gynecol. 2000; 95: 783-785
        • Goff BA
        • Lentz GM
        • Lee D
        • Houmard B
        • Mandel LS
        Development of an objective structured assessment of technical skills for obstetric and gynecology residents.
        Obstet Gynecol. 2000; 96: 146-150
        • Goff BA
        • Lentz GM
        • Lee D
        • Fenner D
        • Morris J
        • Mandel LS
        Development of a bench station objective structured assessment of technical skills.
        Obstet Gynecol. 2001; 98: 412-416
        • Reznick R
        • Regehr G
        • MacRae H
        • Martin J
        • McCulloch W
        Testing technical skill via an innovative “bench station” examination.
        Am J Surg. 1997; 173: 226-230
        • Martin JA
        • Regehr G
        • Reznick R
        • MacRae H
        • Murnaghan J
        • Hutchison C
        • Brown M
        Objective structured assessment of technical skill (OSATS) for surgical residents.
        Br J Surg. 1997; 84: 273-278
        • Winckel CP
        • Reznick RK
        • Cohen R
        • Taylor B
        Reliability and construct validity of a structured technical skills assessment form.
        Am J Surg. 1994; 167: 423-427
        • Goff BA
        • Lentz GM
        • Lee DM
        • Mandel LS
        Formal teaching of surgical skills in an obstetric-gynecology residency.
        Obstet Gynecol. 1999; 93: 785-790
        • Chung JY
        • Sackier JM
        A method of objectively evaluating improvements in laparoscopic skills.
        Surg Endosc. 1998; 12: 1111-1116
        • Lentz GM
        • Mandel LS
        • Lee D
        • Melville J
        • Gardella C
        • Goff BA
        Testing surgical skills of obstetric and gynecology residents in a bench lab setting: validity and interrater reliability.
        Am J Obstet Gynecol. 2001; 184: 1462-1468