Background
Surgical training in the simulation lab can develop basic skills that translate to
the operating room. Standardized, basic skills programs that are supported by validated
assessment measures exist for open, laparoscopic, and endoscopic surgery; however,
there is yet to be a nationally recognized and widely implemented basic skills program
specifically for vaginal surgery.
Objective
Develop a vaginal surgical simulation system; evaluate robust validity evidence for
the simulation system and its related performance measures; and establish a proficiency
score that discriminates between novice and experienced vaginal surgeon performance.
Study Design
In this 3-phased study, we developed the Fundamentals of Vaginal Surgery simulation
system consisting of (1) the Fundamentals of Vaginal Surgery Trainer, a task trainer; (2) a validated regimen of tasks to be performed on the trainer;
and (3) performance measures to determine proficiency. In Phase I, we developed the
task trainer and selected surgical tasks by performing a needs assessment and hierarchical
task analyses, with review and consensus from an expert panel. In Phase II, we conducted
a national survey of vaginal surgeons to collect validity evidence regarding test
content, response process, and internal structure relevant to the simulation system.
In Phase III, we compared performance of novice (first and second year residents)
and experienced (third and fourth year residents, fellows, and faculty) surgeons on
the simulation system to evaluate relevant relationships to other variables and consequences.
Performance measures were analyzed to set a proficiency score that would discriminate
between novice and expert (faculty) vaginal surgical performance.
Results
A novel task trainer and 6 basic vaginal surgical skills were developed in Phase I.
In Phase II, the survey responses of 48 participants (27 faculty surgeons, 6 fellows,
and 14 residents) were evaluated on the dimensions of test content, response process,
and internal structure. To support evidence of test content, the participants deemed
the task trainer and surgical tasks representative of intended surgical field and
supportive of typical surgical actions (mean scores, 3.8–4.4/5). For response process,
rater-data analysis revealed high rating variability regarding prototype color. This
early evidence confirmed the value of a white prototype. For internal structure, there
was high agreement among rater groups (obstetricians and gynecologists generalists
vs Female Pelvic Medicine and Reconstructive Surgery specialists: interclass correlation
coefficient range, 0.59–0.91; learners vs faculty interclass correlation coefficient
range, 0.64–1.0). There were no differences in ratings across institution type, surgeon
volume, expertise (P>.14). In Phase III, we analyzed performance from 23 participants (15 [65%] obstetricians
and gynecologists residents, 3 [13%] fellows, and 5 [22%] Female Pelvic Medicine and
Reconstructive Surgery faculty). Experienced surgeons scored significantly higher
than novice surgeons (median, 467.5; interquartile range, [402.5–542.5] vs median,
261.5; interquartile range, [211.5–351.0]; P<.001). Based on these data, setting a proficiency score threshold at 400 results
in 0% (0/6) novices attaining the score, with 100% (5/5) experts exceeding it.
Conclusion
We present validity evidence relevant to all 5 sources which supports the use of this
novel simulation system for basic vaginal surgical skills. To complement the system,
a proficiency score of 400 was established to discriminate between novices and experts.
Key words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to American Journal of Obstetrics & GynecologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- The nationwide frequency of major adverse outcomes in antireflux surgery and the role of surgeon experience, 1992-1997.J Am Coll Surg. 2002; 195: 611-618
- Patient safety, competency, and the future of surgical simulation.Simul Healthc. 2006; 1: 164-170
- Advancing competency-based medical education: a charter for clinician-educators.Acad Med. 2016; 91: 645-649
- Simulation in surgical education.Clin Colon Rectal Surg. 2012; 25: 156-165
- Implementation, construct validity, and benefit of a proficiency-based knot-tying and suturing curriculum.J Surg Educ. 2008; 65: 309-315
- Development, validation, and implementation of a cost-effective intermediate-level proficiency-based knot-tying and suturing curriculum for surgery residents.J Surg Educ. 2013; 70: 193-199
- FLS and FES: comprehensive models of training and assessment.Surg Clin North Am. 2010; 90: 535-558
- Development and evaluation of a simulator-based laparoscopic training program for surgical novices.Surg Endosc. 2013; 27: 214-221
- Trends in reported resident surgical experience in hysterectomy.J Minim Invasive Gynecol. 2014; 21: 1067-1070
Altman KB, Burrell D, Chen G, Chou B, Fashokun T, Vaginal Hysterectomy Simulation. ACOG Simulation Working Group. Available at: https://www.acog.org/education-and-events/simulations/scog008/simulation. Accessed March 1, 2021.
- Vaginal hysterectomy suturing skills training model and curriculum.Obstet Gynecol. 2019; 134: 553-558
- A novel and inexpensive vaginal hysterectomy simulator.Simul Healthc. 2012; 7: 374-379
- Validation of transvaginal hysterectomy surgical model - modification of the flowerpot model to improve vesicovaginal plane simulation.J Minim Invasive Gynecol. 2021; 28: 1526-1530
- Establishing validity for a vaginal hysterectomy simulation model for surgical skills assessment.Obstet Gynecol. 2020; 136: 942-949
- Teaching vaginal surgery to junior residents: initial validation of 3 novel procedure-specific low-fidelity models.J Surg Educ. 2016; 73: 157-161
- Development and validation of simulation training for vaginal hysterectomy.J Minim Invasive Gynecol. 2014; 21: 74-82
- Validation of an educational simulation model for vaginal hysterectomy training: a pilot study.Int Urogynecol J. 2019; 30: 1329-1336
- Validation of the simulated vaginal hysterectomy trainer.J Minim Invasive Gynecol. 2018; 25: 1101-1106
- Multi-institutional validation of a vaginal hysterectomy simulation model for resident training.J Minim Invasive Gynecol. 2021; 28: 1490-1496.e1
- Assessing current trends in resident hysterectomy training.Female Pelvic Med Reconstr Surg. 2011; 17: 210-214
- Standards for educational and psychological testing. 2009.(Available at:) (Accessed March 1, 2021)
- Validity.in: Linn R. Educational measurement. 3rd ed. American Council on Education and Macmillan, New York, NY1989: 13-103
- What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment.Adv Health Sci Educ Theory Pract. 2014; 19: 233-250
- Reporting guidelines for health care simulation research: extensions to the CONSORT and STROBE statements.Simul Healthc. 2016; 11: 238-248
- Baseline dimensions of the human vagina.Hum Reprod. 2006; 21: 1618-1622
- Curriculum development for medical education: a six-step approach. Vol Baltimore, Md.2nd ed. Johns Hopkins University Press, Baltimore, MD2009
- Constructing hierarchical task analysis in surgery.Surg Endosc. 2008; 22: 107-111
- Basic exercises in surgical technique.Davis and Geck, Wayne, NJ1987
- Development of a model for training and evaluation of laparoscopic skills.Am J Surg. 1998; 175: 482-487
- Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS system.Surg Endosc. 2003; 17: 964-967
- Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery.Surg Innov. 2007; 14: 107-112
- Conception of the Lübeck Toolbox Curriculum for basic minimally invasive surgery skills.Langenbecks Arch Surg. 2018; 403: 271-278
- Many-facet Rasch management.MESA Press, Chicago, IL1989
- Teaching psychomotor skills in the twenty-first century: revisiting and reviewing instructional approaches through the lens of contemporary literature.Med Teach. 2016; 38: 1056-1063
- A cost-effective proficiency-based knot-tying and suturing curriculum for residency programs.J Surg Res. 2007; 141: 7-15
- Essentials in minimally invasive gynecology manual skills construct validation trial.Obstet Gynecol. 2020; 136: 83-96
- Small-sample precision of ROC-related estimates.Bioinformatics. 2010; 26: 822-830
Article info
Publication history
Published online: August 28, 2021
Accepted:
August 24,
2021
Received in revised form:
August 4,
2021
Received:
March 1,
2021
Footnotes
The authors report no conflict of interest.
This project received financial support from the University of Michigan Clinical Simulation Center Research Grant (number U063128), awarded August 15, 2018. The funder played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.
Cite this article as: Schmidt PC, Fairchild PS, Fenner DE, et al. The Fundamentals of Vaginal Surgery pilot study: developing, validating, and setting proficiency scores for a vaginal surgical skills simulation system. Am J Obstet Gynecol 2021;225:558.e1-11.
Identification
Copyright
© 2021 Elsevier Inc. All rights reserved.
