Advertisement

Impact of a labor and delivery safety bundle on a modified adverse outcomes index

Published:January 20, 2016DOI:https://doi.org/10.1016/j.ajog.2016.01.155

      Background

      The Obstetrics Adverse Outcomes Index was designed to measure the quality of perinatal care and includes 10 adverse events that may occur at or around the time of delivery. We hypothesized that adverse outcomes in the labor and delivery suite, including hypoxic ischemic encephalopathy, could be decreased with a combination of interventions, even among high-risk pregnancies.

      Objective

      The objective of the study was to evaluate the impact of a labor and delivery care bundle on adverse obstetrics outcomes as measured by a modified Obstetrics Adverse Outcomes Index, Weighted Adverse Outcomes Index, and Severity Index.

      Study Design

      This is a retrospective cohort study including all women who delivered at our academic, tertiary care institution over a 3 year period of time, before and after the implementation of an intervention to decrease adverse outcomes. Outcome measures consisted of previously reported indices that were modified including the addition of hypoxic ischemic encephalopathy. The adverse outcomes index is a percentage of deliveries with 1 or more adverse events, the weighted adverse outcomes index is the sum of the points assigned to cases with adverse outcomes divided by the number of deliveries, and the severity index is the sum of the adverse outcome scores divided by the number of deliveries with an identified adverse outcome. A segmented regression analysis was utilized to evaluate the differences in the level and trend of each index before and after our intervention using calendar month as the unit of analysis.

      Results

      During the study period, 5826 deliveries met inclusion criteria. Comparing the pre- and postintervention periods, high-risk pregnancy was more common in the postintervention period (73.5% vs 79.4%, P < .001). Overall, there was a decrease in both the Modified Weighted Adverse Outcomes Index (P = .0497) and the Modified Severity Index (P = 0.01) comparing the pre- and postintervention periods; there was no difference in the Modified Adverse Outcomes Index (P = .43). For low-risk pregnancies, there was no significant difference in the levels for any of the measured indices over the study period (P = .61, P = .41, and P = .34 for the Modified Adverse Outcomes Index, Modified Weighted Adverse Outcomes Index, and Modified Severity Index, respectively). Among the high-risk pregnancies, the monthly Modified Weighted Adverse Outcomes Index decreased by 4.2 ± 1.8 (P = .03). The monthly Modified Severity Index decreased by 53.9 ± 17.7 points from the pre- to the postintervention periods (P = .01) and was < 50% of the predicted Modified Severity Index had the intervention not been implemented. The cesarean delivery rate was increasing prior to the intervention, but the rate was stable after the intervention, and the absolute rate did not differ between the pre- and the postintervention periods (28.4% vs 30.0%, P = .20).

      Conclusion

      Overall and for high-risk pregnancies, the implementation of the labor and delivery care bundle had a positive impact on the Modified Weighted Adverse Outcomes Index and Modified Severity Index but not the Modified Adverse Outcomes Index.

      Key words

      Recently, attention has been focused on establishing and reporting obstetrics quality indicators.
      • Friedman A.M.
      • Ananth C.V.
      • Prendergast E.
      • D'Alton M.E.
      • Wright J.D.
      Evaluation of third-degree and fourth-degree laceration rates as quality indicators.
      • Howell E.A.
      • Zeitlin J.
      • Hebert P.L.
      • Balbierz A.
      • Egorova N.
      Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity.
      • Walker S.
      • Strandjord T.P.
      • Benedetti T.J.
      In search of perinatal quality outcome measures: 1 hospital's in-depth analysis of the Adverse Outcomes Index.
      In 2006, Mann et al
      • Mann S.
      • Pratt S.
      • Gluck P.
      • et al.
      Assessing quality obstetrical care: development of standardized measures.
      published a set of 10 outcome measures and 3 quality improvement tools. The 3 quality improvement tools developed included the Adverse Outcomes Index (AOI), Weighted Adverse Outcomes Index (WAOI), and the Severity Index (SI). The AOI is the percentage of deliveries with 1 or more adverse events, the WAOI is the adverse score per delivery (the sum of the points assigned to cases with adverse outcomes divided by the number of deliveries), and the SI designates the severity of the outcomes (sum of the adverse outcome scores divided by the number of deliveries with an identified adverse outcome). Some but not all of the proposed measures have been endorsed by the National Quality Forum.

      National Quality Forum. National voluntary consensus standards for perinatal care 2008: a consensus report. Available at: http://www.qualityforum.org/Publications/2009/05/National_Voluntary_Consensus_Standards_for_Perinatal_Care_2008.aspx. Accessed March 31, 2015.

      Previous studies have evaluated the AOI including the impact of specific interventions on the AOI.
      • Howell E.A.
      • Zeitlin J.
      • Hebert P.L.
      • Balbierz A.
      • Egorova N.
      Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity.
      • Foglia L.M.
      • Nielsen P.E.
      • Hemann E.A.
      • et al.
      Accuracy of the Adverse Outcome Index: An obstetrical quality measure.
      • Pettker C.M.
      • Thung S.F.
      • Norwitz E.R.
      • et al.
      Impact of a comprehensive patient safety strategy on obstetric adverse events.
      Pettker et al
      • Pettker C.M.
      • Thung S.F.
      • Norwitz E.R.
      • et al.
      Impact of a comprehensive patient safety strategy on obstetric adverse events.
      reported an unintended increase in cesarean delivery rates after implementing multiple interventions to improve patient safety at a university hospital. This study, however, did not include preintervention data and trends or account for the transition period following the intervention and thus was not able to account for secular trends. Additionally, the previously reported AOI lacked proven validity in a high-risk population and did not include hypoxic ischemic encephalopathy (HIE) as an outcome.
      HIE is a subcategory of neonatal encephalopathy and is defined as the clinical syndrome of disturbed neurological function in the early days of life in an infant born at ≥ 35 weeks’ gestation. Although uncommon, occurring in 2–6 per 1000 deliveries, HIE can be devastating for families including prolonged intensive care hospitalization and expense. HIE is the most common and costly birth injury claim among medical malpractice lawsuits.
      • Donn S.M.
      • Chiswick M.L.
      • Fanaroff J.M.
      Medico-legal implications of hypoxic-ischemic birth injury.
      The objective of this study was to compare the modified indices including HIE before and after the implementation of our labor and delivery patient safety bundle using robust methodology and to evaluate the impact on cesarean delivery rates. We modified the AOI elements proposed by Mann et al
      • Mann S.
      • Pratt S.
      • Gluck P.
      • et al.
      Assessing quality obstetrical care: development of standardized measures.
      including the addition of HIE, thus creating the Modified Adverse Outcomes Index (M-AOI), Modified Weighted Adverse Outcomes Index (M-WAOI), and the Modified Severity Index (M-SI).

      Materials and Methods

      The protocol for this cohort study was approved by the Mayo Clinic Institutional Review Board. All women who delivered at the Mayo Clinic Rochester from Jan. 1, 2011, through Dec. 31, 2013, were considered. Deliveries were not included in the analysis if the research authorization status was declined for either the mother or infant in accordance with Minnesota law. Delivery of multiple gestations was counted as a single delivery. Pregnancies were categorized as high-risk or low-risk based on the presence or absence of specific risk factors derived from the Eunice Kennedy Shriver National Institute of Child Health and Human Development document on high-risk pregnancy (Appendix 1).

      Eunice Kennedy Shriver National Institute of Child Heatlh and Human Development. What is a high risk pregnancy? Available at: https://www.nichd.nih.gov/health/topics/high-risk/conditioninfo/Pages/factors.aspx. Accessed Jan. 10, 2016.

      At our tertiary care academic center, we perform approximately 2500 deliveries annually and serve as a referral center for an additional 6000 annual deliveries within the Mayo Clinic Health Systems. All providers are salaried Mayo Clinic employees who deliver patients at a single Mayo Clinic-owned hospital with no private or community patients or providers. Prior to 2012, the labor and delivery unit was staffed by a certified nurse midwife and resident staff and supervised by an attending obstetrician who was not required to be on the labor and delivery unit at all times.
      In the spring of 2012, an event review was conducted following a case of HIE. The results of the root cause analysis were consistent with the analyses performed for previous HIE event reviews, and it was concerning to note the frequency of the diagnosis was increasing. In the majority of the HIE cases, there was a demonstrated lack of oversight by the attending obstetrician and ineffective communication among team members.
      Another commonality was the variation in the interpretation of fetal heart rate tracings and the frequent loss of situational awareness in cases of prolonged labor. There was little standardization of practice, leading to a wide variation in approaches based on the team members who were staffing a delivery.
      As a result of these findings, our institution implemented a labor and delivery care bundle with the primary aim of reducing cases of HIE. The care bundle included the following initiatives: (1) dedicated obstetric attending presence on the unit 24 hours a day (the majority of shifts covered by a laborist physician); (2) communication training for all providers including physicians, certified nurse midwives, and nurses; (3) mandatory use of a labor partogram; (4) mandatory fetal heart rate tracing interpretation training (K2 Medical Systems); and (5) regular feedback on individual provider AOI. The unit is staffed 24 hours a day, 7 days a week with obstetricians, certified nurse midwives, anesthesiologists, and consultative maternal-fetal medicine specialists. All women are Mayo Clinic patients cared for under the supervision of attending obstetricians including a minority of patients primarily managed by certified nurse midwives or family medicine attending physicians.
      Maternal demographics and maternal and neonatal outcomes data were collected from a combination of manual extraction from the electronic medical record and diagnosis codes. Manually extracted data included gravidity, parity, gestational age at delivery, mode of delivery, and neonatal outcomes including disposition, birthweight, and Apgar scores. Other variables were extracted from the electronic medical record including maternal age, race, and baseline body mass index (BMI). Diagnosis codes were utilized to identify high-risk factors (Appendix 1), HIE (International Classification of Codes, ninth revision, codes 768.70, 768.71, 768.72, and 768.73), and other adverse events.
      The collected data were utilized to derive the M-AOI, M-WAOS, and M-SI outcome measures including the incorporation of HIE (Appendix 2). HIE was assigned 350 points, 50 points less than neonatal death because of the long-term care impact of HIE. Maternal death was increased to 1100 such that the total elements did not exceed maternal death.
      The primary outcomes of the study were the 3 modified outcomes indices. The secondary outcome was the cesarean delivery rate.
      Statistical analyses were performed using the SAS version 9.3 software package (SAS Institute, Inc, Cary, NC). Baseline maternal characteristics and delivery outcomes were compared between the pre- and postintervention periods using the 2-sample Student t test for maternal age, the Wilcoxon rank-sum test for gravidity and parity, and the χ
      • Howell E.A.
      • Zeitlin J.
      • Hebert P.L.
      • Balbierz A.
      • Egorova N.
      Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity.
      test for categorical variables. Segmented regression analysis, appropriate for an interrupted time series design, was utilized to evaluate differences in the level and trend of each outcome index before and after the intervention, using calendar month as the unit of analysis.
      The outcomes were analyzed for all pregnancies combined, and separately by high- and low-risk pregnancy status. There were a total of 36 monthly intervals: 14 during the preintervention period (January 2011 through February 2012), 4 during the transition period (March 2012 through June 2012), and 18 during the postintervention period (July 2012 through December 2013).
      The outcomes in the transition period were not included in the analysis. For each outcome, the following regression model was fit: E(Y) = β0 + β1 time + β2 intervention + β3 time after intervention + error, where time denotes the number of months sequentially from the first month in the baseline period, intervention is an indicator variable denoting the 2 time periods (before and after the implementation of the intervention), time after intervention denotes the number of months after the intervention, β0 represents a constant term for the level of the outcome at the start of the preintervention period, β1 represents the change in outcome per month (ie, linear trend) during the preintervention period, β2 represents the change in the level of the outcome after the implementation of the intervention, and β3 represents the change in the linear trend after the implementation of the intervention, compared with the trend during the preintervention period.
      The regression models were fit using SAS Proc Autoreg. The Durbin-Watson (D-W) statistic was evaluated for evidence of serial autocorrelation. Because the Durbin-Watson statistic indicated a lack of autocorrelation, the models were not corrected for autocorrelation.

      Results

      During the study period, 7137 deliveries occurred at our institution; 5826 deliveries met inclusion criteria. There were 2349 deliveries in the preintervention period, 621 in the intervention period, and 2856 in the postintervention period. Comparing the pre- and postintervention periods, there was no difference in the mean maternal age in years (29.7 [SD 5.2] vs 29.9 [SD, 5.0], P = .17), obesity (BMI ≥ 30 kg/m2) (23.6% vs 21.8%, P = .20), or median parity (2 [1, 3] vs 2 [1, 3], P = .62) (Table 1). High-risk pregnancy was more common in the postintervention period (73.5% vs 79.4%, P < .001).
      Table 1Baseline characteristics comparing the pre- and postintervention periods
      CharacteristicBefore intervention (n = 2349)After intervention (n = 2856)P value
      BMI was calculated from available height and weight measurements closest to gestational age of 0 (possible weight measurements from 9 months prior to gestational age of 0-12 weeks after the delivery date; height from 1 year prior to gestational age of 0-1 year after the delivery date). BMI was unavailable for 890 (37.9%) in the preintervention group and 608 (21.3%) in the postintervention group
      Age, y, mean (SD)29.7 (5.2)29.9 (5.0).17
      Race, n, %.73
       American Indian/Alaska Native5 (0.2)5 (0.2)
       Asian109 (4.6)145 (5.1)
       Black135 (5.7)168 (5.9)
       White1913 (81.4)2340 (81.9)
       Native Hawaiian/Pacific Islander2 (0.1)1 (0.04)
       Other138 (5.9)154 (5.4)
      Unknown/chose not to disclose47 (2.0)43 (1.5)
      BMI, kg/m2
      Gravidity and parity reports include the current pregnancy
      .20
       < 30.01114/1459 (76.4)1757/2248 (78.2)
       ≥ 30.0345/1459 (23.6)491/2248 (21.8)
      Gravidity, median (IQR)
      χ2 P value is presented for categorical variables, Wilcoxon rank-sum P value is presented for gravidity and parity, and t test P value is presented for age.
      2 (1, 3)2 (1, 3).88
      Parity, median (IQR)
      χ2 P value is presented for categorical variables, Wilcoxon rank-sum P value is presented for gravidity and parity, and t test P value is presented for age.
      2 (1, 3)2 (1, 3).62
      Previous cesarean delivery, n, %413 (17.6)482 (16.9).50
      VBAC, n, %91 (3.9)123 (4.3).43
      High-risk pregnancy, n, %1726 (73.5)2269 (79.4)< .001
      BMI, body mass index; IQR, interquartile range; VBAC, vaginal birth after cesarean delivery.
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.
      a BMI was calculated from available height and weight measurements closest to gestational age of 0 (possible weight measurements from 9 months prior to gestational age of 0-12 weeks after the delivery date; height from 1 year prior to gestational age of 0-1 year after the delivery date). BMI was unavailable for 890 (37.9%) in the preintervention group and 608 (21.3%) in the postintervention group
      b Gravidity and parity reports include the current pregnancy
      c χ2 P value is presented for categorical variables, Wilcoxon rank-sum P value is presented for gravidity and parity, and t test P value is presented for age.
      Trends in the monthly values of the modified indices are shown in Figure 1. Seven of the 5826 deliveries were complicated by HIE; all 7 of these had concurrent events. Thus, the incidence of HIE during the study was 1.2 per 1000 deliveries. Overall, there was a decrease in both the M-WAOI (P = .0497) and the M-SI (P = .01) comparing the pre- and postintervention periods; there was no difference in the M-AOI (P = .43). For low-risk pregnancies, there was no significant difference in the levels for any of the measured indices over the study period (P = .61, P = .41, and P = .34 for the M-AOI, M-WAOI, and M-SI, respectively) (Figure 2).
      Figure thumbnail gr1
      Figure 1Modified indices of groups
      A, Modified adverse outcomes index. B, Modified Weighted Adverse Outcomes Index. C, Modified Severity Index. The dashed lines denote the estimated monthly values from the segmented regression analysis. The period between the vertical blue lines denotes the transition period.
      M-AOI, Modified Adverse Outcomes Index; M-SI, Modified Severity Index; M-WAOI, Modified Weighted Adverse Outcomes Index.
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.
      Figure thumbnail gr2
      Figure 2Modified indices of groups stratified by high- and low-risk
      Modified adverse outcomes index (A), Modified Weighted Adverse Outcomes Index (B), and Modified Severity Index (C), stratified by high- and low-risk. The dashed lines denote the estimated monthly values from the segmented regression analysis. The period between the vertical blue lines denotes the transition period.
      M-AOI, Modified Adverse Outcomes Index; M-SI, Modified Severity Index; M-WAOI, Modified Weighted Adverse Outcomes Index.
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.
      Based on the high-risk pregnancies, at the start of the preintervention period, the average monthly M-AOI was 9.3 (Figure 2). Over the course of the 14 month preintervention period, the estimated mean M-AOI decreased nonsignificantly at a rate of –0.07 ± 0.16 per month (P = .66 for testing whether the slope is different from zero). After the transition period, the estimated mean monthly M-AOI decreased by 2.0 ± 1.7 (P = .24).
      Over the course of the 18 month postintervention period, the estimated mean M-AOI increased at a rate of 0.15 per month (P = .26 for testing the difference in slopes between the 2 time periods). Based on the regression model, the estimated mean monthly M-AOI for the final month of the study period was 9.0 for high-risk pregnancies. However, had the intervention not been implemented and if the trend during the preintervention period were to have continued, the predicted M-AOI for the final month of the study period would have been lower (7.0) for high-risk pregnancies.
      For the M-WAOI among the high-risk pregnancies, the slope or trend in the monthly index was stable across the preintervention period (P = .59), and this slope was not significantly different before and after the intervention (P = .97) (Figure 2). However, at the start of the preintervention period, the average monthly M-WAOI was 4.6, and after the transition period, the estimated mean monthly M-WAOI decreased by 4.2 ± 1.8 (P = .03), indicating an immediate intervention impact.
      Based on the high-risk pregnancies, at the start of the preintervention period, the average monthly M-SI was 39.6 (Figure 2). However, over the course of the 14 month preintervention period, the estimated mean M-SI increased at a rate of 3.1 ± 1.7 per month (P = .07). After the transition period, the estimated mean monthly M-SI decreased significantly by 53.9 ± 17.7 (P = .01).
      Over the course of the 18 month postintervention period, the estimated mean M-SI gradually increased but only at a rate of 0.4 per month. Based on the regression model, the estimated mean monthly M-SI for the final month of the study period was 35.7 for high-risk pregnancies. However, had the intervention not been implemented and if the trend during the preintervention period were to have continued, the predicted M-SI for the final month of the study period would have been considerably higher (exceeded 100) for high-risk pregnancies. Thus, the M-SI was < 50% of the predicted M-SI had the intervention not been implemented.
      Although the difference did not reach statistical significance, there was a decrease in deliveries of < 37 weeks’ gestation during the postintervention period (11.5% compared with 13.2%, P = .07). There was no difference in mode of delivery (P = .30), multiple gestations (P = .11), or neonatal birthweight < 2500 g (P = .43) (Table 2). The cesarean delivery rate was increasing prior to the intervention, but the rate was stable after the intervention, and the absolute rate did not differ between the pre- to the postintervention periods (28.4% vs 30.0%, P = .20) (Figure 3).
      Table 2Delivery outcomes comparing the pre- and postintervention periods
      CharacteristicBefore intervention (n = 2349)After intervention (n = 2856)P value
      χ2 P value
      Mode of delivery, n, %
      For multiple gestations, a single mode of delivery was assigned by the following hierarchy: cesarean delivery if any infant was delivered by cesarean delivery; if no cesarean delivery was performed for any infant but operative vaginal delivery was performed for any infant, then operative vaginal delivery was reported; if no cesarean or operative vaginal delivery was performed for any infant, vaginal delivery was reported
      .30
       Cesarean delivery666 (28.4)856 (30.0)
       Operative vaginal111 (4.7)147 (5.1)
       Vaginal1572 (66.9)1853 (64.9)
      Multiple gestations, n, %71 (3.0)66 (2.3).11
      Gestational age at delivery, n, %.07
       < 37 wks310 (13.2)329 (11.5)
       ≥ 37 wks2039 (86.8)2527 (88.5)
      Neonatal birthweight, n, %
      For multiples gestations, the lowest birthweight was reported.
      .43
       < 2500 g233 (9.9)265 (9.3)
       ≥ 2500 g2116 (90.1)2591 (90.7)
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.
      a χ2 P value
      b For multiple gestations, a single mode of delivery was assigned by the following hierarchy: cesarean delivery if any infant was delivered by cesarean delivery; if no cesarean delivery was performed for any infant but operative vaginal delivery was performed for any infant, then operative vaginal delivery was reported; if no cesarean or operative vaginal delivery was performed for any infant, vaginal delivery was reported
      c For multiples gestations, the lowest birthweight was reported.
      Figure thumbnail gr3
      Figure 3Cesarean delivery rate
      The dashed lines denote the estimated or predicted monthly values from the segmented regression analysis. The period between the vertical blue lines denotes the transition period.
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.

      Comment

      There was a significant decrease in the M-WAOI and the M-SI after implementation of the labor and delivery care bundle. Despite a higher prevalence of high-risk pregnancies after the intervention, a decrease was seen overall and in particular for the high-risk pregnancy population, further affirming the impact of our interventions. However, there was no difference in the M-AOI between the pre- and postintervention periods. The stability of the overall AOI likely reflects the fact that neonatal intensive care unit admission and third- and fourth-degree lacerations are the drivers of AOI, and we would not expect these measures to have an impact by our intervention.
      Recently the American College of Obstetricians and Gynecologists Task Force on Neonatal Encephalopathy developed a list of criteria of neonatal signs and events that determine the likelihood that an acute peripartum or intrapartum event contributed to the development of HIE.

      American College of Obstetricians and Gynecologists, American Academy of Pediatrics. Task Force on Neonatal Encephalopathy. Neonatal encephalopathy and neurologic outcome. 2nd ed. Washington (DC): 2014.

      Key elements include neonatal 5 and 10 minute Apgar scores < 5; fetal umbilical artery acidemia (pH < 7.0 and/or base deficit ≤ 12 mmol/L); magnetic resonance imaging evidence of injury including type of injury and timing; multisystem organ failure; a sentinel hypoxic or ischemic event; fetal heart rate patterns consistent with an acute peripartum or intrapartum event; lack of evidence of other factors that could have contributed to the outcome; and outcome of spastic quadriplegia or dyskinetic cerebral palsy.
      Martinez-Biarge et al
      • Martinez-Biarge M.
      • Diez-Sebastian J.
      • Wusthoff C.J.
      • Mercuri E.
      • Cowan F.M.
      Antepartum and intrapartum factors preceding neonatal hypoxic-ischemic encephalopathy.
      recently published a case-control study evaluating antepartum and intrapartum risk factors for HIE (405 infants with HIE were compared with 293 neurologically normal infants). They found that infants with HIE were more likely to be delivered by emergency cesarean delivery, and there were no cases of HIE among women who delivered by prelabor cesarean delivery. Gestational age ≥ 41 weeks, prolonged rupture of membranes, abnormal cardiotocography, thick meconium, sentinel event, shoulder dystocia, tight nuchal cord, and failed vacuum were independently associated with HIE.
      Our data support the hypothesis that adverse neonatal outcomes including HIE are not attributable to antepartum factors alone.

      American College of Obstetricians and Gynecologists, American Academy of Pediatrics. Task Force on Neonatal Encephalopathy. Neonatal encephalopathy and neurologic outcome. 2nd ed. Washington (DC): 2014.

      • Draycott T.
      • Sibanda T.
      • Owen L.
      • et al.
      Does training in obstetric emergencies improve neonatal outcome?.
      Our interventions were aimed at solely having an impact on intrapartum care and would not be expected to have an impact on antepartum factors. Moreover, impacts overall and on high-risk pregnancies suggest that these data can be applicable to various settings. Because of the implementation of an intervention with multiple facets, it is impossible to determine which specific interventions had an impact on the practice. We hypothesize that the utilization of a partogram to maintain situational awareness in cases of prolonged labor in addition to fetal heart rate tracing training were critical.
      Two recently published studies examined the utility of obstetric quality indicators. Howell et al
      • Howell E.A.
      • Zeitlin J.
      • Hebert P.L.
      • Balbierz A.
      • Egorova N.
      Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity.
      found that the elective delivery at < 39 weeks’ gestation and cesarean delivery performed in low-risk women varied widely across New York City but that there were no correlations between the quality indicator rates and maternal and neonatal morbidity. Furthermore, Friedman et al
      • Friedman A.M.
      • Ananth C.V.
      • Prendergast E.
      • D'Alton M.E.
      • Wright J.D.
      Evaluation of third-degree and fourth-degree laceration rates as quality indicators.
      recently argued that the variation in third- and fourth-degree laceration rates neither vary widely enough nor have the potential for modification required of a meaningful quality indicator. In contrast, our findings support utilizing quality metrics and AOI as a performance improvement tool.
      An important part of our process included regular provider feedback on specific metrics. Operative vaginal delivery and cesarean delivery rates were also tracked and reported. Because cesarean delivery is not included in the AOI, care must be taken to avoid an unintentional increase in cesarean delivery rates because many of the outcomes in the index can be essentially eliminated by performance of cesarean delivery (uterine rupture, third- and fourth-degree lacerations, and birth trauma). Our data show a stable cesarean delivery rate after the intervention.
      Pettker et al
      • Pettker C.M.
      • Thung S.F.
      • Norwitz E.R.
      • et al.
      Impact of a comprehensive patient safety strategy on obstetric adverse events.
      implemented a strategy to improve patient safety including external expert review, protocol standardization, a patient safety nurse and committee, and training in team skills and fetal heart rate monitoring. Their intervention resulted in a significant reduction in AOI. Our intervention bundle differed from Pettker et al with the requirement that obstetricians be physically present at all times during shift duty, use of a partogram to maintain situational awareness, and provision of individual provider’s adverse outcomes as part of annual staff evaluation. However, the same group was able to show a significant reduction in liability claims and payments after the intervention.
      • Pettker C.M.
      • Thung S.F.
      • Lipkind H.S.
      • et al.
      A comprehensive obstetric patient safety program reduces liability claims and payments.
      Nicholson et al
      • Nicholson J.M.
      • Parry S.
      • Caughey A.B.
      • Rosen S.
      • Keen A.
      • Macones G.A.
      The impact of the active management of risk in pregnancy at term on birth outcomes: a randomized clinical trial.
      showed a reduction in AOI with a risk-based preventive labor induction strategy. Compared with previous reports of interventions and AOI, our study utilized interrupted time series, a quasiexperimental analysis including preintervention trends that better establishes the absence of other factors that could explain the observed changed.
      • Pettker C.M.
      • Thung S.F.
      • Norwitz E.R.
      • et al.
      Impact of a comprehensive patient safety strategy on obstetric adverse events.
      Additionally, we included the outcome of HIE and risk-stratified results.
      Our study is not without limitations. Because of the retrospective nature of the study, we cannot account for variation in patient and practice characteristics beyond our intervention. The National Quality Forum has chosen not to adopt the AOI proposed by Mann presumably because of concerns for adverse effects on cesarean delivery rates or variable positive predictive values of indicators based on hospital discharge or administrative data.
      • Walker S.
      • Strandjord T.P.
      • Benedetti T.J.
      In search of perinatal quality outcome measures: 1 hospital's in-depth analysis of the Adverse Outcomes Index.
      • Yasmeen S.
      • Romano P.S.
      • Schembri M.E.
      • Keyzer J.M.
      • Gilbert W.M.
      Accuracy of obstetric diagnoses and procedures in hospital discharge data.
      Walker et al
      • Walker S.
      • Strandjord T.P.
      • Benedetti T.J.
      In search of perinatal quality outcome measures: 1 hospital's in-depth analysis of the Adverse Outcomes Index.
      performed an in-depth analysis of the predictive value of the AOI based on administrative data and manual chart review. They found the positive predictive values of 7 indicators were 86-100% but were lower for the other 3 indicators (neonatal death, birth trauma, and maternal return to the operating room). Because of the observational nature of the study, the Hawthorne effect is a possibility; however, randomizing to this intervention would be impractical and unethical.
      In summary, we utilized a multifaceted bundle to improve intrapartum care and decrease adverse maternal and neonatal outcomes including HIE. We recommend a broad range of metrics beyond the AOI by which to meaningfully compare provider performance including operative vaginal delivery and intrapartum cesarean delivery rates.

      Appendix

      Appendix 1Categorization of high-risk pregnancy based on baseline and pregnancy conditions (alphabetical by column)
      Baseline characteristicsPrepregnancy conditionsPrior pregnancy conditionsCurrent pregnancy conditions
      Age ≥ 35 yAutoimmune diseaseCervical insufficiencyAbnormal placentation
      Alcohol useCardiac diseasePreterm deliveryAlloimmunization
      RaceCoagulation defectPrior cesarean or uterine surgeryAmniotic fluid disorders
      ObesityDiabetesAnemia
      Other drug useEpilepsyAntepartum hemorrhage
      Tobacco useHemologic diseaseChorioamnionitis
      HIVFetal anomalies
      HypertensionFetal malpresentation
      Inflammatory bowel diseaseGestational diabetes
      Liver diseaseGestational hypertension
      MalignancyInsufficient prenatal care
      Pituitary and adrenal disordersIUGR
      PCOSMultiple gestations
      Renal insufficiency/nephropathyPerinatal infection
      Respiratory disease (eg, asthma)Placental abruption
      Thyroid disease (uncontrolled)Postterm pregnancy
      Preeclampsia/HELLP syndrome
      Preterm labor
      Preterm premature rupture of membranes
      Thrombocytopenia
      Unstable lie
      Vasa previa
      HELLP, hemolysis, elevated liver enzymes, low platelets; HIV, human Immunodeficiency virus; IUGR, intrauterine growth restriction; PCOS, polycystic ovary syndrome.
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.
      Appendix 2Definitions and diagnosis codes for Modified Adverse Outcome Index events
      EventModified pointsInclusionExclusionDiagnosis codes
      Maternal death1100All pregnant women who die during the same hospital admission as their deliveryNoneDeath flag during delivery admission
      Intrapartum or neonatal death > 2500 g400Intrapartum death or neonatal death up to 30 dNoneDeath flag for infant; all manually verified
      Hypoxic ischemic encephalopathy350Inborn onlyDiagnosed after 30 d768.70–768.73
      Uterine rupture100NoneNone665.0, 665.1, 665.11
      Maternal admission to ICU65NoneNoneDiagnosis code of 640–677 and one of the following procedure codes 96.04, 96.05, 96.06, 96.7, 93.9, 93.91, 93.93; all manually verified
      Birth trauma60NoneInitial diagnosis after dismissal767.0–767.7, 767.11
      Return to OR/labor and delivery40NoneNoneProcedure codes 69.02, 69.52, 39.98, 75.92, 54.61, 38.86; all manually verified
      Admission to NICU > 2500 g and for > 24 h35Inborns only; ≥ 2500 g

      ICU charge or length of stay > 1 d

      Within 1 d of birth
      NoneRetrieved from manually entered neonatal database
      Apgar < 7 at 5 min25Inborns onlySome cases of extreme prematurity or neonatal death within 1 d excludedRetrieved from manually entered obstetrics database
      Postpartum hemorrhage requiring blood transfusion20NoneTransfusion before delivery or for known issue outside of delivery (cardiac issue, hemophilia, etc)Procedure code 99.04 (transfusion) with diagnosis code of 666.0–666.24 (postpartum hemorrhage)
      Third- or fourth-degree perineal laceration5NoneNone664.20, 664.21, 664.24 (third-degree)

      664.30, 664.31, 664.34 (fourth-degree)
      Apgar, appearance, pulse, grimace, activity, respiration; ICU, intensive care unit; NICU, neonatal intensive care unit; OR, operating room.
      Tolcher et al. Modified obstetrics adverse outcomes index. Am J Obstet Gynecol 2016.

      References

        • Friedman A.M.
        • Ananth C.V.
        • Prendergast E.
        • D'Alton M.E.
        • Wright J.D.
        Evaluation of third-degree and fourth-degree laceration rates as quality indicators.
        Obstet Gynecol. 2015; 125: 927-937
        • Howell E.A.
        • Zeitlin J.
        • Hebert P.L.
        • Balbierz A.
        • Egorova N.
        Association between hospital-level obstetric quality indicators and maternal and neonatal morbidity.
        JAMA. 2014; 312: 1531-1541
        • Walker S.
        • Strandjord T.P.
        • Benedetti T.J.
        In search of perinatal quality outcome measures: 1 hospital's in-depth analysis of the Adverse Outcomes Index.
        Am J Obstet Gynecol. 2010; 203: 336l.e1-336l.e7
        • Mann S.
        • Pratt S.
        • Gluck P.
        • et al.
        Assessing quality obstetrical care: development of standardized measures.
        Jt Comm J Qual Patient Saf. 2006; 32: 497-505
      1. National Quality Forum. National voluntary consensus standards for perinatal care 2008: a consensus report. Available at: http://www.qualityforum.org/Publications/2009/05/National_Voluntary_Consensus_Standards_for_Perinatal_Care_2008.aspx. Accessed March 31, 2015.

        • Foglia L.M.
        • Nielsen P.E.
        • Hemann E.A.
        • et al.
        Accuracy of the Adverse Outcome Index: An obstetrical quality measure.
        Jt Comm J Qual Patient Saf. 2015; 41: 370-372
        • Pettker C.M.
        • Thung S.F.
        • Norwitz E.R.
        • et al.
        Impact of a comprehensive patient safety strategy on obstetric adverse events.
        Am J Obstet Gynecol. 2009; 200: 492.e1-492.e8
        • Donn S.M.
        • Chiswick M.L.
        • Fanaroff J.M.
        Medico-legal implications of hypoxic-ischemic birth injury.
        Semin Fetal Neonatal Med. 2014; 19: 317-321
      2. Eunice Kennedy Shriver National Institute of Child Heatlh and Human Development. What is a high risk pregnancy? Available at: https://www.nichd.nih.gov/health/topics/high-risk/conditioninfo/Pages/factors.aspx. Accessed Jan. 10, 2016.

      3. American College of Obstetricians and Gynecologists, American Academy of Pediatrics. Task Force on Neonatal Encephalopathy. Neonatal encephalopathy and neurologic outcome. 2nd ed. Washington (DC): 2014.

        • Martinez-Biarge M.
        • Diez-Sebastian J.
        • Wusthoff C.J.
        • Mercuri E.
        • Cowan F.M.
        Antepartum and intrapartum factors preceding neonatal hypoxic-ischemic encephalopathy.
        Pediatrics. 2013; 132: e952-e959
        • Draycott T.
        • Sibanda T.
        • Owen L.
        • et al.
        Does training in obstetric emergencies improve neonatal outcome?.
        BJOG. 2006; 113: 177-182
        • Pettker C.M.
        • Thung S.F.
        • Lipkind H.S.
        • et al.
        A comprehensive obstetric patient safety program reduces liability claims and payments.
        Am J Obstet Gynecol. 2014; 211: 319-325
        • Nicholson J.M.
        • Parry S.
        • Caughey A.B.
        • Rosen S.
        • Keen A.
        • Macones G.A.
        The impact of the active management of risk in pregnancy at term on birth outcomes: a randomized clinical trial.
        Am J Obstet Gynecol. 2008; 198: 511.e1-511.e15
        • Yasmeen S.
        • Romano P.S.
        • Schembri M.E.
        • Keyzer J.M.
        • Gilbert W.M.
        Accuracy of obstetric diagnoses and procedures in hospital discharge data.
        Am J Obstet Gynecol. 2006; 194: 992-1001