Advertisement

Can differences in obstetric outcomes be explained by differences in the care provided? The MFMU Network APEX study

Published:March 13, 2014DOI:https://doi.org/10.1016/j.ajog.2014.03.017

      Objective

      The purpose of this study was to determine whether hospital differences in the frequency of adverse obstetric outcomes are related to differences in care.

      Study Design

      The Assessment of Perinatal EXcellence cohort comprises 115,502 women and their neonates who were born in 25 hospitals in the United States between March 2008 and February 2011. Hierarchical logistic regression was used to quantify the amount of variation in postpartum hemorrhage, peripartum infection, severe perineal laceration, and a composite adverse neonatal outcome among hospitals that is explained by differences in patient characteristics, hospital characteristics, and obstetric care provided.

      Results

      The study included 115,502 women. For most outcomes, 20-40% of hospital differences in outcomes were related to differences in patient populations. After adjusting for patient-, provider-, and hospital-level factors, multiple care processes were associated with the predefined adverse outcomes; however, these care processes did not explain significant variation in the frequency of adverse outcomes among hospitals. Ultimately, 50-100% of the interhospital variation in outcomes was unexplained.

      Conclusion

      Hospital differences in the frequency of adverse obstetric outcomes could not be explained by differences in frequency of types of care provided.

      Key words

      See related editorial, page 85
      Obstetric admissions are a leading cause of hospitalization in the United States. Accordingly, there has been an increasing demand for quality measurement from multiple stakeholders. Quality measures typically take 2 forms: (1) outcome measures, such as frequency of peripartum infection, which reflect the actual outcomes, and (2) process measures, such as frequency of episiotomy, which reflect adherence to or avoidance of a given type of care.
      • Lilford R.
      • Mohammed M.A.
      • Spiegelhalter D.
      • Thomson R.
      Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma.
      • Pronovost P.J.
      • Thompson D.A.
      • Holzmueller C.G.
      • Lubomski L.H.
      • Morlock L.L.
      Defining and measuring patient safety.
      However, several uncertainties remain about obstetric outcome and process measures and their ability to represent quality care. There is controversy whether and to what extent hospital differences in outcomes are actually due to differences in the characteristics of their patient population; correspondingly, case-mix adjustment has been used inconsistently.
      • Aron D.C.
      • Harper D.L.
      • Shepardson L.B.
      • Rosenthal G.E.
      Impact of risk-adjusting cesarean delivery rates when reporting hospital performance.
      • Grobman W.A.
      • Feinglass J.
      • Murthy S.
      Are the Agency for Healthcare Research and Quality obstetric trauma indicators valid measures of hospital safety?.
      Also, there is often an implicit assumption that those hospitals that perform best on process measures will have the best outcomes.
      • Draycott T.
      • Sibanda T.
      • Laxton C.
      • Winter C.
      • Mahmood T.
      • Fox R.
      Quality improvement demands quality measurement.
      Yet, this assumption has not been proved in obstetrics.
      In fact, there are several potential contributors to the frequency of adverse outcomes that include patient characteristics (such as maternal age), hospital characteristics (such as the types of obstetric providers or continual availability of interventional radiology), and the types of care that are provided (such as the frequency of cesarean delivery). Although poorly understood, the extent to which each of these categories explains hospital differences in outcomes is important in determining the adequacy of quality measures. For example, if all variation in an outcome were due to differences in patient populations, it would make little sense to use that outcome to represent a hospital's quality. On the other hand, if much of the variation in an outcome were not due to differences in patient populations but were due to differences in a particular process of care, the use of both specific outcome and process measures would be better supported.
      The specific aim of the present study was to assess whether and to what extent hospital differences in the frequency of adverse obstetric outcomes are related to patient and hospital characteristics and to types of care provided.

      Methods

      Study design

      The Assessment of Perinatal EXcellence (APEX) study is an observational study that was designed to assist in the development of quality measures for intrapartum obstetrics care. This study was approved by the institutional review board at each participating institution under a waiver of informed consent. Full details of the study design have been published previously.
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
      In summary, patients who were eligible for data collection were those who delivered on randomly selected days between March 2008 and February 2011 at any of the 25 hospitals in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network, who were at least 23 weeks of gestation, and who had arrived at the hospital with a live fetus. Days were chosen by computer-generated random selection, with enrollment from larger hospitals limited to avoid overrepresentation of patients from these hospitals. The medical records of all eligible women and their neonates were abstracted by trained and certified research personnel at the clinical centers. Patient data that were reported in the chart included demographic characteristics (including race and ethnicity as reported in the chart), details of the medical and obstetric history, types of intrapartum and postpartum care, and obstetric outcomes. In addition, characteristics of the providers who cared for the patients and the hospitals in which they delivered were collected. Maternal data were collected until discharge, and neonatal data were collected until discharge or until 120 days of age, whichever came first.

      Outcomes

      The 5 a priori primary outcomes were (1) venous thromboembolism, (2) postpartum hemorrhage (PPH), (3) peripartum infection, (4) severe perineal laceration, restricted to women with vaginal singleton deliveries with no shoulder dystocia and stratified by spontaneous, forceps-assisted vaginal delivery, and vacuum-assisted vaginal delivery, and (5) a composite neonatal adverse outcome, restricted to term (≥37 weeks of gestation), nonanomalous singleton infants. Additional details regarding the definitions of these outcomes are detailed elsewhere.
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.

      Statistical analysis

      Sample size for the APEX cohort was based on thromboembolism in cesarean deliveries, which was expected to have the lowest frequency (0.175% overall and 0.550% in cesarean deliveries) of the 5 a priori primary outcomes, with techniques that consider the cluster design.
      • Eldridge S.
      • Asby D.
      • Kerry S.
      Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and method.
      • Hsieh F.Y.
      • Bloch D.A.
      • Larsen M.D.
      A simple method of sample size calculation for linear and logistic regression.
      • Lancaster G.A.
      • Chellaswamy H.
      • Taylor S.
      • Lyon D.
      • Dowrick C.
      Design of a clustered observational study to predict emergency admissions in the elderly: statistical reasoning in clinical practice.
      The following assumptions were included: 2-sided type I error = 0.01 and the proportion of deliveries without an associated process measure = 25%. The sample size estimate was based on 30,000 cesarean deliveries. Conservatively, assuming a cesarean frequency of 25%, a total sample size of 120,000 would enable the detection of an odds ratio of 2.75 for the association between a process measure and outcome with at least 80% power for the outcome of thromboembolism. Assuming an odds ratio of 1.5 and assuming event frequencies that ranged from 2.4–8.0% for the remaining 4 outcomes (PPH, peripartum infection, severe perineal laceration in vaginal deliveries, and the composite neonatal adverse outcome in term nonanomalous singletons), power was estimated to range from 83–99%; power was >99% for these 4 outcomes when we assumed an odds ratio of 2.0. Because of fewer than expected thromboembolism events (0.03% overall), this outcome was not further evaluated.
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
      For each of the adverse obstetric outcomes, hierarchical logistic regression with hospital random effects was used to quantify the amount of variation in outcomes among hospitals that was due to (1) patient characteristics, (2) provider and hospital characteristics, and (3) the types of care provided (process measures). The initial regression equation included only the hospitals as random-effect terms. In each successive stage of the model, another level of variables (ie, the patient characteristics, hospital characteristics, or care characteristics) was added as fixed effects. Per the methods used by Synnes et al,
      • Synnes A.R.
      • MacNab Y.C.
      • Qiu Z.
      • et al.
      Neonatal intensive care unit characteristics affect the incidence of severe intraventricular hemorrhage.
      each equation contained a random effects term, and it is the standard deviation of this term that serves to quantify the overall variation in outcome frequency across the hospitals. The difference in the value of standard deviation as each set of characteristics is added to the model then quantifies the amount of variation between hospitals explained by the additional characteristics. Odds ratios and 99% confidence intervals (CIs) for each hospital, with the use of the hospital with the median observed outcome frequency as the referent, were also obtained from these hierarchical models.
      Patient, provider, hospital, and care characteristics that were eligible for multivariable models were selected a priori for each outcome, based on a plausible association with the outcome (ie, face validity). Details regarding the methods and results for selection of the patient characteristics have been reported previously.
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
      The provider and hospital characteristics that were eligible for multivariable models included the specialty of the attending provider, years since the attending provider graduated from medical/midwifery school, nurse-to-patient ratio during the shift that delivery occurred, a hospital's annual delivery volume (expressed in quartiles), the existence of a prenatal electronic medical record, the occurrence of a structured review of laboring patients attended by both nursing staff and attending providers, and the availability of a 24-hour anesthesia service dedicated to the labor and delivery unit. The presence of a 24-hour in-house attending obstetric provider, a 24-hour in-house neonatologist or pediatrician, and a 24-hour in-house interventional radiology service also were evaluated. For each outcome, after the patient characteristics that were selected previously for risk-adjustment were forced into the model,
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
      a backwards selection method was used with a probability value of < .05 to determine which provider and hospital characteristics were to remain in the regression for each outcome.
      After a model that included patient, provider, and hospital characteristics was established, we examined which types of care (ie, process measures) provided, selected a priori, were associated with each outcome. Eligible process measures included elective delivery <39 weeks of gestation without documented lung maturity, cervical dilation at admission among women in spontaneous labor, labor induction, proportion of labor with oxytocin augmentation, maximum dose of oxytocin, duration (minutes) of active stage (5 cm to either 10 cm or to cesarean delivery), vaginal examinations per hour in the first stage of labor, duration (minutes) from complete dilation (10 cm) to start of pushing, duration (minutes) from start of pushing to delivery, vaginal delivery, episiotomy, and type of anesthesia (epidural/regional or general). The process measures were added individually to patient and hospital characteristic-adjusted models that were restricted to women who were eligible for the type of care being assessed (eg, labor induction was not assessed among women with a placenta previa, because women with this diagnosis would not be eligible to receive induction). To facilitate interpretation, process measures that were explored initially as continuous variables were dichotomized for use in the final regression model based on clinical relevance and assessment of plots with the use of a locally weighted scatterplot smoothing technique. Process measures that were associated significantly with a greater frequency of an adverse obstetric outcome were identified and used to derive a composite process measure “exposure score” that was calculated according to the methods by Peterson et al

      Peterson ED, Roe MT, Mulgund J, et al. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912-20.

      as the proportion of the care processes that a patient was eligible to receive that actually were received by the patient. Thus, if a patient received 3 of the 4 care processes that were associated significantly with the outcome of interest, her composite exposure was 75%.
      SAS software (SAS Institute, Cary, NC) was used for the analyses. All tests were 2-tailed. A probability value of < .01 was used to define statistical significance; 99% CIs were estimated when directly testing a hypothesis (ie, examining the association between the process measures and outcomes) and to identify hospital outliers. A probability value of < .05 and 95% CIs were estimated for model building and other descriptive analyses.

      Results

      During the study period, data were collected on 115,502 women and their neonates and on 1797 different delivery-attending providers at 25 hospitals. Characteristics of these patients and their providers and hospitals are provided in Tables 1 and 2. As shown, women were delivered by a variety of types of providers, and these providers had a range of experience. Hospital characteristics, including availability of medical services (eg, obstetric anesthesia), the presence of electronic medical records, and the attendance of providers at structured obstetric patient review, varied as well.
      Table 1Maternal (n = 115,502) and neonatal (n = 118,422) characteristics of the study population
      Characteristicn (%)
      MATERNAL
       Age, y
      <2010,187 (8.8)
      20-24.924,299 (21.0)
      25-29.931,101 (26.9)
      30-34.930,570 (26.5)
      ≥3519,345 (16.8)
       Race/ethnicity
      Race/ethnicity was reported in the chart
      Non-Hispanic white52,040 (45.1)
      Non-Hispanic black23,878 (20.7)
      Non-Hispanic Asian5999 (5.2)
      Hispanic27,291 (23.6)
      Other5083 (4.4)
      Not documented1211 (1.1)
       Body mass index at delivery,
      n = 113,167 with body mass index data; n = 109,773 with prenatal care visit data; n = 113,446 with premature rupture of membranes/preterm premature rupture of membranes data.
      kg/m2
      <2514,242 (12.6)
      25-29.941,268 (36.5)
      30-34.932,088 (28.4)
      35-39.915,088 (13.3)
      ≥4010,481 (9.3)
       Cigarette use during pregnancy11,370 (9.9)
       Cocaine or methamphetamine use during pregnancy830 (0.7)
       Insurance status
      Uninsured/self-pay11,989 (10.5)
      Government-assisted45,125 (39.4)
      Private57,462 (50.2)
       Prenatal care
      n = 113,167 with body mass index data; n = 109,773 with prenatal care visit data; n = 113,446 with premature rupture of membranes/preterm premature rupture of membranes data.
      107,510 (97.9)
       Obstetric history
      Nulliparous46,773 (40.5)
      Previous vaginal delivery only49,865 (43.2)
      Previous cesarean delivery only8872 (7.7)
      Previous cesarean and vaginal deliveries9963 (8.6)
       Any hypertension13,272 (11.5)
       Diabetes mellitus
      None106,706 (92.4)
      Gestational6999 (6.1)
      Pregestational1734 (1.5)
      Anticoagulant use during pregnancy920 (0.8)
       Multiple gestation2815 (2.4)
       Polyhydramnios940 (0.8)
       Oligohydramnios4700 (4.1)
       Placenta previa467 (0.4)
       Placenta accreta158 (0.1)
       Placental abruption930 (0.8)
       Premature rupture of membranes/preterm premature rupture of membranes
      n = 113,167 with body mass index data; n = 109,773 with prenatal care visit data; n = 113,446 with premature rupture of membranes/preterm premature rupture of membranes data.
      6004 (5.3)
       Group B Streptococcus status
      Negative68,918 (59.7)
      Positive24,390 (21.1)
      Unknown22,194 (19.2)
      NEONATAL
       Presentation at delivery
      Vertex111,174 (94.1)
      Breech6010 (5.1)
      Nonbreech malpresentation931 (0.8)
       Gestational age at delivery, wk
      230-2761256 (1.1)
      280-3364282 (3.6)
      340-36610,024 (8.5)
      370-37610,914 (9.2)
      380-38620,723 (17.5)
      390-39637,695 (31.8)
      400-40623,876 (20.2)
      410-4168998 (7.6)
      ≥420654 (0.6)
       Birthweight, g
      <250012,498 (10.6)
      2500-399996,708 (81.7)
      ≥40009186 (7.8)
       Size for gestational age
      Small11,530 (9.7)
      Appropriate97,774 (82.6)
      Large9088 (7.7)
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      a Race/ethnicity was reported in the chart
      b n = 113,167 with body mass index data; n = 109,773 with prenatal care visit data; n = 113,446 with premature rupture of membranes/preterm premature rupture of membranes data.
      Table 2Characteristics of the study population's attending providers and hospitals
      Characteristicsn (%)
      Specialty of attending at delivery
       General obstetrics and gynecology84,057 (72.8)
       Midwife7808 (6.8)
       Family medicine3728 (3.2)
       Maternal-fetal medicine18,954 (16.4)
       No attending at delivery859 (0.7)
      Years since attending at delivery graduated medical or midwifery school
       0-9.9 (includes no attending at delivery)26,717 (23.4)
       10-14.921,793 (19.1)
       15-20.919,880 (17.4)
       20-24.916,248 (14.2)
       ≥2529,428 (25.8)
      Nurse-to-patient ratio at delivery
      Total number of nursing hours worked in Labor and Delivery during the 8-hour shift divided by 8, divided by the number of patient admissions during the 8-hour shift
       <131,781 (27.6)
       1-1.958,263 (50.7)
       2-2.915,804 (13.7)
       ≥39160 (8.0)
      Patient delivered at hospital where prenatal electronic medical record available
       No47,727 (41.3)
       Sometimes35,083 (30.4)
       Yes32,692 (28.3)
      Patient delivered at hospital with 24-hour in-house obstetric anesthesia service
       No13,150 (11.4)
       Yes102,352 (88.6)
      Patient delivered at hospital with 24-hour in-house attending obstetric provider
       No13,823 (12.0)
       Yes101,679 (88.0)
      Patient delivered at hospital with attending providers and/or nurses present for structured obstetric patient review
      Official board sign-out at shift change or other structured patient review.
       No obstetricians present at review21,106 (18.3)
       Obstetricians but no nurses present at review38,052 (32.9)
       Both obstetricians and nurses present at review56,344 (48.8)
      Patient delivered at hospital with 24-hour in-house interventional radiology available
       No79,452 (68.8)
       Yes36,050 (31.2)
      Patient delivered at hospital with 24-hour in-house attending neonatologist or pediatrician
       No neonatologist, no pediatrician12,532 (10.9)
       Pediatrician, no neonatologist4363 (3.8)
       Neonatologist98,314 (85.3)
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      a Total number of nursing hours worked in Labor and Delivery during the 8-hour shift divided by 8, divided by the number of patient admissions during the 8-hour shift
      b Official board sign-out at shift change or other structured patient review.
      The frequencies of the selected outcomes were as follows: PPH, 2.29% (95% CI, 2.20–2.38%), peripartum infection, 5.06% (95% CI, 4.93–5.19%), severe perineal laceration at spontaneous vaginal delivery, 2.16% (95% CI, 2.06–2.27%), severe perineal laceration at forceps-assisted vaginal delivery, 27.56% (95% CI, 25.54–29.57%), severe perineal laceration at vacuum-assisted vaginal delivery, 14.51% (95% CI, 13.34–15.67%), and composite neonatal adverse outcome, 2.73% (95% CI, 2.63–2.84%).
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
      As previously reported, the frequency of the selected adverse outcomes varied widely and differed significantly among hospitals (P < .001 for all).
      • Bailit J.L.
      • Grobman W.A.
      • Rice M.M.
      • et al.
      Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
      The type of care experienced by patients at different hospitals varied widely as well (Table 3).The frequency of labor induction among women who were eligible for such an intervention, for example, ranged among hospitals from 21-37%. Oxytocin at rates >20 mU/min rarely was administered to laboring women at some hospitals; however, this practice occurred in nearly 50% of women who received oxytocin at other hospitals. There was a >20-fold difference in the frequency of delayed pushing among women who reached the second stage and a difference in the frequency of vaginal delivery that ranged from 61-80%. Delivery practices varied as well, with a 50-fold difference in the frequency of episiotomy among women who had a vaginal delivery and >10-fold difference in the use of general anesthesia at cesarean delivery.
      Table 3Observed hospital frequencies of types of obstetric care
      VariableLowest %Median %Highest %
      Labor induction
      In patients with no previa and no history of classic, T, or J cesarean delivery (n = 113,049)
      20.828.237.1
      Dilation ≤2 cm at admission
      In patients at term with intact membranes and spontaneous intended labor with no previa and cervical dilation measured within 1 hour before or after Labor and Delivery admission (n = 46,068)
      6.613.625.9
      Maximum oxytocin ≥20 mU/min
      In patients who received oxytocin in labor (n = 58,228)
      8.717.646.3
      ≥80% of labor augmented with oxytocin
      In patients with spontaneous intended labor who were admitted to Labor and Delivery before delivery (n = 61,157)
      1.010.122.6
      ≥1 hr between complete dilation and initiation of pushing
      In patients who reached complete after intended labor (n = 60,290)
      0.810.921.2
      ≥2 hr between initiation of pushing to delivery
      In patients who reached complete after intended labor (n = 60,290)
      4.49.119.2
      ≥8 hr active phase
      In patients with intended labor who reached active stage (5 cm) with a term nonanomalous singleton pregnancy (n = 71,571)
      2.98.319.2
      <1 vaginal examination per every 3 hr in first stage
      In patients with intended labor who were treated in hospital for >1 hour during first stage (n = 81,826)
      2.921.043.7
      Vaginal delivery
      In all patients (n = 115,502)
      60.670.179.5
      Episiotomy
      In patients with a vaginal delivery and no shoulder dystocia (n = 77,071)
      0.77.035.4
      Epidural/regional anesthesia
      In patients with nonoperative vaginal delivery of a singleton and no shoulder dystocia and who reached complete after intended labor (n = 70,362)
      45.377.789.7
      General anesthesia
      In patients with a cesarean delivery (n = 36,201)
      1.16.514.8
      Elective delivery at <39 weeks' gestation without documented fetal lung maturity
      In patients with a term nonanomalous singleton pregnancy (n = 98,509).
      0.20.512.2
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      a In patients with no previa and no history of classic, T, or J cesarean delivery (n = 113,049)
      b In patients at term with intact membranes and spontaneous intended labor with no previa and cervical dilation measured within 1 hour before or after Labor and Delivery admission (n = 46,068)
      c In patients who received oxytocin in labor (n = 58,228)
      d In patients with spontaneous intended labor who were admitted to Labor and Delivery before delivery (n = 61,157)
      e In patients who reached complete after intended labor (n = 60,290)
      f In patients with intended labor who reached active stage (5 cm) with a term nonanomalous singleton pregnancy (n = 71,571)
      g In patients with intended labor who were treated in hospital for >1 hour during first stage (n = 81,826)
      h In all patients (n = 115,502)
      i In patients with a vaginal delivery and no shoulder dystocia (n = 77,071)
      j In patients with nonoperative vaginal delivery of a singleton and no shoulder dystocia and who reached complete after intended labor (n = 70,362)
      k In patients with a cesarean delivery (n = 36,201)
      l In patients with a term nonanomalous singleton pregnancy (n = 98,509).
      The associations of processes measures (individual and composite exposure score) with the studied outcomes are given in Table 4. Even after adjusting for patient, provider, and hospital characteristics, particular types of obstetric care remained associated with the outcomes of interest.
      Table 4Adjusted odds ratios (99% CI) between the types of obstetric care and adverse obstetric outcomes
      Process measurePostpartum hemorrhage
      Adjusted for age, insurance status, prenatal care, obstetric history, any hypertension, diabetes mellitus, anticoagulant use, multiple gestation, previa, accreta, abruption, birthweight, attending specialty, years since attending graduated medical or midwifery school
      Peripartum infection
      Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, diabetes mellitus, premature rupture of membranes/preterm premature rupture of membranes, group B Streptococcus status, gestational age at delivery, attending specialty, years since attending graduated medical or midwifery school, nurse-to-patient ratio, prenatal electronic medical record present, attending providers and/or nurses present for structured obstetric patient review, hospital volume
      Severe perineal laceration at spontaneous vaginal delivery
      Among women with a singleton delivery and no shoulder dystocia
      Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, birthweight, attending specialty, prenatal electronic medical record present
      Severe perineal laceration at forceps-assisted vaginal delivery
      Among women with a singleton delivery and no shoulder dystocia
      Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, birthweight, electronic medical record present
      Severe perineal laceration at vacuum-assisted vaginal delivery
      Among women with a singleton delivery and no shoulder dystocia
      Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, birthweight, attending specialty, prenatal electronic medical record present
      Composite neonatal adverse outcome
      Among women with a term, nonanomalous singleton infant
      Adjusted for body mass index, cigarette use, cocaine or methamphetamine use, insurance status, prenatal care, obstetric history, any hypertension, diabetes mellitus, premature rupture of membranes/preterm premature rupture of membranes, size for gestational age, attending specialty, round-the-clock in-house attending pediatrician available.
      n105,987110,20568,1441898351589,279
      Labor induction1.20 (1.04–1.37)1.22 (1.13–1.33)1.04 (0.90–1.21)1.05 (0.78–1.42)0.92 (0.70–1.21)1.18 (1.05–1.34)
      Dilation ≤2 cm at admission1.58 (1.37–1.82)
      Maximum oxytocin ≥20 mU/min1.61 (1.33–1.95)1.30 (1.16–1.44)
      ≥80% of labor augmented with oxytocin1.08 (0.78–1.50)1.63 (1.42 –1.87)
      ≥1 hr between complete dilation and initiation of pushing1.67 (1.22–2.28)1.29 (1.04–1.59)1.10 (0.74–1.64)0.94 (0.65–1.34)1.13 (0.89–1.45)
      ≥2 hrs between initiation of pushing to delivery4.02 (3.10–5.23)1.88 (1.51– 2.34)1.21 (0.87–1.69)1.55 (1.15–2.09)1.83 (1.46–2.28)
      ≥8 hrs active stage1.32 (1.08–1.62)
      <1 vaginal examination per every 3 hours in first stage1.18 (1.07–1.30)1.18 (1.01–1.38)
      Vaginal delivery0.19 (0.16–0.22)0.52 (0.47–0.56)0.72 (0.63–0.83)
      Episiotomy1.22 (1.04–1.43)2.47 (2.08–2.93)1.24 (0.87–1.79)1.99 (1.51–2.62)
      Epidural/regional anesthesia0.88 (0.73–1.06)(small no. precludes analysis)0.90 (0.57–1.45)
      General anesthesia3.61 (2.98–4.37)
      Elective delivery at <39 wk gestation without documented fetal lung maturity1.39 (0.67–2.89)
      Composite process measure exposure score (Percentage of care received that was associated with fewer adverse outcomes; referent received 100% of care eligible)0-67%: 4.69 (3.89–5.64)0-57%: 1.88 (1.68–2.11)0-67%: 2.18 (1.88–2.54)N/A0-50%: 2.64 (1.96–3.55)0-67%: 1.65 (1.43–1.91)
      75-83%: 2.25 (1.79–2.83)60-86%: 1.89 (1.70–2.09)75-83%: 1.34 (1.16–1.56)
      Empty cells reflect that this process measure was not assessed for this outcome.
      CI, confidence interval; N/A, not available.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      a Adjusted for age, insurance status, prenatal care, obstetric history, any hypertension, diabetes mellitus, anticoagulant use, multiple gestation, previa, accreta, abruption, birthweight, attending specialty, years since attending graduated medical or midwifery school
      b Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, diabetes mellitus, premature rupture of membranes/preterm premature rupture of membranes, group B Streptococcus status, gestational age at delivery, attending specialty, years since attending graduated medical or midwifery school, nurse-to-patient ratio, prenatal electronic medical record present, attending providers and/or nurses present for structured obstetric patient review, hospital volume
      c Among women with a singleton delivery and no shoulder dystocia
      d Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, birthweight, attending specialty, prenatal electronic medical record present
      e Adjusted for age, body mass index, cigarette use, insurance status, obstetric history, birthweight, electronic medical record present
      f Among women with a term, nonanomalous singleton infant
      g Adjusted for body mass index, cigarette use, cocaine or methamphetamine use, insurance status, prenatal care, obstetric history, any hypertension, diabetes mellitus, premature rupture of membranes/preterm premature rupture of membranes, size for gestational age, attending specialty, round-the-clock in-house attending pediatrician available.
      Supplementary Figure 1 (Appendix) represents the hospital differences in PPH and how those differences are affected by the sequential addition of independent variables in the different categories (ie, patient, provider/hospital, and care). For example, Supplementary Figure 1, A illustrates the odds ratio for each hospital (identified by the numbers 1-25 on the x-axis) for the outcome of PPH that derived from the logistic regression model without any risk-factor adjustment. Hospitals differ significantly from one another (P < .01), and some hospitals have significantly higher or lower odds of an outcome than the reference hospital (ie, 99% CIs do not include 1.0). If patient, hospital, and process characteristics are associated with the outcomes, as they are entered into the regression model, variation among the odds ratios of the hospitals should lessen. If all variation were explained by these characteristics, the odds ratios associated with each hospital would be 1.0.
      The results of adjusting for only patient characteristics are shown in Supplementary Figure 1, B, with the results obtained after the further addition of provider/hospital characteristics shown in Supplementary Figure 1, C, which shows a progressive reduction in the variation of the odds ratios, as illustrated by the hospitals' odds ratio point estimates that have migrated from their original positions and towards the line that represents an odds ratio = 1. However, when care variables are entered into the model, either as a single variable such as labor induction (data not shown) or as a composite exposure score (Supplementary Figure 1, D), the odds ratios that are associated with each hospital largely are unchanged. Graphic representations for the odds ratios that are associated with each stage of the model for the other outcomes are presented in the Supplementary Figures 2-6.
      Table 5 presents the variation between hospitals (standard deviation of the random effects term) that are associated with each stage of the hierarchical logistic regression for each outcome. For infection, none of the interhospital variation was explained by patient characteristics, whereas for the other outcomes 20-40% (percentage difference between the standard deviations) of the hospital's variation in outcomes was related to differences in patient populations. Approximately 20% of the variation in hospital PPH frequency was related to provider/hospital factors. However, for the other outcomes, there was little evidence that interhospital outcome variation was related to provider/hospital factors. In no case did differences in types of obstetric care account for much of the variation in observed outcomes. Ultimately, 50-100% of the interhospital variation in outcomes was unexplained.
      Table 5Variation (σ) in outcome frequency across the hospitals, crude and after adjustments for patient, provider/hospital, and care characteristics
      VariableDenominator size for each outcome, nCrude hierarchical regressionHierarchical regression with patient characteristicsHierarchical regression with patient and provider/hospital characteristicsHierarchical regression with patient, provider/hospital, and care characteristics
      Postpartum hemorrhage105,9870.20 (0.06)0.16 (0.05)0.13 (0.04)0.13 (0.04)
      Peripartum infection110,2050.18 (0.05)0.21 (0.06)0.18 (0.06)0.18 (0.06)
      Severe perineal laceration at spontaneous vaginal delivery
      Among women with a singleton delivery and no shoulder dystocia
      68,1440.15 (0.05)0.09 (0.03)0.09 (0.03)0.09 (0.03)
      Severe perineal laceration at forceps-assisted vaginal delivery
      Among women with a singleton delivery and no shoulder dystocia
      18980.33 (0.13)0.25 (0.11)0.26 (0.12)N/A
      Severe perineal laceration at vacuum-assisted vaginal delivery
      Among women with a singleton delivery and no shoulder dystocia
      35150.20 (0.09)0.15 (0.08)0.15 (0.08)0.14 (0.08)
      Composite neonatal adverse outcome
      Among women with a term, nonanomalous singleton infant.
      89,2790.17 (0.05)0.10 (0.04)0.09 (0.03)0.09 (0.03)
      N/A, not applicable.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      a Among women with a singleton delivery and no shoulder dystocia
      b Among women with a term, nonanomalous singleton infant.

      Comment

      In this study, we investigated the relationship between differences in obstetric care patterns and outcomes among hospitals. Several findings are notable. Despite the fact that the hospitals in the study were either university or university-affiliated and part of a single research network, the frequencies of obstetric practices were vastly different. After we controlled for differences in patient populations and hospital characteristics, several types of obstetric care were found to be associated with adverse obstetric outcomes. Nevertheless, this association did not translate into a capability to explain the hospital differences in adverse outcomes that were found.
      This lack of explanatory power is in contrast to that discerned for care processes in some other disciplines. For example, Synnes et al
      • Synnes A.R.
      • MacNab Y.C.
      • Qiu Z.
      • et al.
      Neonatal intensive care unit characteristics affect the incidence of severe intraventricular hemorrhage.
      examined variation in the frequency of intraventricular hemorrhage among neonates in the intensive care unit. In an analysis similar to ours, after they controlled for patient and hospital factors, they were able to demonstrate that differences in acidosis treatment, vasopressin use, and surfactant use could account for differences in interhospital rates of intraventricular hemorrhage. Similarly, studying adults with cardiac disease, Petersen et al

      Peterson ED, Roe MT, Mulgund J, et al. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912-20.

      demonstrated that adherence to particular types of management (such as beta-blocker use) could explain differences in hospitals' adjusted-mortality rates.
      Process measures, however, have not been well demonstrated to explain interhospital variation in obstetric outcomes. The inability to do so in the obstetric population that we studied has implications with regard to obstetric quality measurement and its interpretation. Process measures quantify adherence to a given type of care. Hospitals often are judged according to their adherence to selected process measures, with the implicit assumption that the hospitals that perform best on selected measures will have the best health outcomes. Yet, Draycott et al
      • Draycott T.
      • Sibanda T.
      • Laxton C.
      • Winter C.
      • Mahmood T.
      • Fox R.
      Quality improvement demands quality measurement.
      have called attention to the fact that this relationship need not hold. Further, they cite examples to illustrate that belief in an inexorable relationship between process measures and outcomes may hinder quality improvement if there is undue focus on process measures, which may be relatively easy to measure, and less attention paid to actual outcomes.
      Our findings support the contention of Draycott et al
      • Draycott T.
      • Sibanda T.
      • Laxton C.
      • Winter C.
      • Mahmood T.
      • Fox R.
      Quality improvement demands quality measurement.
      that, although process measures may be associated with an adverse outcome, the hospitals that perform best on those measures or combinations of those measures do not necessarily have the best risk-adjusted rates of obstetric morbidity. This may be because the labor and delivery process is complex and dynamic, and the evidence base for best practice remains poor. Indeed, the wide variation in the use of different obstetric practices (starting from the time a woman is admitted, continuing through her labor, and present at her delivery) are another manifestation of the lack of consensus for what constitutes best care during many aspects of labor.
      These data do not imply that process measurement lacks any value. Process measurement may provide insight into types of care that hospitals wish to perform more frequently and may help direct internal improvement initiatives. Also, although we believe we have selected and analyzed process measures that are most likely to be associated with variation in outcomes, there are other process measures that exist; we cannot rule out the possibility that these unstudied measures would have a relationship with interhospital variation of outcomes. Nevertheless, such relationships have not been demonstrated, and our findings suggest that the care factors underlying interhospital variation in obstetric outcomes remain poorly understood and that the practice of ranking individual hospital obstetric quality based on frequency of adherence to certain process measures may provide poor insight into which hospitals actually achieve the best outcomes.

      Acknowledgments

      The authors thank the subcommittee members who participated in protocol development and coordination between clinical research centers (Cynthia Milluzzi, RN, and Joan Moss, RNC, MSN), protocol/data management and statistical analysis (Elizabeth Thom, PhD, and Yuan Zhao, MS), and protocol development and oversight (Cathy Y. Spong, MD and Brian M. Mercer, MD).
      In addition to the authors, the following list gives the names of the other members of the Eunice Kennedy Shriver National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network: Northwestern University, Chicago, IL: G. Mallett, M. Ramos-Brinson, A. Roy, L. Stein, P. Campbell, C. Collins, N. Jackson, M. Dinsmoor (NorthShore University Health System), J. Senka (NorthShore University Health System), K. Paychek (NorthShore University Health System), A. Peaceman; Case Western Reserve University-MetroHealth Medical Center, Cleveland, OH: B. Mercer, C. Milluzzi, W. Dalton, T. Dotson, P. McDonald, C. Brezine, A. McGrail; Columbia University, New York, NY: M. Talucci, M. Zylfijaj, Z. Reid (Drexel U.), R. Leed (Drexel U.), J. Benson (Christiana H.), S. Forester (Christiana H.), C. Kitto (Christiana H.), S. Davis (St. Peter's UH.), M. Falk (St. Peter's UH.), C. Perez (St. Peter's UH.); University of Utah Health Sciences Center, Salt Lake City, UT: K. Hill, A. Sowles, J. Postma (LDS Hospital), S. Alexander (LDS Hospital), G. Andersen (LDS Hospital), V. Scott (McKay-Dee), V. Morby (McKay-Dee), K. Jolley (UVRMC), J. Miller (UVRMC), B. Berg (UVRMC); University of North Carolina at Chapel Hill, Chapel Hill, NC: K. Dorman, J. Mitchell, E. Kaluta, K. Clark (WakeMed), K. Spicer (WakeMed), S. Timlin (Rex), K. Wilson (Rex); University of Texas Southwestern Medical Center, Dallas, TX: L. Moseley, M. Santillan, J. Price, K. Buentipo, V. Bludau, T. Thomas, L. Fay, C. Melton, J. Kingsbery, R. Benezue; University of Pittsburgh, Pittsburgh, PA: H. Simhan, M. Bickus, D. Fischer, T. Kamon (deceased), D. DeAngelis; The Ohio State University, Columbus, OH: C. Latimer, L. Guzzo (St. Ann's), F. Johnson, L. Gerwig (St. Ann's), S. Fyffe, D. Loux (St. Ann's), S. Frantz, D. Cline, S. Wylie, P. Shubert (St. Ann's); University of Alabama at Birmingham, Birmingham, AL: M. Wallace, A. Northen, J. Grant, C. Colquitt; University of Texas Medical Branch, Galveston, TX: J. Moss, A. Salazar, A. Acosta, G. Hankins; Wayne State University, Detroit, MI: N. Hauff, L. Palmer, P. Lockhart, D. Driscoll, L. Wynn, C. Sudz, D. Dengate, C. Girard, S. Field; Brown University, Providence, RI: P. Breault, F. Smith, N. Annunziata, D. Allard, J. Silva, M. Gamage, J. Hunt, J. Tillinghast, N. Corcoran, M. Jimenez; The University of Texas Health Science Center at Houston-Children's Memorial Hermann Hospital, Houston, TX: S. Blackwell, F. Ortiz, P. Givens, B. Rech, C. Moran, M. Hutchinson, Z. Spears, C. Carreno, B. Heaps, G. Zamora; Oregon Health & Science University, Portland, OR: J. Seguin, M. Rincon, J. Snyder, C. Farrar, E. Lairson, C. Bonino, W. Smith (Kaiser Permanente), K. Beach (Kaiser Permanente), S. Van Dyke (Kaiser Permanente), S. Butcher (Kaiser Permanente); The George Washington University Biostatistics Center: E. Thom, Y. Zhao, P. McGee, V. Momirova, R. Palugod, B. Reamer, M. Larsen; Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, MD: C. Spong and S. Tolivaisa

      Appendix

      Figure thumbnail fx1
      Supplemental Figure 1Odds ratios for each hospital for the outcome of postpartum hemorrhage
      The following logistic regression models were used: A, unadjusted model; B, model with patient characteristics; C, model with patient and provider/hospital characteristics; and D, model with patient, provider/hospital, and care characteristics. The solid line represents the unadjusted model. The red dots indicate 99% confidence intervals that exclude 1.0; the blue dots indicate 99% confidence intervals that include 1.0.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      Figure thumbnail fx2
      Supplemental Figure 2Odds ratios for each hospital for the outcome of peripartum infection
      The following logistic regression models were used: A, unadjusted model; B, model with patient characteristics; C, model with patient and provider/hospital characteristics; and D, model with patient, provider/hospital, and care characteristics. The solid line represents the unadjusted model. The red dots indicate 99% confidence intervals that exclude 1.0; the blue dots indicate 99% confidence intervals that include 1.0.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      Figure thumbnail fx3
      Supplemental Figure 3Odds ratios for each hospital for the outcome of severe perineal laceration at spontaneous vaginal delivery
      The following logistic regression models were used: A, unadjusted model; B, model with patient characteristics; C, model with patient and provider/hospital characteristics; and D, model with patient, provider/hospital, and care characteristics. The solid line represents the unadjusted model. The red dots indicate 99% confidence intervals that exclude 1.0; the blue dots indicate 99% confidence intervals that include 1.0.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      Figure thumbnail fx4
      Supplemental Figure 4Odds ratios for each hospital for the outcome of severe perineal laceration at forceps-assisted vaginal delivery
      The following logistic regression models were used: A, unadjusted model; B, model with patient characteristics; and C, model with patient and provider/hospital characteristics. (There is no Figure 4, D because none of the care characteristics that were assessed was associated with this outcome.) The solid line represents the unadjusted model. The red dots indicate 99% confidence intervals that exclude 1.0; the blue dots indicate 99% confidence intervals that include 1.0.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      Figure thumbnail fx5
      Supplemental Figure 5Odds ratios for each hospital for the outcome of severe perineal laceration at vacuum-assisted vaginal delivery
      The following logistic regression models were used: A, unadjusted model; B, model with patient characteristics; C, model with patient and provider/hospital characteristics; and D, model with patient, provider/hospital, and care characteristics. The solid line represents the unadjusted model. The red dots indicate 99% confidence intervals that exclude 1.0; the blue dots indicate 99% confidence intervals that include 1.0.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.
      Figure thumbnail fx6
      Supplemental Figure 6Odds ratios for each hospital for the composite neonatal adverse outcome
      The following logistic regression models were used: A, unadjusted model; B, model with patient characteristics; C, model with patient and provider/hospital characteristics; and D, model with patient, provider/hospital, and care characteristics. The solid line represents the unadjusted model. The red dots indicate 99% confidence intervals that exclude 1.0; the blue dots indicate 99% confidence intervals that include 1.0.
      Grobman. Relationship of hospitals' obstetric outcomes and care provided. Am J Obstet Gynecol 2014.

      References

        • Lilford R.
        • Mohammed M.A.
        • Spiegelhalter D.
        • Thomson R.
        Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma.
        Lancet. 2004; 363: 1147-1154
        • Pronovost P.J.
        • Thompson D.A.
        • Holzmueller C.G.
        • Lubomski L.H.
        • Morlock L.L.
        Defining and measuring patient safety.
        Crit Care Clin. 2005; 21: 1-19
        • Aron D.C.
        • Harper D.L.
        • Shepardson L.B.
        • Rosenthal G.E.
        Impact of risk-adjusting cesarean delivery rates when reporting hospital performance.
        JAMA. 1998; 279: 1968-1972
        • Grobman W.A.
        • Feinglass J.
        • Murthy S.
        Are the Agency for Healthcare Research and Quality obstetric trauma indicators valid measures of hospital safety?.
        Am J Obstet Gynecol. 2006; 195: 868-874
        • Draycott T.
        • Sibanda T.
        • Laxton C.
        • Winter C.
        • Mahmood T.
        • Fox R.
        Quality improvement demands quality measurement.
        BJOG. 2010; 117: 1571-1574
        • Bailit J.L.
        • Grobman W.A.
        • Rice M.M.
        • et al.
        Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals.
        Am J Obstet Gynecol. 2013; 209: 446.e1-446.e30
        • Eldridge S.
        • Asby D.
        • Kerry S.
        Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and method.
        Int J Epidemiol. 2006; 35: 1292-1300
        • Hsieh F.Y.
        • Bloch D.A.
        • Larsen M.D.
        A simple method of sample size calculation for linear and logistic regression.
        Stat Med. 1998; 17: 1623-1634
        • Lancaster G.A.
        • Chellaswamy H.
        • Taylor S.
        • Lyon D.
        • Dowrick C.
        Design of a clustered observational study to predict emergency admissions in the elderly: statistical reasoning in clinical practice.
        J Eval Clin Pract. 2007; 13: 169-178
        • Synnes A.R.
        • MacNab Y.C.
        • Qiu Z.
        • et al.
        Neonatal intensive care unit characteristics affect the incidence of severe intraventricular hemorrhage.
        Med Care. 2006; 44: 754-759
      1. Peterson ED, Roe MT, Mulgund J, et al. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912-20.