Can differences in obstetric outcomes be explained by differences in the care provided? The MFMU Network APEX study

During the study period, data were collected on 115,502 women and their neonates and on 1797 different delivery-attending providers at 25 hospitals. Characteristics of these patients and their providers and hospitals are provided in Tables 1 and 2. As shown, women were delivered by a variety of types of providers, and these providers had a range of experience. Hospital characteristics, including availability of medical services (eg, obstetric anesthesia), the presence of electronic medical records, and the attendance of providers at structured obstetric patient review, varied as well.

The frequencies of the selected outcomes were as follows: PPH, 2.29% (95% CI, 2.20–2.38%), peripartum infection, 5.06% (95% CI, 4.93–5.19%), severe perineal laceration at spontaneous vaginal delivery, 2.16% (95% CI, 2.06–2.27%), severe perineal laceration at forceps-assisted vaginal delivery, 27.56% (95% CI, 25.54–29.57%), severe perineal laceration at vacuum-assisted vaginal delivery, 14.51% (95% CI, 13.34–15.67%), and composite neonatal adverse outcome, 2.73% (95% CI, 2.63–2.84%). As previously reported, the frequency of the selected adverse outcomes varied widely and differed significantly among hospitals (P < .001 for all). The type of care experienced by patients at different hospitals varied widely as well (Table 3).The frequency of labor induction among women who were eligible for such an intervention, for example, ranged among hospitals from 21-37%. Oxytocin at rates >20 mU/min rarely was administered to laboring women at some hospitals; however, this practice occurred in nearly 50% of women who received oxytocin at other hospitals. There was a >20-fold difference in the frequency of delayed pushing among women who reached the second stage and a difference in the frequency of vaginal delivery that ranged from 61-80%. Delivery practices varied as well, with a 50-fold difference in the frequency of episiotomy among women who had a vaginal delivery and >10-fold difference in the use of general anesthesia at cesarean delivery.

The associations of processes measures (individual and composite exposure score) with the studied outcomes are given in Table 4. Even after adjusting for patient, provider, and hospital characteristics, particular types of obstetric care remained associated with the outcomes of interest.

Supplementary Figure 1 (Appendix) represents the hospital differences in PPH and how those differences are affected by the sequential addition of independent variables in the different categories (ie, patient, provider/hospital, and care). For example, Supplementary Figure 1, A illustrates the odds ratio for each hospital (identified by the numbers 1-25 on the x-axis) for the outcome of PPH that derived from the logistic regression model without any risk-factor adjustment. Hospitals differ significantly from one another (P < .01), and some hospitals have significantly higher or lower odds of an outcome than the reference hospital (ie, 99% CIs do not include 1.0). If patient, hospital, and process characteristics are associated with the outcomes, as they are entered into the regression model, variation among the odds ratios of the hospitals should lessen. If all variation were explained by these characteristics, the odds ratios associated with each hospital would be 1.0.

The results of adjusting for only patient characteristics are shown in Supplementary Figure 1, B, with the results obtained after the further addition of provider/hospital characteristics shown in Supplementary Figure 1, C, which shows a progressive reduction in the variation of the odds ratios, as illustrated by the hospitals' odds ratio point estimates that have migrated from their original positions and towards the line that represents an odds ratio = 1. However, when care variables are entered into the model, either as a single variable such as labor induction (data not shown) or as a composite exposure score (Supplementary Figure 1, D), the odds ratios that are associated with each hospital largely are unchanged. Graphic representations for the odds ratios that are associated with each stage of the model for the other outcomes are presented in the Supplementary Figures 2-6.

Table 5 presents the variation between hospitals (standard deviation of the random effects term) that are associated with each stage of the hierarchical logistic regression for each outcome. For infection, none of the interhospital variation was explained by patient characteristics, whereas for the other outcomes 20-40% (percentage difference between the standard deviations) of the hospital's variation in outcomes was related to differences in patient populations. Approximately 20% of the variation in hospital PPH frequency was related to provider/hospital factors. However, for the other outcomes, there was little evidence that interhospital outcome variation was related to provider/hospital factors. In no case did differences in types of obstetric care account for much of the variation in observed outcomes. Ultimately, 50-100% of the interhospital variation in outcomes was unexplained.

In this study, we investigated the relationship between differences in obstetric care patterns and outcomes among hospitals. Several findings are notable. Despite the fact that the hospitals in the study were either university or university-affiliated and part of a single research network, the frequencies of obstetric practices were vastly different. After we controlled for differences in patient populations and hospital characteristics, several types of obstetric care were found to be associated with adverse obstetric outcomes. Nevertheless, this association did not translate into a capability to explain the hospital differences in adverse outcomes that were found.

This lack of explanatory power is in contrast to that discerned for care processes in some other disciplines. For example, Synnes et al examined variation in the frequency of intraventricular hemorrhage among neonates in the intensive care unit. In an analysis similar to ours, after they controlled for patient and hospital factors, they were able to demonstrate that differences in acidosis treatment, vasopressin use, and surfactant use could account for differences in interhospital rates of intraventricular hemorrhage. Similarly, studying adults with cardiac disease, Petersen et al demonstrated that adherence to particular types of management (such as beta-blocker use) could explain differences in hospitals' adjusted-mortality rates.

Process measures, however, have not been well demonstrated to explain interhospital variation in obstetric outcomes. The inability to do so in the obstetric population that we studied has implications with regard to obstetric quality measurement and its interpretation. Process measures quantify adherence to a given type of care. Hospitals often are judged according to their adherence to selected process measures, with the implicit assumption that the hospitals that perform best on selected measures will have the best health outcomes. Yet, Draycott et al have called attention to the fact that this relationship need not hold. Further, they cite examples to illustrate that belief in an inexorable relationship between process measures and outcomes may hinder quality improvement if there is undue focus on process measures, which may be relatively easy to measure, and less attention paid to actual outcomes.

Our findings support the contention of Draycott et al that, although process measures may be associated with an adverse outcome, the hospitals that perform best on those measures or combinations of those measures do not necessarily have the best risk-adjusted rates of obstetric morbidity. This may be because the labor and delivery process is complex and dynamic, and the evidence base for best practice remains poor. Indeed, the wide variation in the use of different obstetric practices (starting from the time a woman is admitted, continuing through her labor, and present at her delivery) are another manifestation of the lack of consensus for what constitutes best care during many aspects of labor.

These data do not imply that process measurement lacks any value. Process measurement may provide insight into types of care that hospitals wish to perform more frequently and may help direct internal improvement initiatives. Also, although we believe we have selected and analyzed process measures that are most likely to be associated with variation in outcomes, there are other process measures that exist; we cannot rule out the possibility that these unstudied measures would have a relationship with interhospital variation of outcomes. Nevertheless, such relationships have not been demonstrated, and our findings suggest that the care factors underlying interhospital variation in obstetric outcomes remain poorly understood and that the practice of ranking individual hospital obstetric quality based on frequency of adherence to certain process measures may provide poor insight into which hospitals actually achieve the best outcomes.

The authors thank the subcommittee members who participated in protocol development and coordination between clinical research centers (Cynthia Milluzzi, RN, and Joan Moss, RNC, MSN), protocol/data management and statistical analysis (Elizabeth Thom, PhD, and Yuan Zhao, MS), and protocol development and oversight (Cathy Y. Spong, MD and Brian M. Mercer, MD).

In addition to the authors, the following list gives the names of the other members of the Eunice Kennedy Shriver National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network: Northwestern University, Chicago, IL: G. Mallett, M. Ramos-Brinson, A. Roy, L. Stein, P. Campbell, C. Collins, N. Jackson, M. Dinsmoor (NorthShore University Health System), J. Senka (NorthShore University Health System), K. Paychek (NorthShore University Health System), A. Peaceman; Case Western Reserve University-MetroHealth Medical Center, Cleveland, OH: B. Mercer, C. Milluzzi, W. Dalton, T. Dotson, P. McDonald, C. Brezine, A. McGrail; Columbia University, New York, NY: M. Talucci, M. Zylfijaj, Z. Reid (Drexel U.), R. Leed (Drexel U.), J. Benson (Christiana H.), S. Forester (Christiana H.), C. Kitto (Christiana H.), S. Davis (St. Peter's UH.), M. Falk (St. Peter's UH.), C. Perez (St. Peter's UH.); University of Utah Health Sciences Center, Salt Lake City, UT: K. Hill, A. Sowles, J. Postma (LDS Hospital), S. Alexander (LDS Hospital), G. Andersen (LDS Hospital), V. Scott (McKay-Dee), V. Morby (McKay-Dee), K. Jolley (UVRMC), J. Miller (UVRMC), B. Berg (UVRMC); University of North Carolina at Chapel Hill, Chapel Hill, NC: K. Dorman, J. Mitchell, E. Kaluta, K. Clark (WakeMed), K. Spicer (WakeMed), S. Timlin (Rex), K. Wilson (Rex); University of Texas Southwestern Medical Center, Dallas, TX: L. Moseley, M. Santillan, J. Price, K. Buentipo, V. Bludau, T. Thomas, L. Fay, C. Melton, J. Kingsbery, R. Benezue; University of Pittsburgh, Pittsburgh, PA: H. Simhan, M. Bickus, D. Fischer, T. Kamon (deceased), D. DeAngelis; The Ohio State University, Columbus, OH: C. Latimer, L. Guzzo (St. Ann's), F. Johnson, L. Gerwig (St. Ann's), S. Fyffe, D. Loux (St. Ann's), S. Frantz, D. Cline, S. Wylie, P. Shubert (St. Ann's); University of Alabama at Birmingham, Birmingham, AL: M. Wallace, A. Northen, J. Grant, C. Colquitt; University of Texas Medical Branch, Galveston, TX: J. Moss, A. Salazar, A. Acosta, G. Hankins; Wayne State University, Detroit, MI: N. Hauff, L. Palmer, P. Lockhart, D. Driscoll, L. Wynn, C. Sudz, D. Dengate, C. Girard, S. Field; Brown University, Providence, RI: P. Breault, F. Smith, N. Annunziata, D. Allard, J. Silva, M. Gamage, J. Hunt, J. Tillinghast, N. Corcoran, M. Jimenez; The University of Texas Health Science Center at Houston-Children's Memorial Hermann Hospital, Houston, TX: S. Blackwell, F. Ortiz, P. Givens, B. Rech, C. Moran, M. Hutchinson, Z. Spears, C. Carreno, B. Heaps, G. Zamora; Oregon Health & Science University, Portland, OR: J. Seguin, M. Rincon, J. Snyder, C. Farrar, E. Lairson, C. Bonino, W. Smith (Kaiser Permanente), K. Beach (Kaiser Permanente), S. Van Dyke (Kaiser Permanente), S. Butcher (Kaiser Permanente); The George Washington University Biostatistics Center: E. Thom, Y. Zhao, P. McGee, V. Momirova, R. Palugod, B. Reamer, M. Larsen; Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, MD: C. Spong and S. Tolivaisa