Advertisement

Special statement: Proposed quality metrics to assess accuracy of prenatal detection of congenital heart defects

Published:February 27, 2020DOI:https://doi.org/10.1016/j.ajog.2020.02.040
      Congenital heart defects are a leading cause of neonatal morbidity and mortality. Accurate prenatal diagnosis of congenital heart defects can reduce morbidity and mortality by improving prenatal care, facilitating predelivery pediatric cardiology consultation, and directing delivery to facilities with resources to manage the complex medical and surgical needs of newborns with congenital heart defects. Unfortunately, less than one half of congenital heart defect cases are detected prenatally, resulting in lost opportunities for counseling, shared decision-making, and delivery at an appropriate facility. Quality improvement initiatives to improve prenatal congenital heart defects detection depend on the ability to measure the rate of detection at the level of providers, facilities, or populations, but no standard metric exists for measuring the detection of congenital heart defects at any level. The need for such a metric was recognized at a Cooperative Workshop held at the 2016 Annual Meeting of the Society for Maternal-Fetal Medicine, which recommended the development of a quality metric to assess the rate of prenatal detection of clinically significant congenital heart defects. In this paper, we propose potential quality metrics to measure prenatal detection of critical congenital heart defects, defined as defects with a high rate of morbidity or mortality or that require surgery or tertiary follow-up. One metric is based on a retrospective approach, assessing whether postnatally diagnosed congenital heart defects had been identified prenatally. Other metrics are based on a prospective approach, assessing the sensitivity and specificity of prenatal diagnosis of congenital heart defects by comparing prenatal ultrasound findings with newborn findings. Potential applications, limitations, challenges, barriers, and value for both approaches are discussed. We conclude that future development of these metrics will depend on an expansion of the International Classification of Diseases system to include specific codes that distinguish fetal congenital heart defects from newborn congenital heart defects and on the development of record systems that facilitate the linkage of fetal records (in the maternal chart) with newborn records.

      Key words

      The Problem: A high percentage of congenital heart defects (CHD) are not detected prenatally, resulting in suboptimal care and increased neonatal morbidity and mortality.
      A Solution: We propose quality metrics for tracking CHD detection rates to address this quality gap.
      Congenital heart defects (CHD) are among the most common congenital anomalies,
      • Parker S.E.
      • Mai C.T.
      • Canfield M.A.
      • et al.
      Updated national birth prevalence estimates for selected birth defects in the United States, 2004–2006.
      • Mai C.T.
      • Isenburg J.
      • Langlois P.H.
      • et al.
      Population-based birth defects data in the United States, 2008 to 2012: presentation of state-specific data and descriptive brief on variability of prevalence.
      • Kirby R.S.
      The prevalence of selected major birth defects in the United States.
      • Mai C.T.
      • Isenburg J.L.
      • Canfield M.A.
      • et al.
      National population-based estimates for major birth defects, 2010–2014.
      affecting 0.4–0.8% of newborns
      • Reller M.D.
      • Strickland M.J.
      • Riehle-Colarusso T.
      • Mahle W.T.
      • Correa A.
      Prevalence of congenital heart defects in metropolitan Atlanta, 1998–2005.
      • Dolk H.
      • Loane M.
      • Garne E.
      European Surveillance of Congenital Anomalies Working Group. Congenital heart defects in Europe: prevalence and perinatal mortality, 2000 to 2005.
      • Pei L.
      • Kang Y.
      • Zhao Y.
      • Yan H.
      Prevalence and risk factors of congenital heart defects among live births: a population-based cross-sectional survey in Shaanxi province, Northwestern China.
      • Ficara A.
      • Syngelaki A.
      • Hammami A.
      • Akolekar R.
      • Nicolaides K.H.
      Value of routine ultrasound examination at 35–37 weeks' gestation in diagnosis of fetal abnormalities.
      and are the leading cause of death among infants with birth defects.
      • Benjamin E.J.
      • Blaha M.J.
      • Chiuve S.E.
      • et al.
      Heart Disease and stroke statistics—2017 update: a report from the American Heart Association.
      Unfortunately, less than one half of CHD are detected prenatally,
      • Khoo N.S.
      • Van Essen P.
      • Richardson M.
      • Robertson T.
      Effectiveness of prenatal diagnosis of congenital heart defects in South Australia: a population analysis 1999–2003.
      • Stoll C.
      • Garne E.
      • Clementi M.
      EUROSCAN Study Group
      Evaluation of prenatal diagnosis of associated congenital heart diseases by fetal ultrasonographic examination in Europe.
      • Marek J.
      • Tomek V.
      • Skovranek J.
      • Povysilova V.
      • Samanek M.
      Prenatal ultrasound screening of congenital heart disease in an unselected national population: a 21-year experience.
      • McBrien A.
      • Sands A.
      • Craig B.
      • Dornan J.
      • Casey F.
      Major congenital heart disease: antenatal detection, patient characteristics and outcomes.
      • Pinto N.M.
      • Keenan H.T.
      • Minich L.L.
      • Puchalski M.D.
      • Heywood M.
      • Botto L.D.
      Barriers to prenatal detection of congenital heart disease: a population-based study.
      • Acharya G.
      • Sitras V.
      • Maltau J.M.
      • et al.
      Major congenital heart disease in northern Norway: shortcomings of pre- and postnatal diagnosis.
      • Bull C.
      Current and potential impact of fetal diagnosis on prevalence and spectrum of serious congenital heart disease at term in the UK. British Paediatric Cardiac Association.
      • Friedberg M.K.
      • Silverman N.H.
      • Moon-Grady A.J.
      • et al.
      Prenatal detection of congenital heart disease.
      • Garne E.
      • Stoll C.
      • Clementi M.
      • Euroscan G.
      Evaluation of prenatal diagnosis of congenital heart diseases by ultrasound: experience from 20 European registries.
      • Jaeggi E.T.
      • Sholler G.F.
      • Jones O.D.
      • Cooper S.G.
      Comparative analysis of pattern, management and outcome of pre- versus postnatally diagnosed major congenital heart disease: a population-based study.
      • Parikh L.I.
      • Grantz K.L.
      • Iqbal S.N.
      • et al.
      Neonatal outcomes in fetuses with cardiac anomalies and the impact of delivery route.
      • Ailes E.C.
      • Gilboa S.M.
      • Riehle-Colarusso T.
      • et al.
      Prenatal diagnosis of nonsyndromic congenital heart defects.
      • Quartermain M.D.
      • Pasquali S.K.
      • Hill K.D.
      • et al.
      Variation in prenatal diagnosis of congenital heart disease in infants.
      although detection rates vary, depending on the type of defect,
      • Parikh L.I.
      • Grantz K.L.
      • Iqbal S.N.
      • et al.
      Neonatal outcomes in fetuses with cardiac anomalies and the impact of delivery route.
      ,
      • Quartermain M.D.
      • Pasquali S.K.
      • Hill K.D.
      • et al.
      Variation in prenatal diagnosis of congenital heart disease in infants.
      • Khoshnood B.
      • Lelong N.
      • Houyel L.
      • et al.
      Impact of prenatal diagnosis on survival of newborns with four congenital heart defects: a prospective, population-based cohort study in France (the EPICARD Study).
      • van Velzen C.L.
      • Clur S.A.
      • Rijlaarsdam M.E.
      • et al.
      Prenatal detection of congenital heart disease—results of a national screening programme.
      examiner skill,
      • Tegnander E.
      • Eik-Nes S.H.
      The examiner's ultrasound experience has a significant impact on the detection rate of congenital heart defects at the second-trimester fetal examination.
      and specific population.
      • Tegnander E.
      • Williams W.
      • Johansen O.J.
      • Blaas H.G.
      • Eik-Nes S.H.
      Prenatal detection of heart defects in a non-selected population of 30,149 fetuses—detection rates and outcome.
      ,
      • Lytzen R.
      • Vejlstrup N.
      • Bjerre J.
      • et al.
      Live-born major congenital heart disease in Denmark: incidence, detection rate, and termination of pregnancy rate from 1996 to 2013.
      Prenatal detection of CHD improves neonatal surgical outcomes and childhood developmental milestones and reduces neonatal mortality.
      • Levey A.
      • Glickstein J.S.
      • Kleinman C.S.
      • et al.
      The impact of prenatal diagnosis of complex congenital heart disease on neonatal outcomes.
      • Cloete E.
      • Bloomfield F.H.
      • Sadler L.
      • de Laat M.W.M.
      • Finucane A.K.
      • Gentles T.L.
      Antenatal detection of treatable critical congenital heart disease is associated with lower morbidity and mortality.
      • Calderon J.
      • Angeard N.
      • Moutier S.
      • Plumet M.H.
      • Jambaque I.
      • Bonnet D.
      Impact of prenatal diagnosis on neurocognitive outcomes in children with transposition of the great arteries.
      • Khoshnood B.
      • Lelong N.
      • Houyel L.
      • et al.
      Prevalence, timing of diagnosis and mortality of newborns with congenital heart defects: a population-based study.
      • Brown M.D.
      • Wernovsky G.
      • Mussatto K.A.
      • Berger S.
      Long-term and developmental outcomes of children with complex congenital heart disease.
      • Tworetzky W.
      • McElhinney D.B.
      • Reddy V.M.
      • Brook M.M.
      • Hanley F.L.
      • Silverman N.H.
      Improved surgical outcome after fetal diagnosis of hypoplastic left heart syndrome.
      Prenatal diagnosis facilitates counseling, shared decision-making, and, if appropriate, transfer for delivery at a facility with resources to manage a critically ill newborn.
      • Friedberg M.K.
      • Silverman N.H.
      • Moon-Grady A.J.
      • et al.
      Prenatal detection of congenital heart disease.
      ,
      • Berndl A.
      • Pearce K.
      • Mondal T.
      Fetal echocardiography assists in determining optimal delivery site.
      ,
      • Levy D.J.
      • Pretorius D.H.
      • Rothman A.
      • et al.
      Improved prenatal detection of congenital heart disease in an integrated health care system.
      Quality improvement (QI) initiatives can improve the prenatal detection of CHD.
      • Levy D.J.
      • Pretorius D.H.
      • Rothman A.
      • et al.
      Improved prenatal detection of congenital heart disease in an integrated health care system.
      • Evans W.
      • Castillo W.
      • Rollins R.
      • et al.
      Moving towards universal prenatal detection of critical congenital heart disease in southern Nevada: a community-wide program.
      • Letourneau K.M.
      • Horne D.
      • Soni R.N.
      • McDonald K.R.
      • Karlicki F.C.
      • Fransoo R.R.
      Advancing prenatal detection of congenital heart disease: a novel screening protocol improves early diagnosis of complex congenital heart disease.
      • Evans W.N.
      • Acherman R.J.
      • Ciccolo M.L.
      • Lehoux J.
      • Rothman A.
      • Galindo A.
      Detecting critical congenital heart disease in Nevada.
      • Hautala J.
      • Gissler M.
      • Ritvanen A.
      • et al.
      The implementation of a nationwide anomaly screening programme improves prenatal detection of major cardiac defects: an 11-year national population-based cohort study.
      The QI catchphrase, “If you cannot measure it, you cannot improve it,”
      • Hunter J.
      Myth: if you can't measure it, you can't manage it. August 13, 2015.
      implies that improving CHD detection depends on the measurement of the rate of correct diagnoses. Currently there is no standardized quality metric to measure CHD detection performance.
      In 2016, the Society for Maternal-Fetal Medicine (SMFM) hosted a multistakeholder Cooperative Workshop entitled “Quality Measures in High-Risk Pregnancies” to consider new quality metrics for high-risk pregnancies.
      • Iriye B.K.
      • Gregory K.D.
      • Saade G.R.
      • Grobman W.A.
      • Brown H.L.
      Quality measures in high-risk pregnancies: Executive summary of a cooperative workshop of the Society for Maternal-Fetal Medicine, National Institute of Child Health and Human Development, and the American College of Obstetricians and Gynecologists.
      The Workshop Executive Summary recommended the development of a quality metric for prenatal detection of clinically significant CHD. In this paper, we define clinically significant CHD as any of the diagnoses listed in Table 1, conditions that are considered critical and that typically require surgical repair in infancy to avoid death or major morbidity.
      • Jelliffe-Pawlowski L.L.
      • Norton M.E.
      • Shaw G.M.
      • et al.
      Risk of critical congenital heart defects by nuchal translucency norms.
      These diagnoses comprise approximately 20–30% of all CHD.
      • Reller M.D.
      • Strickland M.J.
      • Riehle-Colarusso T.
      • Mahle W.T.
      • Correa A.
      Prevalence of congenital heart defects in metropolitan Atlanta, 1998–2005.
      ,
      • Pei L.
      • Kang Y.
      • Zhao Y.
      • Yan H.
      Prevalence and risk factors of congenital heart defects among live births: a population-based cross-sectional survey in Shaanxi province, Northwestern China.
      Table 1Critical congenital heart defects considered for quality metrics
      Congenital heart defectICD-10-CM code
      Common truncus arteriosusQ20.0
      Transposition of the great arteriesQ20.3
      Tetralogy of FallotQ21.3
      Common ventricleQ20.4
      Endocardial cushion defectQ21.2
      Pulmonary valve atresia, stenosis, insufficiencyQ22.0, Q22.1, Q22.3
      Tricuspid atresia or stenosisQ22.4, Q22.8, Q22.9, I36.0
      Hypoplastic left heart syndromeQ23.4
      Hypoplastic right heart syndromeQ22.6
      Aortic stenosis, valvular, subaortic, or supravalvularQ23.0, Q24.4, Q25.3
      Ebstein anomalyQ22.5
      Coarctation or atresia of the aortaQ25.1, Q25.2, Q25.21, Q25.29
      Pulmonary artery atresiaQ25.5
      Anomalies of the great veinsQ26
      Any digit 0–9.
      ICD-10-CM, International Classification of Diseases, 10th Revision, Clinical Modification.
      Adapted from Jelliffe-Pawlowski et al.
      • Jelliffe-Pawlowski L.L.
      • Norton M.E.
      • Shaw G.M.
      • et al.
      Risk of critical congenital heart defects by nuchal translucency norms.
      Society for Maternal-Fetal Medicine. Special statement. Am J Obstet Gynecol 2020.
      a Any digit 0–9.
      We propose 2 types of quality metrics to measure prenatal detection of critical CHD. The first type is retrospective, initially identifying newborns with critical CHD and retrospectively determining whether CHD was diagnosed prenatally. The second uses a prospective approach, starting with a cohort of prenatal ultrasound examinations, followed by measurement of the sensitivity and specificity of subsequent diagnoses of CHD. For both approaches, potential uses, limitations, and barriers are discussed.

      Retrospective cohort metric: percentage of newborns with CHD who had a prenatal diagnosis

      This metric begins with a cohort of newborns with CHD and retrospectively evaluates the proportion with a prenatal diagnosis of CHD. The simplified measure specification is:
      Denominator: Number of newborns with a critical CHD diagnosis (Table 1).
      Numerator: Number of newborns in the denominator who had CHD identified prenatally.
      Metric: Percentage (numerator divided by denominator). Theoretical ideal detection is 100%.
      This metric is similar to the one proposed at the 2016 Cooperative Workshop.
      • Iriye B.K.
      • Gregory K.D.
      • Saade G.R.
      • Grobman W.A.
      • Brown H.L.
      Quality measures in high-risk pregnancies: Executive summary of a cooperative workshop of the Society for Maternal-Fetal Medicine, National Institute of Child Health and Human Development, and the American College of Obstetricians and Gynecologists.
      It would most readily be applied at the hospital or hospital-system level. Although it initially appears straightforward, the metric is complicated by several barriers and limitations, as detailed in the critique in Table 2. First, it cannot be tracked using claims or administrative data because there is no specific code for fetal CHD in the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM). Metrics that can be tracked via ICD-10-CM codes or other structured data are generally preferred because they minimize the burden of data collection.
      National Quality Forum
      Measure Evaluation Criteria. 2012.
      Without a specific code, a manual review of maternal charts will be required to determine whether each case of newborn CHD was diagnosed prenatally.
      Table 2Critique of the retrospective metric: percentage of newborns with a CHD-detected prenatally
      IssueExplanationLimitations, barriers, disadvantages
      Population studiedAll newborns delivered at a given hospital during a specified time period
      Likely time period of analysisCalendar yearA small number of cases will result in rates with wide confidence intervals. (For example, a hospital with 1000 births per year may have only 10 CHD cases per year. If 5 are detected, the rate is 50%, but the 95% confidence interval ranges from 20% to 80%).
      Method of capturing denominatorICD-10-CM code list (Table 1) from newborn record• Denominator does not include CHD cases that are not detected by the time of newborn discharge. If eligibility time period is extended to capture diagnoses after discharge, the hospital may not have access to infant outpatient records.

      • Denominator does not include CHD cases that do not result in a newborn diagnosis (eg, miscarriage, termination, transfer for delivery elsewhere).
      Method of capturing numeratorManual review of admitting diagnoses in maternal admitting record and prenatal record• There is no specific ICD-10-CM for fetal CHD; therefore, manual review is required.

      • Numerator is obtained from maternal record, but denominator is defined by newborn record. Health systems may not have effective methods to link maternal and newborn records.
      AttributionMetric tracked at the hospital level• Metric is attributed to hospital, but source of prenatal detection may include a variety of community ultrasound providers.

      • Hospitals have limited ability to influence these community sources and thereby improve quality.

      • As defined, the metric does not require assessing rates for individual ultrasound facilities or providers. If individual rates are assessed, a small number of cases for each facility or provider will result in wide confidence intervals.
      Referral biasReferral of correctly diagnosed cases will affect rates of both the sending and the receiving hospitals• For example, if 50 CHD cases occur within a community hospital’s catchment area and 40 are detected, the actual detection rate is 80%. If the 40 detected cases are referred to deliver at the regional tertiary hospital, this leaves 10 undetected CHD cases delivering at the local hospital. The metric would be 0 cases detected of 10 CHD births at the local hospital, or a rate of 0%, despite an actual detection rate of 80% in the community.

      • Conversely, the delivery population at the regional tertiary hospital population will be enriched by cases sent from the surrounding community hospitals after prenatal detection of CHD. This will inflate the metric for the tertiary hospital.
      Termination biasPregnancy termination in correctly diagnosed cases will result in artifactually low rateFor example, if 50 CHD cases occur within a hospital’s catchment area and 40 cases are detected prenatally, the actual detection rate is 80%. If 30 of the detected cases terminate pregnancy and 10 continue pregnancy, the apparent metric will be based on the 10 births in continuing pregnancies plus the 10 births among the undetected cases. The metric would be 10 detected of 20 total CHD births, or 50% despite an actual detection rate of 80% in the community.
      False-positive diagnosisNot captured• A false-positive prenatal diagnosis of CHD has no bearing on the numerator or denominator of this metric.

      • The metric rate can be increased by increasing the number of diagnoses of CHD, whether accurate or not.

      • A balancing metric would be desirable to detect the extent of over-diagnosis.
      CHD, congenital heart defects; ICD-10-CM, International Classification of Diseases, 10th Revision, Clinical Modification.
      Society for Maternal-Fetal Medicine. Special statement. Am J Obstet Gynecol 2020.
      Another limitation is that prenatal diagnoses are documented in maternal records, but postnatal diagnoses appear in neonatal records. Electronic health record systems rarely provide linkage between the maternal and newborn records to facilitate automated tracking of these diagnoses, so manual linkage will likely be required. Moreover, hospitals may not have access to prenatal ultrasound reports and would thus need to rely on admitting diagnoses, which may be incomplete.
      The likely value of this metric would be to encourage hospitals to develop QI programs to help improve the performance of local practices that provide prenatal diagnostic services for patients who deliver there, including prenatal care clinics and maternal-fetal medicine, radiology, and pediatric cardiology services. This effort could start with a quality-assurance review of cases in which the CHD diagnosis was missed. A focused review may reveal system issues, such as lack of access to prenatal ultrasound, lack of referral for fetal echocardiogram in high-risk patients, or lack of prenatal care (Figure). For patients who had prenatal ultrasound examinations that missed the diagnosis of CHD, the review process could include feedback to all relevant ultrasound providers and a request that images be reviewed for technical quality and accuracy. This feedback alone may help to improve care because many prenatal ultrasound providers receive no systematic information about missed diagnoses and thus have no knowledge about potential quality issues.
      Figure thumbnail gr1
      FigureProspective cohort approach to quality metric for detection of CHD
      The metric is defined as the percentage of all CHD neonates with a diagnosis in the prenatal period (numerator divided by denominator). The remainder of the diagram shows potential quality assurance activities.
      CHD, congenital heart defects; MFM, maternal-fetal medicine subspecialist; OB, obstetrician.
      Society for Maternal-Fetal Medicine. Special statement. Am J Obstet Gynecol 2020.
      Although this metric could be appropriate for hospital-level measurement, it would not provide valid between-hospital comparisons for several reasons. First, it does not necessarily reflect prenatal diagnostic services rendered by the maternity hospital but likely includes services by various community providers. Second, for hospitals with small numbers of cases, the metric will have poor precision (wide confidence intervals). Third, as noted in Table 2, the metric can be biased by referral patterns; a community hospital with a high detection rate that appropriately refers patients to deliver at a tertiary center will have an artifactually low rate on this metric and the receiving hospital will have an artifactually inflated rate. A similar bias would result from pregnancy terminations by women diagnosed with fetal CHD, producing artifactually low apparent detection rates.

      Prospective cohort metrics: sensitivity and specificity of prenatal ultrasound examinations

      The prospective approach defines a cohort of patients who had a prenatal ultrasound examination during a specified time period and then determines the rate of correct prenatal CHD diagnosis. The simplified specification for the main proposed metric is:
      Cohort: All pregnant women who had 1 or more obstetric ultrasound examinations during the measurement period using 1 or more of the following Current Procedural Terminology (CPT) codes: 76805 (basic second- or third-trimester ultrasound), 76810 (basic second- or third-trimester ultrasound, additional fetus), 76811 (detailed fetal anatomy examination), 76812 (detailed fetal anatomy examination, additional fetus), or 76825 (fetal echocardiogram).
      Denominator: Number of patients in the cohort who delivered a newborn with CHD or who had a fetal death (ICD-10 code O36.4 or P95), spontaneous abortion (ICD-10 code O02.1 or O03.xx where x = any digit), or induced abortion (ICD-10 code Z33.2 or O04.xx) after prenatal diagnosis of CHD.
      Numerator: Number of patients in the denominator with CHD diagnosed prenatally.
      Metric (sensitivity): Percentage (numerator divided by denominator). Theoretical ideal detection is 100%.
      This metric is readily recognizable as the sensitivity of ultrasonography as a screening test for CHD. Other test characteristics, such as specificity and predictive values, can be calculated from the 2 × 2 table shown in Table 3.
      Table 3Prospective cohort approach to a quality metric for detection of CHD
      Postnatal findingsPrenatal findings
      Suspected CHDNo suspicion of CHD
      CHDTrue positiveFalse negative
      No CHDFalse positiveTrue negative
      CHD, congenital heart defects.
      Sensitivity metric is true positive/(true positive plus false negative), and specificity metric is true negative/(true negative plus false positive). Positive predictive value is true positive/(true positive plus false positive), and negative predictive value is true negative/(true negative plus false negative).
      Society for Maternal-Fetal Medicine. Special statement. Am J Obstet Gynecol 2020.
      A second metric reflecting specificity can be defined as:
      Denominator: Number of patients in the cohort who delivered a newborn without CHD.
      Numerator: Number of patients in the denominator without a prenatal diagnosis of CHD.
      Metric (specificity): Percentage (numerator divided by denominator). Theoretical ideal specificity is 100%.
      Specificity is a key important balancing metric to guard against the overdiagnosis of fetal CHD. False-positive diagnoses have major medical, emotional, and financial implications, including recommendations for potentially unnecessary tests and procedures such as fetal echocardiography, genetic testing, consultation with pediatric cardiologists and neonatologists, transfer to tertiary facilities, or even pregnancy termination.
      Limitations to the collection and use of these metrics are outlined in Table 4. As with the retrospective metric, the lack of specific ICD-10-CM code(s) for fetal CHD is a major barrier that prevents the use of claims data or administrative data for measure tracking, necessitating burdensome manual record review. As with the retrospective metric, another critical limitation is the general lack of a systematic linkage between fetal diagnoses (which appear in maternal records) and newborn diagnoses (in newborn records). Another limitation is the potential for bias as a result of pregnancy terminations.
      Table 4Critique of prospective metrics: sensitivity and specificity of prenatal ultrasound to detect CHD
      IssueExplanationLimitations, barriers, disadvantages
      Population studiedAll pregnant women with at least 1 prenatal ultrasound examination that includes evaluation of fetal cardiac anatomy
      Likely time period of analysisCalendar yearSmall number of cases will result in rates with wide confidence intervals. (For example, a provider performing 1000 ultrasound examinations per year may have only 10 CHD cases per year. If 5 are detected, the rate is 50%, but the 95% confidence interval ranges from 20% to 80%).
      Method of defining cohortClaims data showing relevant prenatal ultrasound examination performed, including CPT codes 76805, 76810, 76811, 76812, or 76825Cohort can be based on a single ultrasound provider, ultrasound provider group, health system, or payer.
      Method of capturing denominatorICD-10-CM code list (Table 1) from newborn record plus record review of cases of spontaneous or induced abortion after prenatal diagnosis of CHD• Denominator does not include CHD cases that are not detected by the time of newborn hospital discharge. If eligibility time period is extended to capture diagnoses after discharge, infant outpatient records may not be consistently available.

      • Ultrasound provider may not have access to hospital records to obtain newborn diagnoses and may not have access to information regarding occurrence of abortions.

      • Denominator assumes that prenatal diagnosis of CHD was correct in cases of induced abortion, which may not always be true.
      Method of capturing numeratorManual review of ultrasound examination findings to determine whether CHD was diagnosed• There is no specific ICD-10-CM for fetal CHD, so manual review of records is required.

      • Numerator obtained from maternal record, but denominator defined by newborn record.

      Health systems may not have effective methods to link maternal and newborn records.

      • Mother and newborn may be covered by different payor, so even payors may not have ability to link maternal diagnosis with newborn findings.
      AttributionMetric tracked at level of ultrasound provider, provider group, or health planIf individual rates are assessed, a small number of cases for each provider or provider group will result in wide confidence intervals.
      Termination biasRate artifactually elevated by pregnancy terminationsFor example, suppose an ultrasound provider group sees 10,000 women in 1 year, of whom 80 actually have a fetus with CHD. The group misses the diagnosis in 20 cases and correctly diagnoses 60 cases (actual detection rate is 60/80 = 75%), of whom 30 deliver a fetus with CHD and 30 terminate the pregnancy. In addition, the group has 50 false-positive diagnoses of CHD, of whom 25 continue the pregnancy and 25 terminate. The denominator will include the 30 liveborns with detected CHD, the 20 liveborns with undetected CHD, the 30 terminations with correct detection of CHD, and the 25 terminations with false-positive diagnosis (total 105). The numerator will include the 60 correct diagnoses plus the 25 false positives who terminated (total 85) for an apparent sensitivity of 85/105 = 81%, although actual sensitivity was 75%.
      False positivesCaptured using specificity-balancing metricSpecificity metric suffers the same limitations as the sensitivity metric, including lack of a specific ICD-10 code for fetal CHD to define the numerator and lack of systems for follow-up of prenatal findings.
      CHD, congenital heart defects; ICD-10-CM, International Classification of Diseases, 10th Revision, Clinical Modification.
      These metrics could potentially be used by provider groups or health systems to evaluate the diagnostic accuracy of individual ultrasound providers (sonographers or physicians). If significant between-provider variation is found, QI efforts can be focused on providers with low values of either metric. Another potential use might be for a payer to evaluate the diagnostic accuracy of individual providers or provider groups.

      Next steps and actions needed

      Many stakeholders have an interest in knowing about the accuracy of prenatal diagnosis of CHD. Patients want to know whether ultrasound findings can be relied upon. Ultrasound providers want a reputation for high quality because continued referrals depend on that reputation. Payers want assurance that their payments for ultrasound services have value, which depends on accuracy. Thus, we agree with the Cooperative Workshop recommendation that metric(s) reflecting the accuracy of prenatal diagnosis of CHD should be developed.
      • Iriye B.K.
      • Gregory K.D.
      • Saade G.R.
      • Grobman W.A.
      • Brown H.L.
      Quality measures in high-risk pregnancies: Executive summary of a cooperative workshop of the Society for Maternal-Fetal Medicine, National Institute of Child Health and Human Development, and the American College of Obstetricians and Gynecologists.
      The proposed metrics, in principle, can provide an assessment of the accuracy of detection of CHD. However, they are fraught with practical limitations that hobble the ability of hospitals, providers, or payers to actually track them. To minimize the burden of data collection, it should be possible to track metrics using data that are already in the electronic records, administrative records, or claims databases. Claims data based on CPT and ICD-10-CM codes are frequently used for such purposes. Unfortunately, such automation is not currently possible for the proposed metrics because there is no specific ICD-10-CM code for fetal CHD and because it is difficult to link fetal records (in the maternal chart) with newborn records. QI initiatives to improve CHD detection can be done despite these limitations,
      • Levy D.J.
      • Pretorius D.H.
      • Rothman A.
      • et al.
      Improved prenatal detection of congenital heart disease in an integrated health care system.
      • Evans W.
      • Castillo W.
      • Rollins R.
      • et al.
      Moving towards universal prenatal detection of critical congenital heart disease in southern Nevada: a community-wide program.
      • Letourneau K.M.
      • Horne D.
      • Soni R.N.
      • McDonald K.R.
      • Karlicki F.C.
      • Fransoo R.R.
      Advancing prenatal detection of congenital heart disease: a novel screening protocol improves early diagnosis of complex congenital heart disease.
      • Evans W.N.
      • Acherman R.J.
      • Ciccolo M.L.
      • Lehoux J.
      • Rothman A.
      • Galindo A.
      Detecting critical congenital heart disease in Nevada.
      • Hautala J.
      • Gissler M.
      • Ritvanen A.
      • et al.
      The implementation of a nationwide anomaly screening programme improves prenatal detection of major cardiac defects: an 11-year national population-based cohort study.
      but these efforts require a significant investment of resources to perform manual record review. Providers and hospitals are unlikely to invest these resources without external motivation to do so. Payers typically have access only to claims data and so cannot evaluate the metrics as currently proposed.
      A critical first step is to develop specific ICD codes for fetal CHD and indeed for all congenital fetal anomalies. As of 2020, ICD-10-CM has only 3 codes for fetal anomalies (O35.0XXn, O35.8XXn, and O35.9XXn, where n = fetus number). These codes reflect “maternal care for (suspected) fetal central nervous system malformation,” “maternal care for (suspected) other fetal abnormality and damage,” and “maternal care for (suspected) fetal abnormality and damage unspecified,” respectively.
      This lack of specificity is startling in a classification system that is renowned for specificity (eg, codes that distinguish trauma from impact with a parrot [W61.02], macaw [W61.12], chicken [W61.32], goose [W61.52], or duck [W61.62]). There is no such specificity for fetal diagnoses. The same ICD-10 code (O35.8XXn) is used for critical fetal anomalies (eg, tricuspid stenosis) and minor or incidental findings (eg, supernumerary digit). We believe that each anomaly listed in Table 1 should have a specific ICD code reflecting its occurrence in a fetus. More generally, having specific ICD codes for all common fetal anomalies (central nervous system, gastrointestinal, genitourinary, musculoskeletal orofacial) may facilitate a variety of research and QI projects. In the future, the SMFM Coding Committee will recommend specific fetal anomalies that should be considered for addition to the ICD.
      Another critical step is to develop record systems that facilitate the linkage of fetal and newborn findings. Without such linkage, there is no systematic way for ultrasound providers to know whether prenatal findings were confirmed or refuted by newborn examination findings, for newborn care providers to directly review prenatal findings, or for payers to match maternal claims (which include fetal findings) with newborn claims. Such a linkage will require substantial development by vendors of electronic health record systems. The SMFM Informatics Committee has liaisons with several electronic health record vendors and can encourage such development.
      Overcoming these hurdles will allow these metrics to satisfy the Measure Evaluation Criteria of the National Quality Forum.
      National Quality Forum
      Measure Evaluation Criteria. 2012.
      Future development would include pilot testing with interested health systems and subsequent application for endorsement of the metrics by the National Quality Forum.

      References

        • Parker S.E.
        • Mai C.T.
        • Canfield M.A.
        • et al.
        Updated national birth prevalence estimates for selected birth defects in the United States, 2004–2006.
        Birth Defects Res A Clin Mol Teratol. 2010; 88: 1008-1016
        • Mai C.T.
        • Isenburg J.
        • Langlois P.H.
        • et al.
        Population-based birth defects data in the United States, 2008 to 2012: presentation of state-specific data and descriptive brief on variability of prevalence.
        Birth Defects Res A Clin Mol Teratol. 2015; 103: 972-993
        • Kirby R.S.
        The prevalence of selected major birth defects in the United States.
        Semin Perinatol. 2017; 41: 338-344
        • Mai C.T.
        • Isenburg J.L.
        • Canfield M.A.
        • et al.
        National population-based estimates for major birth defects, 2010–2014.
        Birth Defects Res. 2019; 111: 1420-1435
        • Reller M.D.
        • Strickland M.J.
        • Riehle-Colarusso T.
        • Mahle W.T.
        • Correa A.
        Prevalence of congenital heart defects in metropolitan Atlanta, 1998–2005.
        J Pediatr. 2008; 153: 807-813
        • Dolk H.
        • Loane M.
        • Garne E.
        European Surveillance of Congenital Anomalies Working Group. Congenital heart defects in Europe: prevalence and perinatal mortality, 2000 to 2005.
        Circulation. 2011; 123: 841-849
        • Pei L.
        • Kang Y.
        • Zhao Y.
        • Yan H.
        Prevalence and risk factors of congenital heart defects among live births: a population-based cross-sectional survey in Shaanxi province, Northwestern China.
        BMC Pediatr. 2017; 17: 18
        • Ficara A.
        • Syngelaki A.
        • Hammami A.
        • Akolekar R.
        • Nicolaides K.H.
        Value of routine ultrasound examination at 35–37 weeks' gestation in diagnosis of fetal abnormalities.
        Ultrasound Obstet Gynecol. 2020; 55: 75-80
        • Benjamin E.J.
        • Blaha M.J.
        • Chiuve S.E.
        • et al.
        Heart Disease and stroke statistics—2017 update: a report from the American Heart Association.
        Circulation. 2017; 135: e146-e603
        • Khoo N.S.
        • Van Essen P.
        • Richardson M.
        • Robertson T.
        Effectiveness of prenatal diagnosis of congenital heart defects in South Australia: a population analysis 1999–2003.
        Aust N Z J Obstet Gynaecol. 2008; 48: 559-563
        • Stoll C.
        • Garne E.
        • Clementi M.
        • EUROSCAN Study Group
        Evaluation of prenatal diagnosis of associated congenital heart diseases by fetal ultrasonographic examination in Europe.
        Prenat Diagn. 2001; 21: 243-252
        • Marek J.
        • Tomek V.
        • Skovranek J.
        • Povysilova V.
        • Samanek M.
        Prenatal ultrasound screening of congenital heart disease in an unselected national population: a 21-year experience.
        Heart. 2011; 97: 124-130
        • McBrien A.
        • Sands A.
        • Craig B.
        • Dornan J.
        • Casey F.
        Major congenital heart disease: antenatal detection, patient characteristics and outcomes.
        J Matern Fetal Neonatal Med. 2009; 22: 101-105
        • Pinto N.M.
        • Keenan H.T.
        • Minich L.L.
        • Puchalski M.D.
        • Heywood M.
        • Botto L.D.
        Barriers to prenatal detection of congenital heart disease: a population-based study.
        Ultrasound Obstet Gynecol. 2012; 40: 418-425
        • Acharya G.
        • Sitras V.
        • Maltau J.M.
        • et al.
        Major congenital heart disease in northern Norway: shortcomings of pre- and postnatal diagnosis.
        Acta Obstet Gynecol Scand. 2004; 83: 1124-1129
        • Bull C.
        Current and potential impact of fetal diagnosis on prevalence and spectrum of serious congenital heart disease at term in the UK. British Paediatric Cardiac Association.
        Lancet. 1999; 354: 1242-1247
        • Friedberg M.K.
        • Silverman N.H.
        • Moon-Grady A.J.
        • et al.
        Prenatal detection of congenital heart disease.
        J Pediatr. 2009; 155: 26-31.e1
        • Garne E.
        • Stoll C.
        • Clementi M.
        • Euroscan G.
        Evaluation of prenatal diagnosis of congenital heart diseases by ultrasound: experience from 20 European registries.
        Ultrasound Obstet Gynecol. 2001; 17: 386-391
        • Jaeggi E.T.
        • Sholler G.F.
        • Jones O.D.
        • Cooper S.G.
        Comparative analysis of pattern, management and outcome of pre- versus postnatally diagnosed major congenital heart disease: a population-based study.
        Ultrasound Obstet Gynecol. 2001; 17: 380-385
        • Parikh L.I.
        • Grantz K.L.
        • Iqbal S.N.
        • et al.
        Neonatal outcomes in fetuses with cardiac anomalies and the impact of delivery route.
        Am J Obstet Gynecol. 2017; 217: 469.e1-469.e12
        • Ailes E.C.
        • Gilboa S.M.
        • Riehle-Colarusso T.
        • et al.
        Prenatal diagnosis of nonsyndromic congenital heart defects.
        Prenat Diagn. 2014; 34: 214-222
        • Quartermain M.D.
        • Pasquali S.K.
        • Hill K.D.
        • et al.
        Variation in prenatal diagnosis of congenital heart disease in infants.
        Pediatrics. 2015; 136: e378-e385
        • Khoshnood B.
        • Lelong N.
        • Houyel L.
        • et al.
        Impact of prenatal diagnosis on survival of newborns with four congenital heart defects: a prospective, population-based cohort study in France (the EPICARD Study).
        BMJ Open. 2017; 7e018285
        • van Velzen C.L.
        • Clur S.A.
        • Rijlaarsdam M.E.
        • et al.
        Prenatal detection of congenital heart disease—results of a national screening programme.
        BJOG. 2016; 123: 400-407
        • Tegnander E.
        • Eik-Nes S.H.
        The examiner's ultrasound experience has a significant impact on the detection rate of congenital heart defects at the second-trimester fetal examination.
        Ultrasound Obstet Gynecol. 2006; 28: 8-14
        • Tegnander E.
        • Williams W.
        • Johansen O.J.
        • Blaas H.G.
        • Eik-Nes S.H.
        Prenatal detection of heart defects in a non-selected population of 30,149 fetuses—detection rates and outcome.
        Ultrasound Obstet Gynecol. 2006; 27: 252-265
        • Lytzen R.
        • Vejlstrup N.
        • Bjerre J.
        • et al.
        Live-born major congenital heart disease in Denmark: incidence, detection rate, and termination of pregnancy rate from 1996 to 2013.
        JAMA Cardiol. 2018; 3: 829-837
        • Levey A.
        • Glickstein J.S.
        • Kleinman C.S.
        • et al.
        The impact of prenatal diagnosis of complex congenital heart disease on neonatal outcomes.
        Pediatr Cardiol. 2010; 31: 587-597
        • Cloete E.
        • Bloomfield F.H.
        • Sadler L.
        • de Laat M.W.M.
        • Finucane A.K.
        • Gentles T.L.
        Antenatal detection of treatable critical congenital heart disease is associated with lower morbidity and mortality.
        J Pediatr. 2019; 204: 66-70
        • Calderon J.
        • Angeard N.
        • Moutier S.
        • Plumet M.H.
        • Jambaque I.
        • Bonnet D.
        Impact of prenatal diagnosis on neurocognitive outcomes in children with transposition of the great arteries.
        J Pediatr. 2012; 161: 94-98.e1
        • Khoshnood B.
        • Lelong N.
        • Houyel L.
        • et al.
        Prevalence, timing of diagnosis and mortality of newborns with congenital heart defects: a population-based study.
        Heart. 2012; 98: 1667-1673
        • Brown M.D.
        • Wernovsky G.
        • Mussatto K.A.
        • Berger S.
        Long-term and developmental outcomes of children with complex congenital heart disease.
        Clin Perinatol. 2005; 32: 1043-1057
        • Tworetzky W.
        • McElhinney D.B.
        • Reddy V.M.
        • Brook M.M.
        • Hanley F.L.
        • Silverman N.H.
        Improved surgical outcome after fetal diagnosis of hypoplastic left heart syndrome.
        Circulation. 2001; 103: 1269-1273
        • Berndl A.
        • Pearce K.
        • Mondal T.
        Fetal echocardiography assists in determining optimal delivery site.
        J Obstet Gynaecol Can. 2014; 36: 210-215
        • Levy D.J.
        • Pretorius D.H.
        • Rothman A.
        • et al.
        Improved prenatal detection of congenital heart disease in an integrated health care system.
        Pediatr Cardiol. 2013; 34: 670-679
        • Evans W.
        • Castillo W.
        • Rollins R.
        • et al.
        Moving towards universal prenatal detection of critical congenital heart disease in southern Nevada: a community-wide program.
        Pediatr Cardiol. 2015; 36: 281-288
        • Letourneau K.M.
        • Horne D.
        • Soni R.N.
        • McDonald K.R.
        • Karlicki F.C.
        • Fransoo R.R.
        Advancing prenatal detection of congenital heart disease: a novel screening protocol improves early diagnosis of complex congenital heart disease.
        J Ultrasound Med. 2018; 37: 1073-1079
        • Evans W.N.
        • Acherman R.J.
        • Ciccolo M.L.
        • Lehoux J.
        • Rothman A.
        • Galindo A.
        Detecting critical congenital heart disease in Nevada.
        World J Pediatr Congenit Heart Surg. 2019; 10: 702-706
        • Hautala J.
        • Gissler M.
        • Ritvanen A.
        • et al.
        The implementation of a nationwide anomaly screening programme improves prenatal detection of major cardiac defects: an 11-year national population-based cohort study.
        BJOG. 2019; 126: 864-873
        • Hunter J.
        Myth: if you can't measure it, you can't manage it. August 13, 2015.
        (Available at:)
        • Iriye B.K.
        • Gregory K.D.
        • Saade G.R.
        • Grobman W.A.
        • Brown H.L.
        Quality measures in high-risk pregnancies: Executive summary of a cooperative workshop of the Society for Maternal-Fetal Medicine, National Institute of Child Health and Human Development, and the American College of Obstetricians and Gynecologists.
        Am J Obstet Gynecol. 2017; 217: B2-B25
        • Jelliffe-Pawlowski L.L.
        • Norton M.E.
        • Shaw G.M.
        • et al.
        Risk of critical congenital heart defects by nuchal translucency norms.
        Am J Obstet Gynecol. 2015; 212: 518.e1-518.e10
        • National Quality Forum
        Measure Evaluation Criteria. 2012.
        (Available at:)