The HHS-HCC Risk Adjustment Model for Individual and Small Group Markets under the Affordable Care Act

Keywords: risk adjustment, affordable care act, ACA, risk score, hierarchical condition categories, HHS-HCC model, plan liability, predict healthcare expenditures, health insurance marketplaces

HHS-HCC Diagnostic Classification

The basis of the HHS-HCC risk adjustment model is using health plan enrollee diagnoses (and demographics) to predict medical expenditure risk. To obtain a clinically meaningful and statistically stable system, the tens of thousands of ICD-9-CM codes used to capture diagnoses must be grouped into a smaller number of organized categories that produce a diagnostic profile of each person. The diagnostic classification is key in determining the ability of a risk adjustment model to distinguish high from low cost individuals. The classification also determines the sensitivity of the model to intentional or unintentional variations in diagnostic coding, an important consideration in real-world risk adjustment.

The starting point for the HHS-HCCs was the Medicare CMS-HCCs. The CMS-HCCs had to be adapted into the HHS-HCCs for ACA risk adjustment for three main reasons:

1. Prediction Year—The CMS-HCC risk adjustment model uses base year diagnoses and demographic information to predict the next year’s spending. The HHS-HCC risk adjustment model uses current year diagnoses and demographics to predict the current year’s spending. Medical conditions may have different implications in terms of current year costs and future costs; selection of HCCs for the risk adjustment model should reflect those differences.

2. Population—The CMS-HCCs were developed using data from the aged (age ≥ 65) and disabled (age < 65) Medicare populations. For some conditions, such as pregnancy and neonatal complications, the sample size in the Medicare population is quite low, whereas sample sizes in the commercially insured population are larger. HCCs were re-examined to better reflect salient medical conditions and cost patterns for adult, child, and infant subpopulations in the commercial population.

3. Type of Spending—The CMS-HCCs are configured to predict non-drug medical spending. The HHS-HCCs predict the sum of medical and drug spending. Also, the CMS-HCCs predict Medicare provider payments while the HHS-HCCs predict commercial insurance payments.

Risk Adjustment Model HHS-HCCs

There are 264 HHS-HCCs in the full diagnostic classification, of which a subset is included in the HHS risk adjustment model. The criteria for including HCCs in the model are now described. These criteria were sometimes in conflict and tradeoffs had to be made among them in assessing whether to include specific HCCs in the HHS risk adjustment model.

Criterion 1—Represent clinically-significant, well-defined, and costly medical conditions that are likely to be diagnosed, coded, and treated if they are present.

Criterion 2—Are not especially subject to discretionary diagnostic coding or “diagnostic discovery” (enhanced rates of diagnosis through population screening not motivated by improved quality of care).

Criterion 3—Do not primarily represent poor quality or avoidable complications of medical care.

Criterion 4—Identify chronic, predictable, or other conditions that are subject to insurer risk selection, risk segmentation, or provider network selection, rather than random acute events that represent insurance risk.

Following an extensive review process, we selected 127 HHS-HCCs to be included in the HHS risk adjustment model (see Appendix Exhibit A1 for a listing of the 127 HHS-HCCs). Finally, to balance the competing goals of improving predictive power and limiting the influence of discretionary coding, a subset of HHS-HCCs in the risk adjustment model were grouped into larger aggregates, in other words “grouping” clusters of HCCs together as a single condition with a single coefficient that can only be counted once. After grouping, the number of HCC factors included in the model was effectively reduced from 127 to 100.

Model Type

The HHS-HCC risk adjustment model is a concurrent model. A concurrent model uses diagnoses from a time period to predict cost in that same period. This is in contrast to a prospective model, which uses diagnoses from a base period to predict costs in a future period. While a prospective model is used for the Medicare Advantage program, we developed a concurrent model for the HHS risk adjustment methodology because, for implementation in 2014, prior year (2013) diagnoses data will not be available. In addition, unlike Medicare, people may move in and out of enrollment in the individual and small group markets, so prior year diagnostic data will not be available for all enrollees even after 2014.

Data

The calibration sample for the HHS-risk adjustment model consists of 2010 Truven MarketScan® Commercial Claims and Encounter data. The MarketScan® data is a large, well-respected, widely-used, nationally-dispersed proprietary database sourced from large employers and health plans. Employees, spouses, and dependents covered by employer-sponsored private health insurance are included. The MarketScan® sample includes enrollees from all 50 states and the District of Columbia. Although MarketScan® represents the large employer rather than the small group/individual market, we know of no evidence that the relationship between diagnoses and relative expenditures differs significantly in the two markets, holding constant the generosity of plan benefits (essential health benefits and metal level). We compared the age, sex, and regional distribution of the MarketScan® sample to the expected ACA risk adjustment population (Trish, Damico, Claxton, Levitt, & Garfield, 2011; Buettgens, Garrett, & Holahan, 2010). We found that overall they are similar, although the MarketScan® data has more children and fewer young adults, and more sample members in the South and fewer in the Northeast and West than the expected risk adjustment population.³

Sample

An enrollee is included in the concurrent modeling sample if the enrollee has at least one month of 2010 enrollment, is enrolled in a preferred provider organization (PPO) or other fee-for-service (FFS) health plan,⁴ has no payments made on a capitated basis, has prescription drug coverage, and has integrated mental health/substance abuse coverage.⁵ The primary goals of the sample selection criteria were to ensure that 1) enrollees had complete expenditure and diagnosis data, 2) enrollees included those entering (e.g., newborns) and exiting (e.g., decedents) enrollment during the year, and 3) enrollees had health care coverage comparable to the essential health benefits under the ACA.

Expenditures

The HHS-HCC risk adjustment model predicts health care expenditures for which plans are liable, which exclude enrollee cost sharing. This is termed a plan liability risk adjustment model, which has been used in other payment systems, such as Medicare Part C and Part D (Pope et al., 2004; Kautter, Ingber, Pope, & Freeman, 2012). We considered predicting total expenditures and then adjusting to plan liability with a multiplicative plan actuarial value factor. However, this approach may not accurately capture plan liability levels due to the non-linear relationship of plan liability to total expenditures. Although alternative plan cost sharing designs exist, we define a standard benefit (plan liability cost sharing) design for each cost sharing level (platinum, gold, silver, and bronze metal levels, as well as catastrophic plans⁶) using the following elements. Plan liability is zero percent of total expenditures below the deductible, one minus the coinsurance percentage of total expenditures between the deductible and the out-of-pocket limit, and one hundred percent of total expenditures above the out-of-pocket limit. Thus, the standard benefit for each metal level is completely specified by a deductible, coinsurance rate, and out-of-pocket maximum.

Using the 2010 MarketScan® inpatient, outpatient, and drug services files, we summed total payments (submitted charges minus non-covered charges minus pricing discounts), which include enrollee cost sharing. We then trended the 2010 expenditures to 2014 by applying a constant annual growth rate. Once expenditures were trended, the standard benefit design parameters (deductibles, coinsurance rates, out-of-pocket limits) were applied to simulate plan liability expenditures for each metal level. Plan liability expenditures were then annualized by dividing them by the fraction of months in 2010 that each beneficiary is enrolled in the plan (i.e., by the eligibility fraction). Annualized expenditures are the “per member per month” amount multiplied by 12. Annualized expenditures were not truncated.

Finally, plan liability expenditures were converted to relative plan liability expenditures, which are defined as plan liability expenditures divided by a denominator. A relative plan liability expenditure of 1.0 corresponds to the average plan liability expenditure for the calibration sample. The denominator was calculated as follows. For the entire calibration sample, we calculated the mean plan liability for each metal level and then took a weighted average of these means, where the weights were based on a forecasted distribution of enrollment in 2014 across the five metal levels. Going forward, we use the term “plan liability” to mean “relative plan liability.”

In short, we simulated plan liability expenditures for each metal tier from total expenditures for each sample member (that is, we applied different benefit structures to the same sample). An alternative approach would have been to model actual plan liability (payments) for enrollees in MarketScan® plans grouped into ACA metal tiers by the plans’ actual actuarial values. However, MarketScan® provides sufficient plan benefit information to calculate plan actuarial value for only a small fraction of its sample. Also, grouping plans by actuarial value would have led to different samples of individuals for each metal level model estimation, which would have reduced sample sizes for each model and led to differences in unmeasured factors across metal level samples. Simulating plan liability on the full sample for each metal also means that (as intended) the model estimates do not reflect differential induced demand (moral hazard) across metals. For this reason, induced demand is accounted for in the risk transfer formula, as discussed in our companion article.

Demographics and Diagnoses

The HHS-HCC risk adjustment model uses 2010 beneficiary demographics and diagnoses to predict 2010 (trended to 2014) plan liability expenditures for each beneficiary. The demographic factors employed are age and sex. Age is measured as of the last month of enrollment, which in general results in infants aged 0 having been born in 2010.⁷ Age ranges were determined by the age distribution of the commercial population, as well as consideration of post 2014 market reform rules for the individual and small group markets. There are 18 age/sex categories for adults and 8 age/sex categories for children. How age and sex are incorporated into the infant model is described below. Adults are defined as ages 21+, children are ages 2–20, and infants are ages 0–1. The age categories for adult male and female are ages 21–24, 25–29, 30–34, 35–39, 40–44, 45–49, 50–54, 55–59, and 60+. The age categories for children male and female are ages 2–4, 5–9, 10–14, and 15–20.

ICD-9-CM person-level diagnoses from 2010 were used to create diagnosis groups (HCCs) for each beneficiary in the sample. Only diagnosis codes from sources allowable for risk adjustment when HHS is operating on behalf of a state are included in the diagnosis-level file. The goal of the restrictions on source of diagnoses is to improve the quality, accuracy, and auditability of diagnoses used for risk adjustment. For example, clinical laboratory diagnoses, which include “rule outs” and diagnoses not verified by a clinician, were excluded. Allowable diagnoses include those from inpatient hospital claims, outpatient facility claims (hospital outpatient, rural health clinic, federally qualified health center, and community mental health clinic), and professional claims (diagnoses are generally not available on prescription drug claims, including for the MarketScan® data). In addition, diagnoses from outpatient facility claims and professional claims are restricted to those with at least one CPT/HCPCS procedure code⁸ corresponding generally to face-to-face encounters with a clinician.

Subpopulations

Due to the inherent clinical and cost differences in the adult, child, and infant populations, we developed separate risk adjustment models for each group. The adult and child models have similar specifications, with age/sex demographic categories and HCCs (individual HCCs and aggregate HCC groupings) predicting annualized plan liability expenditures.

However, infants have low frequencies for most HCCs leading to unstable parameter estimates in an additive model. Because of this, the infant model utilizes a categorical approach in which infants are assigned a birth maturity (by length of gestation and birth weight) or Age 1 category, and a disease severity category (based on HCCs other than birth maturity). There are four Age 0 birth maturity categories—Extremely Immature; Immature; Premature/Multiples; Term—and a single Age 1 Maturity category. Age zero infants are assigned to one of the four birth maturity categories and age one infants are assigned to the Age 1 Maturity category.

There are 5 disease severity categories based on the clinical severity and associated costs of the non-maturity HCCs: Severity Level 5 (Highest Severity) to Severity Level 1 (Lowest Severity).⁹ Examples of severity level assignments are:

• Level 5—HCC 137 (Hypoplastic Left Heart Syndrome and Other Severe Congenital Heart Disorders);

• Level 4—HCC 127 (Cardio-Respiratory Failure and Shock, Including Respiratory Distress Syndromes);

• Level 3—HCC 45 (Intestinal Obstruction);

• Level 2—HCC 69 (Acquired Hemolytic Anemia, Including Hemolytic Disease of Newborn); and,

• Level 1—HCC 37 (Chronic Hepatitis).

All infants (age 0 or 1) are assigned to a disease severity category based on the single highest severity level of any of their non-maturity HCCs. HCCs not appropriately diagnosed for infants—such as pregnancy and psychiatric HCCs—were excluded from the infant disease severity categories. Infants with no severity HCCs are assigned to Level 1.

When cross-classified, the 5 maturity categories and 5 severity categories define 25 mutually-exclusive categories. Each infant is assigned to 1 of the 25 categories. Finally, there are two additive terms for sex, for age zero males and age one males.¹⁰

Model Estimation

All risk adjustment models are estimated by weighted least squares regression.¹¹ The dependent variable is annualized, simulated, plan liability expenditures, and the weight is the person-specific, sample eligibility fraction. Annualization and weighting—which are equivalent on an annual basis to predicting per member per month expenditures weighting by the number of months each individual is eligible for the sample—appropriately adjusts for months of enrollee eligibility in the sample. Independent variables for the adult model include 18 age/sex demographic categories, 114 HCC diagnosis groups, and 16 disease interactions (discussed below), and for the child model, 8 age/sex demographic categories and 119 HCC diagnosis groups. For the infant model, independent variables include 25 categories defined by birth maturity for age 0, age 1, and diagnostic severity, and 2 age/sex demographic additive terms.

In each adult and child regression model, we include a binary indicator variable for each individual HCC that is not included in an aggregate HCC grouping. In addition, we include a binary indicator for each aggregate HCC grouping. In the latter case, it indicates whether or not the enrollee had at least one HCC in the aggregate HCC grouping.

In addition, we impose coefficient constraints to ensure that the principle that higher-clinically-ranked HCCs in an HCC hierarchy have at least as large incremental predicted expenditures as lower-ranked HCCs is met. Constraints generally have the effect of averaging two or more groups together when, unconstrained, there is a violation of clinical logic.

Disease Interactions

For the adult models, the inclusion of disease interaction terms better reflected plan liability across metal levels and improved model performance.¹² Based on empirical findings, as well as clinical review, we developed a set of eight diagnostic markers of severe illness: HCC 2 (Septicemia, Sepsis, Systemic Inflammatory Response Syndrome/Shock); HCC 42 (Peritonitis/Gastrointestinal Perforation/Necrotizing Entercolitis); HCC 120 (Seizure Disorders and Convulsions); HCC 122 (Non-Traumatic Coma, Brain Compression/Anoxic Damage); HCC 125 (Respirator Dependence/Tracheostomy Status); HCC 126 (Respiratory Arrest); HCC 127 (Cardio-Respiratory Failure and Shock, Including Respiratory Distress Syndromes); and HCC 156 (Pulmonary Embolism and Deep Vein Thrombosis). A severe illness indicator variable was defined as having at least one of the eight diagnostic markers of severe illness.¹³

The severe illness indicator was interacted with individual HCCs and aggregate HCC groupings.¹⁴ The disease interactions that met minimum sample size and incremental predicted expenditure thresholds were included in the model. The incremental predicted expenditures for the disease interactions were categorized into medium and high cost categories. For each category, we included a binary indicator variable in the regression model for whether or not the enrollee had at least one disease interaction in the category. Finally, a hierarchy was imposed such that if an enrollee was in the high cost disease interaction category, he/she was excluded from the medium cost category. In sum, a person can have, at most, one disease interaction coefficient/incremental predicted expenditure. This constraint was imposed because clinical reasoning and empirical evidence indicated that a single one of the diagnostic markers sufficed to distinguish the most severely ill patients among those with the underlying interacted diagnoses.

Predicted Plan Liability Expenditures

For an enrollee in a given metal level plan, the total predicted plan liability expenditures is the sum of the incremental predicted plan liability expenditures (coefficients) from the relevant metal level model. For adults and children, this is the sum of the age/sex, HCC, and disease interaction coefficients.¹⁵ For infants, this is the sum of the maturity/disease-severity category and additive sex coefficients.

Recall that plan liability expenditures were converted to relative plan liability expenditures, resulting in a relative plan liability expenditure of 1.0 for the average plan liability expenditure in the calibration sample. Converting “actual” plan liability expenditures to relatives automatically converts “predicted” plan liability expenditures to relatives. Going forward, we use the term “predicted plan liability” to mean “predicted relative plan liability.”

Model Evaluation

The predictive accuracy of a risk adjustment model for individuals is typically judged by the percentage of variation in individual expenditures explained by the model (as measured by the R-squared statistic). To test the performance of the HHS-HCC risk adjustment models for subgroups, we calculate the expenditure ratio of predicted to actual weighted mean plan liability expenditures, which is commonly termed the “predictive ratio.” If prediction is perfect, mean predicted will equal mean actual expenditures, and the predictive ratio is 1.00. As a rule of thumb, predictive ratios with a margin of error of 10 percent in either direction (0.90 ≤ predictive ratio ≤ 1.10) indicate reasonably accurate prediction (Kautter et al., 2012).

Sample Exclusions

As shown in Exhibit 1, the 2010 data included 45,239,752 enrollees. After all exclusionary criteria were imposed, the concurrent sample comprised 20,040,566 enrollees, which is 44.3 percent of the original sample. Adults, children, and infants comprise, respectively, 71.0 percent, 27.1 percent, and 1.9 percent of the concurrent sample.

Plan Liability Expenditures

Mean simulated plan liability expenditures (annualized, weighted) per enrolled beneficiary ranges from 1.369 (36.9% higher than average) for the platinum cost sharing level to 0.877 (12.3% lower than average) for the catastrophic cost sharing level (shown in Exhibit 2, decomposed by adult/child/infant as well as by metal level). The median ranges from 0.216 for platinum to 0.000 for silver, bronze, and catastrophic.¹⁶ The percentage of individuals with $0 plan liability increases from 16.9 percent for platinum to 83.2 percent for catastrophic. Mean plan liability expenditures are highest for infants (e.g., 2.232 for silver mean plan liability), which is not surprising given infants have costs related to hospitalization at birth and can have severe and expensive conditions that do not occur in adults or children. Adults have close to four times higher mean plan liability expenditures than children (e.g., 1.329 vs. 0.357 for the Silver metal level), which, again, is not surprising given that the onset of most chronic conditions are highly correlated with age.

HHS-HCCs

As shown in Exhibit 3, in the adult concurrent modeling sample, only 19.2 percent of enrollees have at least one HCC, with the vast majority (79.2 percent) of these having only one HCC. This result does not suggest, however, that the HCCs are unimportant in the risk adjustment model. To the contrary, while a minority of the adult sample has HCCs, the majority of expenditures correspond to enrollees with HCCs. Depending on metal level, the percentage of adult expenditures corresponding to enrollees with at least one HCC ranges from 63.4 percent (platinum) to 75.9 percent (catastrophic). Health care expenditures are concentrated in a small proportion of enrollees with serious medical problems, while the majority of the commercial population is relatively healthy. Finally, there is substantial variation by age group in the number of HCCs, with 19.2 percent of the adult sample having at least one HCC, but only 9.1 percent of the child sample. Almost half of the infant sample has at least one HCC, which is to be expected given approximately half of that sample are newborns with associated birth maturity HCCs.

Adult Risk Adjustment Models

The model for each of the metal levels is calibrated on the same adult concurrent sample. Each model includes the same independent variables: 18 age-sex cells, 114 HCCs,¹⁷ and 16 disease interaction terms. Predicted plan liability for each enrollee is the sum of one age-sex coefficient, from zero to many HCC coefficients (individual HCCs and aggregate HCC groupings) subject to HCC hierarchies and constraints/groups, and zero or one severe illness disease interaction term. The model coefficients represent the incremental, not total, predicted plan liability expenditures of each risk marker in the model, given the other risk markers characterizing an individual. The dependent variable for each model is the annualized plan liability expenditures simulated according to a standard cost sharing design for that metal level.

Exhibit 4 shows selected results for the adult risk adjustment models by metal level (for the full results, see Appendix Exhibit A1). The model R-squares range between 36 percent for the platinum model to 35 percent for the catastrophic model. The sample size for each model is 14,220,503, with each age/sex category having between 0.5 million and 1 million observations. Given such large sample sizes, all coefficients are statistically significant at conventional significance levels. The age/sex demographic coefficients are monotonically increasing with age, and higher for females in every age group, but especially in the latter child-bearing years (ages 35–44). These are the total predicted plan liabilities for enrollees without (model) HCCs. In addition, for each age/sex category, the age/sex coefficients are decreasing from platinum to catastrophic. For example, for females age 55–59, the age coefficient decreases by more than half, from 1.054 for the platinum model to 0.443 for the catastrophic model. The lower coefficient reflects the higher enrollee cost sharing and, thus, lower plan liability, moving from the platinum to catastrophic plans.

For the adult silver model, HCC coefficients range from 0.521 (HCC 113, Cerebral Palsy, except Quadriplegic) to 78.175 (HCC 41, Intestine Transplant Status/Complications). For the five most prevalent HCCs, the coefficients are 1.120 (HCC 21, Diabetes without Complications), 1.601 (HCC 88, Major Depressive and Bipolar Disorders), 0.904 (HCC 161, Asthma), 1.120 (HCC 20, Diabetes with Complications), and 0.904 (HCC 160, Chronic Obstructive Pulmonary Disease, including Bronchiectasis).¹⁸ As for the disease interactions, the severe illness high cost and medium cost category coefficients are 12.427 and 2.714, respectively. These amounts are added to the predicted plan liability of individuals who have both a qualifying underlying disorder and one of the diagnostic markers of severe illness.

HCC coefficients decrease by metal level when moving from the platinum model to the catastrophic model, but typically not by a substantial amount, with the majority decreasing by less than half the sample average expenditure (i.e., by less than 0.500). For example, the coefficient for “HCC 130, Congestive Heart Failure” decreases only from 3.790 for the platinum model to 3.594 for the catastrophic model.¹⁹ The differences in the HCC coefficients across metal levels are not as pronounced as the differences in the age/sex coefficients. This occurs because the age-sex coefficients represent the entire predicted liability for persons without HCCs, who are relatively healthy. The plan’s liability for their lower expenditures is greatly reduced by the increase in the deductible across the simulated metal level plans. In contrast, much of the spending for persons with HCCs, especially the more expensive ones, occurs above the plan deductible and even above the plan out-of-pocket maximum, and thus is less affected by the change in cost sharing when moving across metal levels. The upshot is that predicted plan liability, and hence the risk score, are more stable (proportionately) across metal levels for very sick individuals, while predicted plan liability/risk score for healthy individuals is much lower in the bronze or catastrophic plans than in the platinum or gold plans.²⁰,²¹ In other words, plans will incur a significant liability for very sick people even if they have higher lower-end cost sharing; but their proportionate liability for relatively healthy people will be much lower.

Child Risk Adjustment Models

Each of the five metal level models is calibrated on the same child concurrent sample. Each model includes the same independent variables: eight age-sex cells and 119 HCCs.²² Disease interactions were empirically unimportant for the child model and were not included. The dependent variable for each model is the annualized plan liability expenditures simulated according to a standard cost sharing design for that metal level. Predicted plan liability for each child is the sum of one age-sex coefficient and zero to many HCC coefficients, each of which represents an incremental expenditure.²³

Exhibit 5 shows selected results for the child risk adjustment models by metal level (for the full results, see Appendix Exhibit A2). The model R-squares for each of the 5 metal levels range between 31 percent for the platinum model to 30 percent for the catastrophic model. These R-squares are approximately 5 percentage points lower than the R-squares for the adult models. This can be explained partially by noting that less than 10 percent of the child sample has any HCCs, which are the main predictors of individual variation in plan liability expenditures. The sample size for each model is 5,439,645, with each age/sex category having between 362,777 and 921,236 observations. Given such large sample sizes, except for the youngest age/sex categories (age 2–4, age 5–9) for the lowest metal levels (bronze, catastrophic), all coefficients are statistically significant at conventional significance levels.

The age/sex demographic coefficients have a U-shaped pattern, unlike the monotonically increasing coefficients of adults. For example, for males in the silver model, the age/sex coefficients are 0.106 for age 2–4, 0.064 for age 5–9, 0.110 for age 10–14, and 0.191 for age 15–20. Female children are less expensive than male children until ages 15–20, which is perhaps when reproductive health expenses begin to become more pronounced. Similar to the adult model, the age/sex coefficients decrease from platinum to catastrophic.²⁴

For the child silver model, HCC coefficients range from 0.354 (HCC 161, Asthma; and HCC 160, Chronic Obstructive Pulmonary Disease, including Bronchiectasis) to 106.991 (HCC 41, Intestine Transplant Status/Complications). For the five most prevalent HCCs, the coefficients are 0.354 (HCC 161, Asthma), 1.453 (HCC 88, Major Depressive and Bipolar Disorders), 1.882 (HCC 120, Seizure Disorders and Convulsions), 2.198 (HCC 21, Diabetes without Complication), and 1.372 (HCC 102, Autistic Disorder). Three of the five most prevalent HCCs are the same in the adult and child samples. However, the incremental predicted expenditures are markedly different, illustrating the clinical and cost differences among the two populations, which were a major reason for developing separate adult and child models. The child silver model coefficient for “HCC 161, Asthma” is less than half the adult coefficient (0.354 vs. 0.904); the child coefficient for “HCC 21, Diabetes without Complications” is almost double the adult coefficient, perhaps reflecting the greater severity of Type I versus Type II diabetes (2.198 vs. 1.120); and the child coefficient for “HCC 88, Major Depressive and Bipolar Disorders” is relatively similar in magnitude to the adult coefficient (1.453 vs. 1.601). Some other notably higher child versus adult silver coefficients are: “HCC 112 Quadriplegic Cerebral Palsy” (5.223 child vs. 1.681 adult); “HCC 159 Cystic Fibrosis” (12.743 child vs. 9.957 adult); and “HCC 102 Autistic Disorder” (1.372 child vs. 0.974 adult). Finally, like the adult model, the HCC coefficients in the child model decrease when moving from the platinum model to the catastrophic model, but often not by a substantial amount.

Infant Risk Adjustment Models

As described previously, the infant model utilizes a categorical approach in which infants are assigned a birth maturity (by length of gestation and birth weight) or Age 1 category, and a disease severity category (based on HCCs other than birth maturity). Exhibit 6 shows the estimated infant risk adjustment models by metal level. The model R-squares are 29 percent across the five metal levels in the infant model, which are slightly lower than the child model R-squares. The sample size for each model is 380,418, with 90 percent of observations in the “Term x Severity Level 1” category (n=121,841) or the “Age 1 x Severity Level 1” category (n=219,105). The remaining categories (except for the Male Additive terms) each have fewer than 10,000 observations. In fact, sample sizes for a handful of categories are less than 100, which required coefficient constraints to improve statistical precision. Predicted plan liability for each infant is the coefficient of his or her single category [(maturity) x (disease severity)] plus, if male, the coefficient of the Age 0 or Age 1 Male Additive Term.²⁵

For the infant silver model, predicted plan liability for age 0 female infants ranges from 391.387 for the “Extremely Immature x Severity Level 5” category, to 0.998 for the “Term x Severity Level 1” category. Thus, the predicted plan liability for an extremely immature infant with the highest disease severity level is almost 400 times the predicted plan liability for a term infant with the lowest disease severity level. For age 1 female infants, predicted plan liability ranges from 61.217 for the “Age 1 x Severity Level 5” category to 0.333 for the “Age 1 x Severity Level 1” category. The “Age 0, Male” and “Age 1, Male” Additive Terms are 0.574 and 0.094, respectively. Within each maturity level, predicted plan liability is increasing in severity (or is equal when small sample sizes require severity levels to be combined in estimation). Also, for age 0 infants, within each severity level, predicted plan liability increases with greater immaturity.

The infant model predicted plan liability, the (maturity) x (disease severity) coefficients, decrease with greater plan enrollee cost sharing (moving from platinum to catastrophic plans). But, proportionately, the reduction is much larger for the less expensive categories. For example, the (Term) x (Severity Level 5) predicted plan liability falls only from 132.588 (platinum) to 130.292 (catastrophic). But the (Term) x (Severity Level 1) predicted plan liability falls from 1.661 (platinum) to 0.188 (catastrophic). This can be explained by the large difference in deductibles in the standard benefit designs used to simulate plan liability expenditures, which have a much larger proportionate effect on the lower-expenditure categories.

Evaluation

In evaluating the models’ performance we look at both explanatory power at the individual level and under- and over-prediction for subgroups of the population. We evaluate model predictive accuracy using our MarketScan® calibration sample. While we believe that the evaluation results from this very large and nationally dispersed database are informative and representative on average, our evaluation results do not necessarily generalize perfectly to each individual state’s ACA risk adjustment population or plans.

To evaluate the predictive accuracy of the models for individuals, we examine the models’ R-squared statistics. These were between 35 and 36 percent for the adult models, between 30 and 31 percent for the child models, and 29 percent for the infant models (Exhibits 4, 5, & 6). In comparison, the predictive power of demographic-only models is relatively low, generally less than 2 percent. Adding information about diagnoses substantially improves the predictive power of the models. Further, the predictive power of the concurrent diagnosis-based models presented here substantially exceeds the predictive ability for individuals of prospective diagnosis-based models (e.g., the Medicare CMS-HCC risk adjustment model), which typically have R-squared statistics of 10–15 percent.

The R-squared statistics of the HHS-HCC models are within the range of R-squared statistics of other concurrent models predicting expenditures for commercial insurance enrollees (Winkelman & Mehmud, 2007). However, although predictive accuracy is an important goal in model development, the HHS-HCC models are not developed purely to maximize the value of the R-squared statistic. Instead, the HHS-HCC models are intended to balance high predictive ability with lower sensitivity to discretionary diagnostic coding. The latter is primarily achieved by including only a subset of less discretionary HCCs that identify chronic or systematic conditions subject to insurance risk selection rather than being random acute events. In addition, HCCs that primarily represent complications of or poor quality of care (e.g., pressure ulcers) are excluded.

It is also important to assess aggregate predictive accuracy for defined subgroups of health plan enrollees. This analysis evaluates whether the model predicts liability accurately for plans enrolling different types of people, and whether once the model is implemented, plans have any incentives to avoid or enroll certain types of individuals, for example, those with high health care costs or certain medical conditions. In the calibration sample, the models predict mean plan liability expenditures perfectly (predictive ratio = 1.00) for each of the age group subpopulations (adult, child, infant) for each level of plan cost sharing (platinum, gold, silver, bronze, catastrophic). Not only that, prediction is perfect for each of the included demographic (age/sex categories) and diagnostic factors (HCC diagnosis groups) for each subpopulation. This is expected, given the specification and statistical techniques used to estimate the model. However, given their clinical and cost differences, predicting accurately on average for these subpopulations is important. For example, the model accounts for the very high incremental health care costs of children with hemophilia (45.551—relative incremental plan liability estimate in child silver model). Basing risk transfer payments and charges on accurate estimates of the differential costs by subpopulation will help ensure that plans in the individual and small group markets receive adequate payments to treat enrollees with high expected costs.

We also tested the predictive accuracy of the models using enrollee groups sorted into predicted expenditure percentile ranges (0–40%, 40–80%, 80–100%, top 10%, top 5%, top 1%). This set of ratios determines whether the model predictions are accurate at various levels of predicted expenditures; that is, it determines whether expenditures the model predicts to be low are in fact low on average, and whether expenditures the model predicts to be high are in fact high on average. We chose this set of percentile ranges (which we refer to simply as “percentiles”) not only to cover the entire range of predicted expenditures, but to emphasize the higher percentiles that capture the small proportion of high-cost individuals in which most medical expenditures are concentrated. Accurate model prediction is especially critical for these high-cost cases.

For the adult sample, Exhibit 7 presents predictive ratios for percentiles of enrollees created by sorting predicted plan liability expenditures. The adult platinum model predicts well for these predicted expenditure groups. There is less than a 10 percent prediction error in either direction for each of these groups, ranging from lower-cost to very-high-cost individuals. The lower percentiles, 0–40% and 40–80%, are somewhat under-predicted, whereas the highest percentiles (80–100%, top 10%, top 5%, top 1%) are somewhat over-predicted.

The adult models perform adequately across all metal levels, doing especially well for the critical highest percentiles. For example, for the 80–100% percentile, the predictive ratios range from 1.04 (platinum) to 1.07 (catastrophic). The mean actual plan liability expenditures for enrollees in the 80–100% percentile range from 5.012 to 3.944 across metal tiers, which represents, respectively, 64.0 percent to 76.3 percent of overall mean actual plan liability expenditures. Since most of the dollars are in the highest percentiles, it is most important for the model to perform well for these high cost subgroups.

The adult models perform less well for the lowest percentiles, especially for the lower metal levels. For example, for the 0–40% percentile, the predictive ratio for the catastrophic model is only 0.65. However, the enrollees comprising the 0–40% percentile represent only 6.9 percent of overall actual expenditures for the catastrophic metal level. Moreover, the absolute amount of the under-prediction, 0.130 for predicted expenditures versus 0.200 for actual expenditures for a difference of 0.070, is small. The predictive ratio is low, in part, because the denominator of the ratio, 0.200 (1/5 of the average predicted expenditures for the calibration sample), is small for these low-cost beneficiaries, magnifying the absolute prediction error when expressed as a ratio. For the catastrophic metal, as for the other metals, the HHS-HCC model predicts a wide range of plan liabilities across groups, from 0.130 to 30.148 (0–40% percentile vs. top 1% percentile), corresponding to a similar range of actual plan liabilities ranging from 0.200 to 28.948.

The predictive ratios for the child models (Exhibit 8) exhibit the same qualitative patterns as for the adult models, except that the predictive ratios denote less predictive accuracy. For the child platinum model, there is less than a 20 percent error for each percentile (except for the top 1% percentile). Like the adult models, the child model performs less well for the lowest percentiles, especially for the lower metal levels. However, it is important to consider the amount of actual (relative) dollars these percentiles represent. For example, for the catastrophic model, while the 0–40% percentile has a predictive ratio of 0.08, the absolute difference of predicted and actual (relative) expenditures is only 0.049 (predicted expenditures 0.004; actual expenditures 0.053), and only 8.4 percent of overall expenditures of the catastrophic metal level is incurred by the lowest percentile group.

Finally, the infant models perform quite accurately on the predictive ratios for predicted expenditure percentiles (Exhibit 9). In general, there is a 5 percent prediction error or smaller across all percentiles and all metal levels. The two exceptions are the 40–80% percentile for the bronze model (predictive ratio = 0.90) and the 0–40% percentile for the catastrophic model (predictive ratio = 0.80). But again, the dollar amounts of the under-predictions are modest and these percentiles comprise a small share of total actual expenditures, 7.3 percent for the 40–80% percentile for bronze, and 4.2 percent for the 0–40% percentile for catastrophic.

Risk Score Calculation

Below we provide several examples of how empirical risk adjustment model output is applied to calculate an individual’s “plan liability risk score (PLRS)”. We then define the plan average PLRS, which is used in the calculation of transfer payments and charges. In the HHS methodology, the risk score for an enrollee is defined as the predicted relative plan liability expenditure for the enrollee based on the HHS-HCC risk adjustment model for the enrollee’s plan metal level. The predicted relative plan liability expenditures are calculated as follows. For an adult (age 21+), it is the sum of the age/sex, HCC, and disease interaction risk factors in Appendix Exhibit A1; for a child, it is the sum of the age/sex and HCC risk factors in Appendix Exhibit A2; and for infants, it is the sum of the appropriate maturity/disease-severity category and age/sex Additive Term in Exhibit 6.

Based on lower income or certain other qualifying factors, some enrollees in Marketplace plans will be eligible for reduced cost sharing in addition to premium subsidies. An adjustment will be made to the risk score for enrollees in individual market cost-sharing plan variations in Marketplaces (Patient Protection and Affordable Care Act, 2013). Individuals who qualify for cost sharing reductions may utilize health care services at a higher rate than would be the case in the absence of cost sharing reductions. The adjustment for induced demand due to cost sharing reductions will be multiplicative and applied to the risk score.²⁶ Because premiums for all cost-sharing reduction plan variations are required to be the same, despite the increased actuarial value of coverage, we account for the induced demand associated with cost-sharing plan variations as part of the risk adjustment model and not as part of the risk transfer formula.

Exhibit 10 provides illustrative examples of the PLRS calculation, assuming a silver metal level plan. Enrollee 1 is male and aged 56, with two chronic conditions, diabetes with complications and congestive heart failure. Predicted relative incremental plan liability expenditures for these risk factors in the adult silver model are 0.580, 1.120, and 3.587, respectively. Therefore, his predicted relative plan liability expenditure is 5.287, and since he does not have cost sharing reductions (induced utilization factor is 1.00), his PLRS is 5.287. Enrollee 2 is female and aged 11 with asthma. Her predicted relative plan liability expenditures from the child silver model is 0.449 (0.095+0.354). However, she is also a zero cost sharing recipient, so her total predicted expenditures is multiplied by her induced utilization factor 1.12, resulting in a PLRS of 0.503. Enrollee 3 is male and aged 0, with a term birth and severity level 1. His predicted plan liability expenditure from the infant silver model is 1.572 (0.574+0.998), and since he doesn’t have cost sharing reductions, it is his PLRS as well.

Finally, the plan average PLRS, which is used in the calculation of transfer payments and charges, is defined as the plan’s weighted average of individual PLRSs, where the weights are enrollment months. When the plan average PLRS is calculated, all plan enrollees are counted in the numerator, but only billable plan enrollees (parents and the three oldest children) are counted in the denominator (for details, see our companion article on the risk transfer formula).

Disclaimer

The authors have been requested to report any funding sources and other affiliations that may represent a conflict of interest. The authors reported that there are no conflict of interest sources. This study was funded by the Centers for Medicare & Medicaid Services. The views expressed are those of the authors and are not necessarily those of the Centers for Medicare & Medicaid Services.

Exhibit A1. HHS-HCC Risk Adjustment Models—Adult (age 21+).

Exhibit A2. HHS-HCC Risk Adjustment Models—Child (age 2–20).

1. Breyer F, Bundorf K, Pauly MV. Health Care Spending Risk, Health Insurance, and Payment to Health Plans. In: Pauly MV, McGuire TG, Barros PP, editors. Handbook of Health Economics. Vol. 2. Amsterdam: North Holland: Elsevier; 2012. [Google Scholar]
2. Buettgens M, Garrett B, Holahan J. America Under the Affordable Care Act. Urban Institute and Robert Wood Johnson Foundation; 2010. Dec, pp. 1–16. [Google Scholar]
3. Kautter J, Ingber M, Pope GC, Freeman S. Improvements in Medicare Part D Risk Adjustment: Beneficiary Access and Payment Accuracy. Medical Care. 2012;50:1102–1108. doi: 10.1097/MLR.0b013e318269eb20. [DOI] [PubMed] [Google Scholar]
4. Kautter J, Pope G, Keenan P. Affordable Care Act Risk Adjustment: Overview, Context, and Challenges. Medicare & Medicaid Research Review. 2014;4(3) doi: 10.5600/mmrr.004.03.a02. [DOI] [PMC free article] [PubMed] [Google Scholar]
5. Patient Protection and Affordable Care Act. HHS Notice of Benefit and Payment Parameters for 2014, 78 Fed.Reg. 47. Mar 11, 2013. [PubMed]
6. Pope GC, Bachofer H, Pearlman A, Kautter J, Hunter E, Miller D, Keenan P. Risk Transfer Formula for Individual and Small Group Markets Under the Affordable Care Act. Medicare & Medicaid Research Review. 2014;4(3) doi: 10.5600/mmrr.004.03.a04. [DOI] [PMC free article] [PubMed] [Google Scholar]
7. Pope GC, Kautter J, Ellis RP, Ash AS, Ayanian JZ, Iezzoni LI, Robst J. Risk Adjustment for Medicare Capitation Payments Using the CMS-HCC Model. Health Care Financing Review. 2004;25(4):119–141. [PMC free article] [PubMed] [Google Scholar]
8. Trish E, Damico A, Claxton G, Levitt L, Garfield R. A Profile of Health Insurance Exchange Enrollees. The Kaiser Family Foundation; 2011. Mar, (Publication No. 8147) [Google Scholar]
9. Van de Ven WPMM. Risk Adjustment and Risk Equalization: What Needs to Be Done? Health Economics, Policy, and Law. 2011;6:147–156. doi: 10.1017/S1744133110000319. [DOI] [PubMed] [Google Scholar]
10. van de Ven WPMM, Schut F. Guaranteed Access to Affordable Coverage in Individual Health Insurance Markets. In: Glied S, Smith PC, editors. The Oxford Handbook of Health Economics. Oxford, United Kingdom: Oxford University Press; 2011. [Google Scholar]
11. van de Ven WPMM, Ellis RP. Risk Adjustment in Competitive Health Plan Markets. In: Culyer Anthony J, Newhouse JP., editors. Handbook of Health Economics. 1A. Amsterdam: North-Holland: Elsevier; 2000. [Google Scholar]
12. Winkelman R, Mehmud S. A Comparative Analysis of Claims-Based Tools for Health Risk Assessment. Schaumberg, IL: Society of Actuaries; 2007. [Google Scholar]