Comorbid notebook

All results with cell sizes 10 or below need approval to be distributed in any manner.

Jennifer:

• "for the comorbidity analysis, we have only been using 2010 data since it's the most recent we have. The 3 analysis files (all_*) were created using the 2010 claims."
• "the three all_* files will be best for you to work with, so you don't have to deal with the raw claims quite yet. When we asked for the data, we didn't get all Medicare/Medicaid enrollees, we only got those who had an ARV claim or HIV diagnosis at any point. Then we further refine our sample using the algorithm. Due to the nature of claims, we also have to limit to people enrolled for the full year, are fee-for-service (rather than managed care), and are in California the entire year"
• "Regarding the ID variables, bene_id is the Medicare ID so we use it for Medicare only and Duals and msis_id is the Medicaid ID which we use for Medicaid only patients. Throughout the years, Medicaid patients have gradually also been assigned a bene_id, but there are still some people who don't have one. The reason we have bene_id_10 is because we found inconsistencies in IDs when merging across years, this version is needed to merge with raw claims for that year. The bene_id variable allows for consistency in merging across years.… There are a few other caveats with the ID variables we can discuss at a later point when needed."
• The three files have the same variables

These demographic tables use the same sample inclusion criteria as the association analyses described below.

(display-demog "mcare_only")

(display-demog "dual")

These cross-validation analyses of predictive accuracy use all beneficiaries from 2010. The predictors are gender, race, age, age squared, disabled status, and each of the comorbid conditions.

Here, for each of the three datasets and several models, we have the cross-validated mean absolute error (MAE) in predicting total expenditure per subject, in dollars. This means that, e.g., a MAE of $15,000 implies that the model's point prediction of a beneficiary's expenditures would have a mean distance from the true value of $15,000.

Except where marked, each model log-transforms the DV for model-fitting, then exp-transforms the predictions. You can see that this trick increases predictive accuracy (according to MAE) quite a bit.

The best-performing model in each case is unregularized quantile regression. In the case of Medicare-only beneficiares, there is a large improvement upon OLS.

Regularization (as the lasso) is necessary for the more complex quantile regression models because the model-fitting procedure has problems finding a solution otherwise. "Quantile regression w/ lasso, some interactions" interacts age and age squared with each comorbidity, and gender with each race. "Quantile regression w/ lasso, inpatient" uses logistic regression to predict whether the patient had any inpatient costs, and then applies a separately trained quantile regression model depending on each patient's predicted inpatient status.

The last two rows give a kind of lower bound on possible MAE for the demographic IVs and all IVs (demographic variables plus comorbidity flags), respectively. They are simply the MAE achieved when each subject's true value is predicted with the median among all subjects with precisely the same IV values (except for age, since it's continuous, unlike all the other IVs). So, unless age is very informative, we can't get MAEs below these numbers, and we probably won't be able to get MAEs very near them, either; getting a MAE that low in training would almost certainly lead to overfitting in testing.

Here we have two tables of regression, one for each beneficiary status. Outpatient costs, inpatient costs (only among subjects with nonzero inpatient costs), drug costs, and subtotals (the sum of outpatient, inpatient, and drug costs) are fit with quantile regression: the central tendency is the median, the base error is the mean absolute deviation from the median (MAD) and the model error is the mean absolute error (MAE). The probability of nonzero inpatient costs is fit with logistic regression: the central tendency is the mean, the base error is the variance, and the model error is the mean squared error (i.e., the Brier score, which is a proper scoring rule; Brier, 1950; Bröcker, 2009).

For the quantile-regression models, the DVs are untransformed, so each coefficient can be interpreted directly as a linear increase of the conditional median, in dollars.

A few subjects were missing on the urban-rural variable. They were simply dropped.

Among Medicare beneficiaries, we only include subjects with Part D coverage for the full year.

Each row below "Intercept" shows the coefficient for the given variable. age_std is age standardized to have SD 1/2 (per Gelman, 2008), and age_std2 is its square, standardized again.

(display-assoc "mcare_only")

(display-bnc "mcare_only")

(display-assoc "dual")

(display-bnc "dual")

If we chose models on the basis of fit rather than predictive accuracy, then we know a priori that the most complex models would win. This means that for outpatient and drug costs, twin models with all IVs would win (which is similar to the real winning models, non-twin with all IVs), whereas for inpatient costs, all IVs would win (which is the same result we got with predictive accuracy).

We might also compare model selection based on predictive accuracy to model selection based on p-values. In particular, we can use the simple-minded method of trying all the predictors and keeping the significant ones. But how shall I test the significance of the coefficients? quantreg doesn't supply significance tests for models fit with a lasso penalty. Let's use the method of Redden, Fernández, and Allison (2004).

Scott: "Thinking of discussion points: Predictive analytics have not been big in HIV research in part because of smaller data sets we typically collect. However, large data sets, such as Medicaid / Medicare data, are becoming more of the norm than the exception in HIV research. Think of ATN and other large scale studies. NIH push to make data sets publicly available. Virtual cohorts with medical chart data. Basically, predictive analytic tools are needed."

Rob suggested trying a Box-Cox, but this would probably be a hassle to use with quantile regression or lasso linear regression, because it means there's another kind of parameter to fit. Besides, Box-Cox sort of ruins the interpretability of the coefficients, so it's not a good choice when coefficient interpretation is a goal.

I found that I wasn't able to get the sort of improvements on square error that I was able to get on absolute error. My impression is that this is basically a consequence of the high skew of the DVs: a few high values affect square error more than absolute error, so models that optimize mean square error will make a stupider sort of compromise than models that optimize absolute error. This could be discussed in the method paper with a simple, mathematically clean example, and with the RMSEs for outpatient costs.

Arfer, K. B., & Luhmann, C. C. (2017). Time-preference tests fail to predict behavior related to self-control. Frontiers in Psychology, 8(150). doi:10.3389/fpsyg.2017.00150

Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. doi:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2

Bröcker, J. (2009). Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society, 135(643), 1512–1519. doi:10.1002/qj.456

Buntin, M. B., & Zaslavsky, A. M. (2004). Too much ado about two-part models and transformation? Journal of Health Economics, 23(3), 525–542. doi:10.1016/j.jhealeco.2003.10.005

Gelman, A. (2008). Scaling regression inputs by dividing by two standard deviations. Statistics in Medicine, 27(15), 2865–2873. doi:10.1002/sim.3107

Newman, M. C. (1993). Regression analysis of log-transformed data: Statistical bias and its correction. Environmental Toxicology and Chemistry, 12(6), 1129–1133. doi:10.1002/etc.5620120618

Redden, D. T., Fernández, J. R., & Allison, D. B. (2004). A simple significance test for quantile regression. Statistics in Medicine, 23(16), 2587–2597. doi:10.1002/sim.1839