NUR 8150 Module 4 Predictive Analytics Paper: High-Cost Patients, Algorithmic Bias and Fair Use of Data Example

Predicting Cost Is Not Predicting Need: Testing a Care Management Algorithm for Bias in a Mercer County Accountable Care Organization

Student Name

Doctor of Nursing Practice, Thomas Edison State University

NUR-8150: Information Systems and Technology Impacting Healthcare Delivery

Mentor Name

Month Day, Year

Predicting Cost Is Not Predicting Need: Testing a Care Management Algorithm for Bias in a Mercer County Accountable Care Organization

The Program and the Question

The accountable care organization in this paper is invented: a partnership of a hospital in Trenton, thirty primary care practices and a federally qualified health center, that answers for spending and quality across roughly 60,000 Medicare and commercially insured patients. Its care management program assigns nurse care managers to the 3 percent of patients predicted to have the highest needs in the coming year. Patients are chosen by a commercial algorithm that assigns each a risk score.

A care manager at the health center noticed that few of her Black patients with poorly controlled diabetes and heart failure were being referred, while white patients with fewer conditions were. The DNP-prepared director of population health was asked to examine how the algorithm chooses patients and whether it is fair.

What Analytics Can Do

Bates et al. (2014) described six areas where big data and clinical analytics could reduce health care costs: finding the costliest patients, cutting readmissions, triaging, spotting decline early, heading off adverse events and tuning treatment for illness that spans organ systems. They emphasized that organizations need not only algorithms but also registries, monitoring and infrastructure to act on predictions, and they raised policy questions about regulation and privacy. Care management for high-cost patients was the first and clearest of their examples, which is exactly how this organization uses its algorithm.

How Bias Enters an Algorithm

Obermeyer et al. (2019) took apart a commercial risk tool, typical of its industry and applied to millions of people, that health systems use to choose patients for extra care management. Matched on score, Black patients carried more uncontrolled disease than white patients. Its target was next year's bills, not next year's illness; with less care reaching Black patients at equal need, their bills, and so their scores, ran lower. Fixing the target would have nearly tripled the Black share of automatically enrolled patients, to 46.5 percent from 17.7. The authors warned that convenient stand-ins for health, such as cost, can create large biases.

Testing Our Algorithm

The organization's vendor confirms that its risk score is trained to predict next-year total cost. The population health team tests it using two years of claims and clinical data. For patients at each risk score, they compare the number of active chronic conditions, the most recent hemoglobin A1c, blood pressure and kidney function, by race.

The test was designed before the data were examined, so that the team could not choose a method after seeing the results. The analysts grouped patients into risk score bands of one percentile each and, within each band, compared clinical measures between Black and white patients. Patients with missing laboratory values were analyzed separately to make sure gaps in testing did not hide differences. The method was reviewed by a biostatistician at the partner hospital, and the results were shared with the care managers who first raised the concern.

The results mirror the published study. At the same risk score, Black patients had on average 0.9 more active chronic conditions, higher A1c values and more uncontrolled blood pressure than white patients. Among patients with the most chronic conditions, Black patients were less likely to fall in the top 3 percent selected. The team also checks Hispanic patients and patients with Medicaid and finds a similar but smaller pattern for Medicaid patients.

Building a Better Model

The team builds an alternative score that predicts a measure of health need, the number of active chronic conditions that are poorly controlled, rather than cost, while still using claims and clinical data. Rajkomar et al. (2018) showed that deep learning models using a patient's entire record in a standard data format could predict in-hospital death, readmission, long stays and discharge diagnoses across 216,221 adults at two academic centers more accurately than traditional models. The team does not adopt deep learning, which would require infrastructure and expertise it lacks, but takes from the study the value of using standard data formats and richer clinical data rather than claims alone.

When the new score is applied, the share of Black patients selected for care management rises from 14 to 33 percent, closely matching their share of patients with poorly controlled chronic conditions. The total number selected stays the same.

Governance

The paper proposes an algorithm governance committee including a nurse leader, a physician, a data scientist, a compliance officer, a community health worker and a patient representative. Every algorithm that decides who receives care, alerts or outreach must document what it predicts, be tested for differences by race, ethnicity, language and insurance before use and be retested yearly. Patients will be told in plain language that a risk score helps choose who is offered care management, and clinicians may refer patients the score misses.

Measures

Each quarter the team reports who is selected, by race, ethnicity and insurance compared with their share of patients with high clinical need, hospital admissions and emergency visits among selected patients, A1c and blood pressure control, and cost per patient. A balancing measure, the total cost of care for the whole population, will show whether the change affects the organization's financial targets.

Conclusion

This accountable care organization's care management algorithm predicts cost, and like the algorithm in a widely cited study, it selects fewer Black patients than their illness warrants. Big data has clear uses in finding high-need patients, but the choice of what to predict determines who is helped. A model that predicts poorly controlled illness rather than cost doubles the share of Black patients selected without enlarging the program, and a governance committee will make bias testing a standing requirement for every algorithm the organization uses.

References

Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014). Big data in health care: Using analytics to identify and manage high-risk and high-cost patients. Health Affairs, 33(7), 1123-1131. https://doi.org/10.1377/hlthaff.2014.0041

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447-453. https://doi.org/10.1126/science.aax2342

Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., Sundberg, P., Yee, H., Zhang, K., Zhang, Y., Flores, G., Duggan, G. E., Irvine, J., Le, Q., Litsch, K., ... Dean, J. (2018). Scalable and accurate deep learning with electronic health records. npj Digital Medicine, 1, Article 18. https://doi.org/10.1038/s41746-018-0029-1