NUR 8200 Module 5 Care Model Evaluation Paper: The Hotspotting Trial, Medicare Care Coordination and ACO Savings Example

The Forty Percent That Wasn't: Evaluating a High-Utilizer Program Fairly After the Camden Hotspotting Trial

Student Name

Doctor of Nursing Practice, Thomas Edison State University

NUR-8200: Integrating and Evaluating Population Health in Advanced Nursing Practice

Mentor Name

Month Day, Year

The Forty Percent That Wasn't: Evaluating a High-Utilizer Program Fairly After the Camden Hotspotting Trial

The Claim

The Trenton system in this paper is made up: two hospitals and a ring of clinics. Three years ago it launched a program for patients with frequent admissions, pairing nurse care managers with social workers for anyone admitted twice or more within half a year. An internal report, presented to the board, compared admissions in the six months before enrollment with the six months after and found a 40 percent drop. The board wants to expand the program. Before the board votes, its DNP-prepared population health vice president must say whether the evidence holds.

The program has real features worth keeping. Each enrolled patient has a nurse care manager and a social worker who visit at home, accompany patients to appointments and help with housing, food and benefits. Staff describe patients who now have stable apartments, working phones and a primary care clinician they trust. The program costs about $1.9 million a year. The board's question is whether expanding it to a second hospital will reduce admissions enough to justify a larger budget, and the internal report seemed to answer yes.

Why Before-and-After Can Mislead

Patients are chosen for high-utilizer programs at the peak of their use. Many would have fewer admissions afterward even without any program, because crises pass and conditions stabilize. This tendency of extreme values to move toward the average is called regression to the mean. A before-and-after comparison credits the program with that natural decline.

The Hotspotting Trial

Finkelstein et al. (2020) tested the Camden Coalition's care transition program, a widely praised model of intensive, relationship-based support for patients with complex medical and social needs. They randomly assigned 800 hospitalized patients, each with at least one other admission in the previous six months, to the program or to usual care. The 180-day readmission rate was 62.3 percent in the program group and 61.7 percent in the control group, with no significant difference. A before-and-after comparison within the program group would have suggested a 38-percentage-point drop in admissions, but the control group showed a similar drop, revealing regression to the mean.

Broader Evidence

Peikes et al. (2009) reported on 15 Medicare care coordination programs, each randomly assigning fee-for-service beneficiaries, mostly with heart failure, coronary disease or diabetes, to the program or usual care; 18,309 patients were included. Thirteen of the 15 programs showed no significant difference in hospitalizations, one reduced them and one increased them, and none produced net savings after program fees. The handful that looked promising met patients face to face and worked closely with their doctors.

McWilliams et al. (2016) examined Medicare's first Shared Savings Program ACOs using a difference-in-differences design, comparing the change in spending for ACO patients with the change for similar non-ACO patients. In 2013, ACOs that entered in 2012 spent $144 less per beneficiary than expected, roughly a 1.4 percent saving, while those that entered in 2013 showed almost no savings. Physician-led primary care groups saved more than ACOs built around hospitals.

What the Internal Report Can and Cannot Show

The internal report's 40 percent drop resembles the drop seen in both groups of the Camden trial. With nobody to compare against, the report cannot credit the program. That does not mean the program fails; patients and staff describe real benefits, such as housing found and medicines sorted out. It means the board needs a better evaluation before expanding.

A simple check makes the point. The analytics team pulls records for 300 patients who met the program's eligibility rule during the same period but were not enrolled because the program was full. Their admissions also fell, by 33 percent, in the six months after they first met the rule. The program's 40 percent drop may therefore reflect a small effect on top of a large natural decline, or no effect at all; without randomization, the difference of seven points cannot be trusted, because enrolled and unenrolled patients may differ in ways the data do not show.

The paper also asks what else the program might be changing. Housing placements, completed benefit applications and patients' own reports of feeling supported are counted from program records. These are real gains that a readmission measure would miss, and the evaluation will measure them alongside hospital use rather than treating admissions as the only test of value.

A Fair Evaluation

The paper proposes a randomized evaluation as the program expands. With more eligible patients than slots, eligible patients will be randomly offered enrollment or usual care during a 24-month period, which is fair when demand exceeds capacity. The institutional review board will review the design, and patients in usual care will keep all existing services. If randomization is not accepted, a difference-in-differences design comparing eligible patients at the hospital with similar patients at a hospital without the program is the fallback, with its weaker assumptions stated.

Outcomes include readmissions and hospital days over 180 days, emergency visits, deaths, primary care visits, patient-reported quality of life and costs. The program's own process measures, such as contacts per patient and needs resolved, are tracked as well.

Communicating With the Board

The paper recommends that the board continue the program at its current size while the evaluation runs, rather than expand on the basis of a misleading comparison or cancel a program that may help in ways admissions do not capture. The report will explain regression to the mean in plain terms, using the Camden trial as an example from nearby.

Conclusion

A 40 percent drop in admissions after enrollment looks like success but resembles the natural decline seen in the control group of the Camden hotspotting trial, where the program did not change readmissions. Most randomized Medicare care coordination programs did not reduce hospitalizations, and early ACO savings were modest. The paper recommends a randomized or difference-in-differences evaluation before expansion, with outcomes that include patients' own reports, so the board can decide on evidence rather than a misleading comparison.

References

Finkelstein, A., Zhou, A., Taubman, S., & Doyle, J. (2020). Health care hotspotting: A randomized, controlled trial. New England Journal of Medicine, 382(2), 152-162. https://doi.org/10.1056/NEJMsa1906848

McWilliams, J. M., Hatfield, L. A., Chernew, M. E., Landon, B. E., & Schwartz, A. L. (2016). Early performance of accountable care organizations in Medicare. New England Journal of Medicine, 374(24), 2357-2366. https://doi.org/10.1056/NEJMsa1600142

Peikes, D., Chen, A., Schore, J., & Brown, R. (2009). Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA, 301(6), 603-618. https://doi.org/10.1001/jama.2009.126