NUR 7010 Module 3 Data Quality Paper: Testing Flowsheet Data on Restraint Use Before Reuse Example

Is the Restraint Rate Real? Testing Flowsheet Data Before a Hackensack System Reports It

Student Name

M.S. in Nursing, Nursing Informatics, Thomas Edison State University

NUR-7010: Nursing Informatics: Databases and Knowledge Management

Mentor Name

Month Day, Year

Is the Restraint Rate Real? Testing Flowsheet Data Before a Hackensack System Reports It

The Request

The system and every figure in this example are invented. A health system of three hospitals around Hackensack wants to add physical restraint use to its nursing quality dashboard. The proposed measure is restraint days per 1,000 patient days, drawn from the restraint flowsheet in the electronic health record, where nurses document the start of restraint, checks every two hours and removal. Nursing's executive wanted the informatics nurse to confirm that the data were good enough before the first report went to the board.

What Data Quality Means

Weiskopf and Weng (2013) surveyed the published ways researchers test record data before reusing it and distilled five dimensions. Completeness concerns whether a fact about the patient made it into the record at all, and correctness whether a recorded fact is true. Concordance compares sources and elements for agreement. Plausibility judges whether a value is believable alongside everything else known, and currency whether it was recorded when it happened. They also cataloged methods for testing each dimension, including comparison with a gold standard, checks of agreement between data elements and between sources, checks for the presence of expected elements and comparisons of distributions.

Kahn et al. (2016) offered a shared vocabulary that files every data quality check under one of three headings: conformance, meaning that values follow required formats and allowed values; completeness, meaning that expected data are present; and plausibility, meaning that values are believable. Any of the three may be tested internally, against the data set's own rules, or externally, against an outside source. The framework was meant to give organizations a shared vocabulary so that quality checks could be compared and reported consistently.

Records Describe Care

Hripcsak and Albers (2013) cautioned that data in electronic records are shaped by the process of care. A value appears because a clinician ordered a test, documented a finding or billed for a service, not simply because a condition exists. Data may be missing because nobody looked, inaccurate because of how they were entered and biased because sicker patients generate more data. They argued that anyone reusing record data must model the health care process that produced it.

For restraints, the warning was specific. A restraint that a nurse never documented does not exist in the data. A restraint documented as started but never as removed appears to last for days.

The Assessment

The informatics nurse pulled six months of restraint flowsheet data from all three hospitals and ran checks on each dimension, adding a manual chart review of 120 randomly selected patient days on high-use units as a gold standard.

The two largest problems pulled in opposite directions. Missing removals made restraint episodes look far longer than they were, inflating the rate. Missed episodes, which the chart review found mostly in the emergency department holding areas and during transfers, made the rate too low. The net effect on the rate was unknown, and differed by hospital.

Fixes Before Reporting

The informatics nurse recommended four changes before any rate went to the board. The flowsheet would require a removal entry or an active status at every shift handoff, so open episodes could not drift. Restraint orders and flowsheet entries would be linked, with an alert to the nurse if a restraint was documented without an order. The free-text restraint type would be removed in favor of a fixed list. And emergency department holding areas would use the same flowsheet as inpatient units.

She also recommended that the measure be calculated as restraint episodes per 1,000 patient days for the first six months, rather than restraint days, because counts of episodes are less affected by missing removal times.

Decision

The chief nursing officer agreed to delay the board report by one quarter. During that quarter, the system will run the data checks monthly, repeat the chart review on a smaller sample and report the measure internally to unit managers with its known limits. The board will see the rate once completeness of removals exceeds 95 percent and agreement with chart review exceeds 90 percent.

Talking With the Units

The informatics nurse shared the findings with charge nurses on the four units with the highest restraint use before proposing fixes. Their explanations changed the plan. Missing removals happened most often when a patient was transferred while restrained, because the receiving unit started a new episode and nobody closed the old one. Episodes without orders were often short restraints during procedures, ordered verbally and never entered. Charge nurses also pointed out that the two-hour check was charted in batches because the flowsheet required six separate fields for each check. The fixes therefore included closing episodes automatically at transfer with a prompt to the receiving nurse, a quick order entry path for procedural restraints and a shorter check row.

Building the Checks Into Routine

The checks were turned into an automated monthly report that lists open episodes, episodes without orders and free-text entries by unit, so that managers can correct problems while staff still remember the events. The report uses the harmonized framework's three categories, so it can be extended to other nursing measures as the dashboard grows.

Limits

A chart review of 120 patient days is a small gold standard. The chart itself may miss restraints that were never documented anywhere. Linking orders to flowsheets assumes that orders are entered correctly, which is itself a data quality question.

Conclusion

The restraint rate the board was about to see would have mixed missing removals with missed episodes. Weiskopf and Weng's dimensions and the harmonized framework gave a structured way to test the data, and Hripcsak and Albers explained why records of care are not records of patients. Testing first, fixing documentation and setting thresholds before reporting protected the board from a number that looked precise and was not.

References

Hripcsak, G., & Albers, D. J. (2013). Next-generation phenotyping of electronic health records. Journal of the American Medical Informatics Association, 20(1), 117-121. https://doi.org/10.1136/amiajnl-2012-001145

Kahn, M. G., Callahan, T. J., Barnard, J., Bauck, A. E., Brown, J., Davidson, B. N., Estiri, H., Goerg, C., Holve, E., Johnson, S. G., Liaw, S.-T., Hamilton-Lopez, M., Meeker, D., Ong, T. C., Ryan, P., Shang, N., Weiskopf, N. G., Weng, C., Zozus, M. N., & Schilling, L. (2016). A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data. eGEMs, 4(1), Article 1244. https://doi.org/10.13063/2327-9214.1244

Weiskopf, N. G., & Weng, C. (2013). Methods and dimensions of electronic health record data quality assessment: Enabling reuse for clinical research. Journal of the American Medical Informatics Association, 20(1), 144-151. https://doi.org/10.1136/amiajnl-2011-000681