Finding Value: An Approach to Determine Nursing Performance

If we ask a large group of nurses whether they add value to patient care, nearly all would say yes. However, it is excruciatingly difficult to measure nursing care value at both the point of care and across the health care system as a whole. In the past several years, efforts have been made to move towards a value-based health care system, one in which each provider is linked to their outcomes of care individually and collectively. Physicians as well as health care systems are increasingly paid based on quality, efficiency, outcomes, safety, cost reduction, person and community engagement (Centers for Medicare & Medicaid Services, 2019). Hospitals also are measured on patient satisfaction and reimbursement is allocated according to both overall scores and improvement in scores over time. One of the key questions: Where do nurses fit in this evolving value equation? This article demonstrates new data science techniques emerging within acute care settings that link individual nurses to each patient via data from the shift assignment.

The classic business definition of value is the relationship between price and quality. Unfortunately, we do not put a price on nursing care so other methods are needed. Sharon Pappas, chief nurse executive at Emory Healthcare in Atlanta, proposed a definition of nursing value as the health outcomes achieved compared to the dollars spent for that care (Pappas, 2013). We can measure the expenditure of nursing care for each patient by examining which nurses are assigned to each patient. This allows us to calculate a nursing cost metric based on direct hours and associated nursing costs for wages, indirect costs such as paid hours for training time and employer-provided benefits (Welton, Jenkins, & Perraillon, 2018).

Nurse characteristics and patient outcomes

In 2003, Linda Aiken, PhD, RN, professor and director of the Center for Health Outcomes and Policy Research at the University of Pennsylvania, Philadelphia, published a seminal study that demonstrated hospitals with higher proportions of nurses with baccalaureate or higher degrees have better risk-adjusted surgical outcomes (Aiken, Clarke, Cheung, Sloane, & Silber, 2003). This was one of the first articles to link the individual and collective characteristics of nurses to clinical outcomes of care. The interesting question is whether other characteristics are linked with better care, quality and safety. Nurse experience has been a major consideration, especially in specialty areas where increasing medical technologies are leading to more complex care environments and the need for nurses to be prepared at a higher skill level. What if on a night shift at a local hospital all of the five scheduled nurses were new graduates with less than one year of experience? This raises an interesting ethical question to nurse leaders about their duty to not only have the necessary number of nurses needed, but to be sure the scheduled nurses are prepared to deliver high-quality care.

A more vexing problem is how to better monitor and measure both nurse staffing levels and individual nurse performance within the matrix of patient care across time and many different settings. The classic way to do this is by using nurse-to-patient ratios or other averaging metrics, such as the hospital midnight census as a measure of need for nursing care. But these approaches hide the variability and complexity of demand for nursing care, as well as the expertise and preparation of each nurse. To better measure and quantify the value of nursing care, we need to identify the contribution of each individual nurse (Welton & Harper, 2016).

Hospitals and other nursing care settings are increasingly using software to schedule and assign nurses to patients. Because these data are readily assessable in the data systems, they can be combined with other clinically and operationally meaningful data to directly link nurses to patients. For example, consider the overall issue of adequately staffing inpatient units with nurses. Typically this is done by staffing ratios, but there is little information about the quality and characteristics of nurses or the overall skill mix of nurses and nurses’ aides. Conventional unit-based statistics say little about the overall exposure of each patient to the nurses caring for him/her. By capturing the actual nurse-patient assignment, nurse leaders can get a better overall picture of how nurses are linked to each patient. This provides a new tool to understand overall unit and hospital-level trends.

In Figure 1, the assignment data in a study funded by the Agency for Healthcare Research and Quality (AHRQ) at a pediatric academic medical center was used to better understand the mean experience level of nurses during the day and evening shifts summarized by three types of units. The data was collected at the individual nurse-patient assignment, and the unit of analysis is a nurse caring for an individual patient within his or her shift assignment. Patients and nurses are linked through unique anonymized identifiers so it is possible to see every nurse that cared for a patient or select all patients in a particular diagnosis and the associated nurse characteristics. This provides a much more precise measure of the nursing care each patient receives compared to a unit average such as nursing hours or costs per patient day.

 

A clear decrease in the mean experience level of nursing during a night shift is evident compared to a day shift over the three years of data available for analysis. Such an analysis could be helpful in understanding both staffing and assignment patterns and if clinically or operationally meaningful differences exist. Figure 1 clearly shows that patients are more likely to receive care from a nurse with lower experience on a 12-hour night shift compared to a day shift. These patient-level nursing data are potentially actionable on a short time frame from shift to shift and also may be helpful in long-term strategic planning and determining the cost of patient-level nursing care.

In Table 1, these same data are linked to the All Patients Refined-Diagnostic Related Group (APR-DRG) and severity of illness scores for each patient. The APR-DRG is an adaptation of the adult DRG that assigns a single diagnosis for pediatric populations based on primary and secondary diagnoses and procedures (Averill et al., 2003). Overall, nursing experience as distributed across patients by severity of illness on the medical-surgical floors shows relatively stable experience levels with patients having higher acuity. However, the critical care units clearly assign higher-acuity patients to more experienced nurses. Nurses on night shift are two to three years less experienced, on average, than nurses on day shift.

Implications for nurse leaders

The data in this article demonstrates the potential power of multidimensional analysis with more complex data collection and extraction from available electronic health records. Viewing care from a nurse standpoint, a wide variety of analyses that examine the overall characteristics of nurses providing care could be conducted, including producing nurse costing models for each individual patient that can be linked to the reimbursement system. Such analyses would help health care leaders better understand the complex and dynamic interaction of nurses and patients within a particular setting. Short-term or real-time tactical data could show patient nursing acuity levels changing, which would allow front-line charge nurses and managers to adapt assignments or add supplemental nurses during a shift.

From a patient standpoint, future research projects can be constructed that measure the actual nursing care a patient received and pose questions such as: Is there an association between nurse experience levels and patient care outcomes? What are the cost differences of nursing care based on different levels of nurse experience? Patient-level analyses also could be used to examine the cost vs. outcomes of nursing care for patients who stay an extra day, compared to early discharge. These approaches show early promise that could eventually lead to better understanding the unique added value of each nurse.

Acknowledgements

This article was supported by grants from the Agency for Healthcare Research and Quality 1R03 HS025495-01 and the University of Colorado School of Medicine Data to Value (D2V) project.

References

Aiken, L. H., Clarke, S. P., Cheung, R. B., Sloane, D. M., & Silber, J. H. (2003). Educational levels of hospital nurses and surgical patient mortality. JAMA, 290(12), 1617-1623.

Averill, R. F., Goldfield, N. I., Hughes, J., Muldoon, J., Gay, M. J., & Mc Cullough, E. (2003). What are APR-DRGs? An introduction to severity of illness and risk of mortality adjustment methodology. Retrieved from http://multimedia.3m.com/mws/media/1042598O/resources-and-references-his-2015.pdf

Centers for Medicare & Medicaid Services. (2019). CMS hospital value-based purchasing program results for fiscal year 2019. Retrieved from https://www.cms.gov/newsroom/fact-sheets/cms-hospital-value-based-purchasing-program-results-fiscal-year-2019

Pappas, S. H. (2013). Value, a nursing outcome. Nursing Administration Quarterly, 37(2), 122-128. doi:10.1097/NAQ.0b013e3182869dd9

Welton, J. M., & Harper, E. M. (2016). Measuring nursing care value. Nursing Economics, 34(1), 7-14.

Welton, J. M., Jenkins, P., & Perraillon, M. C. (2018). A micro-costing or ‘bottom-up’ approach to measuring nursing costs using data from electronic health records. Nursing Economics, 36(1), 46-51.

About the authors

John M. Welton, PhD, RN, FAAN, is a professor at University of Colorado College of Nursing, Denver.

Jennifer Baird, PhD, MPH, RN, CPN is director of the Institute for Nursing and Interprofessional Research at Children’s Hospital of Los Angeles.

Nancy Blake, PhD, RN, NEA-BC, FAAN is chief nursing officer at Harbor UCLA Medical Center, Torrance, Calif., and assistant adjunct professor at UCLA School of Nursing, Los Angeles.