As the Cheshire Cat tells us, if you don’t know where you are going, any path will get you there. This suggests that if you are not clear about what you, your agency and your team are to “produce”—what the clear outcomes of your work are to be — then it is okay for staff to spend their time on whatever their current practices or preferences of work might be. This leads to a perspective that work—any work— is good, whether it is making a needed difference or not. What we need in our government is a focus on outcomes–the value delivered to the American people by a high-performing federal workforce.
If we consider the importance of outcomes, our search engine steers us quickly toward the field of healthcare. Myriad results discuss global health, clinical trials, infections, epidemics, and emergency services. In medical parlance, measuring outcomes is critical to the effective clinical treatment of disease, which leads to healthy patients. Applying this theory to an organization, rather than an organism, doesn’t charge the theory—healthy organizations measure outcomes.
Defining Outcomes
It is important to start with an understanding of the difference between outputs and outcomes. Outputs are the goods or services produced by programs or agencies, whereas outcomes can be defined as the impact that those outputs have on social, economic or other indicators. Outputs are often easy to measure and track – you can count them up – but they simply reflect the fact that somebody is working. In order to determine the value of a program or project, and to improve the efficiency and effectiveness with which it is delivered, it is essential to measure outcomes.
Outcomes are not activity-based, such as “conduct five tax workshops” or “implement paperless admissions process at 20 hospitals” or “develop a new pest-testing protocol.” None of these statistics reflect results achieved and therefore cannot inform funding and budget decisions, which should be based on the value of the project; not the activities or products of work delivered.
In order to develop outcomes, program personnel must first define success–the specific, measurable, and meaningful results expected of the program. Outcomes can be something that the project wants either to maximize, such as evidence of “increased learning by workshop participants” or minimize, such as “reduce pest damage to fruit.” Some outcomes can be measured with financial indicators. For example, by implementing paperless admissions, the personnel and material costs associated with hospital admissions will be reduced. The outcome, however, isn’t just a cost savings, it is a more efficient and effective admissions process that is faster for patients, results in fewer errors, and improves the patient experience.
Choosing the Right Outcomes
As critical as it is to shift focus from measuring metrics to measuring outcomes, it is even more important to choose the right outcomes. Choosing the right outcomes requires an understanding of the overall agency mission, how the program supports the mission, and what success looks like. What happens when you choose the wrong outcomes? A law enforcement agency had, as part of its duties, the responsibility of moving prisoners between jail and the courthouse. The cost of maintaining the bus fleet, providing drivers and guards, and shuffling the prisoners around was high. The agency found that by hiring an outside security firm, they could manage prisoner transport at a much lower cost. The outcome was measured in cost savings. A few years later, they discovered that the ability of the agency to carry out its primary mission of fugitive apprehension had been compromised. It turned out that prisoner transport was one way the agency law enforcement personnel kept their skills current. Measuring the right outcome–increasing the skills and abilities of law enforcement personnel to apprehend fugitives–would have resulted in a different set of decisions around prisoner transport. Outsourcing the activity reduced costs, but resulted in lost skills needed to execute the mission.
Metrics Versus Outcomes
To ensure you measure what matters: 1) transition from measuring activity to also measuring value; 2) measure contribution, not attribution; and 3) use metrics focusing on business and development outcomes, the value of the project and effective implementation.
Measurement in government is often an act of compliance – counting up the work plan activities completed and monitoring how funds are spent. While ensuring that adequate progress is made on the project and funds are accounted for and expended responsibly is an important aspect of performance reporting, to demonstrate impact and value of a project requires another type of measurement. In short, agencies have a responsibility to track funds and activities; but they also have a responsibility to communicate value through outcome measurement.
Closing
Moving to a performance-based culture that measures outcomes rather than an accountability-based culture that measures metrics is critical. An accountability-oriented measurement system asks, “Did it happen?” It tracks activities and dollars. Data is collected for transparency and rarely revisited.
Performance-oriented measurement attempts to prove theories of change, asking, “Did it work?” and “How well is it working?” It is a value-oriented approach that captures outcomes and progress toward long-term and systemic change and uses data to improve results and demonstrate value delivered.
Outcomes must be specific, measurable, and meaningful. They should be the foundation upon which dollars are prioritized and allocated within our federal budget. Otherwise, like poor Alice, you’ve just spent a lot of money to fall down another rabbit hole.
