IF YOU CAN’T MEASURE IT; YOU CAN’T MANAGE IT
The expression, If you can’t measure it; you can’t manage it, is a derivative of What gets measured; gets managed. Credited to Peter Drucker, a member of the modern business management hall of fame, the expression reduces to a simple aphorism a highly complex idea. Drucker is also credited with the aphorism, Management is doing things right; leadership is doing the right things. This last expression is among the ideas discussed in my finance class because doing things right, relative to doing the right thing, could be costly from an economic as well as a behavioral lens.
Many of the domains we as education leaders manage are difficult to measure, if not unmeasurable.
Using metrics where they add value, and not using them where they don’t, is a balance the most sophisticated of leaders constantly seek. In the business world, where measurement is their drug, everything is measured to prove to someone who's not in the room that what was done was exactly what that someone who is not in the room told everyone in the room to do, according to Liz Ryan, journalist and author of Reinvention Roadmap: Break the Rules to Get the Job You Want and Career You Deserve. In the education world that would be following the money and the strings attached to it, AKA compliance.
Managing a complex modern system like a school district, with a medley of technical, human, and bureaucratic elements, requires measurement to control while understanding the hazards and unsuspected dangers of metrics to manage. The problems arise when using a measure as a metric, sometimes referred to as Goodhart’s Law.
Similar to Campbell’s Law, which has similar implications, its basic principle is, When a measure becomes a metric, it ceases to be a good measure. When managing an education system this is particularly problematic.
Before moving on, consider the terms measure and metric. A metric is an observable value. A measure associates meaning to that value by applying human judgment. The key difference is that a metric is based on standardized procedures, calculation methods and systems for generating a number. A measurement could be taken using a variety of different metrics.
David Manheim writes eloquently about this concept in his essay, Goodhart’s Law and Why Measurement is Hard. In a recent blog of mine on Campbell’s Law, I argued that once a measure becomes a target, it is no longer a good measure. The purpose of my blog was to stir a conversation among my CSUF leadership class students (and all education leaders) relative to the Local Control & Accountability Plan (LCAP) and the metrics that will now drive school leadership behavior. Dr. Manheim replied to my blog with the suggestion that Goodhart’s Law, a close relative to Campbell’s Law, may be of interest to me and my students in this context. He was correct. A researcher in the world of existential risk mitigation, computational modeling, and epidemiology, I was introduced to his work on Reddit™, home to thousands of communities, endless conversation, and authentic human connection. Looking at problems of practice through a variety of lenses is how theory and practice inform each other. It's how we approach problems in the education leadership program at California State University, Fullerton.
Given California’s desire to configure the LCAP as an instrument of compliance, it’s important that education leaders and policymakers learn the lessons of the mistaken metrics of No Child Left Behind (NCLB) legislation.
NCLB was the ultimate example of how a compliance model can have devastating long term effects. Compliance drives measurement. And measurement subsequently drives behavior. The end result of NCLB was an example of Campbell’s Law: the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
Recently retired California State University, Chico Professor and comprehensive data analysis guru Victoria Bernhardt has written extensively on the perils of focusing on compliance. She and her partner at the non-profit Education for the Future Initiative, Brad Geise, have perfected an effective framework for systemic change. Their Continuous School Improvement Model is based on the principle that schools focused only on gaps and compliance can neither innovate nor create a future that looks different from the status quo.
I agree with Bernhardt's conclusion - based on working with hundreds of learning organizations at local, state, regional, national, and international levels - that schools focused on compliance inhibit systemic improvement and limit progress towards excellence and real equity.
Dr. Manheim’s suggestion that Goodhart’s Law is worthy of my and my students attention as we explore the impact of measurement on what’s being measured is based on his argument that the triad of intuition, trust, and complexity are a useful heuristic for understanding where to use measurement or where not to use measurement. Manheim writes, “Measurement replaces intuition, which is often fallible. It replaces trust, which is often misplaced. It finesses complexity, which is frequently irreducible. So faulty intuition, untrusted partners, and complex systems can be understood via intuitive, trustworthy, simple metrics.”
The education system in California now revolves around the Local Control Funding Formula which uses measures via the LCAP to incentivize schools to perform in certain ways. It’s a self-optimization system. Bad metrics optimizes towards undesired outcomes. Goodhart’s Law maintains that the metric intended to measure will not optimize the desired outcome. Education leaders must ensure that the metrics used are effective drivers to help schools meet desired measurable goals. If the metrics diverge from the desired measurable goals the system will not improve.
A shared vision is the engine that drives positive change and keeps learning organizations focused on measurable goals. Good metrics optimizes towards meeting those goals.