← When Metrics Become Targets, Trouble Surely Follows Margaret Thatcher: Britain’s Turnaround Guy →

When Metrics Become Targets Part II: Wicked Problems

Posted on April 9, 2013 by David

Last week I wrote about the cheating scandal in the Atlanta schools and the charging of 35 teachers and administrators with racketeering. The incident has created a furor in Atlanta over the charges themselves and the amount of bail initially required – up to $7.5million in the case of Dr. Beverly L. Hall, the superintendent of the schools district at the time. Indeed, to a Canadian ear, the charges and the bail conditions did sound like the overkill found all too frequently in the American justice system. Around the country the charges added fuel to the controversy about the role of standardized testing in schools.

The debate between the proponents and opponents of standardized testing shows the same, depressing absolutist “either/or” quality found in the wrangling over other political issues in America. The debaters may seldom be quite right, but they are never in doubt. Opponents of standardized testing say that it is damaging equality and equity. Proponents of testing, on the other hand, say that no progress is possible without it. Thomas J. Kane, Director of the Center for Policy Research at Harvard University was quoted as saying that abandoning testing would “be equivalent to saying ‘O.K., because there are some players that cheated in Major League Baseball, we should stop keeping score, because that only encourages people to take steroids.’” That’s really helpful…

Education System Failure as a Wicked Problem

The failure of the education system in America is a classic “wicked” problem. These are the kinds of problems that cannot be abstracted from their context, solved like a Rubik’s Cube puzzle and then put back into context. What looks like a single problem may, in fact be many different kinds of problem masquerading under the same heading. A medical example of this is Type II diabetes, which is not a disease so much as it is a cluster of symptoms produce by a host of different causes. It is the absence of detailed understanding of these causes that leads to rather general treatments – like diet and exercise. Which, on thinking about it, might be just as a good an approach to solving the education problem!

What follows is a quote/précis of the descriptions of a wicked problem from Horst Rittel and Melvin Webber’s classic 1973 Policy Sciences paper on the topic:

At its root a problem can be described as a discrepancy between things as they are and things as they ought to be. The process for addressing the problem starts with a search for the causes of the discrepancy. In the case of wicked problems;

There is no definitive formulation: the information required to understand the problem depends on one’s ideas for solving it. The formulation of the problem is the problem.
There is no stopping rule; the problem is never “solved”. We either run out of time of money or come up with something that is “good enough”.
Solutions to wicked problems are not true-or-false, but good-or-bad. In the Toyota Production System people never talk about “solutions”; they always talk about “counter-measures”, which recognizes that a system reacts and responds to one’s efforts to change it.
There is no immediate or ultimate test of a solution to a wicked problem: this is a consequence of the reactiveness of the system. Every attempted solution sets up waves of repercussions. Short-term results may be favourable, while long run consequences are perverse.
Every solution to a wicked problem is a “one-shot operation”; because there is no opportunity to learn by trial-and-error. One should add here that models, simulations and analogies (simulations drawn from the past) can help one’s thinking, but can never replicate the problem exactly.
Wicked problems do not have an enumerable (or an exhaustively describable) set of solutions because the problems themselves are so ill-defined.
Every wicked problem is essentially unique; there are no classes of wicked problem. Thus the direct transfer of physical-science and engineering thinking into attempted solutions may be positively harmful.
Every wicked problem may be considered a symptom of another problem; there are no logical grounds for deciding the best level on which to tackle a wicked problem. To avoid tackling symptoms one has to address the problem on as high a level as possible.
The discrepancy between “what is” and “what ought” can be explained in numerous ways and the explanation determines the nature of the problem’s resolution.
The planner, unlike a scientist, has no right to be wrong; in the world of wicked problems the aim is not to find truth but to improve some aspect of the world in which we live.

An Ecological Approach

So where does this leave us with wicked problems like the failure of an education system? At a minimum it suggests that we need some practical wisdom – a mental model that acknowledges the importance of context and is capable of handling the central role that it plays in all wicked problems. This is exactly what an ecological model does. More importantly, an ecological model places the actor, planner, manager in a field of tension between means and ends. I call this place the “sweet zone”. Means become ends and metrics become targets only if one leaves this sweet zone and falls into a trap of some kind or another. If the “who” and the “why” of purpose always embrace and contain the “what” and the “how” of methods, we cannot go far wrong. Perhaps this is why the Finnish schools, by always focusing on their purpose of addressing social inequality, end up producing a “good enough” answer to the challenge of achievement.

At the same time, however, the ecological model highlights the system dynamics that make this process of dwelling in the sweet zone so difficult. This is especially true in the case of commercial organizations, as they grow in size and become beholden to the financial community and the metrics that it relies on. As firms become ‘firmer’ and more machine-like, the hard measures tend to drive out the soft ones. It is easy to slip into a spiral trap, where change within the system becomes very difficult and crisis is inevitable.

Laurence J. Peter, author of The Peter Principle, once said, “Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them.” When it comes to wicked problems, like the failure of the American education system, one feels that such a position would be a great advance over the present ones. Wisdom begins with an admission of ignorance.