Goodhart's Law can't be mitigated

Other View vs Mainstream View

There is a non-mainstream point of view that "Goodhart's Law can't be mitigated". Let's take a look into pros and cons of this statement:

1. Pros: Once a measure becomes a target, it ceases to be a good measure. People will inevitably game any system if incentives are tied too closely to a specific metric, leading to perverse outcomes and a breakdown of the system's original intent.
   – Cons (Mainstream View): While Goodhart's Law is a real challenge, there are ways to mitigate its effects, such as using a balanced scorecard approach with multiple metrics, frequently changing the metrics being used, and actively monitoring for gaming behaviors.

2. Pros: Human nature is to optimize for simplicity and personal gain. Any single metric, no matter how well-designed, will eventually be exploited as people find loopholes and shortcuts.
   – Cons (Mainstream View): Thoughtful system design can account for human tendencies and build in checks and balances to limit gaming. Organizational culture and values also play a key role in aligning behaviors with desired outcomes.

3. Pros: There are numerous real-world examples of Goodhart's Law causing perverse outcomes, from Soviet factories overproducing millions of tiny, useless nails to meet quotas, to modern issues with standardized testing in schools incentivizing "teaching to the test".
   – Cons (Mainstream View): There are also examples of metrics and incentive systems that have successfully driven positive behaviors at scale. The key is thoughtful design, frequent iteration, and adaptation over time as unintended consequences emerge.

Terms

• Goodhart's Law: An adage named after economist Charles Goodhart, which states that when a measure becomes a target, it ceases to be a good measure. In other words, when people optimize for a specific metric, it often leads to a breakdown of the system's original intent.

• Gaming the system: Exploiting the rules, policies, and parameters of a system in order to maximize personal gain, often in a way that is against the spirit or intent of the system's designers.

• Perverse incentives: A reward system or incentive structure that unintentionally encourages behaviors that are directly opposed to the desired outcome. Perverse incentives often arise as a result of Goodhart's Law.

Analogy

Goodhart's Law is like a teacher who tells students they will be graded solely on the number of pages they write for an essay. While the teacher's intent may have been to encourage thorough research and in-depth exploration of the topic, the actual result is that students pad their essays with fluff, large font sizes, and overly wide margins rather than focusing on the quality of their arguments and insights.

History

1. 1975: British economist Charles Goodhart pens a paper on monetary policy in which he describes the concept that would later be named after him as Goodhart's Law.

2. 1997: Anthropologist Marilyn Strathern restates the law in a more generalized form: "When a measure becomes a target, it ceases to be a good measure."

3. 2000s-2010s: Many modern examples of Goodhart's Law emerge, especially around standardized testing in education and metrics like click-through rates in online advertising.

4. 2010s-2020s: Increasing discussion of Goodhart's Law in the context of artificial intelligence systems optimizing for specific metrics in unintended and sometimes dangerous ways. Ongoing debate in the AI ethics community about if and how the law can be mitigated in machine learning applications.

How to use it

1. When designing an incentive system or bonus plan at work, consider using a balanced scorecard approach with multiple metrics that balance and constrain each other, such as revenue growth, customer satisfaction, and employee engagement. Review and adjust the mix of metrics each year to keep up with evolving priorities and prevent gaming.

2. If you're a teacher, avoid grading assignments based solely on metrics that are too easy to game, like page length or number of sources cited. Use rubrics that emphasize the quality of thought and arguments presented. Combine quantitative and qualitative feedback to give a more holistic evaluation.

3. In setting personal goals, don't just focus on a single number like pounds lost or dollars saved. Define success more holistically and find multiple ways to measure progress. Regularly reassess your metrics to make sure they're still serving you and adjust as needed. Celebrate qualitative achievements, not just quantitative ones.

Facts

• The cobra effect, where a bounty on snakes in colonial India led to widespread snake breeding and ultimately an increase in the cobra population, is a classic example of Goodhart's Law in action.

• Campbell's Law, coined by social psychologist Donald T. Campbell, is a similar idea applied specifically to the social sciences, stating that the more a metric is used for decision making, the more it will be subject to corruption and gaming.

• Some argue that the rising price of Bitcoin is an example of Goodhart's Law, as it has shifted from being valued for its utility as a currency to being seen primarily as an investment vehicle.

• According to Mario Bunge's Theorem, restated by historian Jerome R. Ravetz, value-free metrics do not exist because "the value and the measure always interact, with the measurement changing the value and the value affecting the measurement."

• Goodhart's Law is often compared to the observer effect in science, where the act of observation necessarily changes the phenomenon being observed.