Is the Human Development Index Really Measuring Anything?

(originally published at thearkansasproject.com)

Alex Cartwright: 

A recent report attempts to compare and rank human well-being in each of the 50 states by means of a new and non-traditional metric – it’s called the Human Development Index. This study and its deficiencies might be of special concern to Arkansas, because our score was the second worst in the entire nation. The report explains, “Human development is about what people can do and be; it is the process of improving people’s well-being and expanding their freedoms and opportunities.” Statistics on life expectancy, graduation rates, and median income are converted into numbers and averaged together to create a state’s HDI. The authors apparently want to obtain a clearer image of citizen’s lives than traditional economic analysis reveals, but the defective methodology they use clouds the study’s results.  In what follows, I explain how the report’s bad methodology leads to garbage outcomes.

The authors emphasize that a state’s high GDP does not demonstrate that its citizens enjoy a high quality of life. But this is hardly a controversial assertion; few, if any, policy experts or economists would argue the point. GDP is a mere accounting measure – it’s designed to measure output. The Soviet Union regularly saw increases in GDP while people starved. GDP measures output, not well-being.

While we understand what GDP shows and doesn’t show, this illustrates the need to use another metric to measure well-being. However, it sure doesn’t show that the three yardsticks the authors chose — life expectancy, graduation rates, and median income – make for good measures.

By using life expectancy to measure quality of life, the HDI methodology assumes that longer lives are better lives.  One’s life expectancy is in part a function of genetic makeup and lifestyle habits, but this report seems to suggest that life expectancy can be altered based on the state you live in. Furthermore, the differences in life expectancies between the states do not vary much. Some states have higher average life expectancies than others, but there are no large differences in life expectancy in the data.

Readers should note that the authors rank states using averages and medians instead of by looking at statistically significant differences. Well-established and widely used standard statistical methods involve taking a series of averages and testing for ‘significance’ between variables. Even though the averages of a measure (say, average life expectancy) in different states might be different, any statistician can tell you that one cannot justifiably conclude that one average is actually greater than another without performing a significance test.

For example, average life expectancy in state A could be slightly higher than state B, but without doing a test of statistical significance, one cannot say with confidence that that life expectancy in state A is higher generally. All three factors that the HDI index utilizes (median income, life expectancy, and graduation rates) are mere averages that the authors use to ‘rank’ the states. It appears that none of the ‘rankings’ on any of the 3 measurements was computed by testing statistical significance – but only by comparing averages. The methodology in this report is fatally flawed because the authors make no use of testing for statistical significance. Without statistical significance, all of the data used in this report (and thus conclusions drawn from them) are mere observations that demonstrate little or nothing.

Median income is not a bad variable to consider when trying to measure well-being, but several other factors should also be considered if we are to consider median income. Many fairly obvious factors that vary between states and contribute to the quality of life were omitted. For example, the authors make no reference to citizen tax burdens. Median incomes, and the well-being that results from a higher median income, can only be accurately evaluated in light of the taxes citizens pay on income, capital gains, purchases, and so forth.

By using graduation rates to measure well-being, the HDI methodology simply assumes that the more educated live better lives. Aside from simply counting graduation rates from school, a more appropriate and accurate way to measure well-being would likely include youth unemployment rates, the average cost of post-secondary education, scholarships available, the amount of debt students with college degrees must carry, and employment opportunities for those with advanced degrees in one’s home state.

Since the HDI seeks to measure how different state environments are conducive to “improving people’s well-being and expanding their freedoms and opportunities” it is remarkable that the authors choose not to look at labor market issues, such as occupational licensing laws (that bar entrepreneurs and new workers from entering an industry). Nor did the authors consider how states protect citizens’ property rights from eminent domains and civil forfeiture laws – or fail to do so.

Using three (unrepresentative at best) yardsticks to measure human development, without any test of statistical significance, is a poor methodology; the results the methodology produces rest on a foundation of sand. For a thorough analysis of citizen freedom in the 50 states based on over 200 public policy items, instead of 3 non-statistically significant measures, someone who was interested in human flourishing could look at the Mercatus Center’s Freedom In The 50 States.