THE Forbes annual Best States for Doing Business Index for 2014 is out, and once again, the political spin doctors are hard at work. I am not going to beat around the bush. I’m not saying ignore it. Read it . . . but don’t put a lot of stock into it.
The problem with the Forbes and other indexes is that people given them far too much weight without understanding the methodology behind them. As Peter Fisher has noted regarding tax and business climate indexes, the methodological differences can yield very different results. In a review of 5 such indexes (the Forbes index not included), Fisher found:
Thirty-four of the 50 states can brag that they are in the top 10 in terms of business climate or competitiveness; they just have to pck which of the five indexes they want to point to. The average state’s best ranking is 26 positions above its worst. Perhaps more importantly, business interests in just about any state can find at least one ranking to support an argument for cutting business taxes to make the state more competitive. In all but eight states, one can find at least one index that puts the state int he bottom half of all states.
Moreover, Fisher notes that;
The underlying problem with the five indexes, of course, is twofold: None of them actually do a very good job of measuring what it is they claim to measure …
Research by Yu Hsing and Michael Budden that looked at business indexes in general, including the Forbes index, concluded that (my emphasis):
The relatively low correlation coefficient between overall ranking and per capita gross state product or the growth rate of civilian employment suggests that it is not a good predictor for state workforce productivity or employment growth.
Jed Kolko and others did find a positive correlation between indexes that measured certain elements and economic growth. However, he and his co-authors concluded that (my emphasis):
The factors that demonstrate the strongest relationship with economic growth are, to a large extent, beyond the reach of policy. These factors include weather and geography, which reflect natural advantages or disadvantages that states and localities are heir to, as well as industry composition and density, which may be the outcome of cumulative long-term policy decisions and cannot be significantly altered within a short time period by policy decisions. Several business climate indexes emphasizing taxes and costs also demonstrate a significant relationship with economic growth, although these relationships are not as strong as the relationship between the non-policy factors (noted above) and growth. None of the business climate indexes emphasizing productivity shows a positive and significant association with employment, wage, or GSP growth.
David Neumark, in examining various business indexes not including Forbes, noted that those positive correlations between tax and costs, and economic growth are also positively correlated with higher income inequality:
There is evidence of a tradeoff between income equality and growth: those states that rank highly on the tax-and-cost indexes, and thus experience faster economic growth, also tend to experience faster growth in income inequality.
The Forbes index most heavily weights business costs, which includes taxes. However, it has not been successful in predicting economic growth, as Sean O’Leary at the West Virginia Center on Budget and Policy Priorities noted in 2011. O’Leary compared the overall rankings, as well as the 6 sub-categories of this rankings, from the 2006 Forbes index and compared them against subsequent job growth from 2006 – 2011. He found there was virtually no correlation between the rankings and job growth.
Beyond the predictive nature of the Forbes and similar indexes, another problem with the Forbes ranking is that its methodology is a little elusive. As Doug Hoffer noted back in 2007, the exact weights Forbes assigns to each category are not known:
Forbes gave a very brief summary of the items included in each ranked category but did not provide the methodology. That is, exactly what measures were used and how were they weighted? Without such transparency, there is no way to evaluate these rankings. So how can anyone comment on them (favorably or otherwise) without knowing the answers to these questions?
For example, from the methodology section (my emphasis):
Business costs incorporate Moody’s Analytics cost of doing business index which includes labor, energy and taxes. Moody’s weighs labor costs the most heavily in its index. We also included a state tax index from the Tax Foundation that launched in 2012 and looks at the tax burden on businesses in each state across different industries. Business costs are the most heavily weighted component in the Forbes Best States for Business.
Weights are mentioned, but exactly how much each component is weighted is unknown. Moreover, the highlighted portion of that paragraph points to one of the methodological problems with the index. Specifically, when the above is read in context with the following from the Economic Climate component:
The economic climate category measures job, income and gross state product growth as well as average unemployment during the past five years.
It appears that income is being treated both positively and negatively given that wages comprise roughly 70% of labor costs. What constitutes income is unknown, so we cannot be sure of the overlap between labor costs and income, but at first blush one could infer that income refers to wages/salaries.
Another issue regarding the index is the issue of reverse causality. As Kolko notes, lower welfare payments might not be a predictor of a better business climate, but rather the result of one–more people employed, fewer people on welfare.
Kolko also brushes up against, but never completely broaches, the issue of controlling for variables between states. For example, as noted during the recent gubernatorial race, the Democrats’ criticism of job growth during the LePage administration was flawed because it did not control for population. Similarly, the Forbes index does not control for variables between states when assessing job and economic growth (i.e., how many of the top 1,000 businesses are located in the state, of which Maine has 0). Maine is not California or New York or Texas. Why, from a policy perspective, compare them on a level playing field?
In short, policymakers should not put a lot of stock into the index. Democrats should not assault LePage for Maine’s low ranking as doing so gives credence to the index, and LePage should not use Forbes’ formula as a paradigm of economic policy. As outlined above, the index, and similar indexes/rankings, is not a great predictor of economic growth, and questions abound about the methodology. Policy shouldn’t be driven by this.