Economic impact studies are everywhere.

Whether it’s to support a new highway project, special tax breaks for solar energy, the building of a civic center or sports complex, or to promote subsidies for Hollywood film producers, you can find an economic impact study, often touting how great the project will be for the state or local economy.

The formula is simple, predictable, and effective. A special interest group that stands to benefit from the project funds an economic impact study that purports to provide hard numbers on the number of jobs, the increase in wages, and the additional output that will be generated by the project or subsidy, and it will do this on an industry-by-industry basis. It makes grandiose claims about how much overall economic growth will be enhanced for the state or region generally. Once the report is completed, the special interest group that paid for the study will tout these results in press releases that will be picked up by the largely uncritical media establishment, ensuring that the political decision makers and others who determine the fate of the project receive political cover.

These studies all have several things in common. First, they typically use proprietary, off-the-shelf models with acronym names like IMPLAN (Impact Analysis for Planning), CUM (Capacity Utilization Model), or REMI (Regional Economic Model, Inc.). Rights to use the models are purchased by professional consulting firms who are hired by the interest groups to do the studies. Furthermore, seldom do those who actually perform the studies have formal training in economics. Instead their expertise is in using one or more of the aforementioned proprietary models. And finally, all of these studies ignore basic principles of economics and, as a result, do not meaningfully measure what they claim to be measuring—the economic impact of the public policies and projects that they are assessing.

What would constitute real economic impact analysis?

To properly assess the impact of any economic activity, whether it’s building a convention center or sports stadium or installing a vast solar power plant, it must first be understood that the project will yield directly observable activities that one can reasonably expect to occur and there will be economic activities that don’t occur but otherwise would. By definition, these impacts, while real, are not directly observable.

The second category is what economists call opportunity costs. Opportunity costs are the result of the fact that all economic activity uses scarce resources that, under normal conditions, would be used for other purposes had the project under consideration not occurred. Opportunity costs, while real, are by their nature related to resource uses that are diverted from economic activities that would otherwise be pursued and are therefore unseen.

Any economic impact study that does not attempt to assess these opportunity costs cannot legitimately be called economic analysis. In fact, not attempting to take account of the latter is considered to be the biggest mistake that noneconomists make when thinking about economic issues. As the 19th-century economist Frederic Bastiat famously pointed out: “There is only one difference between a bad economist and a good one: the bad economist confines himself to the visible effect; the good economist considers both the effect that can be seen and those effects that must be foreseen.”1

Sound-Economics-graphicFor example, let’s imagine that a local government decides that it wants to spend $20 million on constructing a convention center to serve both the local community and possibly outside groups who might use the facility for meetings or conferences. In general, what would a true economic impact study have to take into consideration? Of course, the study would look first at “the seen,” that is, the effect of the $20 million expenditure on the industries that might be directly impacted, such as the construction industry, local suppliers of materials and equipment, labor demand in these industries, etc. These would be immediate effects as the construction begins and is carried out to completion. Of course, local restaurants and hotels might benefit and therefore increase their output as a result of this new business. If labor is paid more in these industries, then these workers will go out and spend some of that money, increasing the demand for other products. These are often called ripple or secondary effects of the $20 million expenditure, and they are what are typically called the “multiplier” effect of the initial spending.

The point is that, at least conceptually, these activities actually occur and can be seen. But what must be realized is that none of them is free. Every dollar that is spent as these “impacts” occur and every resource that is used, including labor, has an unseen opportunity cost. Starting with the original $20 million, the question is simple: What economic activities would have occurred if that money remained in the hands of the taxpayer? It would have been spent on various goods and services or saved in local banks and therefore would have had an economic impact that would also have had secondary effects associated with it. This would have to be subtracted from the visible effects.

During the process of building the convention center, as discussed, local resources will be used. For example, the demand for labor will increase, which means that for some, wages will be increased in the process of bidding labor away from other possible uses. Some local industries unrelated to the construction of the convention center will see their costs rise and will either contract their business or reduce investment in future expansion. This means that other workers — again those not related to the construction of the convention center — will see a reduction in the demand for their services over what it otherwise would be and would face the prospect of lower wages.

The point to be made here is that this would occur while the visible ripple or secondary effects that are being analyzed are occurring. What needs to be understood is that the measurements of visible effects are actually describing how the building of the convention center, or any similar project, is absorbing resources away from other economic activities. A true assessment of the economic impact of this or any other project would have to estimate the losses due to these unseen activities and subtract them from the values associated with the seen activities.

The reality: Studies don’t even attempt to get it right

“As policy makers and the public are often skeptical about economic impact claims, you need credible analysis to demonstrate the effects of your project. State and local stakeholders need to understand how a new development may add to income, output, and employment in their economy.”2 This statement is from a discussion of the REMI economic impact model and captures the essence of what is wrong with all such models. It also helps explain why they cannot accurately be labeled as real “economic” models. Note that the possibility of “new development” subtracting from any of these three variables is not part of the vision. There is a reason for this. The possibility that a new project could cause a net reduction in income, output, or employment is ruled out of the models by their methodology. The “unseen” of opportunity costs go unexamined and therefore unaccounted for.

All major economic impact models use what is called “input-output” (I-O) analysis. This analysis starts with quite complicated tables or matrices showing economic relationships between industries — that is, spending flows from one industry to another as the new spending, often governmental, works its way through the various sectors accounted for in the matrix. In describing how the process works, a U.S. Department of Commerce publication states that “For each industry, an I-O table shows the distribution of the inputs purchased and the outputs sold.”3 Using these tables, predictions are made on how spending that starts in one industry or originates from the government will have effects on other industries—both from initial rounds of spending and then through subsequent rounds generated by the initial round. Typically, policymakers and special interest groups focus on the impacts mentioned above. Those are the final impacts on incomes, output, and employments once the initial spending on the project in question has worked its way though the process of initial and secondary — often called direct and indirect — spending patterns specified in the input-output tables.

Essential to all of this is what is called the “multiplier effect.” The idea is that, as the initial spending works its way through all the interconnections among and between industries included in the I-O tables, its impact is multiplied. The multiplier is a number by which the initial spending is multiplied to generate the final “economic” impact. As described in the Commerce department document cited above, “To effectively use the multipliers for impact analysis, users must provide geographically and industrially detailed information on the initial changes in output, earnings, or employment that are associated with the project or program under study. The multipliers can then be used to estimate the total impact of the project or program on regional output, earnings, or employment.”4

It is through this multiplier process that a dollar spent on a project may, at least within the context of the model, end up “contributing” many more times to “output, earnings, and employment.” One study commissioned by the renewable energy lobby and based on analysis using the the IMPLAN model argues that North Carolina’s $72 million in government subsidies has generated $1.4 billion in new output. This “economic impact” analysis argues that government spending on renewable energy projects has a fantastically high multiplier of over 19.5 Simply put, they claim that spending one dollar yields $19 in economic “benefits” in return.

It should also be noted that none of these studies ever show what economists call “diminishing returns” to that spending, which means that a new dollar of government money spent on renewable energy would continue to generate these kinds of returns for the indefinite future. So, the spurious logic would be to shift all government spending that show multipliers of less than 19 to renewable energy. This is simply to point out that in addition to other basic problems, these studies ignore the first principle of production theory, taught in every first-year microeconomics class — the law of diminishing returns.

Multipliers aren’t manna from heaven

The $72 million didn’t actually cause that much new spending. It transferred investment that would have taken place in other industries into green energy.6

In a professional peer review of the renewable energy study previously mentioned, a team of economists from the Beacon Hill Institute (BHI) at Suffolk University identified the fundamental flaw, not only in this study but in all such studies. Every dollar that flows through the I-O table identified by the economic impact model, which generates the multiplier effect, is a dollar that would be flowing elsewhere, through unidentified, or “unseen” I-O tables. The assumption of the typical economic impact study is that the dollar that flows through the industries identified by model’s I-O tables, and the resources that they are commanding, would not otherwise be used.

But, of course, this is not the case. For example, government subsidies that are used to subsidize the installation of solar panels on hundreds or possibly thousands of acres of land may indeed generate a host of secondary spending flows which I-O tables will attempt to capture. But, for example, very often those solar panels are displacing thousands of acres of farmland. On this land, crops would have been grown — crops that end up on people’s dinner tables. A true economic impact study would have to not only account for the output lost in terms of the crops that are not grown, and the value to consumers of the food that does not get produced, but also the secondary impact of the spending on farm equipment, farm workers, etc. that would also have been generated, as well as the impact that the loss of that spending would have on wages and employment in those industries. This would have countervailing negative multiplier effects that would have to be weighed against, and subtracted from, the “seen” effects of installing the solar farms. Needless to say, the typical economic impact study does not factor any of this in.

Because of this flaw, the economic impact is always going to be positive. In other words, at the end of the day “income, output, and employment” will always be added to and never subtracted from the values for those variables prior to the initial spending or subsidy. As was argued in the peer review noted previously:

… they are robbing Peter [the farmer] to pay Paul [the solar company], and claiming the program increased total spending because now Paul spends more, but they ignore accounting for Peter. … The headline spending and jobs estimates the authors make are based on myopically accounting only for Paul. … The fundamental economic concept of “opportunity cost” postulates that there are alternative uses of scarce resources, and the picture is incomplete if we ignore the path not taken.

This is not only true of the IMPLAN study that is referenced in this peer review, but it is true of all such studies regardless of the model chosen. There is never an attempt to account for the unseen, i.e. opportunity costs, and, as such, they fail the basic test of what would constitute an economic impact study that comports with sound economic science. Indeed, they fail to demonstrate a basic understanding of what constitutes economic analysis.

Economic impact studies are not cost/benefit analysis or a measure of well-being

Even if they are done right, i.e., in a way that is consistent with sound economics, economic impact studies are not a measure of economic well-being or improved social welfare. At best, what they do is to provide a measure of spending flows as they move out of some industries and areas of the economy and into others. They tell us nothing about the relative efficiency of those spending flows. In fact, to the extent that those spending flows would not occur in the absence of particular government subsidies, as is the case for most solar and wind power projects, they are moving resources from more efficient resource use to less efficient uses. This is because, everything else equal, one would expect that purely private sector resource allocation based on entrepreneurial judgments about consumer demand would tend to be more efficient than judgments made by politicians risking other people’s money.

Using the convention center spending project discussed previously as an example, if the impact analysis was done in a way that is consistent with sound economics, it could tell us that employment may increase among local construction firms or for some hotels and restaurants due to investments in those industries generated by the wealth transfer from taxpayers providing the $20 million in subsidies. And if the study attempted to measure the unseen opportunity costs of the project, it would show hypothetical spending flows that would have occurred had the $20 million not been spent on the project but stayed in the hands of taxpayers or went to other government uses. But it can’t tell us anything about whether the uses of resources — land, labor and capital — that result from the subsidy are more efficient in terms of generating valuable output for consumers than the resource uses that would have occurred in the absence of the subsidized project.

5-factsEconomic Impact Studies for Taxpayers and PolicymakersThe economic impact study cannot tell us whether the resulting reallocation of resources will, on net, generate social benefits. Indeed, these studies make no attempt to distinguish between costs and benefits. As a result, what are, from an economics perspective, typically considered to be social costs, are frequently presented by those publicizing the studies as benefits.

Spending on resources of any kind—labor, land, energy, capital equipment, etc.—that is meant to produce a final consumption good or service is a cost and should be considered as such in any economic analysis that purports to be demonstrating how a project will benefit an economy—national, state, or local. Benefits are measured by the value to consumers of the final goods and service that are generated. For the most part, economic impact analysis measures costs, not benefits. What they do is measure payment flows to factors of production. The larger these payment flows and the more resources that are absorbed by the project, the greater will be the economic impact. But, as noted, from a societal perspective, these are costs.

Conclusion: So, what’s a policymaker to do?

In order to determine the actual efficiency of a project, the economic costs of a project, possibly invoking some information provided by economic impact studies, should be subtracted from the value of the final goods and service that these costs are producing. This would tell us whether there are any net revenues associated with the project in the first place. The second step would be to compare these values to the alternative value of the alternative uses of these inputs. That is, to the value of goods and service that these resources would generate in alternative uses, either governmental or through the free market. This is what is known as cost/benefit analysis.

Private businesses always do a private cost/benefit analysis to determine potential profitability before making individual investments. And for governmental projects, this is what policymakers should at least attempt to do. Of course, this is a complicated task when considering social costs and benefits and there are no off-the-shelf models, such as those typically used in economic impact studies, that allow you simply to input numbers and spit out results. Rather, it would require hiring highly specialized economists who truly understand the nature of this kind of analysis.

Ultimately, the fact that real cost/benefit analysis is more difficult and probably more expensive should not be used as an excuse for relying on economic impact studies that, in the final analysis, tell us nothing about social value of government spending projects. This is one case where incomplete information is worse than no information at all, particularly if it is manipulated to mislead the public.


Endnotes

1. Frederic Bastiat, 1995 (1850), “What is Seen and What is Not Seen.” Selected Essays on Political Economy, Foundation for Economic Education, essay can be found at https://admin.fee.org/files/doclib/bastiat0601.pdf

2. From the Regional Economic Models, Inc. (REMI) home page. http://www.remi.com/the-remi-model/topic-areas/economic-development

3. Regional Multipliers, U.S. Department of Commerce, Third Edition, March 1997, p. 1. Found at https://www.bea.gov/scb/pdf/regional/perinc/meth/rims2.pdf

4. Ibid.

5. Jon Sanders, “Nonsense with big Numbers: how to get Legislators to Support Your Industry,” John Locke Update/Research Newsletter found at https://www.johnlocke.org/update/nonsense-with-big-numbers-how-to-get-legislators-to-support-your-industry/.

6. David Tuerck, Ryan Murphy, and Paul Bachman, “Peer Review of ‘The Economic Utility Portfolio, and Rate Impact of clean energy Development in North Carolina’, The Beacon Hill Institute at Suffolk University, April 2013. Found at https://www.johnlocke.org/app/uploads/2016/06/RTIPeerReview20130401A.pdf.