How Not to Approach Empirical Research

Commenting on the ongoing debate regarding Reinhart’s and Rogoff’s spreadsheet and weighting errors, Robert Wenzel writes,

Austrian economics reject empirical data as a method to prove economic theory, for Austrians it is all about logical deductions. Thus, there is not much for Austrians to do, relative to the current Reinhart-Rogoff destruction at the hands of a U Mass graduate student, other than to grab some popcorn and watch with bemusement from the sidelines.

At the risk of being curt, these are probably the worst sentences I’ve read on the whole ordeal. I was a bit disappointed to find Wenzel’s comment quoted at Circle Bastiat, the Mises Institute blog. I’ve already commented there, but there are a lot of people who take what Wenzel writes, and what the Mises Institute buttresses, very seriously. I’m going to reiterate my point. While, at face value, I disagree very little with the sentiment, I find this to be an extremely unimaginative take on the empirical debate over austerity. While it may be true that empirical data cannot disprove theory, this doesn’t mean that the data is irrelevant.

Why can’t data disprove theory? All data must be interpreted. Without theory, history would be largely unexplainable, because the historian (maybe just ‘economist,’ or ‘scientist,’ would be better) wouldn’t be able to draw causal inferences. This creates a dilemma: how do we build accurate theory that can explain empirical phenomena? Most economists, more-or-less, adopt some method of falsificationism, where a model is built and is then tested against the data. The problem is that empirical causal proofs are hard, or even impossible, to discern from complex phenomena (Mises [1998], p. 69; Caldwell [2004] suggests Hayek came to a similar conclusion). Ludwig von Mises’ solution is an a prioristic method, where all economic theory is deduced from a specific axiom: that all human behavior relevant to economic theory is purposeful action. This is close to Lionel Robbins’ definition of economics as a study of human means and ends (Robbins [1932]).

This has led some people, I think erroneously, to deny any role at all to empiricism. The problem is that, ultimately, theory can’t be divorced from reality. The purpose of theory is to interpret the real world, implying that if you avoid empirical research then you’re rendering theory useless. This is especially true when studying complex phenomena, because there are so many conditions present in the real world that we don’t know a priori which theory is relevant and which is not. It can be the case that a theory that our intuition tells us is useful is actually not.

For example, suppose we’re seeking to explain the circumstances of the Depression of 2112. Our intuition tells us that it was caused by malinvestment induced by intertemporal discoordination. In order to know this to be true with any certainty we have to test it against the data: not to disprove it, but to test its applicability. It may turn out that the theory our priors suggested was right can’t actually explain the data we’re looking at. Or, it may be that we find that intertemporal discoordination only explains 30 percent of the data we’re looking at. Maybe we also need to invoke compatible theories to explain other facets of the phenomena in question, such as a monetary glut or banking problemsAll of this requires the tools of history, including econometrics — tools that many aren’t comfortable with, because they’ve erroneously abandoned empiricism with the justification of a priorism.

Another, softer case against a priorism comes from the premise that we are fallible beings, prone to errors in our reasoning. There is a case in being skeptical of the ability of the data to disprove theory, especially as the phenomenon becomes more complex and the data more abundant. There is also reason to be skeptical of our theory if the data seemingly contradicts it. The claim here isn’t that data can disprove theory, rather it’s that there’s no harm in approaching theory critically if we feel that the data clearly contradicts it. This may be the case, for instance, if you find that relatively unregulated and undistorted economies suffer more during depressions under austerity measures than those which benefit from macroeconomic stabilization programs: fiscal and monetary stimulus. If we are error-prone, which means that our theory is likely to be, in part, wrong, then rejecting any path of enlightenment seems counterproductive. This is why I adopt a form of methodological pluralism, or what Caldwell [1989] calls critical pluralism.

A better response to the recent empirical research on austerity and growth is to tackle these interpretations head on (for example, see “Dealing with the Evidence“). I think that many of these studies suffer from omitted variable biases. This is partly the fault of austerity advocates, who have failed to add sufficient nuance to their arguments (although, this isn’t always true). Slow growth isn’t caused only by high government spending, but also by capital consumption, regulation (e.g. inflexible labor markets), and bad governance (e.g. an income distribution that wouldn’t occur in a free market). It could be that a government responds to a crisis by cutting 90 percent of spending, and there may still be slow growth. Does this disprove the theory behind austerity? Well, maybe. It depends on other factors. Again, we are studying complex phenomena. To fully understand these types of events we are going to have to apply a relatively large swath of theory — intertemporal discoordination, inflexible labor markets, government spending, et cetera, on their own aren’t going to explain very much. Austrians should engage the debate by offering multi-causal empirical applications of their causal explanations, otherwise we unnecessarily condemn ourselves to the sidelines.

Another point that isn’t made enough is that we shouldn’t be afraid of changing our minds. It’s always disconcerting to be proven wrong, but, rather than taking it personally, we should accept it for what it is: a process of improving our understanding of the world. Who cares if someone likes you less because you abandoned some key principle of some belief system? We aren’t in a beauty contest. We’re interested in seeking accurate explanations of real world phenomena: ideology and reputation are completely irrelevant.

6 thoughts on “How Not to Approach Empirical Research

  1. gcallah

    “Without theory, history would be largely unexplainable, because the historian wouldn’t be able to draw causal inferences.”

    No, that’s not what historians do. “Cause” in this sense is not part of history.

  2. gcallah

    In fact, here is a better formulation: as long as you are approaching the past with a theory (meaning some abstract model — of course historians have theories such as “Crassus died in Syria”), you are not doing history.

  3. JCatalan

    @google-6db013ce919c76ee1ba4f6266909c3e4:disqus : So, what do historians do? More specifically, what do economic historians do? Are you saying applied theory is the wrong way to go about the study of economic history? I don’t think I’ve ever read a history book where the author hasn’t tried to establish cause. I suppose we could call a historian a pure data collector, but most people want to know why some part of history is relevant and what it all means.

  4. Silvano IHC

    Good post. Sometimes I find unconfortable with Austrians and their blaming credit cycles and public debt separately. As if they were shy in defing private debts a problem. Blame the Fed, blame the ECB, blame FRB, blame banksters but don’t call borrowed money “private debts”. Something similar happens with foreign debt. Indeed, the ABCT in open economies and nation based central banks has much to gain from international macroeconomics.

    1. JCatalan

      I don’t understand. Are you saying we focus too much on public debt, or are you saying we should focus more on public debt? When talking about malinvestment, I’m not sure how much public debt matters. Sure, government can misallocate resource, but what the theory of intertemporal discoordination really says is that profit/loss will be distorted, so there is a high probability that any given private investment will suffer from some intertemporal discoordination. Debt is actually somewhat irrelevant. It comes into play once we look at the recovery, where there may be debt deflation.


Leave a Reply

Your email address will not be published. Required fields are marked *