Tuesday, January 6, 2015

Review: The Limits of Inference without Theory by Kenneth I. Wolpin

The Limits of Inference without Theory by Kenneth I. Wolpin.  Cambridge, MA and London: The MIT Press, 2013, Hardcover ISBN: 9780262019088, $35.00.  192 pages.
This recent book attempts to contribute to the debate about the proper role that economic theory should play in empirical research.  The author, Kenneth Wolpin, is the chair of the economics department at the University of Pennsylvania, and within his lengthy research career has focused on how individuals and families make demographic and labor market decisions.  He is best known for his work in the area of  “structural microeconomics” which attempts to better integrate theory and empirical research.  The Limits of Inference without Theory brings together several of Wolpin's presentations for the Tjalling C. Koopmans Memorial Lectures at Yale in 2010 as they show how both the use of theory can assist with the ability to draw inference from empirical data and how the lack of theory can severely limit what can be inferred.
Wolpin starts by introducing the two major approaches to data inference. Calling them nonstructural and structural methods, respectively, he conveys the limitations eschewing theory can place on data inference as well as the advantages that a more ‘structural’ approach provides.  He defines the nonstructural methods as processes in which theory is not necessary for inferential analysis; an idea often exemplified by the phrase “let the data speak for itself.”  In contrast, structural methods require that in order to be able to make estimates and inferences from the data, the researcher must define a specific theory of human behavior.
        The first section of the book deals with ex-ante policy evaluation and draws heavily from his own research in the areas of wage taxes and school attendance incentives.  Here Wolpin attempts to show that with the use of theory, both researchers and policy makers will be able to use ex-ante policy evaluation, even when there has been no policy variation.  In using a structural approach, he argues, not only can you expand the types of policies considered but also provide new avenues to validate the inferences you draw.
        The next section focuses on how theory can be used to improve existing inference in research.  Wolpin begins by examining research into the impact of unemployment benefits on the duration of unemployment.  He shows that a structural approach allows for better model specification and identification of relevant variables for analysis.  He then uses an example of natural experiments that facilitate inquiry into how school attainment can influence wages. In this particular example, Wolpin demonstrates how theory allows the researcher to clarify the assumptions needed for valid interpretation of the data.  Lastly, he looks at a field experiment that researched the impact of class size on student performance and shows how theory can provide insight into how generalizable the results are to other populations.
        The proper role of theory in inferential data analysis has become more important in modern society and especially in social science research.  Societies now face a mountain of data about almost all aspects of life and are limited not by a lack of data but instead by their abilities to go beyond simple tabulation and summary statistics to find a deeper meaning from the data.  Wolpin does an admirable job of providing examples demonstrating how the strong use of theory extends what can be inferred from data, thereby making a good argument for theory’s ability to improve policy evaluation and empirical research.
        Though this inquiry clearly has strengths, it still lacks a detailed discussion of the disadvantages of the structural approach and the possible advantages of the nonstructural approach.  Wolpin addresses a variety of issues present in nonstructural approaches such as the endogeneity assumptions often found in regression-based approaches and the seemingly random addition of proxy variables without clear justification or concern for proxy variable bias.  Yet he provides no discussion of the limits of inference when using theory, nor does he discuss when a nonstructural approach might be appropriate.     
        Although Wolpin gives a clear description of the research and models he uses as examples, many are still highly technical and complicated.  This leads to another important issue not addressed in this book, that of the implicit assumptions used in many of these models, such as the assumption of individuals being rational, utility-maximizing actors.  An example of such an issue can be found in chapter 3 section 2.3, where the model used to evaluate the impact of school attainment on wages implicitly assumes individuals have only one job.  This assumption is often violated by real life data.  Wolpin weakens his argument for the structuralist approach by not adequately addressing that assumptions in structural models can be non-obvious to people not intimately familiar with such models.  A strong advantage to nonstructural approaches is that the assumptions used to make inferences from the data are explicit and easily understandable for non-technically trained parties such as policy makers.

        This book provides a clear and concise presentation of how a structural approach to data inferences can be used to improve public policy and social science research.  The ideas presented allow the reader to better understand the limitations of the nonstructural approach and how these limitations impact issues such as policy evaluation, empirical research, and even how organizations use data they have collected internally.  Wolpin has provided a variety of reasons why a structural approach to data inference should be preferred along with many examples to ponder over.  This book would be a great addition to a graduate research methodology class and could provide a solid overview of the advantages of the structural approach. It would, however, likely needed to be paired with material more skeptical of the structural approach, such as Charles F. Manski’s Public Policy in an Uncertain World: Analysis and Decision.  Together, such two would provide enough material to engage in a deep and spirited debate of the limits of inference both with and without theory.