Richard Bookstaber is a Wall Street veteran and authority on managing financial risk. His previous book, A Demon of Our Own Design, offered an incisive postmortem of the stock market crash of 1987 and the demise of two prominent Wall Street investment firms, Salomon Brothers and Long-Term Capital Management. Bookstaber revisits the topic of risk and financial crises with a vengeance in his latest book, The End of Theory, which dissects the financial meltdown of 2007–2008 and the Flash Crash of 2010.
In A Demon of Our Own Design, Bookstaber reconstructs the chronology and market dynamics of his example crises. He highlights the precipitating role played by financial engineering “innovations” such as portfolio insurance that he helped develop and popularize. More broadly, Bookstaber identifies complexity and tight-coupling as critical—and continuing—threats to modern financial markets. He argues that market complexity is rooted in increasingly arcane derivative instruments and risk hedging strategies. Tight-coupling refers to interdependencies between derivatives and market liquidity, compounded by excessive leverage and computer trading programs operating at breakneck speed. These “demons,” while intended to improve markets, actually increase their brittleness in the face of rare events, errors, and unanticipated behaviors by market actors in turbulent situations.
In The End of Theory, Bookstaber takes a deeper dive into complexity science, applying it as a cudgel against the foundations of mainstream economics. Bookstaber’s thesis is that the “dismal science” cannot cope with real-world market complexity, rendering it incapable of modeling, much less accurately predicting financial crises. He proposes using agent-based models (ABM) as an alternative framework for helping leaders understand financial crises as they unfold and respond to them.
Bookstaber begins by reviewing classical and modern (neoclassical) economics, outlining the problems these theories were developed to solve—their creators’ aspirations and proposed solutions. He argues that William S. Jevons and subsequent developers of neoclassical economics were motivated by a misguided desire to emulate the mechanistic approach driving physics and other natural sciences. They built their edifice upon deductive inference from a set of mathematical-like axioms called rational choice theory. These core tenets idealize human beings that participate as perfectly rational (market) actors whose preferences regarding possible outcomes of their decision options are complete, consistent, and stable over time. They also possess perfect knowledge about their situation, one another, and the dynamics that drive their actions and markets at large. Third, they always select decision options that maximize their individual benefit (i.e., utility). This idealization renders the behavior of consumers and businesses predictable, enabling formal modeling of all manner of micro- and macro-economic phenomena—at least in normal (i.e., non-turbulent) conditions. Unfortunately, rational choice theory is also wildly unrealistic, akin to the physicist that derives a method for improving milk production on dairy farms starting from the premise of a spherically symmetric cow.
In the second part of his book, Bookstaber marshals “four horsemen”—core features of real-world complex systems —that spell apocalypse for these foundations of neoclassical economics:
● Emergent phenomena: Systems whose behaviors cannot be understood (or predicted) purely in terms of aggregating the behaviors of their components. Thus, the behavior of markets emerges not simply from decisions by individual market actors, but rather how those actors interact with one another and with their environment over time. By contrast, mainstream economics uses “top-down” methods that completely miss the dynamics of emergence in market crises (or even simpler situations such as traffic jams).
● Computational irreducibility: Most large-scale social interactions and critical business decisions can’t be modeled analytically; there are no tractable “closed-end” formulas or algorithmic shortcuts. Instead, these situations have to be lived in real-time. In fact, despite the considerable power of Newtonian physics, no general solutions exist for even a system of only three interacting bodies, only a handful of stable orbital configurations. The dynamics of real-world economic problems are similarly intractable. Thus, mainstream economics depends on its simplifying assumptions as mathematical necessities.
● Non-ergodicity: Ergodicity refers to processes that don’t vary with time, context, or learning; instead, they are stable and reliably uniform. But markets evolve and participants learn and discover and change their behavior. The manner in which we interact with one another and respond to our environment changes is dependent on experience. Economic cycles and market crises never repeat exactly. Mainstream economists treat the world as ergodic, relying on statistical distributions drawn from the past to model the present and future, but history matters for actual markets and market actors.
● Radical uncertainty: We can’t anticipate most contingent events, future experiences, or the evolution of social interactions and markets. Instead, we live in a world characterized by uncertainty, as per economist Frank Knight, where we can’t define dependable statistical distributions. Or as Donald Rumsfeld famously observed: we have to worry about what we don’t know that we don’t know, or unknown unknowns.
Bookstaber cites billionaire George Soros’s theory of reflexivity to reinforce his case. Soros posits that people are fallible in their situational perspectives because they are prone to biases and/or inconsistencies. Reflexivity means that these imperfect views can lead to actions that change the situation in unexpected ways, such as self-fulfilling prophecies. Reflexivity introduces learning and other feedback effects that lead to persistent indeterminacy and uncertainty. Fallibility and reflexivity preclude the possibility of actors possessing “perfect knowledge” about one another and their shared dynamic environment.
Bookstaber’s explanations are clear, if occasionally repetitive, and well-illustrated with interesting examples such as stories by Borges and Kundera. He argues convincingly that complexity and reflexivity conspire to generate surprises and unanticipated risks that invalidate the tenets of neoclassical economics, particularly in financial crises.
His argument is not entirely unique. To put it into context, critics of rational choice theory abound, falling into two broad camps. Psychologists Amos Tversky and Daniel Kahneman spearheaded one line of attack with their theory of cognitive biases. They argued (and demonstrated experimentally) that the intuitions and heuristic rules that people rely on reflexively to navigate everyday situations prove unreliable in economic and other complicated social settings, systematically distorting our judgments and choices. Thus, their critique targets the tenets of coherent preferences and utility-maximizing behavior. Economist Herbert Simon led the second camp, whose attack centers on the “axioms” of perfect knowledge and utility maximization. Extending ideas originated by sociologist Robert K. Merton, Simon argued that as finite and fallible beings, we cannot acquire exhaustive information about complex social environments, propose “complete” sets of decision options, anticipate all possible situational contingencies, project the outcomes of all options against all contingencies, and thus identify optimal choices. Bookstaber aligns with Simon, updating its attack with more recent ideas from complexity theory. Their objection does not center on constraints on cognitive capacity, per se. After all, businesses manage to solve all sorts of optimization problems like dynamic pricing and inventory management perfectly well because they are well-bounded problems that allow for the simplifying assumptions that ground the necessary mathematics. Rather, the core problem is that bounded rationality bares its teeth when it collides with business decisions that involve extended interactions with complex open-ended social environments, like market meltdowns.
In parts 3 and 4 of his book, Bookstaber reiterates the conundrum that complexity poses for neoclassical economics and proposes ABM as an alternative framework for modeling financial markets in turmoil. ABM consists of collections of entities called agents, which represent individual actors such as cars, consumers, or banks. Agents have behaviors, which, following Soros, are either cognitive (relating to understanding their environment), or manipulative (making and executing decisions). Manipulative behaviors include routine processes for interacting with other agents, and decision rules, which dictate when an agent initiates, modifies, or halts specific behaviors such as driving, buying, or trading. Agents operate in environments such as financial markets, which maintain conditions that provide the context or backdrop for agents, their behaviors and decisions (e.g., interest rates, asset liquidity).
ABM contrast with neoclassical economics because they offer a “bottom-up” approach to modeling markets: market behavior emerges from the collective decisions of member agents acting independently, whose decision rules take into account individual agendas, goals, and policies, and perceptions of their current status, market conditions, and other agents. Agents’ decision rules or heuristics are necessarily coarse, ignoring available information about their environment in the name of expediency in the real world. For example, agents representing cars in a basic traffic model rely on simple rules to follow the speed limit and move based on the space between them and the car directly in front of them on the road. Similarly, an agent representing a trading desk of an investment firm makes its market based on internal rules relating to the availability of funding and willingness to hold risky inventory.
The rubber meets the road when Bookstaber presents a notional agent-based model for financial markets. He introduces a set of agent types to represent market participants, including banks/prime brokers, hedge funds, and institutional investors. He also defines a multi-layer network that identifies the relationships between market agents through which three core elements flow: funding, assets, and collateral. This model provides a powerful tool for visualizing the continuing perils in our financial system, where banks in their various roles operate across all three layers of the network, providing the transmission conduits for passing problems in one layer across the entire system as a contagion. Leveraging this schematic framework, Bookstaber reconstructs the progression of market shocks through networks of agents for several types of crises: fire sales brought about by disruptions in the flow of assets and funding among market agents and the 2007-2008 financial crisis precipitated by the bust in the subprime mortgage market. As in his previous book, Bookstaber offers cogent and compelling accounts of what happens and why in these crashes, focusing on the dynamics of market liquidity and how they derail normal equilibrium between actors looking to buy and sell.
What remains in Part 5 is to explain how ABM can help market participants and regulators guide their decisions and redesign market structures to reduce the likelihood, or at least the severity, of financial crises. Bookstaber takes pains to emphasize that his model is schematic and conceptual rather than a concrete model capable of realistic simulations of specific market situations. He argues that ABM should be regarded as “generating plausible possible narratives for how a crisis might unfold.” After all, the four horsemen afflict ABM no less than they do neoclassical economics. However, ABM model crises more like baseball players track and move to intercept hit balls; athletes employ visual heuristics that track the ball and dictate in real-time how where to move and at what speed rather than trying to predict the ball’s endpoint when the ball is first struck using Newtonian calculations. Bookstaber suggests that ABMs offer provisional stories that help decision-makers understand the market dynamics at particular points in time so that they can formulate plans to protect themselves or attempt to dampen market turbulence. These models are “perpetually incomplete” and evolving, accommodating the actual limits of our knowledge.
Bookstaber’s goals for ABM are considerably more modest—and plausible—than the deductive neoclassical models that he attacks with verve. That said, analytic methodologies based on ABM prove to be a tough sell to business decision-makers and regulators (as my own experience as a consultant offering simulation-based solutions will attest). As Kahneman observed, business leaders tend to be unwilling to embrace the unnerving world view entailed by the four horsemen of complexity, much less accept decision support or guidance based on high tech market simulators. Bookstaber also proposes more systemic, market-level changes, namely breaking up banks to disrupt their influence across network layers, and encouraging asset owners such as large pension funds to use ABM to understand the opportunities for profit posed by crises so that they can inject critical liquidity into collapsing markets to stabilize them for everyone’s benefit. Again, while these are worthy ideas, formidable political and psychological barriers exist for their adoption.
Bookstaber’s compact book illuminates the shortcomings of neoclassical economics, the central concepts of complexity theory, and a way forward for thinking about market crises and decisions for responding to them in real time via agent-based simulation models. I recommend this cogent but approachable treatment of the central challenges to the financial markets that shape our wealth and well-being.