Sharon Bertsch McGrayne

how Bayes' rule cracked the Enigma code, hunted down Russian submarines, and emerged triumphant from two centuries of controversy

The Theory That Would Not Die

Gavin

(c) Red Rationality (1987) by Wang Guangyi

A slightly strained oral history of the least romanticised scientists: Bayesian statisticians. She makes up for the long-missing romanticism single-handed! The two-hundred year eclipse of the Bayesian method was much longer than that suffered by even the irrationally-maligned continental drift theory (50 years). And this neglect and opprobrium was suffered by a paradigm now accepted everywhere as powerful and useful in literally all kinds of research. 

She wins us over, particularly with her chapter on the secretive, truculent, omnicompetent genius John Tukey, who used Bayesian methods for elections 40 years before 
  Fivethirtyeight
, with comparable success. But her prose is borderline, with lots of clear but dim-bulb sentences. 

She has one infuriating mannerism: she constantly refers to Bayes' rule, Bayesian logical foundations after Bayes, Bayesian inference, and personalist Bayesian epistemology by the single terrible metonym "Bayes": 

  At its heart, Bayes runs counter to the deeply held conviction that science requires total objectivity and precision. Bayes is a measure of belief.

...even many nonstatisticians regarded Swinburne's lack of care and measurement as a black mark against Bayes itself.

...Bayes, on the other hand, seemed to produce results that corresponded more closely to sociologists' intuitions.

...Wagner took along the youngest and greenest of his three-man staff, Henry Richardson, who had earned a PhD in probability theory all of seven months earlier. He would be Bayes' point man at Palomares.

I suppose she did this to elide away jargon, but it both equivocates between very different entities, hides the complexity of the 'Bayesian' marquee, invites the idea that the frequentists were attacking a logically sound theorem, and produces a whole list of bizarre images, where we see the reclusive Reverend doing all these things: cracking Enigma and Tunny, finding H-Bombs lost at sea, calculating appropriate worker's comp amounts in the absence of reliable data, attributing The Federalist Papers to Hamilton, and blocking 99.9% of the spam email from reaching you (yes, you). 

It is also even more unfair to Laplace than usual. (It was he who developed Bayesianism into the powerful applied framework it is, into more than a single gambler's theorem. Ok, so "Laplace-Coxism" is admittedly even less admissible as a term to which the wise and honest may repair.) 

Grammatical twitching aside this was a fun introduction to an important thing.

She focusses on the soft, social side (and on applications vaguely summarised). There was a huge amount of factional bitching between these serious and cloistered men:

  Attending his first Bayesian conference in 1976, Jim Berger was shocked to see half the room yelling at the other half. Everyone seemed to be good friends, but their priors were split between the personally subjective, like Savage's, and the objective, like Jeffrey's - with no definitive experiment to decide the issue.
In a frustrated circle of blame, Persi Diaconis was shocked and angry when John Pratt used frequentist methods to analyze his wife's movie theater attendance data, because it was too much for the era's computers to handle. But one of the low moments of Diaconis' life occurred in a Berkeley coffee shop, where he was correcting proofs of an article of his and where Lindley blamed him for using frequency methods. "And you're our leading Bayesian", Lindley complained. Lindley, in turn, upset Mosteller by passing up a chance to conduct a big project using Bayes instead of frequency...
Asked how to encourage Bayesian theory, Lindley answered tartly, "Attend funerals".

This human focus means she gives no treatment of Cox's theorem, certainly the most remarkable result in formal epistemology (and probability theory?), and one of the main things which rationally warrants the partisanship and excitement she displays for Bayesian thought throughout. ("Justified fundamentalism", as one great commentator puts it!) It proves that any attempt to use numbers to model belief must be Bayesian or logically equivalent to it. With other results, it raises Bayesianism to the only viable quantitative theory of rationality and of right learning, a behemoth of which Aristotelian logic is a mere special case. No doubt I'm unusual in finding this the most exciting bit.

She's to be applauded for digging out novel examples of Bayesian analysis which were classified or which avoided using the word: early actuarial work, Tukey's US election model, the pre-Three-Mile-Island federal report of reactor safety, and the entire field of operational research. But she is so concerned with emphasising the (genuine) long oppression of the paradigm that she under-emphasises the good reasons to resist Bayesian methods before 1980: they were simply computationally intractable before MCMC. (Which makes the sheer effort put in to shortcuts and approximation methods by ingenious people quite tragic; they just aren't needed anymore, thirty years later.) To her credit, she does mention the parallel dogmatism of the 60s Bayesians and the presumptive overenthusiasm of some people in the last 10 years. 

(The great contemporary frequentist, Deborah Mayo, is able to subtitle her blog "Frequentists in Exile" without being absurd - even though Stats 101 and "Methods for [Social Science]" courses are still everywhere dominated by canned Fisherian tests and frames. She means exile from the philosophy of statistics and probability.)

Insofar as you want to understand the large trends of the present and coming age, you need to know its economics; insofar as you must understand the new economics, you must understand AI; insofar as you must understand AI, you must understand machine learning and decision theory; insofar as you must understand machine learning, you must understand both frequentism and Bayesianism. Insofar as you do not yet have the mathematics to understand Bayesianism, nor the excitement of the promise of a final, real synthesis of objective with subjective, you must read this gentle prose work. Once you are excited by its vague promises, you can find progressively more rigorous people and will have actually have reason to stomach the formalism.

3/5, 4*/5 for those just beginning the march.



(c) African Sonata (c.2000) by Vladimir Kush

Benjamin Snipes

Historical detail but not enough technical detail

Well written, but I suspect most like me would read this book to understand Bayes. This book would have benefitted by using its historical examples to illustrate different aspects of Bayes that make it different from Frequentism, and that would leave readers with some understanding of Bayes rather than just its various historical uses.

The Theory That Would Not Die how Bayes' rule cracked the Enigma code, hunted down Russian submarines, and emerged triumphant from two centuries of controversy

Reviews