Name: Information
ISBN: 9780674013872

Hans Christian Von Baeyer

In this primer for the information age, von Baeyer presents a clear description of what information is; how concepts of its measurement, meaning, and transmission evolved; and what its ever-expanding presence portends for the future.

The New Language of Science

Information

Gavin

Elementary philosophy of science focussing on Wheeler's "Really Big Questions" about the foundations of physics, in particular the 'digital physics'. (The Questions are ‘How come existence?’, ‘Why the quantum?’, ‘It from bit?’, ‘What makes meaning?’) 

Which wasn't what I was expecting from an out-of-print hardback tome by a serious physicist - particularly with that grand title - but still: nice. In fact it's hard to imagine anyone writing out these first steps any friendlier (ok, maybe Ben Orlin). Its technical work feels effortless; think Schroedinger's 
  What is Life?
 (which von Baeyer actually corrects, in passing).

I needed a book on the method / meta-field surrounding mathematical "information", because it has surrounded me: it threatens to encompass science. Just as "energy" eventually became a unifying pillar of all the natural sciences, information has infiltrated that same salient: 

Energy <-> Entropy <-> Information. 

And then into other sciences: vB hints that we should see bits as money as ML-performance as Fisher information as VNM utility, which would seize about half of theoretical science.

Info theory is a core part of a mathematico-philosophical witch's brew: computability, decision theory, computational complexity, Bayesian statistics, digital physics, quantum computing. Which together take big steps towards the naturalisation of logic - or, more, of maths - or more, of thought. (And is information larger than thought?) And/or the dematerialisation of physics?

von Baeyer builds it all up, so we get Clausius (1852) for thermodynamics, Boltzmann (1877) for entropy (inverse info) as a proper physical object, Shannon (1930) for classical info theory, Solomonoff (1960) for algorithmic complexity, Landauer for the shocking physics of computation (1961), Bekenstein (1971) and Hawking for black hole theory (crucial experiments for it-from-bit), Deutsch (1985) for how quantum computing could work. And Wheeler setting the whole new agenda. (I call it new because it hasn't made it into undergraduate philosophy, or physics, or statistics, or ordinary computer science, yet.)

The philosophy is very well done. I really liked his physicist's optimism about reflective equilibrium between science and folk physics:

  Information, too, has been defined operationally. Unfortunately, this technical, bottom-up definition is very restricted, and hitherto bears little resemblence to any of the common, top-down definitions. Eventually the two definitions should converge, but that hasn't happened yet. When it does, we will finally know what information is. 


It impresses me to find a pop science book that has aged this well, over 16 years. It's sad that that's impressive - obviously I'm not reading enough physics and maths. Von Baeyer maybe leans too hard on the physics-is-solid heuristic; he ends up being uncritical about some extremely late-breaking and radical work: the heterodox classical theory of Kahre (2002) and Zeilinger's (1999) grand quietist explanation for QM's weirdness (neither of which I've heard much about since). 
Zeilinger's principle... furnishes an answer to Wheeler's famous question "Why the quantum?" Why does nature seems granular, discontinuous, quantized into discrete chunks like sand..? The answer is that while we have no idea how the world is really arranged, and shouldn't even ask, we do know that knowledge of the world is information: and since information is naturally quantized into bits, the world also appears quantized. If it didn't, we wouldn't be able to understand it. It's both as simple and as profound as that.

A second prediction of QM that is explained... is the randomness of the outcomes of some measurements... if the single bit of information in an elementary system is revealed, then there is no more information left over to answer additional experimental questions... so other independent measurements must have random answers.

Each chapter takes an idea ("heat and entropy", "logarithms and message space", "qubits", "Morse and optimality") and builds it up with little informal proofs and thought experiments. This is nice, but because it has to do everything from scratch it's more of a grab-bag than an argument, and certainly not a "language of science" by the end. For instance, he stops short of one key philosophical outcome of all this technical talk, which is that there are two types of explanations, even though he covers all the ideas you'd need:

1) information compressions (e.g. General relativity explains the force on all of the infinite points in spacetime in one unbelievably terse tensor equation. We can often count the bits used by theories like this, and so solve theory selection!)

2) simplified algorithms, faster ways to reason about the world (e.g. much of computer science)

As you can tell from the number of question marks in this review, I found this stimulating but not conclusive. But it would be foolish to expect a pop book to answer the Really Big Questions, and von Baeyer's reminds us frequently that the current answers he presents are unfinished. So this is step one of a currently unbounded algorithm. Minus a half for not quite taking things as far as they can go.

---

Misc notes

* This would be a pretty good primer for Map and Territory or Quantum Computing Since Democritus, if those assume too much for you.

* Lots of literary illustrations of scientific ideas - Calvino, Wittgenstein, Borges - but it didn't feel forced to me. I suppose it might actually reduce the friendliness, for some readers.

* Is this true?:

  The most important role of noise, however, is as the preserver of our sanity. Without noise, the measurement or observation of a single quantity would requite an infinite memory and an infinite amount of time - it would overload all our circuits. Neither science nor consciousness could exist... noise is a thick blanket of snow which softens the contours into large, rounded mounds we can perceive and sort out without being overwhelmed.


We evolved lots of ways to ignore information. Why would this not happen again? A photosensitive patch arises in that noiseless world; since it is an analogue processor it simply trims off the infinite information by default when it runs out of molecules or reactions(?)

* He calls the Bayesian interpretation of probability "the rational approach" which suits me but let's imagine that's a mistranslation of his meaning "the mental approach", "the personalist approach".


* Gleick handles the social history and applications with unsurpassed skill, but I wanted the mind-bending crunchy side, natural information, digital physics, information as everything.

* "Information is flow of form"

* Solomonoff induction is intractable, another word for practically useless. Does this change the philosophical significance of the above brew? I don't think so - "Here is a way to work everything out; you can almost never use it" is a pretty plausible way for philosophy to end tbf. Does it change its scientific significance? Yes, absolutely - we have to seek approximations of the forbidden ideal or else it has none.

* What's fundamental, thermodynamics or information? Neither?

* Yet another way that info theory eats the life of the mind is the deeply practical "value of information", a way of deciding whether to bother with an experiment (q.v. the master, Gwern).

Information The New Language of Science

Reviews

Highlights