After explaining the main concepts and promises of the Predictive Brain (PB) idea in Part 1 (you may want to read also the comments), it’s now time to explore its boundaries. In this post, I will not drill down into the proposed specific implementations of the idea, I will instead look at how it fits in the general picture, exploring its promise and limitations from the bird’s-eye view. The reason to do so stems from Prof. Friston’s answer to one of the questions at the LSE Event: Is the Predictive Brain theory falsifiable and if so, what kind of data would disprove it?
Friston answered in a way that I’m sure left lots of people unsatisfied, and yet he was absolutely spot-on, and I think that unpacking his answer is useful to fully understand how the PB idea may be useful. We may also learn something about scientific epistemology in the process. Friston’s answer was helpfully summarised by Micah Allen in his Q&A summary, and was something like:
I think that falsifiability is a little overrated. Not all scientific ideas need to be falsifiable. So, no: the general idea behind PB contains an element of tautology, pretty much as Natural Selection. How do you go about falsifying Natural Selection?
Unsurprisingly, the public stirred, and Prof. Holton picked up the prevailing mood by stating that he would find it difficult to endorse a theory that doesn’t try to be falsifiable. This is unfortunate, because both professors were fundamentally right, and I’m pretty sure most people in the public didn’t figure out why. In my case, I’ve been toying with related ideas for a long time, and thus the source of the misunderstanding was immediately apparent.
The short version is: neither PB nor Natural Selection are scientific theories; in fact they are both tautologies, are roughly equivalent and both can be very powerful intellectual keys to help generating fully fledged Theories. In the following paragraphs I’ll explain why they are tautologies, why they are roughly equivalent and hint at why they are nevertheless distinctively useful.
In Logic, Tautology is a proposition that is inevitably true. In the case of Natural Selection, it is frequently claimed that “survival of the fittest” is tautological, because “fittest” actually means, “whatever is better able to survive”, so once unpacked the principle of Natural Selection becomes: “whatever is better able to survive will survive”, who would have thought? I don’t have a problem with this seemingly dismissive interpretation, because Natural Selection is not the same as Evolutionary Theory, it is part of the theory, and at the heart of it, but nothing more than that. In particular, Evolutionary Theory can be seen as the exploration of the inevitable consequences of said Tautology, and at least in its first steps, remains tautological: “whatever is better able to survive will survive: if it is also able to self replicate, it will leave many copies of itself”. Evolutionary theory is not equivalent to these tautologies, it is made of hypotheses of how these inevitable principles manifest themselves in the world, and these hypotheses are in turn what can generate predictions. When you can make predictions, with a little luck you can also test them, and thus you get official recognition as a standard Scientific Hypothesis, which in turn will gradually become accepted as a Scientific Theory (if enough predictions appear to be valid). In the context of evolutionary theory, the foundational tautology has the function of providing a key that is useful to interpret observable reality, and thus generate concepts that help us describe what is happening: genes, replicators, heritability, and so forth. In short: there is nothing wrong in founding a scientific theory on a tautology; actually, you may even think that it is almost necessary.
Back to our predictive brains. I argue that the founding idea of the whole approach is not only a tautology, but that it is in fact essentially the same Natural Selection tautology. It applies to any structure that tends to make its own existence more likely to last (not necessarily reproduce) and can be described as follows: if a structure, by means of its own existence alone, makes it more likely that it will remain intact for more time, this structure will tend to last. I hope this is tautological enough for you.
A chain of Tautologies
In fact, Prof. Friston argues exactly this in his “Life as we know it” article (hat tip: Earl K. Miller via Twitter. Full ref is below), but goes a little further, and shows that such structures, in order to preserve their own integrity, need to appropriately respond to the variations of the surrounding environment. Crucially, in order to do so, they need to have within themselves the ability to be in a state that corresponds to the external situation. For example, using a bacterium as our “self-preserving” structure, a high external pH might require the bacterium to activate some membrane proton pumps. In order to remain intact, our single cell needs to be able to get in the “proton pumps are active” state.
Now we need to recall that a simple thermostat is a system that acts as a predictive layer, it computes the difference between the measured state and the expectation: if the measured temperature is higher than the expected, it outputs zero/off. If the temperature is lower, it outputs 1, or “on”, and switches the heater on. Substitute temperature with pH, the heater with the proton pumps, and you’ll see that there is perfect equivalence. Friston argues that a dynamic structure will be better able to remain intact whenever it possesses the ability to change its internal state in a way that balances the effects of a (potentially injurious) external situation, he adds that “in statistics and machine learning, this is known as approximate Bayesian inference and provides a normative theory for the Bayesian brain hypothesis”. He is simply showing us a chain of tautologies, but the distance between the starting point (what is good at self-preserving will self preserve well) and the end result is remarkable and illuminating. Skipping his mathematical proof, the end results are: to be good at self-preservation it is useful to engage in active Bayesian inference (1), and thus the internal mechanisms of living organisms (the known structures that are best at self preserving) can all be modelled as active Bayesian “predictive” engines (2). The important point for my discussion is not the first conclusion (1), but the second (2) and it is important only because it shows us a promising way to formulate and frame scientific hypotheses. We know how to model Bayesian inference, and we know that it must be possible to model any living organism as a Bayesian inference engine, so we have a first indication that this approach will work.
Running away with the ball
To proceed further, the next question is: OK, we know we can model living organisms in Bayesian terms, but will it be useful? [Here I start plugging in my own thoughts]
I think this question is crucial, and was repeatedly hinted in the comments to my previous post. After all, at this stage we know that (if our line of reasoning is correct) we should be able to use Bayesian machine-learning models to describe the inner workings of pretty much any biological system. To me, this looks immediately questionable: if we expect this approach to work each and every time (P=1, thus information content, in Shannon’s terms, is 0), why should we even bother? Well, I’d say that we should, for the same reasons why building and refining the full Theory of Evolution is useful: we expect to find very reliable knowledge, and may uncover many currently unknown regularities along the way.
Crucially, this is because our starting point is plain old natural selection, which is the metaphorical force that shapes the evolution of biological structures. We have strong theoretical reasons to expect that the dynamic structures generated by natural selection will tend to be more and more efficient Bayesian inference engines. Therefore, it is at least reasonable to expect that we’ll be able to gain great insights by trying to model living organisms in Bayesian terms. We expect this to be possible, and we have good reason to believe that the explanatory power of the approach is going to be high, specifically because we are mimicking the same process that shaped the organisms/brains that we are trying to model.
The most important take home message is: yes, the predictive brain idea is very general. It can be applied to the study of anything biological; actually, it can be applied to any dynamic system that has been shaped by natural selection. However, this happens to be exactly the reason why the approach looks promising: brains are extremely complex, and the predictive brain idea provides a unique/unifying principle that is closely related to what generated such complexity. It may therefore be able to illuminate the function of such overwhelming complexity. True, complexity is in the eye of the beholder, which is us, so anything that may help us to make sense of it should be regarded as good news.
Both Friston’s paper and my humble example with a single bacteria (which obviously does not contain a brain) suggest that the Predictive Brain idea first appeared on the wrong evolutionary end: direct reaction to external conditions surely has evolved before complex signal processing, extensive modelling of the environment, and the ability to dynamically adjust such models (AKA learning). This does lead to some additional interesting considerations that I hope to explore in a future post (and thus return in the neuroscientific fold).
At the Meta level, I have unpacked what I think that professor Friston had in mind when he said that “falsifiability is overrated”. This is important to me as I find that scientific epistemology isn’t as simple as it is usually described, and more importantly, I believe that it is very important to actively explore the limits and promises of the best knowledge-creation system that we have at our disposal. Pretending that scientific epistemology is all about fact versus non fact, and that it concerns only clearly defined boundaries is both misleading and counter-productive. [Note that I’ve explored these issues in some detail, a good starting point would be the Premises category]
Finally, I wish to underline how all of the above seems tightly related to the main intuition that I’ve tried to express in my post about Information. The mechanism that Friston describes in his paper applies to dynamic systems in general, and can be seen as a new way of describing natural selection. All dynamic systems are subject to selective forces, and all of them will tend to evolve in the direction indicated (active inference). Replicators will also emerge whenever complex and dynamic structures are present, simply because they are all subject to the same selective pressures. This could be seen as another way to describe the idea that Darwinism does not apply to biological systems alone, but in general to all complex and not inert structures (an idea that is very close to my proposed notion of information). This is something that Friston hints in the discussion section, phrased as “there are probably an uncountable number of Markov blankets in the universe” (apologies, I’ve decided to avoid the Maths side, so I won’t explain what Markov blankets are, if interested, try here).
Friston K. (2013). Life as we know it, Journal of The Royal Society Interface, 10 (86) 20130475-20130475. DOI: http://dx.doi.org/10.1098/rsif.2013.0475