The predictive brain (part two): is the idea too generic?

After explaining the main concepts and promises of the Predictive Brain (PB) idea in Part 1 (you may want to read also the comments), it’s now time to explore its boundaries. In this post, I will not drill down into the proposed specific implementations of the idea, I will instead look at how it fits in the general picture, exploring its promise and limitations from the bird’s-eye view. The reason to do so stems from Prof. Friston’s answer to one of the questions at the LSE Event: Is the Predictive Brain theory falsifiable and if so, what kind of data would disprove it?

Friston answered in a way that I’m sure left lots of people unsatisfied, and yet he was absolutely spot-on, and I think that unpacking his answer is useful to fully understand how the PB idea may be useful. We may also learn something about scientific epistemology in the process. Friston’s answer was helpfully summarised by Micah Allen in his Q&A summary, and was something like:

I think that falsifiability is a little overrated. Not all scientific ideas need to be falsifiable. So, no: the general idea behind PB contains an element of tautology, pretty much as Natural Selection. How do you go about falsifying Natural Selection?

Unsurprisingly, the public stirred, and Prof. Holton picked up the prevailing mood by stating that he would find it difficult to endorse a theory that doesn’t try to be falsifiable. This is unfortunate, because both professors were fundamentally right, and I’m pretty sure most people in the public didn’t figure out why. In my case, I’ve been toying with related ideas for a long time, and thus the source of the misunderstanding was immediately apparent.

The short version is: neither PB nor Natural Selection are scientific theories; in fact they are both tautologies, are roughly equivalent and both can be very powerful intellectual keys to help generating fully fledged Theories. In the following paragraphs I’ll explain why they are tautologies, why they are roughly equivalent and hint at why they are nevertheless distinctively useful.

Useful Tautologies.

In Logic, Tautology is a proposition that is inevitably true. In the case of Natural Selection, it is frequently claimed that “survival of the fittest” is tautological, because “fittest” actually means, “whatever is better able to survive”, so once unpacked the principle of Natural Selection becomes: “whatever is better able to survive will survive”, who would have thought? I don’t have a problem with this seemingly dismissive interpretation, because Natural Selection is not the same as Evolutionary Theory, it is part of the theory, and at the heart of it, but nothing more than that. In particular, Evolutionary Theory can be seen as the exploration of the inevitable consequences of said Tautology, and at least in its first steps, remains tautological: “whatever is better able to survive will survive: if it is also able to self replicate, it will leave many copies of itself”. Evolutionary theory is not equivalent to these tautologies, it is made of hypotheses of how these inevitable principles manifest themselves in the world, and these hypotheses are in turn what can generate predictions. When you can make predictions, with a little luck you can also test them, and thus you get official recognition as a standard Scientific Hypothesis, which in turn will gradually become accepted as a Scientific Theory (if enough predictions appear to be valid). In the context of evolutionary theory, the foundational tautology has the function of providing a key that is useful to interpret observable reality, and thus generate concepts that help us describe what is happening: genes, replicators, heritability, and so forth. In short: there is nothing wrong in founding a scientific theory on a tautology; actually, you may even think that it is almost necessary.

Back to our predictive brains. I argue that the founding idea of the whole approach is not only a tautology, but that it is in fact essentially the same Natural Selection tautology. It applies to any structure that tends to make its own existence more likely to last (not necessarily reproduce) and can be described as follows: if a structure,  by means of its own existence alone, makes it more likely that it will remain intact for more time, this structure will tend to last. I hope this is tautological enough for you.

A chain of Tautologies

In fact, Prof. Friston argues exactly this in his “Life as we know it” article (hat tip: Earl K. Miller via Twitter. Full ref is below), but goes a little further, and shows that such structures, in order to preserve their own integrity, need to appropriately respond to the variations of the surrounding environment. Crucially, in order to do so, they need to have within themselves the ability to be in a state that corresponds to the external situation. For example, using a bacterium as our “self-preserving” structure, a high external pH might require the bacterium to activate some membrane proton pumps. In order to remain intact, our single cell needs to be able to get in the “proton pumps are active” state.

Now we need to recall that a simple thermostat is a system that acts as a predictive layer, it computes the difference between the measured state and the expectation: if the measured temperature is higher than the expected, it outputs zero/off. If the temperature is lower, it outputs 1, or “on”, and switches the heater on. Substitute temperature with pH, the heater with the proton pumps, and you’ll see that there is perfect equivalence. Friston argues that a dynamic structure will be better able to remain intact whenever it possesses the ability to change its internal state in a way that balances the effects of a (potentially injurious) external situation, he adds that “in statistics and machine learning, this is known as approximate Bayesian inference and provides a normative theory for the Bayesian brain hypothesis”. He is simply showing us a chain of tautologies, but the distance between the starting point (what is good at self-preserving will self preserve well) and the end result is remarkable and illuminating. Skipping his mathematical proof, the end results are: to be good at self-preservation it is useful to engage in active Bayesian inference (1), and thus the internal mechanisms of living organisms (the known structures that are best at self preserving) can all be modelled as active Bayesian “predictive” engines (2). The important point for my discussion is not the first conclusion (1), but the second (2) and it is important only because it shows us a promising way to formulate and frame scientific hypotheses. We know how to model Bayesian inference, and we know that it must be possible to model any living organism as a Bayesian inference engine, so we have a first indication that this approach will work.

Running away with the ball

To proceed further, the next question is: OK, we know we can model living organisms in Bayesian terms, but will it be useful? [Here I start plugging in my own thoughts]

I think this question is crucial, and was repeatedly hinted in the comments to my previous post. After all, at this stage we know that (if our line of reasoning is correct) we should be able to use Bayesian machine-learning models to describe the inner workings of pretty much any biological system. To me, this looks immediately questionable: if we expect this approach to work each and every time (P=1, thus information content, in Shannon’s terms, is 0), why should we even bother? Well, I’d say that we should, for the same reasons why building and refining the full Theory of Evolution is useful: we expect to find very reliable knowledge, and may uncover many currently unknown regularities along the way.

Crucially, this is because our starting point is plain old natural selection, which is the metaphorical force that shapes the evolution of biological structures. We have strong theoretical reasons to expect that the dynamic structures generated by natural selection will tend to be more and more efficient Bayesian inference engines. Therefore, it is at least reasonable to expect that we’ll be able to gain great insights by trying to model living organisms in Bayesian terms. We expect this to be possible, and we have good reason to believe that the explanatory power of the approach is going to be high, specifically because we are mimicking the same process that shaped the organisms/brains that we are trying to model.

Some Conclusions

The most important take home message is: yes, the predictive brain idea is very general. It can be applied to the study of anything biological; actually, it can be applied to any dynamic system that has been shaped by natural selection. However, this happens to be exactly the reason why the approach looks promising: brains are extremely complex, and the predictive brain idea provides a unique/unifying principle that is closely related to what generated such complexity. It may therefore be able to illuminate the function of such overwhelming complexity. True, complexity is in the eye of the beholder, which is us, so anything that may help us to make sense of it should be regarded as good news.

Both Friston’s paper and my humble example with a single bacteria (which obviously does not contain a brain) suggest that the Predictive Brain idea first appeared on the wrong evolutionary end: direct reaction to external conditions surely has evolved before complex signal processing, extensive modelling of the environment, and the ability to dynamically adjust such models (AKA learning). This does lead to some additional interesting considerations that I hope to explore in a future post (and thus return in the neuroscientific fold).

At the Meta level, I have unpacked what I think that professor Friston had in mind when he said that “falsifiability is overrated”. This is important to me as I find that scientific epistemology isn’t as simple as it is usually described, and more importantly, I believe that it is very important to actively explore the limits and promises of the best knowledge-creation system that we have at our disposal. Pretending that scientific epistemology is all about fact versus non fact, and that it concerns only clearly defined boundaries is both misleading and counter-productive. [Note that I’ve explored these issues in some detail, a good starting point would be the Premises category]

Finally, I wish to underline how all of the above seems tightly related to the main intuition that I’ve tried to express in my post about Information. The mechanism that Friston describes in his paper applies to dynamic systems in general, and can be seen as a new way of describing natural selection. All dynamic systems are subject to selective forces, and all of them will tend to evolve in the direction indicated (active inference). Replicators will also emerge whenever complex and dynamic structures are present, simply because they are all subject to the same selective pressures. This could be seen as another way to describe the idea that Darwinism does not apply to biological systems alone, but in general to all complex and not inert structures (an idea that is very close to my proposed notion of information). This is something that Friston hints in the discussion section, phrased as “there are probably an uncountable number of Markov blankets in the universe” (apologies, I’ve decided to avoid the Maths side, so I won’t explain what Markov blankets are, if interested, try here).


Friston K. (2013). Life as we know it, Journal of The Royal Society Interface, 10 (86) 20130475-20130475. DOI:

Tagged with: , , , , ,
Posted in Evolution, Neuroscience, Science
9 comments on “The predictive brain (part two): is the idea too generic?
  1. […] my own attempt to provide a short, non technical explanation (with added links at the bottom). The second part will concern a couple of ideas that I got from the interesting discussion at the LSE […]

  2. ihtio says:

    In this quick note I will only address the idea of linking evolution with predictive brain theory.

    It makes sense to reason that if an organism acquires [better] predictive capabilities, then it will be able to have more chances of spreading its genes. However when you write “All dynamic systems are subject to selective forces, and all of them will tend to evolve in the direction indicated (active inference)” you seem to be implying that all species evolve toward increased complexity (and better predictive functions). This is not the case. Sometimes species lose their eyes or even their brains when they don’t need them. Sometimes species become less complex. I don’t know if anyone has completed any studies comparing complexity of organisms (however one would measure it) in time. There certainly are new species that are more complex and have better predictive capabilities, but they are also new species that are simpler than their “parents” and don’t have such extensive predictive capabilities.

    You wrote “Natural Selection becomes: “whatever is better able to survive will survive””. I don’t think so.
    What is able to survive is more likely to propagate. What survives better than competiton is more likely to propagate. This is evolution.

  3. If you want something fun to play with; Bayes rule and replicator dynamics use the same exact math. You can use it to make fun things like this interpretation of the factored replicator equation.

    I also agree with you on that dismissing something because it is not falsifiable is silly. It is a horrible misunderstanding of Popper.

  4. Sergio Graziosi says:

    Ihtio and Artem,
    it seems I wasn’t at my best on Saturday, and missed a few opportunities.
    Ithio, of course you are right, life is complicated, and the fact that there is a drive towards ever more comprehensive internal modelling doesn’t mean that every route leads to the same result. So if an organism finds its ecological niche in a cave where there is no light, photo-receptors will “evolve away”. This doesn’t negate the fundamental equivalence, as the drive is towards modelling the environment that is relevant for the structure (organism) at hand: no light means there is nothing to keep track of in terms of light-mediated signals. I would suggest you read Friston’s paper, specifically the conclusion: he highlights the fact that one can find nested Markov blankets within an organism, and in the same way you can find others that span multiple organisms. So you can extend the model to a community of bacteria, and realise that the whole is able to track and adapt to changes that a single bacterium won’t be able to handle. For starters, they are quite promiscuous when it comes to swapping DNA, and of course their DNA replication is not precise, when compared to Eukaryotes.
    One of the things that I like about this approach is how it makes it clear that our ways of deciding how to partition reality are rather arbitrary: for example, humans carry and rely on a spectacular amount of symbiotic (commensal and/or parasitic) micro-organisms, and one could very much decide to consider the whole individual as the (strictly) human organism plus all the other ones that live attached to it.
    There are lots and lots of other complications, such as the availability and discoverability of better modelling solutions, the energetic cost, and of course, the prediction horizon. There is also the awkwardness that comes from direct competition: if you can predict your competitors, you will gain the upper hand, so this ultimately requires either to keep becoming better predictors, at a fast rate, or to give up and become unpredictable yourself.
    As I’ve said, life is complicated. I should probably have provided some hints about all this in the OP, thanks for keeping me on my toes.

    Artem: once again, thanks for the links, I knew about them, and I know that at some planning stage I though “I should see how they fit in”, but then I forgot. I was also expecting you to jump at me pointing out that my treatment of the topic is indeed dangerously close to a simple evolutionary ladder view (as Ihtio points out); I should have known better :-/.
    Well, that’s why it’s good to have other people checking my reasoning.

    • ihtio says:

      I don’t see a thing that I wouldn’t endorse myself in your reply. I can comfortably see organisms as tight ecosystems (human organism + bacteria in the intestines, etc.), I have no problem with eyes being “evolved away”, I really like the notion of bacteria / ant colonies exhibiting behaviors and mind-like phenomena that are on the level of the entire colonies, but not on the level of individual agents (watch the talk about bacteria colonies:

      My analysis (I don’t want to say “critique”) of the Predictive Brain theory lead me to the conclusion that it would be ok to say that predictive capabilities are greatly advanced in very complex organisms and the reorganization of brain structures and functions is as it is (on the evolutionary perspective), because it was strongly linked with these predictive capacities. Then we could probably formulate questions and directions that would look like “if the brain is so strongly linked with prediction, then we could look for this and that…”.
      However this is not what is happening with Predictive Brain theory. PB claims that the fundamental mode of functioning is prediction. In other words: prediction is not emergent, but fundamental. It’s like saying that perception, attention, categorization is fundamental. Well… This is a grand claim. I don’t see any grand arguments for that.

  5. […] this idea on Sergio Graziosi’s blog: The predictive brain (part one): what is this about? and The predictive brain (part two): is the idea too generic?, and on Conscious Entities: Minimising Free Energy. In this post I present you my take on this very […]

  6. […] that Bayesian logic can work within a single subject or across generations (in the latter case, we call it evolution)[3]. Thus, we reach Artem and colleagues evolutionary model, where, because of the game rules, a […]

  7. […] and will, without exceptions, tend to generate new self-sustaining norms (because you know, natural selection operates on all structures). Thus, the waste that comes with too much experimentation is a clear example of the second type of […]

  8. […] interpret it, is necessarily shaped by what we have and haven’t experienced already (this is tautological, that’s why it matters). Thus, it is not sufficient to realise that we are blind to our own […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow me on Twitter

All original content published on this blog is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Creative Commons Licence
Please feel free to re-use and adapt. I would appreciate if you'll let me know about any reuse, you may do so via twitter or the comments section. Thanks!

%d bloggers like this: