My good friend Tommy Blanchard wrote a great article the other week, on the question “Are humans (and biological organisms) simply wetware machines?”
He gave a really good look at it with illustrations and everything, highly recommend reading it here:
I just wanted to add some more interesting - for me - context to this point. In fact, I kind of want to do a loosey-goosey, unscientific look at what I’ve learned about the utter chaos of how the brain is put together by evolution. I think it really illustrates just how incredible these systems are, but also how mind-bogglingly complex and seemingly illogical, and how it might compare to a brain if we designed it like we might a computer.
Prepare thyself.
So the boss comes in to the office one morning. He rushes over to your desk.
“Nicholas!” he says, breathlessly. “I’ve had a revelation!”
Uh oh. It’s never good when the boss has had a revelation.
“Since the brain is just too complicated to figure out, we should design a new one from the ground-up. It will be modular, maintainable, and debuggable!”
You convey your misgivings for such a project, but their mind is made up. You’re now project lead for Brain 2.0
What now?
Well you might start by analysing the existing behaviour of the current brain. In software engineering, when needing to do this, we often study the behaviour of the “legacy” version to identify each discrete “capability”, “feature” or “flow”, and map it to its own objective. Brain 2.0 should at the very least match the existing “legacy” behaviour.
To know if it’s working, we have to formulate test criteria which the legacy brain should pass consistently to guide our work. The end goal should be for Brain 2.0 to also pass all those tests consistently.
An example set of functions and features for a brain might consist of:
* Sensory input processing
* Language processing
* Autonomic systems
* Threat detection
* Motivation state management
* Bodily resource management
* Memory
* Motor function
…and so on.
Under each of these “feature buckets” we’d have to break it down into constituent parts based on what we understand of each set of behaviours or needs. For instance, “sensory input processing” is a very broad set of things. It includes vision, olfactory, tactile, proprioceptive and auditory inputs. We’d have to break each of these down into discrete modules.
I’m already looking at it like:
“Create a base schemata for modelling input data from a sensory organ, each sensory organ then extends that base with its own variations and needs.”
This way, we have a basis for shaping the data for all sensory organs, and we can even add entirely new ones arbitrarily. Each sensory organ could have its information passed through a single module which would route the information to the correct destination for processing.
So let’s say we get all this working, and now sensory inputs are coming in to the brain and informing it about the environment in which it finds itself. Furthermore, we have magically made motor control work, so our Brain 2.0 can respond to its environment via control of a physical body.
But we’re running into problems.
Something we need to consider is how to turn this discrete sensory information into a unified world model, but before we even get to that, we’re faced with a more serious challenge: compiling a world model directly from sensor data is slow. It takes time for signals to travel up nerve fibres, and yet more time to get filtered and processed, making it too slow to be viable when rapid near-reflex responses are sometimes required to avoid danger.
To solve for this, we can borrow an idea from the legacy brain: its predictive model.
The real human brain doesn’t feed sensory information directly into consciousness. Instead, it runs a continuous on-going prediction of the world, and uses sensory data only as a comparison to measure how accurate the prediction was, and to make corrections when necessary. The prediction is able to run ahead of the sensory input processing, making it ideal for rapid responses to potentially volatile environments since it doesn’t have to wait around for the data.
So that’s what we’ll do with Brain 2.0! We’ll include a module for that as well!
Notice what we’re doing here?
We’re designing architectures, circuits, modules, patterns, and structures. We’re grouping things into neat, discrete boxes. We’re wiring things up cleanly and logically.
The real human brain is nothing at all like this.
Despite the way we often talk about it as though it were a modular architecture, it’s not really. It is a chaotic mess, worse than some ancient decades-old banking code base which has never been refactored. Change one thing, and you get unexpected side-effects all over the place, in ways that make no sense at all.
Some people call evolution “The Blind Watchmaker”; quite frankly I think that’s an insult to blind people and watchmakers.
It’s more like the proverbial infinite monkeys on typewriters, except also they’re blind and mildly deranged, have a total of 3 fingers, and for some unknown reason become obsessed with pressing only 4 keys on the typewriter: A, C, G, and T (or sometimes U instead of T, but no one knows why)
Said monkeys never get even close to Shakespeare.
Now, the universe didn’t really know what to make of this when it saw the crap these monkeys were coming up with. So in some strange act of madness, it invented a mechanism to interpret these long streams of 4-letter nonsense, and associated each letter with a nucleotide molecule: Adenosine, Cytosine, Guanine, and Thymine (or Urasil, depending on the monkeys particular whim)
It is truly a remarkable achievement, then, that from such building blocks came all we see before us.
Indeed, the legacy brain is a billions-of-years-old project. Every time evolution has demanded a new feature, or even a bug fix, our infinite blind and semi-deranged monkeys almost never clean up the old mess. Instead, they simply patch over the top of it. Everything becomes a patch on top of another patch. They live and die by one golden principle: if it increases survival and/or reproduction, ship it immediately and patch it over and over and over again.
In digital computers, we would have electrical signals crossing wires and circuit pathways, transmitting digital information: a stream of 1s and 0s. By interpreting the specific sequences of 1s and 0s in a known way, we can derive its true meaning. Each processor that receives data knows how to interpret these signals in that way.
In the wetware brain, things began much more simply.
The earliest neural networks are thought to have been made out of highly multifunctional generalist cells, which combined sensory functions, inter-cellular communication and muscle contraction all into one, similar to the epithelial muscle cells still found in the modern cnidarian. Communication between these cells was necessary for rapid adaptive injury and stress responses. When damaged, a cell would release a flood of cheap, abundant signalling molecules into its surroundings to trigger coordinated, defensive/repair responses in neighbouring tissue.
It was a rather crude but effective method of volume-transmission, and a bit like a mesh-network of light switches; damaged cell releases a messenger chemical, chemical binds to receptor on a neighbouring cell, cell activates a prepared response, and in turn releases its own chemical messenger molecules to recruit other neighbouring cells, and so on.
This rapid cell recruitment and activation strategy proved highly successful, and the blind monkey that happened to stumble upon this particular configuration with its 4-letter alphabet was made to breed a whole new lineage of infinite blind monkeys.
Then the scope creep began.
These ancient multifunctional cells began specialising, separating the responsibilities of sensory receptor and muscle contraction into their own distinct cell types. Because these newly specialised sister cells became physically separated, and the traditional chemical messenger approach used before was too slow for communicating over larger cellular distances, a new type of cell which would bridge the gap between sensor and muscle contractor was needed: thus, the electrically-active neuron was born.
Complex functionality first evolved within localised systems. Instead of a central brain suddenly appearing, small groups of these proto-neurons would connect up together in various parts of the body to control specific local reflexes.
The Carribean Box Jellyfish, for example, has no central processing for sensory inputs, despite having 24 eyes. Instead, it has 4 separate “sensory ganglia” dangling off its body, each one processing input from 2 high-fidelity camera-like eyes, and 4 simpler ambient-light-sensors.
These ganglia don’t communicate with each other, and have no world model. They continuously fire a regular pacemaker signal which controls the contractions of surrounding musculature for swimming. When one ganglia recognises an obstacle in the visual data from its 6 connected eyes, it changes the rate of the pacemaker signal, causing it to steer away.
Over time, these networks grew to span even longer distances, and as organisms became more complex and evolved more sophisticated limbs and musculature, there was an evolutionary need for coordination and synchronisation.
The Starfish’s approach to this problem was very simple. Each arm kept its localised network of nerve cells with all their localised sensory processing, but added peer-to-peer networking via a broadcast ring.
When one of the arms detects food, a signal gets sent out to the other arms and degrades over distance. So the nearest neighbour arms will get a strong impulse, while the arms further away will get a weaker or no signal. The starfish will then crawl towards the food.
The successes kept coming, so evolution went and scaled this up to an absurd level. Enter: the Octopus.
Octopi have around 500 million neurons (a small fraction of a humans 86 billion), yet are highly intelligent. Their central brain consists of only about 10% of all its neurons, while the optic lobes have around 30%. The rest is distributed across its many infinitely-flexible arms. In fact, the central brain doesn’t control the limbs at all. Not directly. Instead, it might make strategic decisions and set objectives - “lets hunt”, “run away”, or something more specific like “grab that target” - while the local nervous system in each arm determines exactly how the kinematics for moving that arm will work to achieve the objective.
The brain doesn’t coordinate movements either; each limb talks directly with other limbs to coordinate movements, something called “self-organised embodiment”. The signals from sensors on each arm are also processed locally, deciding whether something is food, a rock, or dangerous, before sending that information to the central brain for a strategic decision.
This decentralised architecture is especially advantageous in the Octopus, as each arm has infinite degrees of freedom, and the processing required to compute all the kinematics for all the arms in one place would be overwhelming. Thanks to this decentralisation, in theory an octopus could have any number of additional arms added to it, without needing to change anything about the central brain.
Interestingly, in software engineering, decentralisation has always been held up as a pillar of “good architecture.” We like the idea of being able to swap out components at will, and allow the system to reorganise itself according to certain rules. Change to something at a lower level of abstraction should not require major changes to things at higher levels of abstraction, and vice versa.
In fact, it seems that invertebrates in general tend to follow this more decentralised architecture in their neural networks, while vertebrates opted for heavily centralised architectures where everything is hard-wired into the central brain, which micro-manages all downstream activity.
You might, like me, wonder why. Wouldn’t this be more brittle?
It is.
We know this because of conditions like phantom limb syndrome.
The human brain has a rigid map of the body hard-wired into itself, which cannot handle changes in configuration. This map is called the “cortical homunculus”. If a limb or other body part is removed or is disconnected from the brain or body, the neurons representing that body part in the homunculus can end up searching for other nearby neurons to connect to instead.
Due to this rewiring, some amputees can feel sensations in their phantom limb when touching an area of the body adjacent to the phantom on the cortical homunculus; for example, due to their proximity, some people with foot or leg amputations can feel sensations in their phantom when their genitals are touched.
(Neurologist V.S Ramachandran speculated this proximity might explain foot fetishes)
Octopi have no such hard-wired body map.
There are, however, some advantages to having such a rigidly centralised brain, which have served us well. The big headline one is imitation learning and mimicry.
Humans are really good at “monkey see, monkey do.” Since the central brain has that unified knowledge of where each part of our body is located in physical space, our visual system can be presented with a shape and map it onto the cortical homunculus. So when we see someone else move in a certain way, we can mimic them pretty much immediately, something an Octopus simply cannot do as effectively.
Over time, certain branches of the evolutionary tree really pushed hard on the brain centralisation track, and new features began to proliferate. If you look closely, you can see an interesting pattern forming: there seems to be inhibitory systems everywhere, apparently bolted on as an after-thought.
For example, signals between two neurons can only flow in a single direction for a given synapse; there is almost always a strict division of labour between who the sender is and who the receiver is. However, at some point, evolution seemed to discover a need for the receiving neuron to have some control over how much signal it is being sent, so although it can’t send signals back in the other direction, it can do something called “retrograde inhibition”, telling the sender neuron to essentially “shut the fuck up” by throwing endocannabinoid molecules at it.
In fact, this inhibitory bolting-on seems to happen at every level of abstraction.
Lets take a look at something called the “direct pathway”, a bunch of ancient structures located deep in the brain which are heavily involved in voluntary motor movements. Even when we’re at rest, the motor cortex is continuously trying to get us to move, firing off signals non-stop for every possible movement we could make.
The reason we’re not in constant motion at all times is thanks to a structure in the basal ganglia acting as a gate; by default, it fires a constant inhibitory signal to block motor movements. When we decide to move an arm to reach for a coffee cup, the cortex talks to the striatum, and the striatum sends a selective inhibitory signal to the gate, causing it to stop inhibiting those necessary motor signals.
It inhibits the inhibitor.
Then there’s the “indirect pathway”, which adds an extra inhibitor that inhibits the inhibitor of the inhibitor!
It’s inhibitors all the way down!!
It’s like if you were writing to a friend to tell them what colour the sky was, and so you write:
“The sky where I am is not not blue, what colour is it where you are?”
and your friend replies:
“The sky here is not not not green.”
In fact, our entire prefrontal cortex, one of evolutions most recent additions to the brain, is basically a giant slab of inhibitory neurons, helping to moderate and modulate our base impulses of fear, aggression, and drive for instant gratification. Without it, we would do the very first thing that popped into our heads moment to moment.
Now, I’m not one of those militant atheists determined to argue the point, but I will say that perhaps those folks posting youtube videos about how a flagellum is evidence of intelligent design should take a closer look at the brain.
The last point I’m going to make shows how truly incredible all of this is, because despite the sheer chaos and utter lack of any design whatsoever, the efficiency and power of biological neural networks are truly astounding.
One of the most challenging things when it comes to actually simulating the biological brain on a computer is how much raw computational power it seems to require.
Let’s compare, say, simulations for a fruit fly vs a mouse brain.
So the fruit fly is only around 150,000 neurons and 50 million synapses, and if you run it as a simple stripped-down leaky integrate-and-fire network - like a typical AI model - you could run it comfortably on a laptop. However, to run a full simulation of the brain’s biology at real time speed would require about 100 Teraflops of computing power from a cluster of modern GPUs all running in parallel.
You could technically do it with a home rig, if you had a motherboard with enough ports and data transfer bandwidth. It would require a ton of electricity, as well as making a lot of noise and heat, since the sheer communication overhead of tracking 50 million synapses forces us to throw massive amounts of hardware at the problem just to prevent the memory bottlenecks from slowing the simulation down.
A mouse brain is around 71 million neurons, and 100 billion synapses; the neuron count is nearly 500 times larger, with 2,000 times more synapses.
You couldn’t simply add more GPUs to your home cluster, because the bottleneck is in transferring the neuron data in these enormous matrices between all the separate compute devices, and then having the new data be re-integrated again at each step. The extra communication overhead for this would utterly overwhelm any cloud data centre.
So first, you would need to obtain an enormous empty warehouse, like an aircraft hangar. Then you’d need to fill it with the very latest, most specialised, most powerful unified hardware. If we take for example the NVIDIA GB200 NVL72, which is essentially a rack-mounted 72-core GPU supercomputer on a chip, we get about 2.88 petaflops per unit. It has fully integrated memory, so inter-chip communication won’t become bottlenecked as easily.
We’d likely need a full 200+ petaflops - with a P - to make this work.
That means we’d need at least 70 racks of these.
Each NVL72 draws 120kW of power. Running 70 of them at once would require 8.4 megawatts of electricity, continuously. But hey, at least you’d have one of the most powerful supercomputers that exists on this planet today.
Which is kind of crazy, considering the biological mouse brain uses only about 8 milliwatts of power. The computational simulation of it requires over 1 billion times more energy to achieve.
How in Darwin’s name did these infinite, blind, semi-deranged monkeys pull off such absurd levels of energy efficiency?
Each neuron is like a single processor core operating in parallel. The fruit fly brain has 150,000 of these. That’s likely more parallel processing power than every single electronic device within a 1km radius of your house combined. The flip side is that neurons are not capable of everything that a single electronic CPU can do (which is a lot), however they don’t need to be.
A lot of computational math is also obtained for free just by the nature of the organic chemistry happening at the cellular and synaptic layers.
Digital hardware burns massive amounts of power to do large scale mathematics using rigid, binary logic gates. Evolutionary biology does it using fluid dynamics.
The sprawling branches of a neuron act like microscopic, saltwater-filled cables with little holes that can open and close. When a synapse fires, it opens a hole at one end, letting a wave of voltage carried by ions rush in. The membrane is slightly leaky, so this wave naturally decays as it travels, giving the brain spatial filtering for free. If multiple waves fire in quick succession, they physically stack on top of each other in the fluid volume before leaking out, which gives the brain temporal integration.
They leverage the natural electrical resistance of the cell membrane and the passive diffusion of ions, with positive and negative voltage ripples crashing into each other at the junctions of these branches, physically adding and cancelling each other out in real time.
It gets complex calculus practically for free just by letting physics sort it out.
Lastly, no central integration is required for biological neurons. They get their inputs, produce their outputs, and it just gets sent to the next set of connections. Each neuron can add or reconfigure or remove synaptic connections they have with other neurons in response to this, so this serves as fully integrated weighted memory. A computer, however, needs to keep shuttling the outputs from processors over to memory storage and back again to use as inputs just to simulate synaptic input/output.
It’s a lot.
All this is to say that while we can technically simulate biological brains on computers, it’s the many differences between logical design and evolutionary free-for-all which make it challenging.
While we can use analogies to computers or modular architectures and the like to help us learn about the brain, we should do so with eyes wide open to the fact that it obscures just as much as it reveals.
That’s all for today, thank you so much for reading all the way to the end. I hope I’ve been able to inspire you in some way with new thoughts, ideas and possibilities. If so, please consider upgrading to a paid subscription if you haven’t already, or drop me a donation over at ko-fi:
I’ve been working hard on content for a complementary Youtube channel where I hope to deliver more digestible video versions of this stuff, as soon as the first set of episodes are ready, you’ll all be the first to know.
Many thanks to everyone who has a subscription or made a donation at any point, you literally keep this show on the road.
Until next time!
Get full access to A Chemical Mind at chemicalmind.substack.com/subscribe