Shithousery/ˈʃɪthaʊsəri/ (n.)

Poor, but not unambiguously illegal, behaviour, especially in competitive sports. Disruptive or underhand tactics intended to gain an unfair advantage.

Aboyne/əˈbɔɪn/ (v.)

To beat an expert at a game of skill by playing so appallingly that none of his clever tactics or strategies are of any use to him.

—Douglas Adams & John Lloyd, The Meaning of Liff

Prelude: the final

Against all expectations, not least of which yours, you are in the grand final. Things broke lucky: you had a good draw, your opponent in the quarter-final tore a hamstring and in the semi-finalist had an off-day, you got the bounce of the ball and somehow prevailed, and here you are. They’re singing the anthems, but you know you’re up against it. Your opponent is the reigning champ: younger, taller, faster, fitter, and more skilful, in every facet of the game. Just hands-down better. In a straight shootout, you don’t stand a chance.

There’s one arrow in your quiver, but even that’s bent: you’re crafty. Streetwise. You’ve got game.

So what are you going to do? Just sit there and take a pounding?

Now, where would be the fun in that?

The 2026 FIFA World Cup Final

There was an extraordinary moment towards the end of the 2026 FIFA World Cup Final between Spain and Argentina. It was missed by the television cameras, VAR, and all of the little accelerometers embedded in the $1200 ball.

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

It was captured, accidentally, on an iPhone by a fan standing high in the stands behind and to the left of Argentina’s goal in the second half, as he filmed down the ground toward the Spanish end. Regulation time was over. We were three minutes into an alloted four minutes of stoppage time. It was nil-all and extra time was beckoning.

Having just lost an aerial challenge, Argentina’s Enzo Fernández sprints to recover the ball as Spanish defender Pau Cubarsi comes out to compete for it. Cubarsi gets there first, on the halfway. So late, and so poor, is Fernández’ challenge that Cubarsi is catapulted spectacularly into the air. Referee Slavko Vinčić immediately shows Fernández a yellow card — really, it should have been a straight red — but since Fernández has already been booked, he is sent off anyway.

But this is not the interesting incident. Twenty-five metres behind the tackle, the iPhone catches Argentine left back Nicolás Tagliafico watching, unengaged in the action, a good ten metres from any other player. Closest, in fact, is referee Vinčić, but he, like everyone else has his back to Tagliafico, mesmerised by Cubarsi’s dramatic rear somersault.

Within an instant — literally, before Cubarsi has even hit the ground, which he does in a tremendous heap — Tagliafico collapses, waving his legs in the air and clutching his knee, for all the world like he has been felled by a huntsman’s axe. No Spaniard is near enough by to even bother protesting innocence. No-one touched him. There is no-one within cooey of him, who even could have touched him. Tagliafico — a painted ship upon a painted ocean — writhes and gurgles, as if his Earthly struggles are at an end. It is perfectly absurd. At first, with all the other commotion on the pitch, no-one even notices.

Like everyone else, the iPhone zooms in as the referee strides towards Fernández — already vainly protesting that is all some ghastly misunderstanding — and brandishes the succession of cards. Indignantly, Fernández refuses to quit the field.

The prone Tagliafico is now out of shot, but enter Argentina’s captain Lionel Messi. He marches from the Spanish half toward the referee — the farthest and fastest he moved all day — motioning towards Tagliafico. Even now, only 13 seconds has elapsed since Fernández’s foul. Messi remonstrates with the referee, repeatedly gesturing towards Tagliafico, for another fourteen seconds.

To his credit, Vinčić does not buy it for an instant. We cannot see what happened to Tagliafico in the meantime and no other cameras appear to have captured it, but the BBC’s interactive 3D match experience does capture it. You can highlight Tagliafico, zoom in on him, and watch him roll around until, apparently, being told to stop by the linesman.

Though it failed, this is still shithousery of the highest order. To apprehend a scenario and so quickly react to it requires not just presence of mind but instinct. You cannot learn this.

Likewise, from Messi: to even notice Tagliafico, some fifty metres away the other side of an explosion of action is a testament to the maestro’s unparalleled vision and game awareness. To read and comprehend his signal as astutely as he does suggests spooky entanglement. It speaks to a deep, innate feel for the game, and all the means of prevailing, fair and foul.

For deep shithousery.

Some have this, some don’t. Argentina has it. English teams, by and large, don’t.

“Shithousery”?

Shithousery is gamesmanship: to play just within the rules—or in the unspoken voids where there are none, or even against the rules—but in the certain knowledge you will get away with it. To “shithouse” is to infringe morally, but not legally. You most often see it in sports but is just as common in politics and business.

To shithouse is to sledge. To dive. To run down the clock by taking the ball to the corner flag. But also to grandstand. To take credit for someone else’s work. To brown-nose the boss. To impugn. To ask, “have you stopped beating your wife?”

It can go wrong. Following media reports of child abuse linked to an unidentified Tory official, plainly thinking it an excellent piece of shithousery, Sally Bercow tweeted to her 50,000 followers:

Why is Lord McAlpine trending? *innocent face*

This impugned the conservative peer with the ostensibly plausible deniability of an open question that was only loaded for those plugged into British political minutiae — but when it proved unfounded, Lord McAlpine successfully sued.

In any case, shithousery is inevitable, whenever we fix rules in advance with arbitrary limits that do not carve nature at its joints. Rules that presume all actions are witnessed, and all players have honourable intent. But referees are unsighted. Touchlines are arbitrary. Less gifted players do what they can to win: they do not always play fair. If you’re facing an overwhelming opponent, sometimes you have no choice.

Preconfigured rules of the game are blind to this “underdog’s dilemma”. They presume each team will continue to press for goal until the final whistle, but sometimes keeping the ball from the opponent, however you can, is all you need to win.

In a sense, going a goal up and parking the bus is a form of shithousery.

Each year governing bodies — of sport and in life — realign the rules to shut down newly discovered routes to the shithouse, but the scope for imaginative gamesmanship passeth all understanding. It is, in any case a forlorn task: to shithouse is, in a sense to exploited misconceived rules. No rules have more misconceptions than new ones.

We may not, therefore, like shithousery, but we are stuck with it. This article makes the case that, instead of righteously disdaining it, when confronted with it, we should return the favour.

Shithousery as tactic in a finite game

By taking their shithousery to a different level, Argentina demonstrate how finite games — competitive, bounded, zero-sum games like football — are most comfortably won at the margin of legality. Argentina may not have won the World Cup this time, but they have before, and they got a good deal closer here than, by rights, they ever should have.

They did this not by being good, but bad. This is not a criticism. It is a fact of life, and “good” teams need strategies to cope with it. One strategy is to reply in kind.

As in business and life, there are “formal” and “informal” modes of play. Formal ones we can see, and track, and record: goals scored, tackles made, possession and territory held, passing accuracy and so on. But they can paint misleading pictures. “The data” absolutely fail to explain why Spain could not wrap up the game in the first 20 minutes.

Spain plays beautiful, elegant, rational, orderly football. Their players are settled, stable, calm, and efficient. They don’t panic. They press high, tackle cleanly, and make good shapes.

Argentina did little of any of this. They were awkward, disorderly, disruptive, destabilising and chaotic. They were pugilistic from the first whistle. They left in elbows and studs and risked bookings — and until the 93rd minute got away with it. Spain were good. Argentina were bad.

Here is what I mean by “good” and “bad”:

The same goes for England in their semi-final. By this measure of good and bad, England were at least as good as Argentina — but the South Americans were on a different level when it came to being wilfully bad. Argentina never really looked like losing, even when a goal behind, and even though they never led until deep stoppage time.

For, despite the obvious chasm in formal quality between the teams — you did not need to be Franz Beckenbauer to see it — still Argentina got within 14 minutes of a penalty shoot-out for the title. And penalty shootouts, we know, are bad. They are chaotic, capricious, random, cruel, awkward, and disruptive. Be in no doubt: Argentina backed themselves to win on penalties. Argentina’s supreme, functional, effective badness almost won them a fourth World Cup.

Teams who can be good and bad are most likely to succeed. You cannot indulge the light if you do not exploit the dark.

As we will see, this concept of effective bad has a more general application.

Good and bad in politics

Warning: this section touches on contemporary politics, a subject JC tries to steer well clear of. No opinion is advanced here about any political issue. This is about behaviour and shithousery — something which some are better than others at — but over which no party has a monopoly.

Arthur: Then who is your lord?

Old Woman: We don’t have a lord.

Arthur: What?

Dennis: I told you. We’re an anarcho-syndicalist commune. We take it in turns to act as a sort of executive officer for the week. But all the decisions of that officer have to be ratified at a special biweekly meeting, by a simple majority in the case of purely internal affairs, but by a two-thirds majority in the case of more–

Arthur: Be quiet! I order you to be quiet!

—Monty Python and the Holy Grail

The same goes for adversarial politics: if you are always “good”, in the sense described above, you are predictable. If you can be anticipated, you are vulnerable to a crafty opponent who knows what you will do, keeps you guessing or cannot be trusted to do the right thing. Who is given to shithousery. This is the “craziest guy in the room” strategy.

This, by the way, is a vital difference between unbounded, positive-sum, “infinite” games, like trade and diplomacy, where both sides stand to gain by continuing to “play”, and “finite” ones, like party politics or litigation, where there are fixed rules, fixed players, a loser and a winner. I have touched before on finite and infinite games, a devising of cultural anthropologist James P. Carse.

“Trust” — supposing other contestants will do what they have undertaken to — is a vital component of an infinite game. In finite games, trusting your opponents to behave how you want them to is a marked weakness.

This is true within organised political systems: the success of many populist and authoritarian politicians is due in large part to their being the “craziest guy in the room”. Their range of actions is not limited by the predictably elegant, patient, rational and nuanced responses of a traditional politician, always trying to strike some kind of balance: to the contrary, upsetting the applecart confers an advantage, as “good” opponents scramble to restore order, thereby forcing themselves into reactivity, forever responding to the opponent’s moves rather than dictating the terms of the contest.

In a political finite game, exclusively “good” players risk always on the back foot.

It is even more true between organised political systems, where the rules of engagement are less clear: take the European Union: an organisation conceived specifically as a system to prevent regional international conflict: to replace the bad incentives of war with the good incentives of cooperation. The EU functions as a bulwark, too, against threats from the East. But the EU is a good organisation. It is profoundly, constitutionally, rule-making and rule-following.

Would-be antagonists, like Mr. Putin — and to a lesser extent, Mr. Trump — know this. They know exactly what the European Union will do, and what it is almost certain never to do.

For its part the European Union does not have the same confidence about Mr. Putin or Mr. Trump. It has no idea what they will do next. Nor, perhaps, do they. The more unexpected their actions are, the less confidence their “good” counterparts on the world stage can have. The range of “good” responses is therefore, for the same reason, limited. The more limited it is, the more predictable it becomes.

So the EU’s natural reaction when confronted by an unpredictable and volatile force outside its boundaries is to try to placate it, to reason with it — to strive for consensus with it, and failing that, to appeal to the rules and referees of the game. These are “good” strategies. They don’t work well against shithousery.

For, as far as there are any rules of geopolitics, they are opaque, such referees as there are are invested with meagre powers, and smart players exploit the natural opacity and ambiguity of life at the fringes of acceptable “play”. But players do not have to broadcast their real ends, tactics or strategies.

The European Union, being by nature a ruleset, does broadcast its ends, tactics and strategies, real-time, as they develop: that is a fundamental “good” of an open democracy, so it must.

The range of things the European Union can do is conditioned by an ancient, deeply regretted history of violent regional conflict, and a recent history of enlightened convergence through benign constitutional bureaucracy. The range of things it will do is tighter even than that, and is conditioned by rigorous, transparent “good” process.

European constitutional bureaucracy is absolutely supreme to “executive fiat” — there is no meaningful “leader” of the EU (the EU “president” is a largely ceremonial role that rotates between member states in a fashion strikingly redolent of an anarcho-syndicalist commune, at least as envisaged by the Pythons). The American constitution plainly (and despite many protestations to the contrary over its history!) is not really like this, and the Russian constitution definitely is not.

The plausible preparedness to subvert “rules of engagement” is an enormous advantage in a finite political game. It may explain Mr. Trump’s surprising success in foreign policy. Mr. Putin has Western Europe “taped”, but really does not know what Mr. Trump will do, possibly because Mr. Trump does not either. Mr. Trump has, thereby, unshackled himself from the conventional bounds of U.S. Foreign policy. It is unthinkable for any European leader to do anything like that.

The EU can’t be the craziest guy in the room. It finds itself flanked, on either side, by crazier guys.

OODA loops

The OODA loop concept comes from aerial dogfighting but applies to any situation of combat where you are in a constant cognitive cycle between taking in what is happening (observing), synthesising a theory of what’s going on and what your opponent doing (orienting or synthesising), figuring out how to respond (deciding) and then doing it (acting) — ideally, before your opponent gets through her decision cycle, works out what you’re doing and changes up what she is planning to do to you. The OODA loop is the basic unit of combat psychology.

Combatants who can get through that decision cycle first have a distinct advantage because they can then dictate the forward terms of combat, and thereby play to their own strengths, continually changing up while their opponents are still reacting to their last move: they can “get inside their opponents’ OODA loop”.

Organisations that are by nature rule-based and process-focused are at a natural disadvantage against scrappy upstarts unless they can somehow create room inside their own constitutions for disobedient behaviour.

This has implications for data modernism.

Data modernism

I like to think(right now, please!)of a cybernetic forestfilled with pines and electronicswhere deer stroll peacefullypast computersas if they were flowerswith spinning blossoms.

—Richard Brautigan, from All Watched Over by Machines of Loving Grace (1967)

The current trend towards data modernism — an aspiration to “solve the world” with data and encode the future, so the human race need no longer struggle, but may lie in cybernetic meadows dreaming of electronic sheep — seems to swing ever further in the direction of a “good”, calm, solved, rational world — whilst, at the same time, the public stage fills with “bad” actors: would-be populists playing silly buggers and running rings around staid, “good” traditionalists. This is some kind of cognitive dissonance, but the message may at last be filtering through that the dark arts are still with us, and there is nothing the computers can do about it.

The upcoming Clacton-on-Sea by-election points the way.

For one way to seize the initiative from a grandstanding populist who has got inside your OODA loop is to refuse to follow your script, and thereby to get inside his.

When under pressure about electoral funding irregularities from various mainstream media sources, Reform UK leader Mr. Farage — a past-master invader of OODA loops, great and small — threw out a doozy: he did what no-one expected and resigned his constituency seat precipitating a by-election.

And then I thought: why should I be judged today or in the future by Sky News and their ilk? Why should they be the people that decide my fate when, as I repeat, I’ve done nothing wrong?

I’ve thought about it hard, and I’ve decided: today, I will resign as a Member of Parliament for Clacton-on-Sea, thereby forcing a by-election, which should happen, I hope, in short order.

Now, I’ve decided that the people of Clacton should be the judges of my actions. This will be a people-versus-the-establishment by-election. It’s a chance to stick two fingers up to the entire establishment. To frankly tell them where to go.

—Mr. Farage’s press statement announcing his resignation

Here Mr Farage explicitly shifts the terms, and subject, of the debate. He reframes it from an allegation that he, personally, has breached disclosure rules that apply to all politicians where, presumably, he assesses he has a low chance of success, to a question of unseen forces mobilising against this awkward representative of the people — this selfless teller of truths to power. The accused is re-rendered as victim.

But he is not alone: like Spartacus, Mr. Farage identifies with his electorate. He invests it with his own suffering. The way he tells it, it is the people of Clacton-on-Sea who are impugned by these allegations: he is merely their conduit. He makes his own embarrassment their struggle and their opportunity to stick it to the man.

Whatever you think of the substance, the form is clever. This is good oratory.

Shadowy establishment forces should not be judges in their own cause. Rather, the people should decide. Specifically this carefully chosen local constituency with whom Mr. Farage knows he is unusually popular.

Mr. Farage has tried to re-render a dry procedural inquiry — albeit one with significant consequences — as a popularity contest. The rules do not allow the subject of allegations unilaterally to change the terms of inquiry, of course, so the investigation will continue — but Mr. Farage’s opponents will be at least temporarily disoriented by the turn of events. And he has repointed the public debate and bought himself some time to formulate further tactics to deflect from that dry central issue.

This is what, in an Argentinian footballer, we might recognise as “shithousery”. With the referee’s view impeded, having left his studs in on a tackle, Mr. Farage has gone down in a heap and is rolling around clutching his ankle, while his opponents protest too much, belatedly accusing him of foul play no-one noticed at the time.

Not only has Mr. Farage chosen a most favourable audience to be judge in his cause — he has done so knowing how his main opponents are likely to react, and knowing that that will further play into his hands. His mainstream opponents are numerous — there are five of them — divided, disorganised, misaligned, and expected by constitutional convention to oppose each other as much as they oppose Mr. Farage. As long as they obey the rules, follow conventions and play to type, Mr. Farage knows he cannot lose this by-election.

Remember the distinction between “good” and “bad” here. Mr. Farage is the “bad” one. He is the wily old fox. Like an Argentine footballer, he has formidable conventional skills: he is as astute on the ball as any of his opponents. But unlike them, he is prepared to play at the fringes; dabble in the dark arts; push the limits of what he can get away with. Like Argentina, Mr. Farage presumes he has an edge — not through superior skill, but because he is the craziest guy in the room. His opponents won’t have the gumption to do the sorts of things he can do. It seems a safe bet: they never had the gumption before.

Count Binface and the last man (standing)

Novelty Candidate: We’re for the compulsory serving of asparagus at breakfast, free corsets for the under-5s and the abolition of slavery.

News Interviewer: I’m sure many moderate people would respect your stand on asparagus, but what about all this extremist nonsense about abolishing slavery?

Novelty Candidate: Oh, we just put that in for a joke! See you next year!

—Blackadder, Dish and Dishonesty

As these occasions tend to, this by-election has drawn a number of novelty candidates. Novelty election candidates are a venerable British tradition: some come and go; some almost become part of the, well, establishment. The Monster Raving Loony Party has haplessly—they might say “lucklessly”—contested elections for nearly half a century.

Another, active since 2017, is Oxford-educated satirist Jonathan Harvey, who plies his trade as “Count Binface” and, to the closest approximation, resembles Darth Vader, only with a plastic bin on his head. His policies include nationalising model railways and Adele — sensible policies for a happier Britain — and extremist nonsense like halting the sale of arms to repressive regimes.

In their way novelty candidates, too, are “Argentines”, only they tend not to be very good at football. They are a class of subversive political disruptors who thumb their noses at all electoral conventions, if not just for the sake of it, then in mute protest at the electoral system’s intrinsic absurdity and, perhaps, for the thrill of lurking behind a prime minister in an Elmo costume while he delivers a sober acceptance speech.

System incentives are such that, however meaningful their metaphysical point, novelty candidate “noise” is drowned out by the mainstream parties’ “signal”. They don’t register strongly enough on the political radar to attract any serious candidate’s attention, let alone get inside an OODA loop.

My enemy’s enemy is my friend

But this time, it may be different. For this time, Mr. Farage may have underestimated his opponents’ capacity to get inside his OODA loop. For all his popularity with one part of the electorate, another, larger, sector finds Mr. Farage positively objectionable.

To date, he has managed his unpopularity by dividing it into pieces comfortably smaller than his popularity and thereby conquering it, because each of those smaller pieces, once disconnected and put against each other, fight like ferrets in a sack.

But for all their quarrels elsewhere, a solitary issue does unite Labour, Conservatives, Liberal Democrats the Greens and even Reform breakaway party, Restore UK: that is their mutual opposition, personally, to Mr. Farage.

Usually that would not be nearly enough to unite them. But this is an odd scenario. But for the fact that it concerns Mr. Farage’s constituency, this is an inconsequential by-election. The Government’s sitting majority is such that there is less to be gained by any of the other parties from winning Mr. Farage’s seat — it wouldn’t change the voting dynamic in the House of Commons at all — than there is from Mr. Farage losing it.

When Mr. Farage won it in 2024, he captured 46% of the vote, representing a handsome majority over a bunch of scrapping ferrets. Taken together, those ferrets from other mainstream parties captured a small outright majority of the electoral votes cast, evenly distributed between them. If they could buck convention, and swallow some pride, the ferrets could deliver Mr. Farage a bloody nose, without needing to gain any more net votes.

But how to persuade scrapping ferrets to unite against a common enemy? However appealing in theory, it will be a cold day in hell before a diehard Labour voters cast a ballots for a Tory, and vice versa — that is what “diehard” means — so the prospect of all the mainstream parties miraculously agreeing on, and then persuading their constituents to vote for, a single “unity candidate” amongst them seems rather beyond the pale.

Unless the unity candidate represented no threat to any of them. The ideal fellow would have meaningless policies, or none at all, beyond uncontroversial stuff like nationalising Adele.

Enter Count Binface, who quickly spotted this opportunity and presented himself as that unity candidate. In a rare example of political sophistication, the mainstream parties all cottoned on to the opportunity and stood their candidates — if not their party machinery — down. There will be no Conservative, Labour, Liberal Democrat, Green or Restore candidates taking on Reform UK in the Clacton-on-Sea by-election 2026.

Mr. Farage will contest his seat with one “realistic” opponent: a man with a bin on his head.

Mr. Farage’s reaction — to complain — has been heavily ironic: in not playing by rules that Mr. Farage has subverted, the rest of the system is running scared.

“If they believe in the democratic process they put up candidates, but they won’t do it because they’re scared of the result, and they know that I’d beat them.”

In fact, in standing down, the mainstream parties are respecting Mr. Farage’s wish that “the people of Clacton should be the judges of my actions” without the distraction of policy, the economy, or the government’s track record since the last election. They are saying,

“Let this be entirely about you, and your constituency’s opinion about you. In having no policies and not representing a mainstream organisation, the single substantive candidate running against you is, effectively, a no-confidence vote. This way, the question becomes: Does the electorate have continuing confidence in Mr. Farage or not?”

Nevertheless, political pollsters and pundits say Mr. Farage is likely to win the by-election. But, by vanquishing only a man with a plastic bin on his head, Mr. Farage is deprived of his moral victory. His opponents have outdone him at his own game. By playing at the margins of convention, by acting contrary to type, by defying predictions — by eschewing their usual elegant, rational, clear, sideways-passing football and thereby disrupting Mr. Farage’s own destabilising game, the mainstream parties may learn a lesson here. There is a place for shithousery in the beautiful game.

What this means for England

JC’s posse never tires of reminding him he knows nothing about football and so should shut up about it, but perhaps there is a lesson for England’s footballers too: to embrace the dark arts, and get inside their opposition’s OODA loop.

England has an ill-starred history of being on the wrong end of shithousery: Diego Maradona’s Hand of God in 1986, Diego Simeone’s theatrical collapse on David Beckham in 1998; Cristiano Ronaldo winkishly engineering Wayne Rooney’s dismissal in 2006, Italy’s all round cynicism to prevail in the 2021 Euros.

At some point, England must learn its lesson: if you can’t beat ’em, join ’em.

Thanks for reading! This post is public so feel free to share it.

We have hit the crazy summer season where JC will be bouncing around a bit more randomly than normal, and newsletters might become somewhat intermittent. Rest assured, much is going on behind the scenes at the Jolly Contrarian. Including the rollout of a new consulting arm!

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Wither voice recognition?

I’m trying something new for this post, and that is using the Wispr Flow voice recognition application for dictating rather than writing.

This does the useful but unglamorous job of voice recognition (a technology that is well developed) and overlaying some AI pattern matching to fix up and properly punctuate dictation. JC’s nasal mumbling comes out looking more presentable.

For those who never learned to touch type or who regularly input significant data on a mobile device, voice recognition should be a real productivity opportunity.

It is odd it has not caught on more quickly to date.

Using AI to overcome some of the obvious limitations of dictation (that includes making errors, ums, ahs, and contradicting yourself: that is to say, applying a small amount of limited editorial control) is an ideal application for AI. It saves a good deal of incremental time. It is obviously testable and correctable at the time, so little or no risk of hallucination.

Wispr Flow works pretty well. You can train it to correct my annoying verbal and linguistic tics. As do all lawyers, I tend to overcomplicate my sentences. It would be great to train AI Assistant to take my turgid rambling output and rectify it by reference to specific rules I specify.

For example, “convert all nominalisations into direct verbal constructions”. “Replace all weak verbs with strong ones and avoid equivocal language like I think that — and this seems to be —”. These features help me eliminate awkward aspects of my natural speech.

Getting the rules just right is important, but you can overdo it. Sometimes the output is nothing like what I said and feels generic and “sloppy”.

It includes a handy feature: you can select your output text, and it will polish it according to your preferred rules. This will directly challenge Grammarly and other spell‑checking applications.

All this does prompt the question: why isn’t voice recognition more prevalent? Accurate voice recognition is hardly new. Maybe these tools will send it over the top.

But the ongoing deprecation of office workspaces has something to do with it too.

After all, forty years ago, voice recognition was the norm. We didn’t call it that: we called it “dictation”. We didn’t use tech—beyond cassette recorders. The intelligence was real: a human typist.

In hindsight, having one employee dictating and another typing, and the inevitable iterative process between them to finalise a document, looks highly labour-intensive.

Collapsing that down to a single employee composing and typing was less so—once the office manager [for younger readers, that’s the chief operating officer—Ed] got over the idea that “we don’t pay lawyers to type”.

But dictating without having to type is surely quickest of all. So where is it? Why isn’t Wispr Flow eating the world?

One factor may be the concurrent changes in the physical workplace that came with new technology: dedicated offices gave way to shared offices, then cubicles, then open‑plan spaces, and finally hot‑desking.

Over that time we became quieter and more timid. Noise is distracting, though on a busy trading floor it is a fairly pleasant, comforting hubbub. Speaking aloud can feel slightly embarrassing.

At the same time, dictation — indeed, all oral communication — has deprecated sharply. Desk phones have disappeared, replaced by Teams and Zoom. Phone calls — once ad-hoc, bilateral, analogue affairs — have given way to prearranged online meetings. Calling someone out of the blue, without arranging it in advance, is now the height of rudeness.

This, incidentally, is one advantage—probably the only advantage, beyond the saved ground rental—of our post-COVID generational shift towards remote working: suddenly we all have our private spaces back, and voice recognition is a realistic prospect.

Local and distant networks

The above touches on the crux of the digital divide: since digital networks arrived in our working lives — the world-wide web broke its academic borders in the early 90s — the nature of our networks has changed.

Networks in the pre-digital era

The pre-digital era was based upon small, rich, accretive local networks. These networks were multilateral: they involved people interacting, in person, in the workplace, or in physical meetings. The longer such a network lasted the better it got, as relationships formed, ways of working evolved and “institutional capital” accumulated. Hence these were rich, accumulative networks.

They were augmented by a thin layer of what I will call remote networks, which only got used were the local ones weren’t suitable. These these could be electronic — phone, fax, telex or telephone — or physical — snail mail or courier — but in any case were mainly bilateral in nature, and didn’t have the quite same “capital accumulating” nature.

Building relationships was central to business in the pre-digital era.

Local networks in the pre-digital era

Local physical networks had distinct advantages over remote ones. They were immediate and multilateral — people who were on hand could convene quickly —the communications were rich — not just “text” but “speech”: imbued with relationships, institutional knowledge, body language and context — multilateral — as many people as you needed could attend — and instant — but at the same time informal: dialogue was not systematically captured, logged, audited, or recorded, allowing candour, correction and the safe removal of faux pas, irrelevance and error before creating any formal minute or record of the meeting — and free — in the sense that there was no incremental internal cost to having already-engaged employees meet together. (Employee cost attribution even today is minimal; in 1990 it was non-existent) since the employees were already on the payroll.

Only where a physical meeting was not possible, would you opt for a remote communication. These were comparatively poor — usually bilateral, and at best (phone communication) conveying a fraction of the information of an in-person meeting; telex and fax even less so — or significantly delayed — telex and fax might take hours, mail correspondence days — and in any case incrementally expensive. Telephone and fax costs, postage and courier were meaningful costs per communication. Unlike labour costs, these were not baked in, and could be specifically accounted for.

Therefore before the world wide web every business — rightly — would prefer the local rich, free networks available to it — over remote, poor, expensive bilateral networks.

We organised ourselves accordingly. Firms organised into offices and branches, where people with a need to quickly interact could do so. Offices organised into compact districts where one could easily meet customers, counterparties, advisors and competitors. We built complex, rich, local, interlocking networks. We built relationships, reputations and trust. Much of the strength of the network was implicit — the shared history, knowledge, and common interests.

By contrast, it was hard to build reputation and trust over a telephone line, and impossible by fax.

So, these were the critical features of a local network: immediacy, speed, convencience, informality, memory, trust, and subtlety. These features are hard — impossible, really — to quantify.

But before 1990, not everything in business needed to be quantified. There was a costs to doing business, and that was that. The wealth of local networks is “illegible”. You can’t see it in the financial accounts. It may was well not be there.

That doesn’t mean it isn’t there, however. So:

Local, physical, rich, fast, cheap, invisible networks good.

Remote, electronic, impoverished, slow, expensive, visible networks: bad.

Networks in the post digital era

When the World Wide Web burst onto the public consciousness in around 1993 — I remember attending a seminar about it — everything changed, but nothing did.

Local networks didn’t change at all.

Everything changed about remote ones.

Digital, not analogue

Remote networks became now digital, not analogue. Whereas analogue communications occupied physical space and took time, and where they could be copied at all, quickly lost fidelity, digital communications occupied no space, could be copied, recorded or moved over any distance instantly and at zero marginal cost.

Whereas analogue communications had to be embedded in a “substrate” — paper, mainly — digital data needs no substrate. It travels unaccompanied.

Infinite audit

While digital solved the problem of incremental cost and created an “opportunity”: infinite audit.

As we will see, where there is “infinite audit” you no longer need “trust”: you have all the receipts.

But trust turns out to be quite important. Everything, always, being on the record has a chilling effect on candid communication. Things you might once have said are no longer said. This can be a boon and a bane.

There is one fork of the information revolution still cantering towards a fully trustless network: crypto.

High bandwidth

As the physical network has built out and chips have grown faster, bandwidth has exploded. In 1990 it took ten minutes over a 56k dial-up modem to download a day’s worth of email. In 2026 we can stream virtual reality to a mobile device in the scottish highlands.

Multilateral

In a distributed digital network, everyone is online — connected, and able to send and receive — all the time. Because communications are instant, costless and copied with no loss of fidelity, digital remote networks can be multilateral: we can quickly convene an all-hands video conference.

Before the internet, remote networks — phones, faxes and letters — were a poor relation.They were expensive, slow, of poor quality, and cussedly bilateral.

Now the remote networks are fantastic. They are fully audited, instantaneous, high-bandwidth, free and multilateral.

Not only do they outperform the analog remote networks in every possible regard, they seem also to outperform the rich local networks.

COVID as a live-fire experiment

For the longest time, Chief Operating Officers around the world [for older readers, that’s the office manager—Ed] wondered about the stubborn cost of maintaining an office in the CBD. They had systematically rolled back employee benefits — “bring your own device” is one thing: but even for McKinsey, “bring your own premises” seemed a step too far.

Then came COVID, and we were all bounced into trying it whether we liked it or not. That live fire experiment ended more than 4 years ago, but things have not yet gone back to normal.

There is an odd confluence: this suits many employees — especially us older ones, who quite like working out of the box room — and it rather suits the office managers, as it gives scope for downsizing premises — but you might wonder whether it rather misses the point about what is so good about rich local networks.

I might return to the question of rich local networks and poor distant networks. All complex systems finds their own configuration, and if we participants don’t like where it ends up, all we can do is change our own behavior and see if that corrects it.

If we prefer distant networks over local ones in our daily activities; if we accept the premise that cost, convenience, and auditability outweigh the value of slowly built relationships, trust, and wisdom, we should not to complain when the system treats us poorly.

Wispr Flow redux

Just wrapping up with Wispr Flow. Interestingly, the longer I wrote this newsletter, the less I relied on Wispr Flow, though I’m back to using it now for this closing section. I’m not sure whether this reflects a preference for typing or simply familiarity.

Wispr Flow has a neat “polishing” function it does not depend on voice recognition. You can apply it to any text you’re editing, and you can also create your own rules and AI prompts for it.

For example, I was exasperated at first that Wispr Flow did not handle “smart” punctuation (curly quotes and so on): I am a bit anal retentive about curly quotes, en-dashes and thing like that — but you can set a polishing prompt to convert everything to them.

It strikes me this is how we should be using AI: not to replace human effort in toto, but rather to take away the boring faff of interacting online every day — getting typing and punctuation right.

Thanks for reading! This post is public so feel free to share it.

So remember, when you’re feeling very small and insecure—How amazingly unlikely is your birth.And pray that there’s intelligent lifeSomewhere up in space, becauseThere’s bugger all down here on Earth.

— Galaxy Song, Monty Python’s Meaning of Life, 1983

This piece started out as a book review, of Professor Brian Klaas’ recent book Fluke: Chance, Chaos and Why Everything We Do Matters. It quickly would up as a rumination on a much, much bigger topic for which Fluke, is really only the occasion, not the subject, for it sets off one of the many bees in the JC bonnet.

The bee in question is the presently fashionable world-view among tech-bros and people who spend too much time online that everything can be computed. This “computationalist” outlook, and its best practical example, the omnipresent Turing machine, has wormed its way into contemporary intellectual life and is at risk of getting stuck there. This, I think, would be a tremendous pity.

The computationalist view runs more or less as follows:

* Turing machines are, by design, deterministic.

* You can model the human brain as a Turing machine.

* Therefore, the brain is a Turing machine.

* Therefore, the human brain is deterministic.

* Therefore, human agency is an illusion.

Professor Klaas himself is a good case study, because much of his book is wise and — excuse the pun — enlightening, but he still winds up mistaking the map for the territory and concluding that since we can be made to seem like machines, we are machines, and are therefore constrained as machines.

This seems like a typically dusty academic debate, but it has real-world implications for not just the hypothetical freedoms of metaphysical will, but the real-world freedoms of thought and action. Where you stand on this question makes a political and cultural difference to how you feel about where other people should be allowed to stand on it. We will turn to that at the end.

My readers keep me going. For more posts like this — and even better ones — consider supporting the JC. Only half a pint a week and you get to feel like a Florentine arch-duke in 1470, and I get to feel like Sandro Botticelli. Plus, you get access to Premium JC and all kinds of neat stuff.

A big “but”

I have been grappling for some time with what is so troubling about the modern techno synthesis — End of History millenarianism, truth, data modernism, determinism and the oddly illiberal fix these logical ideas put us in — but have struggled to put my finger on it.

In Fluke: Chance, Chaos and Why Everything We Do Matters, “disillusioned social scientist” Brian Klaas helps. But not as he means to.

This is a silly book. It makes a great show of being interesting, but then steers off a cliff in its final chapter, which presents as a big “but”. As G.R.R. Martin puts it, “Everything before the word ‘but’ is horseshit.”

You might think I am being unfair or uninformed in my opinion, and so I might — it would hardly be the first or last time — but even Professor Klaas would have to admit it is not my fault: I can hardly do otherwise. It is written in the stars.

So, you shouldn’t blame me, but if you do that’s not your fault, either: that too, is written in the stars. Indeed, Professor Klaas would oblige himself to concede, as he asks us to, that all of creation — all its galaxies and nebulae, all its tragedy and comedy, its infinite majesty and infinitesimal frippery exactly as it appears, right down to this silly article about that silly book — has been coming, unerringly and ineluctably, since the dawn of cosmic time itself.

Which means I have an excuse — nay, a compulsion — to talk about metaphysics.

God, mind and free will

“And if we are to prevent the lights going out on our lives once more, we should ask ourselves crucial questions.

Where are we? How did we get here? Why did we come? Where do we want to go? How do we want to get to where we want to go? How far do we have to go before we get to where we want to be? How would we know where we were when we got there?

Have we got a map?”

—Rowan Atkinson & Richard Curtis, Marcus Browning MP, 1980

Anyone with a tertiary education who didn’t take at least one philosophy course missed the point of the exercise. The sine qua non of the university experience is, surely, stage one metaphysics. Its three great questions — how did we get here, how does consciousness work, and do we have free will? — are not a million miles away from those posed by the hon. Marcus Browning.

“God” and “mind” really collapse into “free will”, and that boils down to how you feel about the causal principle: has every action in the cosmos a complete and identifiable set of reliable mechanical causes, or is the universe some how unpredictable, biddable and capricious?

Those who embrace the causal principle are broadly classed as “determinist”. Since, given enough information, one can precisely calculate the outcome of any interaction between any objects in the universe, the interaction of composite objects must be a function of their components.

Since classical mechanics assumes that, besides elementary particles, all objects are composites it follows that the behaviour of everything can be reduced to the behaviour its components. It is not quite turtles all the way down, though: it stops at these subatomic particles. Their behaviour, theoretically, can be traced back to the singularity whence we, and the whole of creation, came. There is, therefore, nothing new under the sun. All was inevitable, Q.E.D.

If this is right, certain popular beliefs about the cosmos cannot be true. There cannot be a non-material interventionist God: if there were, some events would not have a material cause. Humans cannot have free will: all human actions are the product of traceable electrochemical impulses in the brain.

While much of the West has acclimatised to the first idea, the second one is taking some getting used to.

If this all seems a bit desolate, there are a broad range of non-determinist philosophies that might suit you better. Beyond “non-determinist,” these philosophies have no catch-all label. If it would not lead to confusion you might call them “casual” as opposed to “causal”.

They include most types of mysticism and religion and any belief in magic or the occult. They also take in the widely traduced “continental” philosophies of relativism and postmodernism, but also some “rationalist” parts of the intellectual landscape: classical economics, much of behavioural and social science, systems theory, chaos theory and — though it might horrify Professor Dawkins to acknowledge it — evolutionary biology. These are all pragmatic, “emergent” philosophies that take the world as they find it. While they might find their explanations in granularities, they tend not to extrapolate actual predictions from them.

Science’s insistence on analytical rigour in inferring causal principles from observed regularities, seems an incontestably good and valuable thing: it has vouchsafed flight, sanitation, aqueducts, viticulture and so on. But if we say, as Professor Dawkins does:

“Show me a relativist at 30,000 feet and I’ll show you a hypocrite”.

We throw quite a lot of baby out with the mystical bathwater. Like the idea we can have free will.

So we are at a bit of an impasse. To enlightened liberals — often the same people as the modern rationalists — free will is also a generally good and valuable thing. All modern polities — not to mention all the ancient ones — depend on it for their basic coherence as social structures.

There is no room for any immaterial agency inside a human that can override the eternal logic of cause and effect. To act a propos nothing but “animal spirit” must, at some synaptic level, violate Newton’s laws about conservation of energy.

This is a great paradox: if we have free will, science is falsified. If we do not, the entire premise of civilisation is.

Thus, the “free will versus determinism” debate presents a dilemma that has animated stage one philosophy classes as long as anyone can remember. Every now and then, someone comes along and tries to sort it out for once and all — the late Daniel Dennett spent the last thirty years of his life trying to do so, for the determinists, but — in my mind at any rate — fell a long way short. Contending that freedom of thought and expression is an illusion is a hard sell.

Nor is it one that, in the final analysis, makes a great deal of difference. Either the causal principle is true, or it isn’t, and we could have free will, or none, and it would not make any difference to how we must interact with the universe. Plato had this much right: we have only shadows on the wall of the cave to go by. It doesn’t much matter what causes them.

That is why “God, mind and free will” is only a stage one course: it illustrates philosophy’s gift for paradox and at the same time, its deeply-rooted irrelevance. The debate is titillating but, at the same time, stale: causal principles seem to hold, and we seem to have moral agency, and humankind has managed the theoretical conflict ever since we first looked up at the stars.

The transparently silly answers the paradox throws up suggest it is a silly question. So, for the longest time the scientists and humanities have kept out of each other’s way. But developments on our ways of being occasionally reopen the debate. The alarming advances in “compute” are doing that now: there is a generation of technologists who are once again yearning to solve the universe.

Professor Klaas is here to help them.

The great divide

In modern academic discourse, there are two strands of thought: the icy rationalism of the STEM disciplines and the post-modernist mumbo jumbo of the humanities. This is, of course, an outrageous generalisation, but it also, kind of, isn’t.

Occasionally, shots ring out across the aisle: mendacious physicists fox humourless sociology journals into publishing PoMo-tinged gobbledegook, while philosophers and historians ruffle biologists’ feathers by pointing out that their discipline cannot be as rational as they would have us believe.

For all that, there is buyer’s remorse on either side: a clique of humanities academics, uncomfortable with Post-modernism’s reductio ad absurdum that nothing means anything, have organised themselves into a rationalist stance, huddled around the theory of evolution. Prominent among them are linguist Steven Pinker and late philosopher of mind Daniel Dennett but their high priest — if you’ll forgive the expression — is biologist Richard Dawkins. This group is outspoken in its criticism of religion, relativism and other ostensibly magical accounts of human ingenuity.

You may wonder, as I do, whether there isn’t something ironic about this, given the contingency of “Darwin’s dangerous idea”. As Professor Dawkins explains it, the evolutionary process takes us away from an imperfect now; it does not converge upon a perfect later.

It was Professor Dennett who did the most to integrate evolutionary concepts into the humanities, sheeting them back to information theory by means of that algorithm — the “universal acid” that explains everything. Evolution is profoundly algorithmic — that is Darwin’s Dangerous Idea, in a nutshell — and so, of course, are computers: Turing machines. So if this is how evolution works, and this is how computers work, why not brains?

At the other end of the STEM spectrum, information theorists and cyberneticists strive to apply their inorganic learnings to human systems. They work backwards towards the meatware, noting the calculable “Shannon entropy” of the symbol strings by which humans communicate. Cybernetics is a positivist sort of systems theory that strives to solve things from the middle — to solve the management conundrum — but there are better forms of systems theory that start at the edges of the network, with agents and their immediate interests, and let the middle of the system look after itself. See the end-to-end principle.

Now, computation is a bounded, mathematical, calculable, zero-sum endeavour. It is fully causal. It is intensely deterministic. Therein lies the rugged beauty of the Turing machine: fast, cheap and utterly predictable. It doesn’t make mistakes. It doesn’t pick up the wrong end of the stick. It can’t. It is obliged to follow a determined causal chain of reactions. All steps can be reverse-engineered. This makes it quite a neat nomological machine.

This compels a worldview where everything is directly, bi-directionally causal: assuming a given operation, a machine state N’ can be computed by reference to machine state N that preceded it, and vice versa. One can journey up and down this chain of reactions indefinitely without losing fidelity or risking a different outcome.

It would be great — some think — if we could quantise and optimise human intelligence the way we optimise computing. But while we understand pretty well how Turing machines work we don’t, very well, understand how brains do. In particular, we don’t understand how consciousness works. We don’t know what it is. We don’t even know if it is.

Professor Dennett was not the first to see in Turing machines a framework for understanding human consciousness, but he pushed the idea harder and more successfully that anyone before him.

But this is not so much to anthropomorphise machines as to robomorphise humans. For a Turing machine is a weak metaphor for human intelligence. Turing machines are not like humans: as George Gilder drily noted, that is why we build them. Humans are inconstant, slow, easily bored and they take up space. They are hopeless in the environments in which machines excel. But in the chaotic, unbounded, inchoate complex environments in which humans excel, machines — even generative AI — are hopeless. It is precisely human flexibility, imagination — our freedom from causal constraints that gives us this edge. Humans can imagine. Machines cannot.

There is a flip side: as a result, humans are inconstant. They can “misinterpret”. They can pick up the wrong end of the stick.

Fluke

This is the category error Professor Klaas makes in the last part of his book, having made a great show of avoiding it in the first three quarters. There is much to agree with in his groundwork — he talks perceptively about the pervasive contingency of complex systems, but it turns out that he has deliberately taken up what he must regard as the wrong end of the stick to make a point. Because all computer operations are provably causal — garbage in, garbage out — and because computers superficially resemble brains, the temptation is to infer that humans are unavoidably causal too. This Oolon Colluphid-style “puff of logic” prioritises causality. It rules out God, the mischievous supernatural and all those big-shirted French post-structuralists — but jettisons the principle of free will, too.

Hence, if true, the inevitability, from the Big Bang, of this review. I had no choice but to read Professor Klaas’s book, and no choice but to be exasperated by it. If there is a God, she is mendacious indeed. But she is also as pointless as the universe she has created.

Though the universe seems disinclined to random irregularities — it conforms, after a fashion, to our cosmological models — our sample size is minuscule in a spacetime of incomprehensible proportions. The data upon which we found our assumptions of universal causation is, for all intents, nil. And it is vulnerable to our own evolutionarily-conditioned selection bias. What we see is all there is. We have evolved to truck in regularities. Ostensible irregularities we suppose, without evidence, can be explained by as yet undetected causes. But even that is a contingency.

So, this is all a bit wishful — for those who don’t find it utterly desolate.

Still, Fluke starts off brightly. There is good discussion of path dependence, contingency and convergence. Our existence here is the product of a colossal sequence of flukes. Things would not have had to be very different at any point in our evolution for us not to be here at all.

For such a glib observation, Professor Klaas hammers it hard. Before long, we see why. He turns to chaos theory and invokes the familiar metaphor of Amazonian butterflies. But the lesson of chaos theory is not that “a butterfly’s wing-flap caused the Ottoman Empire to collapse”, but that for all we know a butterfly’s wingflap could have.

That is illustrated by the marvellous contingencies Klaas illustrates — but he is looking out his rear window at them, as he drives away, by which stage, they are no longer contingencies. They are crystallised histories. They are no longer emblematic of chaos. They are now — to the extent they were recorded at all — fixed historical data. The contingent process — our observation of these data — is complete. What remains can be represented, without much loss, as a string of symbols. It is but a single path between two points in a garden not just of forking paths, but infinite, random, interconnections. To choose one path is to forgo all the others. This is the opportunity cost of existence.

The string of symbols we are left with have no meaning until we run our information processing apparatus over them. It matches against familiar patterns and creates a narrative. In this way our symbol strings produce meaning. This is not a binary, Turing-style symbol-processing operation, but an imaginative one. It, too, is contingent, a willed extraction of one meaning from a vast plurality of possibilities. We fox ourselves that the story we chose is the “true” one. That is the pattern we draw on the side of the Texan barn. But we are not Turing machines. It is not inevitable until it is made.

At least with history there are data. We have at least something to draw our fancy casual chains over and declare sacred and “true”. This is not true of the future. Here our uncertainty is “aleatory” — it has not happened, so we cannot know it — and not merely “epistemic” — it has happened, unobserved, so we don’t know it.

In a complex adaptive system filled with intentional agents, building that narrative and generating consensus about what has already happened is hard enough. Predicting what will happen next is much harder and, as the horizon recedes, quickly becomes hopeless. Complex adaptive systems adapt. They have autonomy. Intentional agents intend. They have agency. Pretending they are mechanical ducks for the sake of a commitment to causation gets us nowhere.

Professor Klaas even states this, outright: he cites a neat experiment in which social scientists were presented with sets of historical input data about human subjects and asked, based on their own published theories, to predict behavioural outcomes. They were, of course, systematically unable to get anything right. Physical scientists laugh up their sleeves at social science, but the outcome would be no better were you to ask a biologist, in a double-blind test, to predict the present form of juramaia sinensis, a tiny, nocturnal, insectivore from the Jurassic era. And nor would the mechanical physicists do any better. We are not puzzled that professors of ballistical mechanics are no better at cricket than anyone else.

Here we must be careful not to make another category error. There is, thereby, little practical difference between “unknown events in the past” — epistemic uncertainties — and “unknowable events in the future” — aleatory uncertainties. But — and this is rather the point — there are different things to take from that similarity. Professor Klaas proves it by drawing an opposite conclusion to the one I would have: rather than taking it to illustrate how unreliable historical data is, he takes it to show that the future that is no less set in stone than the past.

At this late stage Professor Klaas declares, as might a newly-minted philosophy undergraduate, that there are two alternatives: either the causal principle holds, the cosmos flies by calculable wire, and the reason we can’t better predict it is due to our own inadequacy and lack of suitable data, or there is free will, science is worthless and the universe is random.

Doing without causation seems unthinkable, and the cosmos appears to be ordered and science seems to be worthwhile, so Professor Klaas, with more exhilaration and less regret than I would, embraces causality and gives free will the old heave-ho.

The thing is, on their face, both are plainly preposterous positions. Professor Klaas has boxed himself in with a silly stage one a priori thought experiment.

Given that he calls his blog “the garden of forking paths” — named for a Borges short story— seems to be all about contingency and unpredictability of complex systems, so it is a bit baffling that Professor Klaas still concludes that everything is pre-ordained.

Since, even if so, we have no practical ability to predict what will happen, and we seem to have autonomy, and in any case there is no way of knowing, then what do we achieve by saying “well, it is all predetermined”. How does that even help us?

The computationalist view, the infinite, and the end of political freedom

“It wasn’t infinity, in fact. Infinity itself looks flat and uninteresting. Looking up into the night sky is looking into infinity — distance is incomprehensible and therefore meaningless. The chamber into which the aircar emerged was anything but infinite, it was just very, very, very big, so big, that it gave the impression of infinity far better than infinity itself.”

—Douglas Adams, The Hitch-Hiker’s Guide to the Galaxy

The question is not without its practical consequences. In one sense it is a desolate view, hankering to get to the end to the journey we define ourselves by being on: life. The computationalist view offers the same desolate resolution as do the Abrahamic religions: an end state, where everything is solved, everything is managed, a benevolent higher intelligence silently ensures everything is optimised, and no-one wants for anything. Religions promise that in an afterlife: computational techbros promise it here, if not now, then in the foreseeable future. But a universe without problems to be solved, lots to be improved or games to be played — with no riddles, mysteries or hazards — of unlimited accommodation is hell as much as it is heaven. Utopia and dystopia are the same. No-one would like such a world, and we can already see what happens when we approach it: people wilfully ruin it.

Francis Fukuyama — he has had some bad press, including from me — captured it well:

Experience suggests that if men cannot struggle on behalf of a just cause because that just cause was victorious in an earlier generation, then they will struggle against the just cause. They will struggle for the sake of struggle. They will struggle, in other words, out of a certain boredom: for they cannot imagine living in a world without struggle. And if the greater part of the world in which they live is characterised by peaceful and prosperous liberal democracy, then they will struggle against that peace and prosperity, and against democracy.

Heaven would be awful. The Devil has all the best tunes.

But there’s a more insidious implication, too. A causal, solvable universe implies a single truth. It means those who have that truth are justified in suppressing anyone who promotes a different idea. Scientism leads to this: we see harmless forms of it in Richard Dawkins’ grumpy (and uninformed) tracts against religion. Elsewhere, the religious are just as guilty of the same kind of intellectual imperialism.

But, either way, it is an illiberal instinct: to assert a single solution — let alone to enact it — is to stop people from living their lives as they would choose. Citizens should be free to make their own mistakes: who knows? We might all learn from them. Preventing it might deprive us of the serendipities that have characterised scientific discovery over centuries. In any case we should not wish ourselves to a utopian endpoint that none of us would like much if we got there. We are not non-player characters. We are not hopeless invalids better hooked up to a pleasure machine for our own wellbeing. We do not want to spend an eternity making small-talk with do-gooders. We came here to struggle and make sense of things.

I’ve wittered on enough and have to finish this now to get out the door to a talk by Iain McGilchrist, whose marvellous book The Master and his Emissary gives a much more hopeful perspective on this very question.

See also

* Robomorphism

* The Master and his Emissary

Thanks for reading! This post is public so feel free to share it.

“I once got all the way from Glasgow to Edinburgh without a ticket. I walked.”

— Sid Snot, The Kenny Everett Video Show.

“There are more network use-cases in heav’n and earth, Horatio, than are dream’t of in your philosophy.”

— Shakespeare, Spamlet I, vi

An iron-fisted Romanian

“The Bickerings,” ancestral home of the Contrarian clan, is freezing old pile in Squatney Green. It is cold enough, but made worse on account of the JC’s missus, the Contesă Birgită von Sachsen Rämmerstein, who controls the central heating with an iron fist.

The Contesă grew up in a stone castle in the high Transfăgărășan, her father was a tyrant and she has therefore grown accustomed to a chilly ambience. The family was grand but impecunious, and she habitually regards any attempt to put temperatures into double figures as evidence of immutable moral decay. “Eef you are cold,” she is fond of saying, “you should put on a hat.”

I am, by these standards, weak. I am often tempted into defiance when she is not looking. Until now my meagre resistance has been mainly useless: the Contesă is gimlet-eyed, and immeasurably helped by our central heating system which was designed about the time they built the computers for the Apollo programme, and it has similar functionality. While it can, I am told, schedule and regulate temperatures this requires an advanced facility with algebra that I, alas, do not have.

Nor will the Contesă countenance my occasional suggestions that we upgrade to a modern central heating system with an intuitive user interface. That would involve massive expenditure and, besides, capitulate to my lack of Transylvanian fibre.

But recently things have changed. I have identified a way of fitting inexpensive replacement valves on our radiators. They are wifi-enabled and fitted with a smart thermostat. They can be programmed, controlled and adjusted from an app.

I used the meagre allowance the Contesă grants me and bought a set of smart valves. As the northern hemisphere winter grinds its saturated way to a squelchy close, retailers are trying to shift their inventory before the spring arrives, the world warms up and it is too late. The valves are currently on sale. I bought seven and I got a bargain: they were half price.

Thanks for reading! This post is public so feel free to share it.

The problem with central heating systems

Until there was the internet, the problem with upgrading a traditional central heating system was exactly that: it is a centralised system. It has a heavy structure. There is a single central brain, a designed-in “nervous system” and it is integrated and not articulated: if you want to upgrade any part, you need to upgrade the lot.

The brain controls two systems: a water system, that sends hot water from the boiler out to spur radiators around the house, and an electrical system that measures temperatures around the house with remote thermostats and sends that information back to brain. The brain has a “preferred setting” from which it controls how much water it should send out to the radiators. If the thermostats say, “it is too hot” the central system shuts off. If they say, “it is too cold” the central system opens up. There is no great intelligence in the system: it has some kind of a time scheduling function and a temperature gauge, and that is it. More sophisticated systems divided the house into temperature zones, each controlled by a single thermostat.

But beyond that, to micro-manage their local environment, users would have to manually adjust the radiators. Each has its own analog thermostatic valve connected to a switch that gates the pipes running into the heater. If it opens, water flows in. If it closes, water stops. But the manual valves are not connected to the central brain: if a radiator’s local valve is fully off, the radiator will not come on, whatever the central system tells it. The electronic thermostats that talk to the central system’s brain are overriden by the manual ones that do not. On the other hand, if the central system thinks the zone is too hot, it won’t send any water to the radiators, so it won’t matter how the local radiator valves are set.

The system is, therefore, something like a binary logic gate: a radiator heats only if both the electronic and the manual valves open. It is what lawyers, and grammarians, would call conjunctive: an “and,” not an “or”.

It all takes quite a lot of — well — plumbing and wiring to install such a system, and therefore quite a lot of disruption if you want to replace it. The electronic thermostats are hardware-controlled and connected by cable, chased into the walls of the house. God forbid should I suggest we move a thermostat and upset the Contesă’s Farrar & Ball™ elephant spunk™ skim coat wall finish.

Since our control panels were designed in the late 60s, they have little of the functionality we are used to these days. They were not designed to be upgraded. They are not modular. Their programming is hard-coded into ugly little devices dotted around the house. Not just ugly, but dysfunctional: they hail from a time before “user experience” was any kind design criteria. There are four buttons, embossed with hieroglyphics I don’t understand, and a small liquid crystal display panel that displays different hieroglyphics that I don’t understand either.

It isn’t clear what any of them do. How we originally programmed them is now lost to posterity, and for some years now we have just tolerated the meagre assistance they provide in the depths of winter. For the Contesă, this is business as usual. Over the years I have invested in knitwear. The heating comes on when it deigns to come on, goes off when it deigns to go off and that is that. The Contesă and I shuffle around our frigid house, wrapped up in mittens and scarves.

The problem is solvable because of the ingenious design of the valves. They accord with a principle of network design called the “end-to-end principle”. It is quite unintuitive but, when you get your head around it, utterly brilliant. The design of the internet is fastidiously based on the end-to-end principle.

But — and this is the beautiful thing about design — the internet’s construction in the 1960s long preceded theory that made it viable. The end-to-end principle explaining why the internet works was not identified or formalised until 1984.

How to design networks

When creating a network of dispersed “users” — call them “endpoints” and the system a “distributed network” — you have design choices to make. Different network designs have different pros and cons and different consequences for scaling, efficiency and task management. It is all rather mathematical.

Direct point-to-point networks

The simplest, in theory, is to link every endpoint in the network directly. We can see this rapidly gets complicated. With a two endpoint network there is one link. Adding a third endpoint, requires two new links. Adding a fourth requires three. The problem grows arithmetically as you add new users. Given a total userbase of N, the number of new connections needed to add a single user is N - 1. The more endpoints, the more links required to add a single new user.

The application for which the network is used is important. If all users will be interacting with all other users all the time, this may be the maximally efficient design. An example of this kind of network is a high-performance computing GPU cluster used for AI training: here the point is parallel processing, where every node exchange data directly with every other node on the “network” (a series of gates on a graphics processor) at maximum speed with minimal latency. But it is a pretty unique case. There aren’t many cases where a point-to-point network is a great design choice.

Most human networks are not like that. We only have a certain amount of personal bandwidth. We can only read one book at a time, or watch one film at a time. Our interaction with a given network is highly selective, and in fact unique: how I experience and interact with London is unique: I go to the Cherry Tree in Ost Finkelstein for my apples. The Contesă goes to an odd little Russian shop to get ingredients for her borscht. She does not need a link to my greengrocer. I don’t need a link to her cabbage purveyor.

In this case a fully-connected network becomes progressively harder to scale and less efficient. The more endpoints in the network, the less likely user is to communicate along a given link. A directly linked network, therefore, contains a great deal of redundancy.

Hub and spoke

Another way of designing networks is a hub and spoke model where local users are connected to a single large hub which has a much greater bandwidth connection to other hubs, to which other local users are connected. This is how, for example, railway networks work: There are a small number of “nodes” — stations — and these have limited set of very-high bandwidth connections between them. Endpoints — passengers — must make their own way to a node. But “adding new users” is therefore, from a “hub and spoke” network’s perspective, a low-cost, low complexity activity. It carries a predictable, low marginal cost. building additional hubs and connectors between them — that is, rails and tunnels — is obviously more expensive, but it is a one-time expenditure that happens infrequently and supports a greater capacity to handle users on the network. It is much, much less wasteful than a point-to-point network.

But hub-and-spoke models have some odd inefficiencies of their own. For one thing, connection routes on the network may be much longer and more complicated than is needed to cross the physical distance between user endpoints in real space. The London Underground is famous for this sort of thing. Visitors who take the journey from Wood Lane, on the Circle Line, to White City, on the Central Line — which takes about three quarters of an hour via Liverpool Street, or over half an hour with two changes, via Notting Hill Gate and Edgware Road —deposits them across the road from where they started.

Furthermore, knocking out a single hub can break the whole network, at least for anyone connected to it, or depending on it for a through link to another person.

The hub-and-spoke model is, nonetheless effective in most cases, at least where nodes are not very close to each other. Airlines run a similar arrangement, with regional airports feeding central hub airports like Heathrow and Chicago, which handle long-haul flights between them. Postal services, too, are hub-and-spoke models, often with several layers of hubs arranged as spokes around each other.

But typical social networks are not like that. In urban communities a lot of different networks live on top of each other. There are all kinds of random intersections and interconnections between disparate networks. It is all very fluid. There’s no central control: networks arise and die back as individuals need and use them. These networks don’t have any intelligence of their own: all the intelligence lives within the individual members of the communities. At network endpoints, in other words. Community members figure out which networks to join and what to use them for.

Neither the point-to-point or hub and spoke networks are efficient when people are often close to each other and sometimes distant, and where network needs are constantly in flux. In a dynamic, fluctuating community users need something that can do a bit of both.

Mesh network

There is, as Tony Blair once said, a third way. (There are doubtless others, but I don’t think you would thank me for embarking on a comprehensive survey of all network ontologies.) In this case, there are a great number of nodes, and most endpoints function as nodes too. the only difference between a true endpoint and a node is that an endpoint only has a single connection. Because there are countless nodes, nodes are not all interconnected but, instead, connected only to nearby nodes. Distant nodes are only indirectly connected through one or more intermediate nodes.

Now there are any number of indirect connection paths between any two nodes. The more nodes in the network, the more possible connection paths between them.

This solves all three of the problems identified above, and quite quickly. Firstly, it is easy, and cheap to add new nodes and endpoints to the network — each needs a small number of connections,: it may be as few as one, so the “arithmetic increase in cost to connect an additional user” problem does not exist. The network is easy to scale. The marginal cost of adding users is static, and it is borne by the connecting user, not the rest of the network. User pays.

Secondly, it solves the “single point of failure” problem of a hub-and-spoke model. As a mesh network scales, what does increase, geometrically, is “the number of potential connections between any two points”. The bigger the network, therefore — the more nodes it has, and mesh networks tend to have a lot — the more robust it is. The more resilient to failure. This means that there are no single, or significant points of failure. If you knock out a node, that only impacts that node, and any endpoints connected only to that node.

This is, indeed the fundamental problem that the U.S. Department of Defense’s Advanced Research Projects Agency — DARPA — was trying to solve when it formulated the principles for the ARPAnet, on which the modern internet was founded. The goal was to create a network that could sustain operation during its partial destruction, such as by nuclear strike. A mesh network is largely immune to targeted attack. If you want to knock out the network you must take out all its nodes. The more nodes the network has, the harder it is for a single impulse to destroy it.

Thirdly, it solves the hub-and-spoke model’s “stupid-way-of-crossing-the-road” problem, too: since all nodes are connected directly to other local nodes and will always be connected to the ones closest to it, there will never be a need to go from Wood Lane to White City via Liverpool Street.

Problems with mesh networks

Of course, nothing is perfect and mesh networks have their disadvantages too. For one thing, the route any signal takes across the network is likely to be circuitous.

That is a problem if what you are sending is somehow secret. Everyone in the communication chain will get to see it. It’s also a problem if you are a control freak or, for some other reason, you need a predictable route. A mesh network is all very-seat-of-the-pants, make-it-up-as-you-go-along and ad hoc.

Furthermore, should there be a time or cost implication of sending a message, then mesh networks can be quite inefficient. The larger one gets, the more expensive, and slow, sending “content rich” messages becomes.

But there has been an information revolution in the last 40 years. Electronic signals move down a wire at the speed of light. Speed was not the constraint it once was.

But the resource impact of sending a message across a node — not speed of communication, but volume and format of information sent — presented another problem.

The variety of human communications

There is a down-side to there being an almost infinite number of pathways across a network. It means, to route a given message, every one of those pathways needs to be able to handle the message.

Say you built a physical “mesh” network that employed those cute little Citroën Amis to shuttle your messages between individual loading bay nodes on the network. The vehicles are smart, they drive themselves, using an algorithm to determine which nodes to use on the network pass. As long as you are transporting small people and the odd parcel it will work serviceably well. But if you want to transfer a live dolphin, the network cannot manage. You would need to re-engineer the whole network, and every point on it, to cope. You are stuck. You would have to start again.

Unless you can figure out a way of working around the chunkiness implicit in a live dolphin.

So, whatever your network topology there is always a design decision to be made: what is the universe of items that can conceivably be transported across this network?

It is an optimising function, rather like the one we take when buying a car. We know most of car our journeys will be short and involve one occupant with little luggage. For these, a Citroën Ami would be perfectly adequate. Better, in fact, as long as our friends don’t see us. You don’t need a Land Rover with a snorkel to get around the Hampstead Garden Suburb.

But there will be times when we need to collect the kids from karate practice, take old furniture to the dump, or go off-roading in Wales. It is worth “solving” for these contingencies. But every now and then it might be useful to have a minibus, or a tractor. But we don’t optimise for these extremes: we just hire in the equipment, or the man with a van, as we need it. The “network” has its limits.

Designers of physical networks — even for mesh networks — must do the same exercise. They will optimise for known use-cases, but cannot be expected to predict future use-cases that might come along as technology develops. This is a shortcoming of all models of network design — if you build tunnels that are only ten metres wide, that forever precludes putting eleven-metre wide vehicles on your railway. So, along with a rail network (hub-and-spoke) there is a road network, which is much more like a mesh. The railway is very good at certain transport functions — passenger commuting, or hauling coal around – but not good for nipping up to the highstreet to collect your dry cleaning, or ingredients for borscht.

Because the link count in a mesh network is so large, and chaotic, capacity constraints are a particular limitation. This leads to different arrangement of structure and intelligence. For a hub-and-spoke network there is a real advantage to heavily engineering and controlling the central parts. It doesn’t matter of some things can’t go on the railway because there are aways other networks: the road, sea, and air, that can accommodate them. So railways and their designed-on rolling stock are heavily engineered to work together, and closely controlled by a centralised, intelligent monitoring system.

But central control of a system has its drawbacks. It is a single point of failure.

Any London commuter will know that a central signalling failure can lead to widespread disruption. End users can’t work around it unless they get off the network and use the roads — being a different kind of engineering proposition. The engineering of roads is minimal, and while in urban settings they are controlled, it is lightly. If all the traffic signals go down, the network functions: drivers just have to be a bit more careful.

In any case there are two design principles: engineering and intelligence in the middle, or intelligence and engineering at the edges.

A railway is a heavily engineered, centrally controlled, intelligent network. All the intelligence is in the middle, and the edges are really easy. You don’t need any particular kit to ride a train other than a ticket. You can just sit there. You just have to remember where to get off.

A road is simple, mainly dumb network, with little central intelligence. All the complication, design and intelligence is “at the edges”. Users must bring their own vehicles, and they have to operate them. They have to figure out where to go, by which route, and how to operate their vehicle. The road network is mainly passive. It just sits there. You have to worry about where you are going. The road doesn’t care.

Internet as a dumb network

So there are smart networks and dumb networks. What about the internet? You could be forgiven for presuming the world wide web—surely the most sophisticated distributed network in the known universe—is highly intelligent. In fact, it is not. It is a supremely dumb network. That, indeed, is its very brilliance. The world-wide web could hardly be stupider. All the brilliance is at the edges.

This is partly a function of its genealogy. They built the digital world wide web on a network that was already there, that was designed with a completely different use-case in mind: analog telephone signals.

A traditional telephone mouthpiece worked by converting sound waves into an analog electrical signal—a continuously varying voltage describing those sound waves that travelled to the exchange, passed through a series of switches and down another wire to the other caller, where the receiver’s ear piece speaker does the reverse: converting the analog signal back into sound waves. An analog system was a continuous pipe. The exchange would physically dedicate a continuous electrical circuit between callers for the duration of the call. It was like a private, dedicated tunnel. It persisted whether anyone was speaking. It was inefficient for data. The internet wanted to send binary digits — lots of ones and zeros — down the pipe. It did that by converting them into audible tones that the phone line was expecting. That is the famous modem noise — youngsters probably don’t remember it, but for people of about JC’s age it was a thing of marvel and wonder.

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Reality TV competitions like the BBC’s Traitors offer valuable insights into group dynamics and decision making under situations of uncertainty.

Thanks for reading! This post is public so feel free to share it.

For those bewitched by the unravelling convictions of postmasters, LIBOR rate setters and antenatal nurses Traitors is a superb, but unsettling model. It shows how easily we — be we contestants, viewers, witnesses, prosecutors, judges, juries or poundshop Poirots — can be mistaken, even about very obvious things. How inevitably, where the facts and social dynamics before us are inchoate or contrived, or calculated to mislead, we will be misled. The same cognitive habits and heuristics, that serve us well as we navigate our ordinary worlds of straightforward surfaces and familiar social relationships, lead us astray when we are asked to play strange games of misdirection with unfamiliar participants.

These are “games” not in the senses of parlour games like bridge or chess or even poker, but language games: hermeneutic constructs built from artificial conditions and contradictory and only partially disclosed rules. Where players can reach their objectives only obliquely, while appearing to head in the opposite direction. There are some parlour games like this: Secret Hitler — great fun as long as you don’t mind being accused of fascism — has a similar dynamic.

There are other common situations like this in our lives. The workplace — often a nest of sharp-elbowed misdirections — is one. So is any political organisation: the clue is in the name. The criminal justice system is another.

Traitors is a fully-designed exercise in the wilful perpetration of injustice. Of the twenty-two initial contestants, three — according to the canonical rules — “deserve” to be banished. The others are pure in heart, if not in deed. But even that is a misdirection: all players in the game have the same object— to win it — and that, whether you are traitor or faithful, involves eliminating all the other players. The trick is to to be seen to do as little of the eliminating as possible. The winner is the best at misdirection.

How to play

For those living under a rock, the comatose and the deeply uninterested in popular culture, here are the principles of the game. Calling them “rules” is a bit of a stretch.

Twenty-two contestants convene at a neo-gothic castle near Inverness, hosted by Claudia Winkelman. Having bonded briefly, the players are sat around a round-table and blindfolded whereupon, theatrically, Winkelman assigns 3 of them the secret role of “traitor”. The remainder are the “faithful“.

The traitors will shortly meet in private, so have certain knowledge know who each other are and therefore, who are faithful. The faithful know neither. They only know their own status and, about that, to other faithful they are unreliable witnesses. This is a key information asymmetry. It gives traitors an enormous advantage.

For the faithful, their putative objective — we will come to why it isn’t their actual objective — is to identify and eliminate traitors. They have one opportunity each day to do that, during a banishment session convened at the round table where players debate who seems suspicious. At the conclusion of the roundtable debate players vote to eject one of their number, based on whatever meagre information presented to the table that best persuaded the congregation. But all participants, including the unknown traitors, participate in the roundtable. The odds are therefore somewhat stacked against faithfuls even in the use of their own most powerful weapon.

Each person who pleads her own case, or casts aspersions about another’s, is an unreliable witness. The roundtable is like a jury in every respect but one: “jurors” are players, guilty and innocent, and therefore also participants in, or witnesses to, the crimes alleged. Each has a stake in the outcome in a way a juror does not.

Given the paucity of information, players’ pet theories are inevitably bunk. This is obvious. Everyone can see it. I’ve lost count of the numbers of times I’ve heard people say, “it’s mad they all get het up because the elimination process is basically random.”

When, occasionally, they do stumble on the truth — an inspired aspect of the show is the confessional segments wherein individual players disclose their innermost suspicions to the audience —contestants usually trip over it. Being no surer of themselves than any other players, those who are onto something are usually talked out of action.

There is another elimination mechanism. The traitors, most days, confer in a secret conclave to agree upon the “murder” of a faithful. Murders take place unwitnessed and off-stage: the traitors leave no direct evidence: If the faithful want to catch a traitor, they must use their powers of deduction and inference on the strength of whatever weak circumstantial evidence they can find that give the traitors give away: tics, oral slips, guilty looks, conspiratorial behaviour — that kind of thing.

In the meantime the characters participate in “missions” where they must cooperate to add money to the prize pool. Here, traitors’ and faithfuls’ interests are aligned. This is in some ways clever, but in others, a weakness in the show’s format: it would be better if the traitors stood to benefit by jeopardising the faithfuls’ prize pool somehow. It might give faithful more concrete material to go on at the roundtable.

As it is, there is precious little: before and after the mission contestants have time to interact, air their suspicions and eke out information about each other but only from an inert “data set” that does not really contain any useful information. The game is carefully constructed to avoid traitors ever leaving unambiguous evidence of their identities, except by accident. As long as the traitors have been circumspect, there will no meaningful clues from which anyone could draw a sound conclusion.

In any case, at least one of the traitors is certain to survive to the “final five”— the rules are, literally, rigged as the game progresses to ensure this: television schedules, and not game dynamics, require it. But players should nonetheless factor it in: there will be traitors at the death. There must be. The game would not work without them.

A game of chance

What is fascinating is how players approach a situation in which they must make important decisions with almost no reliable information. Traitors is a show about deception, and it perpetrates its own deceptions on the players, in plain sight, from the outset. It tells them their objective is to identify and eject traitors. But it is not: traitors will in any case “respawn” if their numbers dwindle. Each player’s objective — traitor and faithful alike — is simply, and only, to survive. They should do nothing that jeopardises that objective, including displaying skill at identifying traitors, and thereby presenting an apparent threat. The best strategy is to keep your head down, keep your opinions to yourself, and say nothing unless spoken to. Be the zebra in the middle of the herd.

For Traitors is a game of ostensible, but not actual, strategy. It is, by contrast, a game of pure chance.

At the outset there is maximum uncertainty: players have nothing to go on but resting probabilities. These are easy enough to calculate: once their roles are nominated, each faithful should know there is a 3 in 21 chance — that’s 1 in 7, for the hard of mathematics — of any other player being a traitor.

And then the game commences. Of the 19 original innocenti, at least 15 must, by the rules of the game, be thrown under the bus. They will not all be murdered: more than half will be ejected by the faithful at the roundtable. Murder victims are necessarily faithful — the traitors cannot murder each other — but it is a necessary consequence of the game that most round-table ejectees will also be faithful. They are “murdered” too, only by a council comprising a large majority of faithful.

All that really differs between traitors and faithful, therefore, is their means of killing other players: traitors by murder, faithful by banishment. Presuming they don’t cheat, “traitors” are no less “deserving” of success than “faithful”.

For the thing is: as far as players have any control over outcomes, the elimination process isn’t just basically random: it’s completely random. Players, like viewers, must know this, yet they persist in believing they can anticipate and even influence outcomes — and, for the sake of watchability, just as well: if they did not, the show would not work. The game obliges players to willingly suspend their own disbelief. For all the good their uninformed machinations do them they would be better, and happier, were they to leave things to chance.

If faithful players can discipline themselves into thinking in terms of probabilities, they will note some reliable posterior information does emerge as the game goes on: as the roles of eliminated players are evealed — all murdered are ipso facto faithful, the banished declare their allegiance as they depart — so remaining faithful can update their “priors” somewhat, though, again, their information is incomplete. Faithful players’ odds systematically shorten — get worse — as the game progresses and contestants are whittled down. Elimination overweights the faithful, so as players disappear, the higher the probability that remaining competitors are traitors. By the time of the final five, at least one and probably two of the players must be traitors — that can be deduced from the fact of ongoing nightly murders. This means, for a given faithful, the “traitor ratio” amongst the remaining players increases over the game from about 14% to between 25% and 50%. That is an inevitable consequence of game play.

Most seem unaware of this. They must surely know it, at some level, but if they do, they don’t seem to care. For despite it, remaining players form strengthening bonds. Their sense of “ordeal camaraderie” is at least as strong as their willingness to suspect their comrades are traitors. Thanks to their improbable longevity surviving players, whatever their allegiance, have more in common with each other than any player has with a fallen comrade.

There is a good reason for this: when push comes to shove, individual faithful are no less incentivised to murder — or, by the end, guilty of it — than traitors. Traitors is a gave of push and shove. It is also a fantastic illustration of just how hard it is to make good decisions in times of uncertainty.

No killer facts

The game is carefully constructed so that the faithful are never presented with unambiguous evidence of treachery. It can happen, but only by a traitor’s unforced error. As long as the traitors are not careless, they can avoid leaving direct clues and the faithful must form suspicions based on inferences that are basically bunk.

This, as Traitors series across the world — there are editions in the US, Ireland, Australia and even little old New Zealand — has consistently illustrated, is incredibly difficult to get right, except by fluke. It’s little wonder: the faithful are (mostly!) perfect strangers to each other. They don’t know how each other behave in normal social situations, let alone times of social stress or the state of prolonged contrived deceit that Traitors forces them into.

Their suspicions are usually wildly wrong. Viewers, who know who the traitors are, find the faithfuls’ utter guilelessness at the same time mesmerising and exasperating. We howl at our televisions. We clutch our heads in exasperation. “How can you possibly miss it?!”

But this is a perfect example of hindsight bias. Of course it is obvious when you know who the traitors are. We are deceiving ourselves if we think we could do better.

The game environment is highly artificial: all players, not just traitors, are motivated to lie and disguise their true opinions in ways they ordinarily would not. A faithful who believes she is “onto” a traitor will keep her opinions from the traitor, but will readily share them with others — who may include the traitor’s confederates. This obligation to engage in duplicity leads even the faithful to spin and perpetuate dishonesties the same way traitors do. This is a neat design feature: the natural advantage the faithful would otherwise have, of having nothing to hide, is extinguished.

Some are better at this then others, but the cognitive load in trying to draw inferences from minimal available information often manifests in erratic behaviour, as faithful scrabble helplessly to get some purchase on who is who and what is what in the game.

This erratic behaviour is often mistaken as “traitorous” and those exhibiting it banished. Usually, it is quite the opposite: with their superior information and greater sense of jeopardy, traitors tend to have a much better “game plan” than faithful. They are generally more careful and rational because they do have a plan. This, ironically, tends to stave off suspicion! The faithful tend systematically to banish each other on dismal pretexts, while the traitors continue to get away with murder, literally, undetected and even unsuspected.

As the game unfolds players tend to form alliances. Across the Traitors’ regional franchises, the way they do this differs in a way that, amusingly, reinforces cultural stereotypes: the the Brits are self-effacing, charming, polite and deferential, especially at the beginning. They tend to eject players who are not polite. Irish are cheerfully idiomatic in their interactions. Australians, from the first morning, are brutal.

The British “celebrity” edition of Traitors, in 2025, was a nadir Britishness. Random British contestants are pretty bad at detecting traitors, but do they tend to get some. British celebrities, as you might expect from a bunch of luvvies, grovel disingenuously to each other at all times, in contrived mutual deference, and prove therefore quite useless when it comes to identifying traitors.

There are some learnings from this. An obvious one: in situations of epistemic uncertainty, when people you cannot trust are motivated to present a particular view of the world, we are really bad at figuring out who is telling the truth. Worse even than a choice at random.

This has real-world implications. Traitors might seem contrived, but it tracks the commonplace. For most of us, complex situations of factual uncertainty where conflicted agents spin facts to suit their own agendas, is an everyday experience. This is how parliament works. It is how the media works. It is how most workplaces work. And it is, explicitly how justice works: the “traitors’ dilemma” is exactly the scenario faced by a criminal jury. Who is faithful? Who is a traitor? Who is spinning? What is relevant? What is a red herring?

Like the faithful, jurors have limited information to go on. It may not be everything. It may be wrong. It may invite prejudicial inferences that are not justified.

Misconceptions

Traitors is so beguiling because it is based on a couple of misdirections. For one thing, the faithful are not the good guys: the “faithful” and “traitor” labels are a misdirection. There are no innocents in Traitors. The inevitable probabilities of the round table gives the lie to the idea that the “faithful” are really the good guys. Over a series there will be some 12 round tables. A banishment at each is compulsory. The dynamics of the game require contestants, faithful or not, to winnow themselves down to three finalists. The faithful have no power to save each other to avoid this.

This means the faithful must compete for survival against each other just as fiercely as they must against the unseen traitors. The traitors, conceivably, could all make it to the final. They have slightly more incentive to be collegiate than do the faithful, which is ironic.

Over the course of the game, the faithful typically eliminate more of their own than do the traitors. The familiar refrain, “I’m faithful, 100%” is not quite the ringing endorsement of probity its seems. Being faithful just means you intend to eliminate people in public, not private. The faithful is, in no sense, a “team”.

Unconscious bias?

In recent times, collated game statistics across five seasons of Traitors have prompted questions as to whether the collective decisions made in roundtables and the ”turret” reveal the unstated, even unconscious, prejudice?

Banishment data from early rounds invites the inference that there is mild bias against minorities and older players, who are often ejected first.

We should not be surprised at this. It does not prove prejudice. Firstly, in a novel situation of great uncertainty, informed decision making us impossible: literally there is no information. The players know the baseline probabilities — there’s a 1 in 7 chance of another player being a traitor, so a given contestant is, most likely, not a traitor. This is a dissonance though, because the players also know that three definitely are traitors.

A good Bayesian uses what information she can find to provisionally improve those odds. This is a subjective process, to which she will bring all her life experiences. A person who appears easy to trust has a marginal advantage. Here “in-groups” and “out-groups” might make a difference. We are all, instinctively, inclined to trust those with whom we are familiar — those our accumulated experience of the world tells us are likely to share our experiences, impressions and values — and those we form an interpersonal connection with.

These will often be people who most resemble us — by age, sex, cultural background, occupation, interests, geographic origin.

This is no kind of positive discrimination against those who don’t resemble us — they keep their base line odds — but a concession towards those who do.

Those common connection points are often cultural. Ethnic, religious and racial identities often follow cultural ones.

I can illustrate this with my own “minorityship”: though I live in the UK, I am from New Zealand. There are not many Kiwis in the UK — come to think of it, there aren’t that many in New Zealand either — but in the UK we make up about 0.1% of the population, though, like sand in a picnic rug we do tend to get everywhere.

Though my own connection with Aotearoa is slim — I’ve spent the vast majority of my life in London — should I encounter another New Zealander in the UK, we will quickly connect. We have shared experiences. We can make assumptions about how each other will think. We’re also likely to have been to school with each other’s cousins but that is a different story. The connection might not last — some kiwis are jerks — but all other things equal it is a good starting basis.

This is exactly what is happening on traitors. The great majority of contestants are under 45. The cast reflects the ethnic diversity of the UK, which is predominantly Caucasian, and geographic make up: there are always a couple of Welsh and Scottish but a majority from England. We should expect these people to instinctively bond with in-groups, the same way ex-pat New Zealanders do.

No surprise, the “bias” effect in the data wears off after a few days, by which time participants have got to know each other and have adjusted their perceptions based on actual evidence. We are natural Bayesians. We update our priors.

As the game wears on contestants get no better at picking traitors, however. They consistently allow obvious confirmation bias to override their better judgment. We are astounded at their credulity. We should let it tell us more about our own.

Complex system

Traitors is a perfect model of a complex system. Not only are their autonomous agents making uncontrollable decisions and stark, but shifting asymmetries in information, but the “rules” of the game are opaque and amorphous. Some are disclosed late, others are never disclosed and some change without warning or notice. Generally the rule changes are engineered to favour the traitors, but not always. Secret traitors are introduced. Players are unexpectedly ejected before the game starts, and then reintroduced, just as unexpectedly, later.

Players are therefore in a situation of uncertainty, not risk. Risks you can manage; uncertainty you cannot. The players’ efforts to manage uncertainty and work each other out are doomed not only to fail, but to sow seeds of doubt and resentment in other players. This rancour ossifies into factions, and hostile subgroups. Of course, the traitors merrily stir up this rancour. The net effect is that the faithful get even worse at guessing traitors than random.

Players may as well be in a lottery, where an elimination is drawn from the group, and a murder victim selected from the faithful, at random each day. If they all resigned themselves to that fate, they could relax, enjoy the game, enjoy each others’ company, and let fate’s cold hand decide, without blaming it on any player. This would completely spoil the spectacle for viewers, of course: who wants to watch a bunch of random strangers having a nice time in a Scottish castle?

We, and the networks, can therefore be grateful it never occurs to any of the participants that they have no control over the game. They carry on as if they can beat the game, and each other, with their cunning. Even after the Faithful have ejected eight of their own and just one traitor — even when players they profess to be convinced are lying repeatedly turn out not to be — it never occurs to anyone to abandon the psychodrama and just draw lots.

Traitors as a model for the workplace

Similar group dynamics exist in the workplace, especially where it comes to promotion and preferment.

If you can influence outcomes with certainty, it informs how you “play the game”: being political may pay off — forming and then tactically defecting on alliances, exaggerating your role on things you were involved with, and taking credit for things you were not — even if this destroys relationships with those whom you are outmanouevring — makes sense. It is — should be —management’s job to impose incentives and structures inside the system that discourage this kind of behaviour. Most management fails to.

For if you can’t influence outcomes — if the rules are shifting and unclear — if the decision-makers to whom you appeal are themselves subject to just the same game-playing and caprice, whose fortunes, like yours, may ebb and flow — then you cannot know whether your gamesmanship, like that of a “faithful” in a game of traitors — will pay off or sink you.

You are better to let the river take you where it will, building as you go enduring and healthy relationships around you. Being useful, agreeable and unthreatening is a sensible tactic for a safe but unspectacular career. Most people in professional services have long since figured that out.

The workplace is different from Traitors and Squid Games in an important respect: Traitors is a finite game; the workplace is an infinite one. There is no equivalent to Traitors’ known common general objective of elimination. There is no end-point at which a player wins. At work, the objective is just to keep playing. Relative advantages are often transient.

We think we know “the rules of the game”, but the game is complex, the rules are opaque, and they continually change with the continually changing market outside and internal organisations and priorities within. From where most of us sit, the “rules” — if there even are any — that govern our advancement may as well be random.

This is what propels the experimental finding from 2010 that organisations that promote people at random do no worse than those with extensive performance appraisal processes.

Curiously, “the rules being random” may be a better outcome either way, if it leads to staff prioritising cooperation, collaboration, informal relationships and trust over “playing the game”.

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Welcome to a new experiment in gratuitous discursion about music, culture and modernity.

Thanks for reading! This post is public so feel free to share it.

This is a little different that the usual JC fare, and in lieu of any strengths plays to my weaknesses, which is the ability to get utterly sidetracked by the simplest questions.

On a train up to see my daughter Antagonista I happened across U2’s album The Unforgettable Fire. A guilty, embarrassing pleasure, but halfway though the six-minute, two-chord epic Bad it struck me what a magnificently great song, and performance it is, and that called to mind U2’s legendary, notorious performance of Bad — all ten minutes of it — at Live Aid.

I have a theory that Live Aid was a fundamental cultural touchpoint for people of my generation — that it changed the world in ways we do not often acknowledge – and this pulls on a few JC strings, and I started writing it up as a sort of appreciation of rock bands, guitar rock, digital delays, ambient music, plonkers, irony one of the great vocal performances — and and realised it would work much better as full audio surround sound experience.

So here it is. Hope you enjoy! If this goes well I might do a bit more of this sort of thing.

The original text — though it isn’t half as much fun — is here.

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

There is a Spotify playlist of the forty seven — FORTY SEVEN FOLKS — songs name-checked and sampled in this podcast!

Sightseeing trips to the ice

In the 1970s New Zealand’s national carrier Air New Zealand began operating “sightseeing” flights to Antarctica. The flights would depart from Auckland, fly south over the remote Balleny Islands and onto the ice continent, passing over Capes Hallett and Adare and from there, down McMurdo Sound to the ice continent. There they would circle for an hour or so, to give passsengers a good look at Antarctica, before turning around. They returned to Auckland via Christchurch in the South Island for a refuelling stop: the airline’s fleet of DC-10 wide-bodied jets did not quite have the range to make it back to Auckland comfortably in one go. The round trip took about 12 hours.

At 8 am on 28 November 1979 flight TE-901, the final flight for the season, left Auckland with two hundred and thirty-seven passengers and twenty crew. At the controls was an Air New Zealand veteran, captain Jim Collins. All was well, though visibility was not great. As he approached Antarctica shortly after midday, Collins reported cloud cover at 5,000 metres, and requested permission from local air traffic control at the McMurdo Sound naval base to drop below the cloudbase to give passengers a better view. McMurdo granted it.

Soon after that exchange, McMurdo Base lost contact with the DC-10. When it had not reappeared on New Zealand air traffic control systems a couple of hours later, officials began to fear the worst.

Over the afternoon, planes were scrambled from McMurdo Base. They traced TE-901’s last known position. They followed its scheduled flightpath. They found nothing.

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Hours passed. News of the missing plane began to spread. As the Auckland evening drew in, relatives began to reconcile themselves to the idea of disaster. Air New Zealand issued a grim deduction: the plane could not possibly still be in the air: it would long since have run out of fuel.

Then, at about midnight — the sun does not set in Antarctica in late November — the crew of a US Navy Hercules spotted a dark smear on Ross Island, some forty-five kilometres east of the plane’s scheduled flight path. They investigated. They quickly realised it was the remains of an air crash. The aeroplane’s tail section, but little else, was intact. It bore Air New Zealand’s distinctive koru. Wreckage stretched for hundreds of metres up the lower slopes of Mt. Erebus, a 3,900 metre volcano.

The Hercules radioed back to base. They had found TE-901. There would be no survivors.

A commercial airliner with two-hundred and fifty seven souls on board had flown at an altitude of 400 metres into a mountain the size of the Eiger.

At the time it was the fourth worst crash in the history of powered flight. It remains New Zealand’s worst single peacetime loss of life.

The crash recovery, at such an inhospitable location, was harrowing. Many on the rescue teams were permanently traumatised. The New Zealand Government created a special civilian award for gallantry for those who went down to the ice.

The government initiated an investigation. The cause of the crash remained baffling. The plane’s captain, Jim Collins, was an experienced and conscientious pilot. He had reported no trouble. The plane’s telemetry indicated it was functioning perfectly. The flight recorders indicated the cockpit seemed harmonious, though there was some discussion about visibility. But photographs recovered from passenger cameras indicated good visibility below the plane, and it had been the basis on which Collins’ was granted permission to descend below the plane’s minimum safe altitude of just under 5,000 metres. That altitude had been set a good kilometre above the highest point of the surrounding terrain, which was the crater of the active volcano Mt.Erebus, on Ross Island to the east of the Sound.

Yet, despite all this, the airliner had been nearly fifty kilometres off course and was flying lower than 500 metres above sea level.

Someone, or something, had plainly gone very wrong. Quickly — the consensus, after 45 years, is far too quickly — the question turned to who. As is so often the case in “human error” investigations, the first person in the frame was the operator: Jim Collins. Captain Collins, as is also so often the case, was in no position to argue.

Whether or not Collins was at fault, there is a deeper point. It ought to be a well-understood operating risk in any complex operation that people make mistakes. Human error is not an exception to the operation of a system but an inevitability. Yet, when failure happens, our instinctive response to find the wrongdoer.

It need not be that way.

Moonshot

When President Kennedy fired the starting gun on the space race in 1962 — at which point the Reds had a lap head start — the Apollo Programme was on an absurdly tight schedule. The farthest it had then got was Alan Shepard’s suborbital Mercury flight in May, 1961. That lasted fifteen minutes and reached an altitude of less than two hundred kilometres from Earth. It wasn’t even, truly, in space. Now Kennedy promised that NASA would have a man on the moon by the end of the decade.

The moon is nearly four hundred thousand kilometres from Earth. The mission would take ten days. The challenge NASA faced was, literally, an order of magnitude greater than its greatest achievement to date.

Then, in January 1967, disaster struck. A terrible fire during a launchpad test killed three astronauts in NASA’s Apollo programme. The immediate cause of the accident was a stray spark from exposed electrical wiring, which ignited in the pure oxygen environment of the sealed capsule. The crew did not stand a chance. They were bolted in: it would have taken them a minute and a half to open the door.

Programme director Gene Kranz shut down the blame game before it could get started with his famous “tough and competent” speech.

Spaceflight will never tolerate carelessness, incapacity, and neglect. Somewhere, somehow, we screwed up. It could have been in design, build, or test. Whatever it was, we should have caught it. We were too gung-ho about the schedule and we locked out all of the problems we saw each day in our work.

[...] Nothing we did had any shelf life. Not one of us stood up and said, ‘Dammit, stop!’ I don’t know what Thompson’s committee will find as the cause, but I know what I find. We are the cause! We were not ready! We did not do our job. We were rolling the dice, hoping that things would come together by launch day, when in our hearts we knew it would take a miracle.

[...] From this day forward, Flight Control will be known by two words: ‘Tough’ and ‘Competent.’ Tough means we are forever accountable for what we do or what we fail to do. We will never again compromise our responsibilities. Every time we walk into Mission Control we will know what we stand for. Competent means we will never take anything for granted. We will never be found short in our knowledge and in our skills.

Here are two divergent responses to disaster. The first, and often instinctive, is to find a single root cause in an individual—to isolate a “bad apple”. The second is exemplified by Kranz. He shifted the focus immediately from who to blame to what to fix: How can we redesign the system—the spacecraft, the procedures, the culture—to ensure this never happens again?

The Erebus disaster would, unfortunately, take the first path.

Investigation

Within a few months of the crash New Zealand’s chief air accidents investigator Ron Chippindale issued a preliminary report. It was thorough — something like 90 pages — and pulled few punches. Chippindale attributed the crash to pilot error, more or less upon the legal principle res ipsa loquitur — “sometimes, things speak for themselves”:

The probable cause of this accident was the decision of the captain to continue the flight at low level toward an area of poor surface and horizon definition when the crew was not certain of their position and the subsequent inability to detect the rising terrain which intercepted the aircraft’s flight path.

If you can’t see a mountain in front of you and you fly into it, that’s on you.

But Jim Collins was a fastidious pilot. He had been briefed on the route. He had studied it diligently. His family had watched him, the night before departure, marking up the route on his own personal atlas, which he took with him on the plane. And besides: it made no sense. Why would an experienced pilot drop through the published minimum safe altitude if he didn’t know where he was and couldn’t see where he was going?

One of Mr. Chippindale’s other observations — made in passing and of little consequence — caught the attention of the pilot’s association:

The flight planned route entered in the company’s base computer was varied after the crew’s briefing in that the position for McMurdo on the computer printout used at the briefing was incorrect by over 2 degrees of longitude and was subsequently corrected prior to this flight.

[...]

Some diagrams and maps issued at the route qualification briefing could have been misleading in that they depicted a track which passed to the true west of Ross Island over a sea level ice shelf, whereas the flight planned track passed to the east over high ground reaching to 12450 feet [about 3,800 metres] above mean sea-level.

That “high ground” was Mt. Erebus. But even at McMurdo Sound’s extreme southern latitude, “two degrees of longitude” is no small difference: it is about 50 kilometres. The route Jim Collins had been briefed on would take him well to the west of Mt. Erebus: if he was going by his maps, he would have been over McMurdo Sound. But still: even from 50 kilometres, mountains the size of Erebus are hard to miss. If visibility was poor, the plane should not have been at a dangerously low altitude. As far as Ron Chippindale was concerned, the operating cause of the disaster remained that Jim Collins flew his plane into a mountain.

Air New Zealand quickly swung in behind this narrative. The pilot’s union did not.

Another thing to bear in mind: the Antarctic flights were unlike ordinary commercial flights for a couple of reasons. Firstly, being a round-trip, they didn’t have a fixed destination waypoint. Planes navigate via “waypoints”. The final one is usually the air traffic control tower of the destination airport. Needless to say, you have to reach a runway waypoint exactly: near enough is not good enough. But TE-901 was not aiming at a landing strip, but rather just a marker it would fly around before heading home.

Secondly, it was a sightseeing trip: the point of the journey was to have a look around, so the pilots might be expected to take a little more, well, latitude than one might expect on a normal commercial route, especially if visibility was patchy.

And thirdly, it was, well a sight-seeing trip. That meant being closer to the ground that the DC-10’s normal cruising altitude of 10,000 metres. There is no chance of hitting any mountain when you are 10 kilometres in the air, but it is not much good for sightseeing, especially when there is a layer of cloud below. Pilots on the Antarctic route had standing permission to descend to a “minimum safe altitude” of 4,900 metres — still well above Mt. Erebus’s summit — and it emerged during later evidence that they regularly dropped well below even that to give passengers a view of the Sound as they approached.

That typo

There is a little frisson here: the “error” mentioned in Mr. Chippindale’s report had been coded into the computer coordinates and the Airline’s briefing notes for more than a year. The previous Air New Zealand flights had all taken this route, incorrect though it was flying down McMurdo sound, some 43 kilometres west of the approved flightpath.

It was, in fact, a better route for sightseeing, precisely because you could fly low over the sea-ice. The approved route went right over the top of Mt. Erebus, obliging the pilots to stay much higher and having to contend with the small matter of flying over the crater of an active volcano!

It is possible — no-one knows — that Air New Zealand knew of the error, and kept quiet about it on account of its somewhat difficult relationship with air traffic control at the McMurdo Station, which was not used to commercial flights and did not like having to “babysit” them. Had McMurdo known about the error, they may have withdrawn permission for Air New Zealand to fly it.

There is some controversy as to what prompted the action, but in the hours before TE-901’s departure, an Air New Zealand flight controller noticed the “error” in the intended flight co-ordinates — they had been miskeyed as “164°48’ east” instead of “166°48’ east” — and corrected them back to the approved civil aviation route.

It is possible that the real error was this correction to the original error, and the ideal route was down McMurdo Sound all along. As far as Captain Jim Collins was concerned, that was where he was meant to be.

In any case, the correction was never communicated to Captain Collins or his crew. The DC-10’s navigation system would take the plane directly over Mt. Erebus, and not 43 kilometres to the west.

At first this non-communication of such a large correction seems outrageous, but it is an oversight of exactly the same nature as the one apparently made in the cockpit. The controller must have assumed it was an insignificant change. It would be, as long as the plane stayed at its permitted altitude. Seeing as it realigned the plane with its approved flight path the controller may have also assumed his action simply conformed to the route Captain Collins was expecting to fly.

And — had he even taken his thought process that far — he might have figured that, if all else fails, Collins would be able to see that the flightpath had been shifted. Mt. Erebus is nearly four kilometres high: it is usually quite hard to miss.

Visual meteorological conditions

On 28 November 1979 there was a heavy layer of cloud over McMurdo Sound at the programmed altitude. Captain Collins sought from McMurdo Air Traffic Control, and was granted, permission to descend below the cloud base to give the passengers a better view. On his co-ordinates he, and local air traffic controllers at McMurdo Station, believed he was flying over McMurdo Sound, so there was little risk. McMurdo ATC had a radar that could “let down” the plane 400 metres, so approved the descent as long as Collins maintained “visual meteorological conditions” — weather conditions good enough to maintain separation from other aircraft and obstacles by sight. Collins confirmed that he did. Passenger photographs recovered from the crash site confirm there was good visibility across the ground. McMurdo’s radar system never managed to lock onto the DC-10.

Had it been clear, he would have seen Mt. Erebus ahead. But it was heavily overcast, and when a snow-covered landscape blends into a flat white sky the horizon disappears in an unusual condition called a “sector whiteout”. This is different from the whiteout that you might experience when skiing, in which you lose all sense of up and down. It is less obvious, and more insidious.

In a “sector” whiteout clouds above clear air reflect light preventing any snow-covered features from casting shadows. The sky, snow, and horizon blend together. There is no contrast to reveal slope. Rather than obscuring your vision it affords an apparently clear view. A colossal mountain is indistinguishable from flat ice. Captain Jim Collins expected to see flat ice. He looked out his window, and that is what he saw. He had no idea he was flying straight into a mountain until his ground indicator warnings sounded. By then it was too late.

So a highly unlikely combination of factors — an unrecognised programming error, its non-communication, a cloud layer and the visual conditions it created — contrived to create a disaster. Had any of the factors not been present TE-901 would have returned safely.

Normal accidents and system glitches

The Erebus crash was a textbook example of what Charles Perrow called a “normal accident”: a catastrophic failure mode of a highly complex, tightly-coupled system. It is literally a textbook case: it is in Perrow’s book. Perrow called them “normal” accidents because they arise during normal passages of expected operating conditions. No major error, oversight or sabotage is needed to cause them: just a confluence of unusual circumstances and the sort of ineradicable misapprehensions that comprise the human condition and from which we all from time to time suffer.

Normal accidents are not, in the final analysis, really “failures” as such: rather, they unwelcome operating modes that arise as a function of sheer system complexity: when non-adjacent parts of the system that aren’t meant to interact do, then there is a potential for non-linear reactions. The Erebus disaster is a classic case.

Catastrophic failures are all too easy to recognise when they do happen.

But what happens when the malfunctioning complex system is not a tightly coupled power station or commercial aviation system, but one of bureaucracy and law? Here human judgment and misapprehension is just as vital. What happens when the failure is not a sudden explosion or a plane crash on a distant mountain, but a series of loosely coupled misapprehensions, misalignments of interests, and clouded judgments? When they happen not in seconds, but over years? Where each error is not recognised as a failure but, rather, as a success? What if errors are taken as validations of a system that is in fact malfunctioning?

These would not look like “accidents” at all: they would present — much later — like latent defects. In the meantime they may have been built on and integrated into foundations. Those afflicted by these everyday misapprehensions might not be blamed, or vilified, but rewarded for carrying out their appointed function. These calumnies may only unravel years later. When they do, there will be the same hue and cry, and those responsible will be, in the same way, eviscerated.

These we might call “system glitches” — failures that don’t look like failures, and so are allowed to repeat, without a fixed end point.

What might one of these look like? The best recent example is the Post Office Horizon IT scandal: the prosecution over fifteen years of literally hundreds of sub-postmasters for non-existent fraud. The prosecutions had been wrought by a sprawling, incomprehensible “complex system” — a complex combination of software, software vendors, investigators, inhouse lawyers, Post Office executives, externally appointed solicitors and barristers and the court system — seemed to be working properly. It identified fraudulent behaviour, successfully extracted compensation for it and punished those the system held responsible.

Subpostmasters tend to be “pillar-of-the-community” types, often having taking the role out of a sense of public duty. They tend to notably lack criminal records, nor any motive for petty fraud.

But it is only when we step back to ask a bigger question that the possibility of a system glitch emerges: can it really be true that nine hundred generally upstanding individuals should start independently defrauding the Post Office in strikingly similar ways, just as the Post Office introduced a state-of-the-art accounting system designed to detect fraud?

Catastrophic accidents typically only happen once. The lesson is learned, systems are updated, protocols introduced, and the complex system adapts. They tend, therefore, to be highly unusual events. The system gradually gets safer.

The Post Office Horizon IT scandal shows us that this need not be true for system glitches. They may not recognised. They can happen over and over again. A false prosecution that looks like a fair one will not prompt any change in behaviour. If the same circumstances come up, we should expect the same outcome.

Hunter becomes the hunted

This is the most insidious system effect of all: the self-justifying hunt for another scapegoat to explain how the system made scapegoats. Once revealed, the system glitch becomes obvious. Of course these public-spirited subpostmasters weren’t siphoning away money! That is absurd! Now, we must make an example of those villainous few who made it their business to prosecute them! With this new resolve, the system reconfigures itself to find some new “bad apples” to replace the subpostmasters in the public stocks. Commissions of enquiry are convened. King’s Counsel appointed. The shellacking is broadcast on the internet for all the world to see. It is a modern-day public flogging.

But is not this to commit the same category error the system made in the first place? Aren’t we looking for simplistic, linear causes of a complex, non-linear failure?

The lawyers who prosecuted subpostmasters were, in their own way, acting within their expected function in the system.

They were presented with what appeared to be clear evidence of theft, from a respected institution, backed by what they were told was infallible technology. Their role in the system was to prosecute apparent crime; that is what they did. The system incentivised success, not scepticism. They were operating with their own “local” information, unable to see the wider pattern that would have revealed the truth.

This is not to excuse malicious or knowingly dishonest conduct. But it is to doubt that many of the hundreds of people involved in prosecutions over 15 years were malicious or knowingly dishonest. Most were, like the rest of us, beset by misapprehension, cognitive bias and ordinary human fallibility. That is all it took. They were unsighted components in a sprawling, malfunctioning system failure. Now that failure has manifested itself, the system turns on those same components and treated them the same way they had treated the subpostmasters. We find this to be a satisfying type of retributive justice, but it serves to misdirect attention away from a system that keeps glitching.

What if the real villain is not a person, but the pathological dynamics of the complex system?

Human organisations as complex systems

Complex system/ˈkɒmplɛks ˈsɪstəm/ (n.)

A self-organising system of autonomous individuals, components and subsystems, which interact in non-linear ways and whose behaviour cannot be reliably predicted from the behaviour of the individual parts. The rules, boundaries and components of a complex system are typically not well defined, may themselves be complex systems, and may change unexpectedly.

The thing about human organisations is that they are complex systems. The Post Office Horizon IT scandal wasn’t caused by one bad actor — nor even a lot of them — but by the unlikely conjunctions and interactions of actors doing, generally, what they were there to do, and were expected to do, only mediated by the peculiar motivations, incentives and information gaps that cross-cut any component’s interactions with the rest of the system.

In this way the system itself contrived to distribute a latent failure across multiple institutions and people. Each component operated within its own domain and according to its own incentives and with its own limited information. The Post Office needed to boost its results. Fujitsu needed to demonstrate Horizon’s reliability. Fujitsu employees knew there may be bugs but were disincentivised by their employer from flagging or escalating them. Isolated subpostmasters could not see that others were experiencing the same impossible discrepancies. Lawyers presumed an institution as august as the Post Office had well founded concerns and played their role in optimising the success of what they took to be thoroughly justified prosecutions.

The individuals interacting with this “system” — and their subsystems — broadly did not have the information needed to assemble apparently innocuous local warning signs into a coherent picture of wide-scale injustice. Now that we do have that information — thanks to the intervention of other complex systems, like the fourth estate — it is hard to put ourselves back in the purblind position that the actors were in at the time.

This is not to excuse, but to explain: many are understandably anxious to isolate and punish a villain, but that is to make exactly the same category error that this glitching system has made: to look for a single root cause of a complex and unpredictable interaction. The right reaction, on seeing in-house lawyers squirming under cross examination at the Post Office Horizon Inquiry was to think, “there but for the grace of God go I”.

This does not mean there should be no consequences: it just means they should be target at the people who are meant to be accountable for these systems. There is little to be gained from blaming operators, even when they are still here to speak for themselves. Accountability belongs with those empowered to design, maintain and change the systems we operate. For the Post Office, that was Paula Vennels and the Post Office executive. For Erebus it was Morrie Davis and Air New Zealand’s executive. In both cases they notably baulked at accepting a responsibility that was clearly theirs. In stark contrast, Gene Kranz and the NASA flight control team accepted their responsibility and acted on it. They are rightly held up as exemplars of how accountability is meant to work.

Air New Zealand’s executive massaged the picture to deflect their accountability. Responsibility lay not with a pilot following his brief, even if he was mistaken, but with airline management allowing a system to persist where simple misapprehension could result in disaster. Human error is, in any system, inevitable. Likewise, final responsibility for the Post Office scandal lies not with those pursuing the prosecutions, however short-sightedly, but with those who set their incentives and encouraged that short-sightedness, discouraging internal challenge and ushered in those outcomes regardless of their plausibility.

Complex systems have a “mind” of their own. Especially those that comprise human agents who, literally, have minds of their own. We can try to harness them, but the opportunity for chaos, swamping our best laid plans to remain in control, is never far away. Those who commission systems remain responsible when, as they will, they play up.

We tell ourselves we are in control: all to often, the gods have other ideas.

Memoriam

The Erebus disaster could have been something that bound a small nation together. Instead, it became a story about blame, anger and recrimination. A quest for the truth that went badly off course.

— Michael Wright, White Silence, episode 6.

The Erebus recriminations did not stop with the Chippindale report. The uproar was such that the government commissioned a royal enquiry. It would be chaired by a High Court Judge, Mr. Justice Peter Mahon. Mahon rejected Chippindale’s report and exonerated Collins and his crew, finding that the airline’s administrative mistakes — especially that typo and the failure to communicate its correction to the crew — were the true operating causes of the crash.

The Judge was unusually scathing about the corporation’s dissembling after the crash, theatrically describing the totality of Air New Zealand’s evidence as an “orchestrated litany of lies”. That meaty phrase has become part of the New Zealand canon.

The mountain continued to claim its victims. Air New Zealand’s combative chief executive Morrie Davies abruptly retired in the wake of the Mr. Justice Mahon’s report in, he claimed, “an attempt to remove a focus point from this current controversy and hasten the company’s recovery.”

The Royal Commissioner himself would be next in line. The airline sought judicial review of Mr Justice Mahon’s findings — especially his description of the corporation’s evidence as “an orchestrated litany of lies”. New Zealand’s Court of Appeal ruled that as the airline had not been offered an opportunity to respond to the allegations before they were published there had been a “breach of natural justice”. Rather anaemically, the Privy Council then upheld that ruling. Consequently, Mr. Justice Mahon felt forced to retire. Though he had been sharply criticised by the courts, he remained markedly popular with the New Zealand public until his death in 1986.

Even forty years after the disaster Air New Zealand was still too wounded to allow its then outgoing chief executive, Christopher Luxon — now the nation’s Prime Minister — to contribute to an excellent Radio New Zealand podcast about the episode.

In forty years, no-one took the “Apollo programme” approach, to accept that the system had failed, and that everyone who was responsible for the system had some share of the responsibility for the accident. The emphasis on apportioning blame on individuals was misplaced and counterproductive: damage had been done. What was required was to explain what had happened, to give a full account, to learn from it, and to honour the memories of the dead.

For a small country in the middle of the Southern Ocean, New Zealand has had its fair share of tragedies. By and large, it is good at remembering them and honouring those who fell. There are national memorials to the Ferry Wahine, the country’s major earthquakes and its occasional tragedies of human conflict. But, even forty-five years later, the only national memorial to New Zealand’s greatest peacetime tragedy is the decades of unseemly wrangling about who was responsible and some memorable judicial phraseology.

In 2017, Jacinda Ardern’s New Zealand Government committed to building a National Erebus Memorial. Eight years later, though a former Air New Zealand Chief Executive now holds Ardern’s old job, it has yet to be built. Perhaps that is the signature of that same old system glitch, still running through the Erebus affair. Perhaps it’s just a sign of our times. But in that same period of eight years, Gene Kranz’s Apollo Programme put a man on the moon.

Without a concrete memorial we have only Peter Mahon’s words, but they have an enduring resonance. Mahon understood the system dynamics at play, and turned a beautiful phrase. The closing paragraph of his Report of the Royal Commission, which came to him as he surveyed the crash site in Antarctica, captures perfectly the idea of the “normal accident”:

By a navigational error for which the air crew was not responsible, and about which they were uninformed, an aircraft had flown not into McMurdo Sound but into Lewis Bay, and there the elements of nature had so combined, at a fatal coincidence of time and place, to translate an administrative blunder in Auckland into an awesome disaster in Antarctica. Much has been written and said about the weather hazards of Antarctica, and how they may combine to create a spectacular but hostile terrain, but for my purposes the most definitive illustration of these hidden perils was the wreckage which lay on the mountain side below, showing how the forces of nature, if given the chance, can sometimes defeat the flawless technology of man. For the ultimate key to the tragedy lay here, in the white silence of Lewis Bay, the place to which the airliner had been unerringly guided by its micro-electronic navigation system, only to be destroyed, in clear air and without warning, by a malevolent trick of the polar light.

Thanks for reading! This post is public so feel free to share it.

Resources

Sorry for the lack of news recently — I’ve been tinkering around with a spanner in the underbelly of the great steampunk machine which is the JC. The website’s performance — how quickly it serves pages — had fallen off a cliff. It had even started refusing to serve pages at all. The more I looked into it the more I realised I had to do something. The problem turned out to be twofold:

Bots

Firstly, bots. There are a lot of random webscraping devices roaming the dark netherworlds of the Grid that just incessantly hit unsuspecting websites. They have always done this, but it is getting worse. There are good webscrapers — Google does this so that its search engines work — sort of bad ones — all the AI engines just harvest text for training material, which is kind of annoying — IP fundamentalists get very wounded about it — but it is also nice to think my bloviations might form part of the forthcoming memeplex forging a contingent path to our future in the digital simulacrum — and baaaaad ones — these come from China, mostly, and Russia, and God knows what they want with the JC but, until recently, they were making up about 90% of all traffic and slowing down everything for everyone else.

No more. Good webhosts can help with this, and JC now has measures in place to throttle these bots. One is just blocking traffic from China altogether. Sorry Chinese ISDA fans!

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Moi

Secondly, to put not too fine a point on it, me.

I have designed and built the JC in a decidedly evolutionary way. I got stuck with certain design decisions I made — or didn’t make — early on and that was that. It is like the famous XKCD cartoon:

This is a common feature of all big organisations of any longevity. At the heart of every investment bank, be assured there will be some Wang server kicking out ASCII code on an orange screen that is absolutely fundamental to the risk management systems, and which no-one dares remove for fear of unintended consequences. In smaller ways it is true of the JC. As I’ve gone along I have configured things in certain ways that seemed sensible at the time, but in hindsight not so much.

The nature of evolutionary adaptation is that ostensibly good decisions get sedimented into the architecture, overlaid with more and more stuff, and it gets harder and harder to go back and fix them because of all the knock-on consequences. Usually, it is not until much later that ostensibly good decisions reveal themselves to have been bad decisions.

Some were good decisions, once: whether a decision is good or bad is situational — we tend to forget this: hindsight conquers all. But unintended knock-on effects aren’t always obvious. They depend on how the future unfurls. (That figure of speech is wrong, by the way: it implies the future is already there waiting for is, like a rolled-up flag, and just needs to be revealed. But the future is not like that: it needs to be invented).

I have a piece upcoming about non-catastrophic failure modes in complex systems — “systems glitches” — that can lie latent in a system for a long time, and the longer they lie, the more they are normalised and start to look like normal functioning. The Post Office Horizon debacle is a good example. So, I think, is the Healthcare Serial Murder phenomenon. But there was one right under my nose.

This is also what happens when non-experts dabble. I am, first and foremost, a windbag: I am no web developer. And I made the choice a long time ago to use the MediaWiki platform, which is really good at being a knowhow wiki, but not very good at user rights management, founded as it is on the premise that information should be free, and the assumption that lots of people would be contributing and not just one.

I “solved” the problem of rights management by creating two wikis, and having them talk to each other. The premium wiki pulls all the free content from the main one, and then overlays it with extra juicy stuff. (This is mainly of interest to people involved in legal negotiation — especially ISDA — so if you are here for the Otto Büchstein’s cod Shakespearean melodrama fear not, you are not missing much. But if you want even more about Indemnifiable Taxes then, boy oh boy, is the Premium subscription for you!)

Anyway this two-wiki approach — which as far as I can tell, I invented by myself — has created some performance issues. Because requests to view one page creates all kinds of demands to load content from the other wiki. Because of the complex interplay of transcluded templates.

This is much more interesting than it sounds.

Templating

The thing that first sold me on Mediawiki is its really cool templating feature. Over my life parents, siblings, teachers, lecturers, bosses, children and most long-sufferingly, my spouse have railed and bemoaned my impulsiveness, lack of attention to detail, goldfish-like attention span, general slapdashery and a tendency to get easily carried away. (Also fecklessness and verbosity but they’re not important right now). I tend to be all over the place, as anyone who has tried to arrange a meeting with me more than ten minutes into the future will know.

Apparently, by the way, this is a classic trait of ENTPs. We also tend to build self-organising systems around ourselves and get into set routines, because force of habit is a great way of remembering to do things.

THERE I GO GETTING DISTRACTED AGAIN.

Mediawiki has this system of dynamic templating that is brilliant for building self-organising systems. I got really into them. They are what makes the site so apparently organised and so content-rich, but if you get too carried away with them they can become quite an albatross.

I got quite carried away.

Templates are essentially reusable gobbets of text. There is a different area on the wiki called the Template namespace where you can create little slugs of text, name them, save them down and then “call” them in the main pages of the wiki by writing their names inside {{double curly brackets}}.

At first blush this is just a time and labour saver: if I create a template called “QBF” with the text string, “the quick brown fox jumps over the lazy dog”, then I can “call” that text string on any page in the wiki by typing “{{QBF}}” and saving the page.

So if I type:

“my mother called me yesterday to tell me {{QBF}}.”

That would render as:

“my mother called me yesterday to tell me the quick brown fox jumps over the lazy dog.”

This is called “transcluding”. The term was coined by Ted Nelson in his 1982 book Literary Machines to describe the idea of a document including portions of another document by reference, rather than by copying the text.

Transcluding probably seems unremarkable. But it does two things: it saves keystrokes and it ensures consistency. Every time you get the same words, the same capitalisation, the same form: you can be confident it will always be “jumps” and not “jumped”. It saves you having to think about it. It compensates for a lack of attention to detail. It dramatically reduces faff. I have a template {{otto}}, for example, that returns the text “Otto Büchstein” because it is a pain in the arse remembering how to put the umlaut over the ü. And as all metal fans know, umlauts are a matter of, um, grave importance.

But there is more. Templates can have variables in them. Variables are designated by triple curly brackets. So I could set my {{QBF}} template up as follows:

“the quick brown {{{1}}} jumps over the lazy {{{2}}}”

Now if I “call” {{QBF|orang-utan|moon}} it will render as:

the quick brown orang-utan jumps over the lazy moon

I can quickly create a magical universe of all kinds of wonderful quick brown things jumping over lazy things. I can add more variables for the adjectives! There is a world of possibilities! I could rewrite the template as

The {{{1}}} {{{2}}} {{{3}}} {{{4}}} over the {{{5}}} {{{6}}}

But of course, at some point you might as well just not use a template. But as you can see, Mediawiki here is functioning as a symbol processor. It is a sort of Turing Machine.

Now, I dare say even that seems unremarkable. Get your kicks, JC!

But here is the thing: you can put templates within templates: you can have templates calling templates calling templates. So there is multilayered dimension here.

And you can put those variables inside template calls. This means you can use variables to tell the outer template which inner template to call.

So if my outer template calls a template (with double curly brackets) and I put a variable (triple curly brackets) in that inner template’s name then I can specify which inner template I call by the variable I put in the outer template call. This is conditional logic. We really are talking about Turing Machine style processing here.

Let’s say my template “{{clause}}” calls the text “{{isda {{{1}}}}}”.

If I type {{clause|1}}, the template will assemble {{isda 1}} — let’s say, that is the text of Clause 1 of the ISDA Master Agreement — and will return the text of that template. If I type {{clause|2}} it will assemble {{isda 2}} — Clause 2 of the ISDA — and will render that clause. If I have set up templates for all the clauses in the ISDA Master Agreement — and I have — then I have created a very easy way of pulling any clause I want on any page. The variables in the “outermost” template control which templates I want to render on the page.

This is very, very powerful. It means you can, basically, programme the wiki. There is essentially no limit to how complicated you could make this. This is what I did.

For example, the “owners manual” template, {{nman}} has only three variables (agreement, edition, clause) but calls twenty-two different inner templates, and some of those are dynamic and call their own inner templates. A single call, such as as {{nman|isda|2002|2(a)}}, may return as many as seventy or eighty subtemplates:

All of the components of this page are separate calls. The out template {{nman}} calls twenty-two different inner templates.

It all looks pretty good on the outsider — well, it does to me — and it makes creating new pages really easy: I just type {{nman|agreement type|edition|clause number}} and the whole page appears, with all the contextual material surrounding it, links to podcasts, further reading and so on — but to do all this the inner Turing Machine — Mediawiki as a symbol processing engine — is having to do an awful lot of work.

Now computers are good at accurately and quickly doing an awful lot of work, as we know. But there are good ways of doing a lot of work, and bad ways, and plainly my cack-handed system design over the generations was a stupefyingly bad way.

If that were not enough, when I set up the premium JC a couple of years ago, I created this barmy double wiki structure, so now there are a set of complicated “interwiki” calls between the main wiki and the premium wiki. Everything drives off a single page with a single call on it on the main wiki. When that call gets to the premium wiki, the underlying templates are subtly different and call on different templates, add different things in, thereby giving you lovely premium customers all that super extra content that you know and love.

But it also has to do something else: when it has imported wiki-linked text from the main site, it has to repoint those links to the equivalent pages in the premium wiki — otherwise, every time a user clicked on an imported link the user would be sent back to the main wiki.

This started causing problems

This was all working, but it was getting increasingly slow and unmanageable. For one thing, it was really hard to remember, or figure out, how the templates worked, and updating them was getting really hard — for example, when I started including links to podcast recordings on three different platforms.

And all of this was creating quite a lot of stress on the server. The two wikis were repeatedly pinging each other with API calls (whatever they are). All I could see was the bad performance — pages would hang or just take ages to load — and no obvious way to fix it.

It looked like, after 15 years of evolutionary decision-making, I had run out of road. Had I reached the limit of what I could achieve, given my decade-and-a-half breadcrumb trail of bad decisions.

There was also “key person risk”. If I got hit by a bus, it would be incredibly hard for anyone — even a Mediawiki specialist, and there aren’t many of those — to untangle what the hell I had done.

The JC had become a phantasmagoric, four-dimensional Heath Robinson machine. It was expiring under its own weight.

Enter the chatbot

And this is where Claude comes in. It could, I suppose, have been any large language model, but I have used Claude — especially Claude Code — a version that you install in a terminal environment which can directly see all the code you want to work on — and which you pay directly for in tokens.

I couldn’t ask a human mediawiki developer this stuff for fear of watching her pour gasoline on herself and set herself on fire. A virtual mediawiki developer might be less, er, volatile.

But are LLMs any good at developing Mediawiki sites? Well, it’s all relative, and Claude’s benchmark here is me, but my answer to this question is an unequivocal yes. Claude has been a game-changer.

Firstly I downloaded both sql databases, and the entire codebase for both wikis — this is gigabytes of information — and said, more or less, “help”.

Claude thought about it for a while then came up with nine suggestions, in order of bang-for-buck and complexity, that would help.

Simple things like caching webpages and automating links and audit between the the two sites. I am slowly working through them, but little by little the site’s performance has improved. You should notice that pages — especially on the premium site — load a lot quicker. Hope so!

The next phase is to do some actual content pruning: there are a number of offshoots, bright ideas, abandoned projects that I never got round to killing off. The total number of articles on the site should start to drop in the next couple of weeks.

Fear not, {{otto}}’s complete works are safe.

(See what I did there?)

The JC’s difference engine

Claude has helped with another component: “diffs”.

For a while there has been a feature comparing equivalent clauses in different editions of the same agreement — the idea being that it is useful to see how, say, Section 6(e) of the ISDA has evolved since the Children of the Woods thought of it all those untold generations ago. Ok: not the actual Children of the Woods — I didn’t go back as far as the 1985 Code — there are Burmese Junglers who would sooner give up than try to use that — but at least to the age of the First Men, those pioneers from the city of Salomon, who fired the One Agreement in the shadow of the Iron Mount.

I set up two-way comparisons between the three agreement types using Mediawiki’s inbuilt code comparison. Even with transcluded templates this was quite labour-intensive and very manual, for reasons I won’t bore you with, and they kept breaking and, every time the wiki updated, disappearing.

Enter Claude: he came up with some javascript that does it so much better and quicker — giving you a lightning quick, genuine redline, and, with the push of a button you can toggle between side-by side comparisons, and backwards and forwards comparisons.

I call it the JC’s “Difference Engine”. Cool, huh?

What is so surprising is how quickly you can create, essentially, a deltaview programme, that works better than an application you might pay hundreds of pounds a year for.

Doubtless there will be more improvements and enhancements to come as I think what else I can get Claude to do to help.

This all has cost me about £9 in LLM tokens so far. Seems to me like a good investment.

AI use case yes — but business case?

I expect this will be the enduring, and benign, use-case for AI. For now, we are in a new “wild frontier” where technology has escaped its garden and got into the hands of ordinary people, who can use it freely, creatively and really with minimal cost to improve their own lives, businesses (or make baby Trump videos — get your kicks). We normies don’t have the same outlook, much less agenda, as techbros, and it may well be (briefly) liberating for us to be able to commission technology programming to do what we think would be neat, rather than what a techbro thinks I think would be neat: they are quite different things.

History says this period of blissful frontier-like anarcho-syndicalism won’t last: rent-seekers, corporates, governments and systematisers will be along shortly to monopolise and monetise this opportunity, degrading the possibilities it offers an untethered imagination — so make the most of it while you have it.

Will it make the human race redundant? Hardly. Just as every technology before it, it extends and widens the range of things we can plausibly do. It will make possible things that would simply have been uneconomic, or infeasible, just too plain dreary to bother doing before. These new things will all create their own new opportunities, as they ever have done.

Playlist

Well, it had to be either seventies TV or — TRON. And with the magnificent news that not only is there a Tron sequel in the offing — here’s the JC’s review of the last one — but that Trent Reznor and the Nine Inch Nails have done the soundtrack and it is AWESOME. So without further ado here is - TRON ARES: ORIGINAL MOTION PICTURE SOUNDTRACK.

(For some reason this is not rendering in Chrome Browser. I hope you can see it. If you can’t , try clicking for the Spotify album here.)

Content should start to resume as I spend less time on on database reconfiguration. Happy autumn, folks!

JC

Thanks for reading! This post is public so feel free to share it.

Normal accidents

In 1999, organisational sociologist Charles Perrow wrote the masterly book Normal Accidents: Living with High-Risk Technologies. In it, Perrow proposed that certain complex human systems — the Three Mile Island nuclear power station was his prime example, but there were others — are so complex and the interaction of components so tightly coupled that certain modes of operation cannot be anticipated nor, if they happen, effectively stopped before they spin out of control. Therefore, from time to time, systems like this will suffer catastrophic failures. Meltdown at Three Mile Island was unavoidable. It was only a matter of when.

Thanks for reading! This post is public so feel free to share it.

Perrow called such failures “normal” accidents: they do not arise from error, malfunction or malice but from unexpected interactions of system components during normal operation. Normal accidents are an occupational hazard of running the system. Says Professor Perrow:

It is normal not in the sense of being frequent or being expected—indeed, neither is true, which is why we were so baffled by what went wrong. It is normal in the sense that it is an inherent property of the system to occasionally experience this interaction.

If you operate a system like that, you must accept that, even without anyone being seriously “at fault”, the system will occasionally fail in ways you cannot anticipate or, therefore, avoid: such failures are an emergent property of the system’s design:

Though the failures were trivial in themselves, and each one had a backup system, or redundant path to tread if the main one were blocked, the failures became serious when they interacted. It is the interaction of the multiple failures that explains the accident.

These failure modes are not true of simple systems.

Take a bicycle, for example. The “system” is a machine: it has clear boundaries. It is comprised of static components that cannot think, much less change themselves and that interact in linear ways. Absent component failure, we know exactly how a well-designed bicycle will behave. Even if its components fail, we still have a good idea how it will behave: the components fail in predictable ways, and with predictable, containable, outcomes.

But complex systems do not have clear boundaries. They are dynamic. They comprise multitudes — multitudes of autonomous, decision-making agents and volatile substances. Mutlitudes of complex subsystems. None can be delineated. None is static. These systems can, and usually do, change their own configuration over time and depending on a situation, without input from the designer.

Subsystems and agents (and subsystems of agents) “think” for themselves. They have lives of their own. They may behave irrationally or mistakenly. They may interact unpredictably. They are especially likely to do this in edge cases and at times of unusual stress.

These are the times when operators want the system’s designed-in fail-safes and backups to work — but also the time at which they are most likely not to: when they are most likely to impede safe operation.

Failure modes that can’t be anticipated can’t be avoided or designed out. Only once they happen and are recognised for what they are can the system can be redesigned to prevent them happening again.

Non-catastrophic normal accidents

In Normal Accidents, Professor Perrow focused on an unusual subcategory of normal operating modes that cause the catastrophic failure of the whole system they are part of — these are self-destructive failure modes. Examples such as nuclear power stations, airliners, chemical factories and financial institutions.

It is hard not to notice when your failure mode is catastrophic: there is usually a big crater where your system used to be.

But not all “normal” system failures are catastrophic. As long as it seems to be generating good outcomes, a system can be in a non-self-destructive failure mode indefinitely. System operators will happily continue to operate it.

And one of the prime features of a complex system is its “sorcerer’s apprentice” tendency to “misbehave” — to play up; to do something other than what the operator expected. The system theorist John Gall called a complex system’s tendency to antics “systemantics”.

We should expect complex systems to produce results that look acceptable while being subversively insidious. Such “latent” failures may go unrecognised for a long time. Until they are, they cannot be fixed. Until then they are liable to repeatedly throw off bad outcomes.

Asbestos as a case study

Asbestos is a naturally occurring mineral. Humans have used it at least since the Stone Age, but it came into widespread use during the industrial revolution, where its insulating and fire-retardant properties made it valuable. In encouraging its use, the “system” appeared to be functioning well.

By the beginning of the 20th Century the “negative health effects” of asbestos were becoming apparent: the first recognised death was in 1906, and “asbestosis” was first diagnosed as a formal illness in 1924. But its true danger was not fully appreciated. Regulations increased over the middle of the century, but asbestos was only finally banned in the 1980s.

Of course, the catastrophic health effects only manifest years after exposure. Once the health risks of deteriorating asbestos were fully realised its “latent failure mode” was obvious and asbestos was prohibited in new building projects removed, carefully, from existing structures.

Because the construction “system” did not immediately recognise the failure mode, it tolerated (and repeated) the accident. It was not a mistake: it was not an error. There was no malice: this was just an misunderstood bad outcome of the system’s operation.

But asbestos was a fairly central failing in the “construction-industrial complex”. Other “latent normal failure modes” may be peripheral. They may therefore lie dormant for long periods, providing apparently trouble-free system operation, before being triggered.

They may only be set off by unlikely interactions between usually isolated system components.

The Post Office Horizon scandal

The Post Office Horizon scandal is an instructive case in point. Partly following suspicion of endemic financial mismanagement in its branch network, in the late 1990s the Post Office introduced “Horizon”, a state-of-the-art computer accounting system built by Fujitsu, across its UK operation.

As the Horizon system was rolled out, it appeared to confirm management’s worst fears. Up and down the country, a pattern emerged of cash shortfalls in branch ledgers. Based on Fujitsu’s assurances that the system was robust, Post Office management concluded that Horizon data indicated widespread fraud among sub-postmasters managing local post office branches. That this conclusion was unintuitive as a matter of common sense — the sorts of people who act as sub-postmasters tend to be “pillars of the local community” and while there might exceptions, one would not expect sub-postmasters as a group to have common tendencies to fraud — Post Office managers preferred the data they were given and commenced prosecution and enforcement action.

Notwithstanding strident complaints from many sub-postmasters that the Horizon system was malfunctioning, the Post Office held its course. As early prosecutions succeeded, management’s early suspicions appeared vindicated and the Horizon system’s reliability validated. This made subsequent challenges to the system even harder. The sub-postmasters’ complaints increasingly fell upon deaf ears.

In the end, the Post Office prosecuted nearly one thousand sub-postmasters over fifteen years, imprisoning more than two hundred, and convicting and fining hundreds more.

Of course, much later, it turned out the Horizon system was at fault, just as the sub-postmasters had alleged. If not all then the overwhelming majority of prosecutions were outrageous miscarriages of justice.

Tellingly, in most cases the Post Office pursued private criminal prosecutions rather than referring matters to the police or the Crown Prosecution Service. These developed into their own cottage industry, involving teams of investigators, Fujitsu consultants, middle managers, in-house lawyers, as well as external solicitors and barristers. This was a complex system, itself composed of complex subsystems, each operating according to its own private priorities and defending its own interests. This prosecution system became opaque: it was so complex as to be impossible for any single actor to see the whole picture and get an appreciation of how it could be contriving bad outcomes:

Fujitsu employees were incentivised to suppress criticisms of the Horizon system that would put them in conflict with Fujitsu management.

In-house teams, acting on instructions from Post Office middle management, went out of their way to ensure they handled prosecutions expeditiously, shielding upper management from interactions with sub-postmasters regarded as “troublemakers”, and withholding from their management chain mounting evidence that the Horizon system was malfunctioning.

External advisors presented the Post Office’s interests as favourably as was possible in litigation, at times using tactics that stretched — but didn’t quite break — limits of acceptability, delaying and withholding material from defendants that was directly relevant to their cases.

Seen in isolation, on localised assumptions the postmasters were guilty, individual actions within the system were understandable, even if not entirely honourable. Each was insignificant in the wider scheme of things — it would be hard to point to one “bad apple” as being directly causative of a miscarriage — but that is what they caused in the aggregate.

Incentivised loose coupling and “system glitches”

Nothing is perfect, neither designs, equipment, procedures, operators, supplies, or the environment. Because we know this, we load our complex systems with safety devices in the form of buffers, redundancies, circuit breakers, alarms, bells, and whistles. Small failures go on continuously in the system since nothing is perfect, but the safety devices and the cunning of designers, and the wit and experience of the operating personnel, cope with them. Occasionally, however, two or more failures, none of them devastating in themselves in isolation, come together in unexpected ways and defeat the safety devices—the definition of a “normal accident” or system accident.

— Charles Perrow, Normal Accidents: Living with High-Risk Technologies

The Post Office Horizon scandal thus represents something different from what Charles Perrow had in mind: here, the “system” was loosely coupled: many of the independent agents and gatekeepers in the system did have time and opportunity to intervene — the process unfolded over decades — but, because of their incentives and limited view of the broader picture, didn’t. Rather, information emanating from the different parts of the system, all conditioned as it was by incentives, narratives and biases at play had the effect of reinforcing existing preconceptions.

There was a loosely coupled chain-nonreaction here: Because one gatekeeper didn’t intervene, other similarly unsighted gatekeepers took it as read that the coast was clear, everything was in order and so, notwithstanding any of their own scruples, they didn’t need to intervene either. Indeed, doing so might pose more questions than it answered. It might be career-limiting behaviour. The safest course was stick to the original instructions.

Important point: no malice, skulduggery nor conspiracy was required for this outcome. Indeed, that is what is so insidious about the process: each step seems so innocuous. There are no red flags that anything is wrong: the system appears to be functioning normally. If anyone were acting with obvious malice, others in the system would quickly recognise it and adopt a more critical disposition. They may — incentives permitting — even call it out, though history tells us whistle-blowers are routinely ignored: the ruins of many a broken empire are littered with prescient warnings disregarded.

So, terminology check: If a “system accident”, in Professor Perrow’s sense, is “a catastrophic implosion caused by an unexpected chain reaction of tightly-coupled components”, perhaps we should call these latent “non-catastrophic defects in a complex system caused by unexpected non-interactions between incentivised loosely-coupled components” something else: I suggest “system glitches”.

The failure mode here arose not because of uncontrollable tight-coupling, but what you might call “incentivised loose coupling”: where each subsystem’s decision-making discretions were shaped by a limited, and pre-coloured, view of the whole picture. Where needed interventions were suspended by institutional pressures, incentives and inadvertent information filtering processes bearing upon the wider system.

“Incentivised loose coupling” is insidious because the system accidents it creates are often the product of omission as much as commission. They seem preventable: indeed, the system appears designed to prevent exactly the scenarios that arise: it just doesn’t. Many of the affected gatekeepers are only there in the first place as fail-safes to stop this kind of thing happening.

To be sure, there are some misapprehensions here. Executives commonly suppose the inhouse legal team operates, at some level, as the business’s conscience. But not being in the “operational stack”, the legal department does not see the daily flow of business — it is involved by exception — so is poorly positioned to fulfil this role. In any case, delegating business judgment to legal would send a terrible message to the front office: that ensuring prudent business practice was somebody else’s problem.

In any case: any of these gatekeepers could have, and had the system worked as intended, should have stopped the bad outcome. But the system wasn’t out of control the way a melting-down reactor might be. Instead, the system remained defiantly, inexorably, insistently, perversely in control: it was just latently misbehaving. This was classic “systemantics”.

Indeed, it was not the speed but the glacial slowness of the failure that was critical here: the Post Office Horizon debâcle unfolded over fifteen years. Had it happened over a weekend, someone might have noticed something was wrong.

Somebody Else’s Problem fields

“An SEP,” he said, “is something that we can’t see, or don’t see, or our brain doesn’t let us see, because we think that it’s somebody else’s problem. That’s what SEP means. Somebody Else’s Problem. The brain just edits it out, it’s like a blind spot. If you look at it directly you won’t see it unless you know precisely what it is. Your only hope is to catch it by surprise out of the corner of your eye.”

― Douglas Adams, Life, the Universe, and Everything (1982)

Slow-moving system accidents can be hard to spot in a large commercial firm: given the short half-life of role occupancy — like sharks, modern executives are compelled to keep moving, and good ones tend not to be in situ long enough to identify (or take the rap for) slow-burning errors — and the short timeframes of corporate decision-making — success is measured in quarters, not decades — undertakings like the Horizon prosecutions would have seemed almost stationary, and fully controlled, making it harder to recognise them as latent system failures.

Individuals could have been — but likely will not be — forgiven for assuming that, if something was wrong with the information they were being given, somebody else would have surely picked it up. It was, as Douglas Adams put it, “somebody else’s problem”.

The Post Office Horizon Inquiry ran for over three years. It has yet to report. The oddly unsatisfying evidence of the dozens of gatekeepers called as witnesses — Post Office and Fujitsu executives, middle management, investigators, engineers, inhouse lawyers up and down the chain and external lawyers — illustrates the folly of looking for human causes of such a latent system failure. Lots and lots of “SEPs” contributed to the Post Office Horizon debâcle.

Every one of the subsystems worked, within tolerances, according to its own rules of engagement. Any shortcuts and variances that individuals took — overlooking technical glitches and assuming certain fact patterns in cases of ambiguity — were justifiably-intentioned exercises of “wit and experience”, albeit coloured by the priorities and interests driving those individuals.

No particular malice or intentional foul play was needed among any of the operators. No-one privately believed they were prosecuting innocent sub-postmasters.

There are, I dare say, “somebody else’s problem fields” just like these in every organisation on the planet.

There is one other thing to say. Unlike the tightly-coupled normal accidents Professor Perrow had in mind — which, by definition, are extremely rare “tail events”, unprecedented before they happen, painfully obvious when they do, and which tend to “resolve” themselves, by self-destruction, as soon as they occur — loosely-coupled latent system failures need not be rare — see asbestos — by definition will not be obvious and will tend to continue indefinitely until someone notices them.

Latent failures are therefore inherently more likely than catastrophic failures.

So what should we look for if we wanted to find some?

Next time, in our crime and punishment thread: a proposed case of latent system error: the healthcare serial murder cases. Isn’t it a bit weird that so many serial murders are doing exactly the same thing?

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Radio: live transmission

You may have noticed in recent months that a few JC newsletters have been breaking out into audio format. I’m on Spotify. I won’t deny this has been a personal ambition for a while now. With Substack, it is surprisingly easy to go nationwide.

My personal preference for “discretionary content” these days — how I read for pleasure, that is — is audio: like everyone I spend all the time at work reading, usually off a screen, my eyes are going bung, so for pleasure I would much rather listen to audiobooks, and podcasts and find it increasingly exasperating when authors do not publish recorded versions of their material.

By the way: if you find a narrator as good as Martin Jarvis (e.g., Oliver Twist), Wanda McCaddon (e.g., The Guns of August), or David Horovitch (e.g., The Good Soldier Svejk), the dusty old classics come to life.

I suppose it is a sign of our attention-depleted times that sitting down quietly to read a text is impossible with all the distractions. In my case, it tends to send me to sleep.

Also, audio is the most economical way of consuming literature. No doubt this comes at great cost to budding authors, but Audible’s policy of selling any book on its site for one £6 credit is a pretty hard value to pass up.

So, I started recording a few JC newsletters.

I find it is a useful discipline, reading a newsletter out: you quickly work out what works, what doesn’t, and when your logic doesn’t flow — which is quite often, in my case.

I have some decent recording kit, too, so there is scope for a bit of fun with mixing and so on, though it is still quite the time-sink if, as I had suspected, no one is going to be listening to me drone on in my nasal mid-Pacific accent.

So imagine my surprise when recently a reader told me not only did he listen to the recorded versions, but he put it on his car and his wife had to listen to it too!

“I also quite like the podcast format and often listen to them when I am driving (sometimes against the wishes of my wife, who is my usual passenger)”

I am so sorry — all I can say is, you poor thing!

Aside from inflicting my droning on your spouses, loved ones and pets, I thought I should ask — does anyone else prefer the podcast format?

If so, I will lean into it a bit more. I have a vague plan to do some more free-form sessions about the key clauses of the ISDA — not so much read-outs but discussions around the points in premium articles — but if this is how folks best enjoy content, I will change my approach. Do let me know, either in the comments or by DM or whatever.

This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber.

Fictitious cases, AI, negotiable cows and access to justice

“Was the cow crossed?”

“No, your worship, it was an open cow.”

These and similar passages provoked laughter at Bow Street to-day when the Negotiable Cow case was concluded.

—Board of Inland Revenue v Haddock (1932)

In Messing v. Bank of America N.A., 792 A.2d 312, a case about presentation of a bank check in 2002, the seven-seat Court of Appeals in Maryland acknowledged the celebrated English case of Board of Inland Revenue v Haddock (1935):

In that case, the protagonist, Mr. Haddock, after some dispute involving uncollected income-taxes owed, elected to test the limits of the law of checks as it existed at British common law at the time. Operating on the proposition that a check was only an order to a bank to pay money to the person in possession of the check or a person named on the check, and observing that there was nothing in statute or custom at the time specifying that a check must be written on paper of certain dimensions, or even paper at all, Haddock elected to tender payment to the tax collector by a check written on the back of a cow.

When Mr. Haddock was then arrested in Trafalgar Square for causing an obstruction, and summoned by the revenue for non-payment of income tax, Mr. Haddock said it was “a nice thing if in the heart of the commercial capital of the world a man could not convey a negotiable instrument down the street without being arrested,” and instituted procedings against the metropolitan constabulary for false imprisonment.

So the court held:

It cannot be unlawful to conduct a cow through the London streets. The horse, at present time a much less useful animal, constantly appears in those streets without protest, and the motor-car, more unnatural and unattractive still, is more numerous than either animal. Much less can the cow be recorded as an improper or unlawful companion when it is invested (as I have shown) with all the dignity of a bill of exchange.

Board of Inland Revenue v Haddock is, of course, fictitious: it was written by the great humorist A.P. Herbert for Punch magazine. You can now find it in collections such as Uncommon Law: 66 Misleading Cases, a volume which every student of the law should have on her shelf.

In any case, judicial recognition of disputes that illuminate the fundamental essence of things, even if they are imaginary, is hardly new. It scarcely matters whether the Court knows they are imaginary — the Court in the Messing case was careful to state that it did — it is surely the underlying legal principle that is important: a merchant should be able to bear a negotiable instrument in the city of London without undue harassment.

And so we come to the recent news that a legal firm has been caught out in its submissions for judicial review of a local authority’s decision — JC’s own local authority the London Borough of Haringey, as it happens — not to grant accommodation to a homeless applicant.

In a fulsome and legally correct submission, the applicant’s counsel cited a number of judgments by way of precedent, all of which were made up.

Now: this was not malicious but, most likely, a “hallucination” by a large language model, which the lawyer used to assist in her research. The lawyer used ChatGPT, or something like it. (The barrister has not formally admitted this, and seems to have dissembled about it when challenged in Court, but it seems pretty obvious this is what went on.)

However, the finding which I can make and do make is that Ms Forey put a completely fake case in her submissions. That much was admitted. It is such a professional shame. The submission was a good one. The medical evidence was strong. The ground was potentially good. Why put a fake case in?

The response from the profession here has been predictable and reactionary: the lawyer has been roundly castigated, and the case taken to sound a warning as to the dangers of artificial intelligence. There have been many thought pieces along these lines, along with some fairly unsparing public criticism by the judge in the case.

Commentators are as disapproving as they are baffled: why invent these cases when, in fact, they were advanced to support arguments that were in any case correct, and could be found in genuine authorities?

There’s a simple answer to this: researching by LLM is pretty good, is getting better, and it is a lot quicker than manual research of databases to which a young lawyer might not have access in any case.

In a profession costed and billed on a time-and-attendance basis, AI is not just a smart choice: if you want research, it is the only choice for a customer on a budget, as presumably a homeless man being represented by a pro bono law centre would have been.

It is a good choice, in fact, for any customer on any budget. It is not like, these days, a buyer of legal services has any choice: if you believe Magic Circle firms do not use LLMs to support every part of their operation every day, I have a bridge to sell you.

To say, as some learned commentators have, that “AI should not be used for such serious legal research—that is, the legal research on which others will rely” is absurdly Luddite: about as sensible as warning a lawyer off Google.

If the proposed alternative is to trawl through online data sources, case reporting services and law reports and texts, do not expect that the case of a homeless man represented by a pro-bono community law centre will be researched at all.

The answer is, as with any research tool, to use it with care, experience and in conjunction with other sources.

Part of the problem is the poor public access to decided cases. Given that decided cases form part of the common law — they are a primary source for the law we are all deemed to know — ignorantia juris non excusat — it is a matter of some scandal that those cases are not all available, in full, in a catalogued database. Fewer than half of all reported decisions make it onto the free British and Irish Legal Information Institute (BAILII) website:

There is a restriction on the number of English cases from Divisions of the High Court which can be added to the BAILII database which arises from the fact that the shorthand-writers who transcribe judgments which have been given verbally (as opposed to judgments handed down on paper) may own the copyright in the transcribed version of the judgment. Copies of judgments in published law reports may also be subject to copyright. This prevents the judgment being added to the BAILII database without the consent of the shorthand-writer or law report publisher. BAILII, being a free web site, has no funds with which to acquire a licence to copy and display the transcripts. BAILII continues to press for the system to be changed so that all judgments may be freely available.

This makes it hard for a busy young lawyer to check case references. The law of the land should not be under copyright. That it is, and that the profession is not more bothered about it than it seems to be, is something we should be agitating about.

No injustice was done. The court was not, substantively, misled.

LLMs are a fact of life. We should be more constructive about working out how they can be put to use, bringing research and legal analysis to people who have, historically, not had it. The legal teams have been caught out here, but the profession’s reaction — disciplinary procedure — seems counterproductive. That the case involves those representing a homeless man on legal aid makes the profession’s reaction all the more regrettable.

Lucy Letby

My fascination with the legal and systems issues thrown up by this case continues. Nothing quite ready to push publish on yet, but here are some of the issues:

Intent, motive and motivation: it is trite law that you do not need a prove a motive to convict for murder, but where the evidence that there has been a murder in the first place is ambivalent, doesn’t the lack of any motive, motivation, or even a pre-identified disposition to similar murderous acts weigh quite heavily against the prosecution?

Was it legalese wot did it? I have access to transcripts of key legal submissions in the case: in particular, defence applications for further disclosure of how charges were selected, seeking orders that the evidence of the prosecution’s lead witness be excluded (or that he be declared hostile) and the Defence’s final submissions to the jury. One thing that strikes me is that the barristerial language — and length — in which these submissions were rendered might have made it all a bit hard to follow, even for the judge.

Conspiracy theories: Is anyone making up conspiracy theories? If so, who? And what is a conspiracy theory anyway, and how do you tell them apart from real theories?

Eyes peeled on the Jolly Contrarian Crime and Punishment channel for that one.

Law Commission consultation: criminal appeals

For anyone generally interested in the criminal justice procedure — how could you not be, as a lawyer, after the Post Office Horizon scandal and the LIBOR appeals? — The Law Commission is engaged in a wide-ranging consultation on the criminal appeals procedure, including the Criminal Cases Review Commission and, I suspect, expert evidence (even though it has published and been ignored on that subject in the last decade).

They have made it very easy to engage with the consultation at whatever level of detail you would like to.

Here is where you can do that.

Go well!

Thanks for reading! This post is public so feel free to share it.