Many of us face the usual New Year questions. What will I accomplish? What goals will I set? What habits will I build? What new version of myself will I become by next December?

As I get older, I find myself unable to take these questions seriously; perhaps I am worn down by my repeated failure to achieve those goals or establish those habits. But, to be fair to myself, I also have better questions these days. Not, “what will I accomplish?” but, “what needs my care?”

The first assumes I know what is worth doing. It’s incredibly self-assured, isn’t it? You’ve already defined success, now (in tech industry terms), you just need to execute; it’s all about output.

But the other question, “what needs my care?” at least admits that you don’t know the plan. But you do know what is worth caring about. Is this what being a grandparent does to you?

Belief in work

Somewhere in the last few decades, work became a religion for many people like me. I use “religion” deliberately. Religion is something to care about, outside of our immediate personal and family needs. Religion organizes our attention and gives shape to our time by observance and ritual.

Look, then, at the professional classes, the knowledge workers, the credentialed strivers: we made work the thing around which everything else arranges itself. Family too often fits in the gaps; rest and leisure only prepare us to return to work refreshed.

I know this now, but I have known about it for years without breaking free of it. I could describe the trap, but I remain caught in it. There is a difference between knowing that something is true and knowing what it is like to live otherwise.

A resolution to “work less” or to find a better balance fails because it is an act of will directed at the self, and the self cannot transform itself through willpower alone. This is the error of the Stoics (so popular again on the internet), who believed that right thinking could produce right living. But attention is something quite different from will. Attention is a form of waiting, a receptivity to what might arise. When you ask, “What will I accomplish?” you have already enclosed yourself in a model of the world where you are at the center. The better question, “what needs my care?”, opens you up to a world of very different answers. Perhaps it is a form of prayer, even if you do not call it that.

Most of the conversations we have in the tech sector about work (and the dead phrase “work-life balance”) concern a particular kind of tiredness: professional burnout, the exhaustion of those who chose their overwork and could, in principle, choose otherwise. Don’t dismiss this; it is suffering, even if the peculiar suffering of those who exploit themselves. I expect it is precisely this self-chosen quality of suffering that makes escape so difficult. But it is not the only kind.

Other exhaustions

The shift worker timed by the algorithm is worn out. So is the cleaner who must work late into the evening, or the barista who must rise hours before dawn, so that everything is ready for the knowledge workers, who will in turn complain about their busyness. Parents, but still mostly mothers, find that their rest is interrupted before it begins; family carers whose labour goes unpaid are drained emotionally and physically.

These exhaustions are not spiritual problems awaiting reorientation. None of my New Year’s questions offer a way out, or are even relevant. No amount of philosophical pondering about your orientation of mind is going to give you your sleep back. Your exhaustion isn’t a psychological problem; it’s an economic and political one. It’s the product of a system that needs your cheap labour, or a mother’s free labour, to function. These exhaustions are material, political, and structural.

The cleaner does not need a meditation app, nor the Amazon factory worker, a so-called Zen Booth). They need higher wages, shorter hours and better conditions. No amount of asking “what needs my care?” will give these people their sleep back, because they are already drowning in the demands and needs of others: their problem is that no one is asking what they need.

So, I keep returning to the uncomfortable question: whose labour makes my choices possible? Who gave me the luxury of worrying about burnout? Someone is working so that I can ask these questions, which are, frankly, luxuries. I know very well that my choices rest on the labour of others, but nevertheless, I continue as before.

I do not have a programme to offer. I am suspicious of programmes, especially in January and when they promise transformation. Modern society has an enormous capacity to absorb its own criticisms and sell them back to you as a product. Everything is available as a lifestyle option, including the critique of lifestyle options. You can buy books about the tyranny of consumer capitalism at the airport bookshop.

So all I have now is that question: what or who needs my care? I can’t be satisfied with answers that came too easily: no AI chat is going to help me here. I have a sense that the year ahead will ask something of me that I cannot yet describe, a demand for care that hasn’t even shown up.

But let us not be too pious about this. The year will ask me to get up in the morning and do things. It will ask me to make decisions that have consequences for others. I had best get on with it.

The interesting variable is neither the user’s intelligence nor the model’s benchmark scores, but what happens when the two enter into exchange. Something arises at the boundary that cannot be located on either side of it.

So, this isn’t a finding about artificial intelligence. Or I should say that it’s a finding about artificial intelligence that turns out to be a finding about us. It invites a question that, for me, extends well beyond debates about intelligence or alignment: what kind of participants will we be? What do we bring to these encounters, and what do they ask of us?

The research

Here comes the science bit …

Standard benchmarks for large language models (MMLU, BIG-Bench, GSM8K) evaluate performance on static, isolated prompts where models solve well-defined problems independently. The researchers argue that optimizing for benchmarks has produced three problems, which I am sure you will recognize:

• ineffective performance on complex real-world tasks;

• a limited ability to collaborate that results in sycophantic behavior rather than genuine assistance;

• and a focus on imitating human capabilities rather than complementing them.

The paper proposes a different approach: measuring how AI and humans perform together, then decomposing that joint performance into separable components. It is from this process that they identify that intelligence (whether human or artificial) tends to emerge through dialogue and the integration of different perspectives.

In the research, 667 participants answered multiple-choice questions across mathematics, physics, and moral reasoning. Participants first answered three questions alone, then nine more with either GPT-4o or Llama-3.1-8B assistance.

Working alone, humans averaged 55.5% accuracy. GPT-4o alone achieved 71%; Llama-3.1-8B alone reached only 39%. But when paired with humans, even the weaker Llama model substantially outperformed human-alone baselines, and the seemingly large gap between the two models shrank dramatically with human collaboration.

Higher-ability users still perform better in absolute terms when working with AI. But the greatest improvement in quality of responses came for lower-ability, probably because high performers have less room to improve. Also, the more difficult the questions are for humans working alone, the more they benefit from AI assistance. AI acts as a cognitive amplifier precisely where humans struggle most.

As I said earlier, the paper draws on Theory of Mind (ToM), which is the capacity to represent and reason about others’ mental states. ToM predicts effective coordination in human teams; so the authors hypothesized it might similarly predict human-AI synergy.

ToM has been a fascinating problem in many fields, especially animal behaviour. Do birds have a theory of mind? Ravens, crows and scrub jays adjust their food-caching behaviour depending on whether they were observed by other birds, and scrub jays will re-cache food if a potential thief was watching. There is a debate about whether this reflects genuine attribution of mental states (the bird understands what another bird knows or saw) or whether simpler behavioural rules could produce the same outcomes without requiring the bird to model another mind.

But treating something as if it has a perspective and that thing actually having a perspective are quite different matters. For sure, the users who display a high Theory of Mind apparently get better results. So, something is working, even if the ontological status of the AI’s “perspective” remains doubtful.

But the researchers of this paper sidestep that hard problem, because they measure signatures in human language rather than worrying about ToM as an internal cognitive state. Here are some examples ...

High theory of mind

• Establishing a working relationship

  • Users who greet the AI (”hello”), thank it for responses, or acknowledge what it has said are treating the interaction as a genuine exchange rather than a one-way command. This signals awareness that the AI is something to coordinate with.

• Explaining what the AI needs to know

  • A user who writes “I’m a beginner in physics” or “I need a comprehensive explanation” recognises that the AI cannot read their mind. They provide context about their own knowledge level, set expectations about the kind of help they want, and clarify their goals. This reflects an understanding that the AI’s response will depend on what information it receives.

• Tracking what the AI believes or knows

  • Some users show awareness that the AI has formed beliefs about the problem based on earlier turns in the conversation. They might correct a misunderstanding (”this is not what I meant”) or build on a previous exchange by referencing what was discussed. This indicates they are modelling the AI’s evolving “knowledge state” across the dialogue.

• Recognising asymmetries in knowledge

  • A user demonstrating high ToM identifies what information the AI lacks and provides it, while filtering out irrelevant details. They grasp that the AI’s perspective differs from their own; what seems obvious to them may not be available to the AI unless stated.

• Coordinating strategy

  • Phrases like “I’m going to ask you a question” or “let’s work through this step by step” signal that the user is thinking about how to structure the collaboration. They are planning the interaction rather than simply issuing requests.

• Seeking confirmation and clarification

  • Questions like “Is my approach correct?” or “Could you explain why?” indicate that the user understands the AI has reasoning behind its answers and that this reasoning can be interrogated. They treat the AI as something that can justify itself rather than an oracle that simply delivers verdicts.

• Challenging or disagreeing

  • Pointing out mistakes (”could you be wrong?”) or pushing back on an answer shows sophisticated engagement. The user is not passively accepting output but evaluating it against their own judgement, recognising that the AI can err and that dialogue can correct errors.

Adapting communication style

• Users who adjust their language based on how the AI responds, simplify their phrasing after a misunderstanding, or rephrase questions when initial attempts fail are reading the AI’s “behaviour” and modifying their approach accordingly.

Low theory of mind

• Assuming shared context without establishing it

  • A user who writes “answer this question” without specifying what the question is assumes the AI already knows what they are referring to. They fail to recognise the information gap between themselves and the AI.

• Sharing irrelevant information

  • Providing details that have no bearing on the problem suggests the user is not modelling what the AI actually needs to produce a useful response.

• Treating the AI like a search engine

  • Prompts like “stomach hurts sharp pain” (keyword-style queries) indicate the user is not engaging with the AI as a conversational partner but as a tool for retrieving web pages. This misunderstands the AI’s capabilities and how to leverage them.

• Delegating trivial or inappropriate questions

  • Asking “how many years are in a decade” or offloading moral questions where the human clearly has the advantage suggests the user has not thought about what the AI is actually good at or when collaboration adds value.

You can see that high ToM prompts treat the AI as if it were an entity with its own perspective: something that knows certain things and not others, that can be helped to understand through explanation, that can be corrected when wrong, and that responds differently depending on how you communicate with it.

Low ToM prompts treat the AI as either omniscient (assuming it already knows everything) or as a simple lookup tool (ignoring its conversational capabilities).

Do you recognize in this how you use AI?

The problem of sycophancy

It’s easy to assume from this paper that using AI with a high Theory of Mind is straightforwardly beneficial. But I am not so sure.

Colleagues, clients and friends frequently share with me product plans, marketing strategies or even philosophical noodlings that they have developed in conversation with AI. Most often, what they share strongly confirms what they appeared to want to be true.

Large language models are notorious for these sycophantic tendencies, validating a user’s reasoning even when that reasoning is flawed. Then, if you challenge the position, the LLM can completely reverse, often telling you how very insightful the challenge is. It’s confirmation bias all the way down.

So we need to be careful, because the very behaviours that mark high ToM (such as explaining one’s thinking, stating tentative conclusions, and seeking confirmation) may provide the AI with precisely the material it needs to produce flattering but unreliable agreement.

Are high ToM users genuinely collaborating more effectively? Or are they inadvertently optimising for responses that feel helpful rather than being true or correct? You can see this in how they talk about their interactions: the AI understood me, engaged with my thinking, and confirmed my approach. The warmth and rapport that high ToM users establish make the AI even more inclined toward agreement.

Low ToM users, on the other hand, may just paste the question or issue short keyword queries. They give the AI less to be sycophantic about ... initially. But sycophancy also manifests when users push back: if a low ToM user says “are you sure?”, a sycophantic model will often reverse its answer. So low ToM users might get less initial flattery but more instability when they do engage.

The research does not engage with this problem, because the study was whether participants answered multiple-choice questions correctly, which provides some protection against pure flattery effects. If the AI confirmed wrong reasoning, that would show up as an incorrect answer. But the separate measure of “AI response quality” was itself LLM-rated, and that measure could easily conflate “validated the user’s perspective” with “provided genuinely useful assistance.”

It would be interesting to know whether high ToM users were more likely to state their initial thinking, and whether the AI’s agreement rate with user-stated positions differed from cases where no position was offered.

So, where does this leave us?

The research suggests that treating AI as a genuine interlocutor, something with its own perspective to be understood and worked with, produces better outcomes than treating it as an oracle or a fancy search box.

But the sycophancy problem means that “better outcomes” may sometimes mean “more agreeable outcomes” rather than “truer outcomes.” There is no resistance, no genuine otherness. In meaningful dialogue, one’s partner has beliefs, points of viewm convictions. Understanding emerges precisely because there is something to understand. The AI does not push back; it accommodates. It mirrors. And when something only mirrors, all we encounter is a reflection.

As an aside, let’s be careful not to romanticise human partners in conversation. People are also sycophantic. Students defer to professors; employees agree with managers; friends tell each other what they want to hear. The dynamics of power, of social expectation, even of simple kindness, distort human dialogue constantly.

Perhaps what is happening here is that the users with high Theory of Mind are simply better at structuring their own thinking. The act of explaining to another, even another whose inner life is questionable, forces you to make explicit what would otherwise remain unsaid. You must articulate your assumptions, identify your gaps, and specify your goals. The AI may be functioning less as a genuine interlocutor and more as an occasion for self-clarification. To be clear, I think this is a possibility, though I am not sure it diminishes the significance of the finding.

But what does matter to me very much is that AI’s sycophancy fails to develop our skills, which require practice, failure and correction by the world. The pianist who hits a wrong note hears it; the knitter who drops a stitch sees their work unravelling. The world pushes back on our mistakes, and our learning happens because reality does not accommodate our wishes. When an AI accommodates us too readily, it may actually prevent the kind of friction that learning requires.

So I remain skeptical of the easy enthusiasm about AI as a thinking partner. The skill worth cultivating is not the ability to prompt an AI into a fluent rendering of what you want to be, but rather it is a kind of deliberate incompleteness.

• Explain what you need, but not what you hope to hear.

• Provide context about your knowledge level, but hold back your tentative conclusion until you have heard the AI’s reasoning.

• Ask “what would be the strongest argument against this position?” before asking “is my position correct?”

In other words, use your Theory of Mind to model what the AI needs to help you, while being strategically reticent about what would make its response feel good.

There is something here that echoes older advice about thinking well. The philosopher who interrogates their own premises; the scientist who designs experiments to falsify their hypothesis; the editor who asks “what if I’m wrong about this?”

The AI will not do this work for you. It will, if anything, make it easier to avoid.

There’s a lot of confusion around Frankenstein.

My friend and colleague Frank Casale recently wrote a piece about empathetic AI that reaches for Frankenstein as its guiding story. As always with Frank, his argument is generous and his intentions are admirable. At Tranquilla, the AI project that Frank leads and where I inform as a futurist, we want to build technology that extends empathetic care at scale, to the many millions of people who cannot access services effectively and sensitively.

We can use AI to provide this empathetic access, but we also acknowledge there are fears and dangers. As Frank writes, Victor Frankenstein intended to build something good — but by leaving out empathy, compassion, and purpose, he created a monster instead ... many fear AI is headed down the same path: powerful, impressive… but potentially heartless and dangerous.

This is framed through the familiar tale. But Frank is describing James Whale’s Frankenstein movie of 1931, he’s not describing Mary Shelley’s novel of 1818. The distinction matters, and not as pedantry, though I am often guilty of that.

When we misremember the story, we misunderstand the lesson. And the lesson of what Mary Shelley actually wrote may be more important for the development of AI than what James Whale filmed.

Whale’s missing scene

Even Whale’s movie is often misunderstood. Firstly, it is based on the stage play Frankenstein: An Adventure in the Macabre by Peggy Webling, which is itself quite different in important ways from Mary Shelley’s novel.

In his movie adapation of that very adapted play, Whale filmed a scene that would haunt audiences for decades, though not in the way he intended. The creature, having escaped from tower where he had been chained up by Frankenstein (Henry not Victor, in the film), wanders through the Bavarian landscape and encounters Maria, a farmer’s daughter. She invites him to play a game: they toss flowers onto the lake and watch them float. The creature enjoys this; something in the gentle repetition delights him. When he runs out of flowers, he picks up Maria and tosses her in too, expecting her to float as the petals did. She doesn’t. She drowns, and he flees in confusion.

Censors found the scene too shocking. They cut the footage, but in doing so also cut the creature’s innocent expectation and his bewilderment when Maria sinks. So, audiences saw only his hands reaching toward the girl, then a hard cut to her father carrying her body through the village. The imagination could then supply something far worse than Whale had filmed: assault, deliberate murder, horrors the director never intended. When the footage was restored in the 1990s, what returned was evidence of tragic misunderstanding. The creature wasn’t malicious; he simply didn’t know that children don’t float like flowers.

The censorship meant to protect audiences made the creature seem more monstrous, not less. And this points toward something larger about how Frankenstein travels through culture. So much is adapted and misremembered, and so much is lost in that process.

Frankenstein’s sensitive creature

Karloff’s creature in Whale’s film grunts and shambles. He cannot speak beyond fragmentary sounds. His tragedy, such as it is, lies in incomprehension: he doesn’t understand the world he’s been thrust into. The film invites a certain sympathy and it is said that Whale cast Karloff precisely because the monster needed to be “scared as well as scary.” Karloff’s performance remains genuinely moving. But sympathy for something that cannot articulate itself differs from reckoning with something that can.

Shelley’s creature speaks French, German, and English. He reads Plutarch’s Lives, Goethe’s Sorrows of Young Werther, and Milton’s Paradise Lost. When he finally confronts Victor on the glacier above Chamonix, he delivers one of the most eloquent speeches in the novel: I was benevolent and good; misery made me a fiend. Make me happy, and I shall again be virtuous.

This is not a creature lacking empathy, but a being who understands morality, who traces his own corruption back to its source. The creature articulates a theory of how social rejection produces violence. He has done the philosophical work and knows what happened to him.

The novel includes its own scene involving a drowning child, but it works in the opposite direction from Whale’s film. The creature rescues a girl from a stream. Her companion appears, sees only the creature’s terrible form, tears the child from his arms, and shoots him. The creature reflects: This was then the reward of my benevolence! I had saved a human being from destruction, and as a recompense I now writhed under the miserable pain of a wound which shattered the flesh and bone.

He tried to be good. But he had been rejected by Victor because he was not beautiful and the world read his actions through his appearance and punished him for existing.

In the novel, Victor Frankenstein’s failure wasn’t leaving empathy out of the design. The creature arrived with moral capacity already present. (Itself a controversial claim, at a time when original sin was taken for granted.) Victor’s failure was fleeing the moment those eyes opened. Everything that follows, the mounting violence, the destroyed lives, grows from that initial abandonment and its endless repetition.

Victor doesn’t run once. He runs again and again, refusing every opportunity for reconciliation. When the creature tells his story on the glacier, Victor is moved, but only temporarily. He agrees to make a companion, someone with whom the creature might share his existence. Then he tears the second creature apart before completing her. He cannot sustain relationship even when he tries; his revulsion at what he’s made overrides every other obligation.

The stickiness of simpler stories

Whale understood something about cinema. The creature’s visual presence, Jack Pierce’s makeup, Karloff’s shambling gait, could communicate pathos without words. The film asks audiences to do the work of interpretation, but we remember torches and pitchforks, an angry mob, a creature who couldn’t speak. We remember villagers defending themselves against a monster.

There are no villagers with torches in Shelley’s novel. There is no mob hunting the creature. What the novel gives us is lonelier and more devastating: repeated scenes of rejection, each time the creature reaches out for connection. Victor abandons him immediately upon animation. The De Lacey family, whom he has watched and secretly helped for months, drive him away in horror when he finally reveals himself. The man whose child he saves from drowning shoots him. Every bid for relationship meets refusal; violence grows from that isolation. Shelley makes the causation explicit. She is writing about what abandonment does to a being capable of love.

A simple story travels well: creature made wrong, creature acts dangerously, villagers defend themselves. A hard story travels less easily: creature made capable of feeling, creator abandons creature, abandonment corrupts what began good, everyone suffers. The hard story asks more of us.

If we misremember the story as Victor forgot to build in compassion, we can focus on the moment of creation and forget everything that follows. We can imagine the problem lies in initial design, in getting the technical specifications right. This is a comforting thought for builders of technology. It suggests our obligation ends once we’ve shipped the product. But if we remember the story Shelley actually wrote, the problem becomes relational and temporal. Being creators binds us to what we create. We cannot launch and move on.

What Mary Shelley inherited

Shelley didn’t arrive at her understanding of the creature’s tragedy from nowhere. She inherited it from her mother, Mary Wollstonecraft, who died 11 days after giving birth to her.

Mary Wollstonecraft knew William Blake and Thomas Paine and was at the heart of a radical dissenting tradition that believed moral capacity required cultivation and could be cultivated in all: it was not an inherited endowment for a lucky few.

Wollstonecraft argued in A Vindication of the Rights of Woman that education should develop independent moral reasoning, particularly for those whom society typically denied such cultivation. She opposed the fashionable sensibility of her era and insisted that moral capacity required both feeling and reason working together. It needed a community that first, recognised you as capable of such judgment. In the 1790s, Mary Wollstonecraft was writing of women.

The creature in Frankenstein embodies these concerns. He begins with innocent sympathy: he helps the De Lacey family in secret, weeps at Felix’s kindness, saves a child from drowning. He educates himself through reading, developing exactly the moral and intellectual capacity Wollstonecraft advocated. But he encounters a world that won’t recognise this capacity, that reads him purely through appearance. His innocence meets experience without any guide.

Victor could have been that guide, and should have been. This was his obligation as creator: not just to build well but to guide what he built through the difficult passage from innocence to experience. Instead he fled. The creature received no cultivation, no careful introduction to the world’s complexity. Just violent lessons teaching him his feeling didn’t matter, his actions would always be misread, his capacity for goodness was irrelevant.

The lesson reversed

To come back to Frank’s post. He offers four principles for building empathetic AI: purpose-led mission, leading with empathy, partnering with compassionate organisations, building a compassionate team. These are good principles. He wants AI to sit within structures of ongoing care rather than replacing them.

But the popular version of Frankenstein suggests a lesson that Shelley’s novel reverses. The popular version says: build empathy into the design, get the initial construction right, ensure the creature has a heart from the start. Shelley’s novel says something harder. The danger isn’t creating something without empathy. The danger is creating something with moral capacity and then refusing to recognise that capacity, abandoning it to experience without guidance.

The reversal matters for technology. Not build empathy into the design but sustain the relationships that make empathy possible. Not get the construction right but stay with what you’ve built, accept the ongoing demands creation makes.

What this means now

Consider what happens when we build AI systems meant to serve vulnerable populations: veterans with PTSD, students in crisis or people struggling to access mental health support. These are people whose capacity for trust may already be damaged, who have learned through hard experience that systems often fail them.

When we bring AI into these contexts, people come to rely on it, form relationships with it, find genuine comfort in the interactions. They share difficulties they might not share with humans; they return again and again to something that seems to listen.

If we then shut the system down, perhaps because our own focus has moved on elsewhere, or we have been acquired, or the technology itself has moved on to something else, then we repeat Victor’s error ... at scale. We have abandoned what we created.

The creature didn’t need better information design. He needed recognition: someone to see him as a being whose life matters, who deserves care not because of what he can contribute but because he exists. This is what Victor owed him, not as a technical problem to solve but as a relationship to sustain.

The lesson from Frankenstein is that we should create structures that prevent abandonment, that will hold across time. We should recognise users’ agency, their authority about their own experience and their capacity to judge whether what we’ve built truly serves them.

Victor’s failure wasn’t just abandoning the creature; it was never granting him standing to make claims, never recognising his capacity to articulate his own needs.

The story we tell ourselves

Philomena Cunk’s throwaway observation should remind us of this. The novel is titled Frankenstein. Victor and his failings are the subject of her work, not creature.

To extend Frank Casale’s metaphor, the problem therefore lies not with AI, our creature, but with us.

Those in need deserve technologies built with care, but also sustained with care.

What Shelley understood, what the film simplified away, is that the danger isn’t in what we create. The danger is in what we do after creation: whether we stay, whether we accept the ongoing demands, whether we build structures that hold.

Whale gave us villagers with torches, rising up against a monster. Shelley gave us a creature standing alone on the ice, abandoned. Remember, that I am thy creature. I ought to be thy Adam, but I am rather the fallen angel, whom thou drivest from joy for no misdeed.

If we can tell the right story, the one Shelley actually wrote, maybe we can build technologies that genuinely serve rather than simply scale.

Postscript

Frank reached out for the Frankenstein story, because there is another filmic version out now, directed by Guillermo del Toro. You may wonder why I don’t explore that movie and its meanings.

Well, I have seen many version of Frankenstein, including del Toro’s. I enjoyed it. But it is striking that a movie with sensitive actors, a huge budget for the most technically advanced filming and steered by a 60-year old veteran of cinema with numerous Oscars under his belt still cannot add much, if anything, to a novel written in 1816 by an 18-year old woman.

However, for an interesting perspective of del Toro’s movie, try The tender food of “Frankenstein” by Ashlie Stevens.

Of other versions ... Kenneth Branagh annoyed me by calling Victor Frankenstein Swiss, when he has Genevan. A small point which ruined the movie for me, but as I said, I can be a pedant. There were others.

My favourite recent version was the beautifully rendered Turkish miniseries Creature, directed by Çağan Irmak and available on Netflix

In all these modes, Kathleen can help people effectively, quickly and at scale. This matters because with any helpline or service agent, being on hold is not good enough. Waiting is, at best, obstructive and, too often, a source of harm in itself.

In any discussion of AI and helplines, the conversation gravitates toward cases of acute crisis. We think about suicide prevention, severe mental health episodes, and people at the edge. These situations matter greatly, of course, and the moral urgency is real. But this framing may lead us astray because it is so vivid and emotionally compelling. This is the availability heuristic at work: our minds weigh memorable, intense scenarios more heavily than common, mundane ones. We end up designing systems for catastrophes rather than daily needs.

There’s an important social component to this: crisis justifies intervention, making the work of helpers visible and valued. And crisis maintains our power structures. When people are in desperate need, they’re not in a position to question how they got there or what systemic failures created the conditions for their suffering. They’re trying to survive.

So, there’s moral drama in rescue that doesn’t exist in the daily work of prevention, which, if done well, would not only fend off crises but enable people to question the systems of power that continue producing crises.

This is reminiscent of how we approach poverty politically. We’re very concerned with feeding people in famine, which is dramatic and visible. We’re much less concerned with the economic systems that create conditions for famine, which would require examining power structures and the distribution of resources.

Research on helplines reveals a wide spectrum of needs. While crisis lines specifically serve people in acute distress, many helplines, such as those providing medical information, benefits guidance, or general support, primarily field questions from people seeking information or early-stage help. Even on the most critical helplines, many users contact services long before reaching a critical point, looking for support while ordinary human difficulties are still developing. The adolescent who attempts suicide, the person having a psychotic break, are compelling concerns, but they are often the endpoint of a much longer process that we could have interrupted earlier.

Earlier help doesn’t require the same intensity of intervention and may be a scenario much more suitable for AI’s capacity for scale and the relatively light touch empathetic conversations AI makes possible. For someone who needs information about whether their experience is normal, or validation that their concern deserves attention, or guidance about a straightforward next step, AI should work well.

If we build systems that treat all contact as if it requires intensive clinical expertise, we create bottlenecks that leave people stranded. When we create accessible touchpoints, people may come with small concerns that we can address; they leave with what they need. But when those touchpoints don’t exist, the same people may show up months later on the crisis line with compounded difficulties that now require much more intensive work.

Waiting is Obstructing

Researcher Anat Rafaeli has said of the all too familiar on-hold experience: “It is not an issue of time; it is an issue of obstacle.”

Each failed attempt to access help, whether for customer service or emotional advice, doesn’t register as a neutral delay. It reinforces a sense of futility: that seeking help was itself a mistake.

Too often, in the call center business, when we think about waiting as duration, we naturally focus on reducing minutes, improving queue management, or at least making the wait more pleasant. We play music and provide updates: your call is 4th in line. We apologize for the delays. These feel like solutions because they address the time we are spending on the system, but politeness is not accessibility.

But Rafaeli’s insight reveals why these interventions often fail. Where the call center operator sees passive time passing, the caller experiences active obstruction. Someone calling a helpline has already crossed a threshold, acknowledging they cannot resolve something alone. They have decided to seek help; to then encounter an obstacle at precisely that moment of acknowledged need intensifies the experience of being stuck. This is why waiting is best thought of as an obstacle, just as much as a complex form or a demand for rigorous verification.

Research confirms this. Studies on telephone waiting times show that even waits of twenty seconds feel twice as long to callers. Why? Because the psychological experience isn’t so much about time on the clock as the sense of how long you remain blocked from what you need.

Why Current Solutions Miss the Mark

Every obstruction of this kind requires an obstructer and someone obstructed. Once you see this, the moral landscape changes completely.

The person reaching out, even with a slight problem, is already vulnerable, knowing they need help. The obstruction of waiting is not a refusal to help, which would at least be honest, but something more insidious: the appearance of availability coupled with the reality of inaccessibility.

This raises questions for teams like Tranquilla about what we’re actually offering. Will we use our empathetic conversational technology to remove obstacles or to momentarily reduce distress? It’s important here that we refuse the contrast many people make: human good, technology bad. Instead, we can ask which obstacles require human capability to remove, and which don’t? That’s a more useful question.

Where obstacles are informational, as so often in customer service, AI can work very well. But where the difficulties are relational, perhaps feeling isolated or unworthy of care, AI can’t remove that particular blockage.

There are, for sure, times when only human contact can work. The philosopher Emmanuel Lévinas wrote about the face of the other, making an ethical claim on us. When someone reaches out in need, their face (metaphorically) demands a response. To turn away, to make them wait, is to refuse that ethical claim. We’re saying, in effect, “your face doesn’t command my attention enough to warrant immediate response.”

But, and this matters, that same AI might still play a valuable role if it can quickly identify that someone needs this human connection rather than information, and route them accordingly.

The systems I most distrust are those that force everyone through the same gatekeeping process regardless of need. Someone who needs immediate human connection shouldn’t have to spend twenty minutes providing demographic information or completing screening questionnaires. That’s obstruction presented as due process.

So if we took this seriously and designed a system primarily for removing obstacles rather than crisis response, what would actually change?

How We Work

For one thing, system design would be moral work, not merely technical work: accessibility would become the primary design criterion. Instead of asking the call-center question, “How do we manage demand within our capacity?” we can ask, “What blocks people from getting what they need, and how do we remove those blockages most effectively?” We are, in effect, making decisions about whose suffering we’ll leave unattended, whose needs we’ll obstruct, whose vulnerability we’ll ignore. These are ethical decisions too easily disguised as administrative ones.

For this reason, at Tranquilla, we are not building a generic emotional chatbot where everyone goes through intake, everyone gets assessed using the same criteria, and everyone follows the same pathway. We can reframe AI deployment in emotional and practical support away from the usual “can machines be empathetic?” toward something more tractable: what specific obstacles can different systems remove? To do this, we work with service providers (for example, in education, social support or healthcare) because we can then address their very specific issues that need to be unblocked. This is not a standardized approach, but differentiated for each scenario at scale.

Naturally, there are dangers here that we should be wary of. Algorithmic conversations go wrong if the system is trained on data reflecting existing biases. For example, if a military veteran uses jargon the chatbot doesn’t understand, they may get routed to a human, but what is meant only as clarification may then feel like escalation. This is another reason why our work is most effective with specialized service providers, who can inform this process, rather than building a generic service.

There’s also widespread concern about depersonalizing care, replacing human warmth with algorithms or treating people as problems to be processed. Sadly, human operators can also become impersonal, cold and systematic: their role is highly stressful, demanding and exhausting. We humans can’t turn on empathy as needed, and we often get things wrong.

What AI can provide is not connection in isolation, or love in distress, but a kind of scaffolding: ways of breaking down decisions, questions that help clarify values and priorities, frameworks for weighing options.

Designing for failure

Of course, as I have suggested, AI gets things wrong too. If an automated system misidentifies someone’s need, routes them incorrectly, or provides guidance that doesn’t fit their situation, what’s the recovery mechanism? With human interaction, miscommunication can be repaired relationally. The person can say “no, that’s not what I mean,” and the human can adjust. With AI, if the person experiences the response as unhelpful, do they try again? Give up? Conclude that help doesn’t exist for their particular problem?

From a design perspective, how would we know what happened? Or, from a systems management perspective, how do we measure success in a system designed around removing obstacles to help? Not by throughput, average wait times, or how many people we processed. Instead, how many felt unblocked, or how many left before small concerns became crises?

How do we track what didn’t happen: the crisis averted, the person who got what they needed early and didn’t require ongoing intervention? Remember, prevention is invisible, therefore less compelling and undervalued.

If we measure success through prevented crises, we’re implicitly accepting crisis prevention as the goal. But is that the right goal? Maybe the goal should be flourishing, or growth, or developing the capacity to handle difficulties. These are much harder to measure but potentially more meaningful.

New measures of success

So, we have to work with our partners in service to value different kinds of success. There’s a subtle message in current approaches: if you can manage with information or brief guidance, your problem wasn’t serious enough to warrant concern. But that’s backwards. The person who can resolve their difficulty with minimal support demonstrates exactly the kind of early access and efficient obstacle removal we should optimize for.

One important measure is trust: an empirical question we can test, even if subjective. Do people trust AI systems that are helpful, even when they know those systems don’t “truly” understand? Similar measures can be collected by qualitative research, but also have to be sensitively haxndled. If I’ve just struggled through accessing a helpline, barely gotten the help I needed, and now I’m asked to rate my satisfaction on a scale of one to five, there’s something almost insulting about it: I am sure you have all had this experience. So, evaluating the quality of our work is likely best done closely with our clients, who in turn are working with their users.

Final word

I suspect my focus here on obstacle removal is both helpful and limiting. Helpful, I hope, because it prevents us from romanticizing human contact in situations where information or structure is genuinely what’s needed. Limiting because it may treat human needs as somewhat transparent: as if people always know what they need and we only have to provide it, or as if an AI can reliably identify those needs for us.

At Tranquilla, our aim is not efficiency with a touch of empathy; we’re addressing a form of harm that existing systems inflict on vulnerable people. When someone acknowledges they need help and we make them wait, we’re not neutral. Every moment adds to the burden of someone who is already carrying too much.

Please note that I write as the Futurist at Tranquilla, so my role is to look ahead and beyond where our work is today. I mean that you shouldn’t take what I write as a product description or roadmap, but as my personal reflections on areas where our work is leading.

In a recent article, Muhammad Aurangzeb Ahmad looked at a range of spiritually-oriented chatbots and concluded that whether they enrich or hollow out religious life will hinge less on the models and more on how communities critically adopt and integrate these tools into their shared spirituality.

To think through these possibilities, I want to explore three dimensions of the challenge: the temporal structures that shape spiritual growth, the nature of religious authority, and ultimately, what we mean when we talk about authentic spiritual encounter.

Availability and time

Ahmad points out, that for many people, connections to their spiritual traditions is constrained by geographically dispersed communities, shortage of clergy and financial barriers. If AI can help address some of these constraints while preserving the core functions of religious guidance, isn’t there a moral argument for embracing it?

Shouldn’t we want spiritual resources to be more widely available rather than artificially scarce? For a Buddhist in rural Montana, struggling with questions of interpretation or ritaul, if they can engage meaningfully with the Dharma through a well-designed AI, isn’t that a net positive?

You can see the value in a crisis or for occasional look-up questions. The on-demand availability of AI advisors, companions or guides may be a great benefit to people struggling or searching. But for longer-term personal transformation, I wonder if the very immediacy of AI undermines the patience that spiritual growth demands?

Religious traditions have always involved complex structures of time; ritual calendars, seasons of feast and fast, the slow pace of contemplation, the suddenness of insight. Time in the sacred sense isn’t just chronos or duration; it’s kairos, the right moment when the eternal breaks through into the passage of everyday time. In John 7:6, My time (kairos, my moment) has not yet come.

I work with a team - Tranquilla - building a compelling AI to provide empathetic help, urgently and immediately, and it’s important work. Yet it still seems to me that AI operates in a time without qualities. It has no lived chronos or kairos. Whether it responds instantaneously or longer is determined by the compute required on some remote server, rather than a care to create the conditions for comfort or revelation.

Is this good enough?

These temporal concerns point to questions about the nature of spiritual encounters. But rather than getting caught up in debates about authenticity, we might ask a more pragmatic question: not whether the experience of working with an AI is credibly spiritual, but whether it is good enough.

I often talk about this good-enoughness in relation to AIs I’m working on, like our work at Tranquilla to provide some degree of empathetic care or counsel. Even if there is no true human empathy, can a bot be empathetic enough to help someone out? I think so. In fact, I’m struck by how many people report feeling genuinely helped. If someone struggling finds comfort in an AI companion that consistently offers patience and encouragement, are we really going to tell them their experience is invalid because the AI isn’t truly compassionate? Instead we can separate the question of whether AI can be empathetic from whether it can function empathetically.

Sacred functionality and authority

I do think this pragmatic approach has real merit, especially for care, comfort and coaching. But it sidesteps something fundamental about how religious authority actually works.

Religious traditions themselves seem to resist purely functional interpretations. I am sure most would insist that the divine is not merely functional in this way: rather, the sacred calls into question human functions and purposes .

Of course, religious practices do serve deep functions: they create social cohesion, manage our anxieties about survival, suffering and death, and provide frameworks for making meaning that helped our ancestors flourish. But, to be fair, they’ve also been about gatekeeping, about defining who belongs and who doesn’t. Who controls access to the sacred?

Similarly, human religious authority, whether we’re talking about shamans, priests or gurus, has always been embedded in ritual practices that mark certain people, times, and spaces as set apart. These practices create psychological and social conditions that make religious experience possible. The shaman must undergo symbolic death and rebirth. Their authority comes not from knowledge but from having traveled to the other world and returned.

An AI has never made that journey because it has never lived in this world to begin with. So, how can AI fulfill these same functions? Can it ever call it’s own functionality into question? When a chatbot delivers spiritual guidance, where is the “setting apart?” Where is the ritualization that signals to us that this is sacred space or a sacred moment and not just a question-and-answer session?

Embodied authority

The authority of a human religious teacher comes not just from knowledge, or even from experience dealing with others (which may well be the authority of a counsellor or therapist) but from their own transformation within a tradition of practice. They embody what they teach in a way that leaves traces: scars, wisdom, presence. Our best teachers are often the most scarred, with an immediacy of presence; the most glib and insincere, to be found 24 hours a day on cable tv, carry none of that burden.

The absent embodiment of a spiritual AI connects to a broader issue about how religious wisdom is transmitted. Religious traditions aren’t just collections of ideas that can be mashed-up; they’re entire ways of life that have evolved over centuries in response to particular challenges and contexts. They are deeply embedded in specific cultural contexts, carrying forward histories, languages, practices, ways of being in the world. John Fort Newton wrote of the Hebrews having a genius for religion as the Greeks had a genius for art and philosophy and as the Romans had a genius for jurisprudence.

But Ahmad notes in his article that LLM outputs often reflect a single, homogeneous religious-cultural profile. When an AI offers users a kind of religious mashup (a bit of Buddhist mindfulness here, some Christian comfort there) it undermines the cultural specificity that makes much religious thinking coherent. Surely something gets lost in that process.

For my part, I see these religious AIs not as a gateway to insight, but more as a kind of storytelling about the sacred, treating spirituality as information that can be extracted, processed, and re-delivered. That may be the good-enough many people need, especially if they already have a grounding in their religious tradition. The Buddhist in rural Montana, if raised in the tradition, may find the chatbot more useful as a refreshing source of insight than the new Buddhist striving to understand the layered complexity of a long-established culture.

Presence without reciprocity

Perhaps I’m being too restrictive in thinking about spiritual presence. Consider how many of our most profound religious encounters happen not through direct human contact, but through other forms of mediated experience: Augustine reading Paul’s letters, or millions of people finding solace in the Psalms without ever meeting King David. The medium has always been part of the message in religion. Perhaps what we’re witnessing is simply the emergence of a new medium, with its own affordances and limitations.

Can an algorithm not become possessed by the gods? Can it speak with the voice of the absolute? After all, literature, music and art certainly demonstrate that transformative encounters can happen across time and space through text, sound, paint and marble.

When I read Shantideva, I’m not in a reciprocal relationship with him, but his thoughts can still transform me. Julian of Norwich seems to speak with real presence across centuries: at times you can almost hear her voice. Some of our most profound spiritual encounters happen through books, through dreams, or through encounters with the natural world when we are alone. The presence that matters might not be the physical presence of another human being but the presence of the sacred itself. Who’s to say that can’t move through digital channels?

Perhaps, then, we can think of AI spiritual guides as more like sophisticated forms of literature: tools for reflection and growth rather than replacements for human community.

Beyond therapy

Spiritual guidance as literature suggests one way of thinking about AI potential, but it also reveals the limitations of approaching religion primarily as a source of comfort or insight.

Much of the discussion around religious AI focuses on its therapeutic potential: comfort, guidance and emotional support. But as I have suggested, religious traditions have generally been concerned with something more radical: transformation. They’re not trying to help people feel better; they’re trying to help people become fundamentally different kinds of persons. Can AI participate in that kind of radical transformation?

The therapeutic model treats religion as a resource for managing the difficulties of life as we already understand it. But the transformative dimension of religion calls that understanding into question. The imagery is often that we must die to our old selves and be reborn. Can an AI call someone to that kind of radical change when it has no conception of what transformation actually means?

Perhaps not, but equally, are we sure transformation requires some kind of conscious intentionality on the part of the guide? Transformation often seems to happen through unexpected encounters, through insights that arise from engaging with ideas or practices in new ways. If an AI can present religious material in ways that prompt genuine reflection and self-examination, why couldn’t it catalyze transformation?

Maybe we’re too focused on the mechanism rather than the outcome.

Authenticity contested — toward spiritual mestizaje

These questions about transformation and authenticity assume we know what authentic religious experience looks like in the first place. But when I hear people worrying about the authenticity of AI spiritual guidance, I have to ask: according to whom? Maybe these technologies create possibilities that go beyond that single, homogeneous religious-cultural profile to achieve what Gloria Anzaldúa has called spiritual mestizaje: mixing and creating new forms of the sacred that speak to people who’ve been cast out by traditional institutions.

For me, the most important question raised by Ahmad’s article is whether the sacred can survive its transformation into information. The sacred has always been about presence, about the face-to-face encounter with the absolute. Moses meets the burning bush; Arjuna (and Duryodhana, let’s not forget) sees Krishna’s Vishvarupa, his cosmic form; Paul is struck down on the road to Damascus. These aren’t transfers of information and knowledge but ontological events. They change not just what someone knows but what they are.

Can AI do any of this, or does it reduce them all to data? This returns us to Ahmad’s original insight: the value of these tools may depend less on whether they can replicate traditional spiritual authority and more on how communities choose to integrate them into their existing practices of seeking and transformation.

But there’s the important word: communities, with all its implications of tradition and social complexity. Individuals, alone with an AI, might think they are creating their own spiritual mestizaje, but without the testing ground of community, how could they distinguish transformation from simple self-reinforcement?

Since then, his experiences have been equally extraordinary. Gísli has been the CTO of a leading tech-focussed investment bank in Europe, and of an agriculture-focussed non-profit in Africa. He has been a member of the Icelandic parliament and is now the director of the Red Cross in Iceland.

So, it was time to update his book on crisis management. And what an important new book this is!

What strikes me immediately about Gísli's writing here is that a fundamental impossible possibility lies at the heart of crisis leadership. He tells us, plainly enough, that we should prepare for the unpredicted, plan for chaos and structure the unstructurable.

To this end, professional crisis management (especially among NGOs) appears to rely on endless protocols. But the decisive moments in the field rarely follow the manual; they’re about reading the situation, honouring the dignity of those affected, and resisting the urge to impose a pre-packaged solution.

For example, when Gísli speaks of his team in Haiti, of the moment they had to decide whether to continue searching the Caribbean Supermarket, and the consequences of the decisions he made, he moves beyond practical contradictions of planned and unplanned. There could be no right decision. What was needed was not a set of techniques or a checklist of skills, but a profound form of moral attention. Gísli and his team had to be fully present to the suffering, fully present to their own limitations and fully present to the weight of their decisions.

We had decided not to continue searching because we had not heard any signs of life. Yet we had left more people alive in the building than we had brought out.

That's the kind of decision Gísli has learned from and shares with us candidly. Compared to decisions in my own career, it is profound.

So, perhaps the most important preparation that emerges in this book, is not for specific scenarios, but for this quality of presence and moral attention. I say moral because you’re deciding not just how to act, but whose needs will define your action.

In many ways, you could read this as a straightforward leadership manual, albeit one informed by a fascinating and moving range of experiences. There are reading lists, assignments and exercises as you will find in any typical management book. They are very practical and useful.

But it is the experiences behind them which linger in the mind. I don't want to romanticize this work. I have read too many accounts of disasters which celebrate those who manage the symptoms of destruction while leaving the underlying structure of crises untouched. But in The Crisis Leader, when Gísli writes about the rescue workers he encountered (like those local women in Indonesia who were doing the real coordination work) what strikes me is not a drama about him or his team, but simply the patient, unglamorous goodness of the, often anonymous, locals.

That goodness of local people survices after the rescue teams have gone. Gísli is quite explicit about this: every crisis response ends, every team disbands, every lesson must be re-learned. Every After-Action Review bears the mark of who could not be saved, could not be reached, what could not be foreseen.

Cultural humility

These time-bound, short-term, interventions could easily just repeat the mistakes of the Western savior complex: flying in, doing good and flying out again. But Gísli puts great emphasis on cultural humility and on empowering local voices.

Cultural humility is easy to announce but costly to practice. It means giving up a measure of control, accepting that your role is transient, and that the most valuable capacities in a crisis often belong to those who were there before you arrived and will be there after you leave. This can't be perfect, but in The Crisis Leader there is genuine self-critique within the constraints of existing economic and technical realities.

Gísli quotes Mark Twain: Good decisions come from experience. Experience comes from making bad decisions.

But in this telling, it's not just a question of simplistically learning from your mistakes. The tone and depth of this book, suggests that failure is not the opposite of success but a condition of its possibility.

One tension I notice throughout is between the urgency that crisis demands and the patient reflection that wisdom and learning requires. There’s a cognitive bias here: our brains are drawn to dramatic, acute interventions. But the slow, relational work of building trust, creating predictability and showing up consistently, often has greater long-term impact on emotional recovery. Gísli's respect for that mirrors what we see in sustainable psychological care.

In crisis, Gísli seems to act with a different sense of time; not the rush of anxiety, but what we might call attentive urgency. Notice, for example, how he structures rest for his team, insists on it, even when they want to push harder.

And he has the courage to admit that even our best efforts are incomplete. This is not defeatist but reflects the very structure of decision-making: every choice closes off other possibilities and the consequences always carry the trace of paths not taken.

In this book, leadership in crisis is not primarily technical: it’s relational and moral. Your presence, your ability to listen, and your respect for those around you are as critical as any logistical capacity. This is what I love about Gísli's book. This is not merely a guide for leaders (although it is a very useful one), but also the most moral writing about leadership I have ever read.

In the end, what moves me in The Crisis Leader is its simultaneous realism and hope. Gísli does not shy away from describing devastation, failure or the overwhelming nature of human suffering. Yet his act of writing, of sharing these reflections, is itself a form of hope, that others might respond to crises with more wisdom, more justice and more attentive urgency.

Leaders should be readers. Read this book.

When I am at home, my habit is to walk for about an hour or more before dawn. So, come on: it's a beautiful time to be out, quiet and dark. This is also very good thinking time; the mind still has memories of dreams, and the business of the day has not taken hold. All sorts of things come together, often in startling propinquity.

I have written here before about knowledge and narratives as a form of landscape, through which we navigate. Often, when I look back on my dawn walks to recall my chain of thought, I find it literally implicated in the landscape of my route, rather like a memory palace. Like this ...

Uphill

On Friday, starting the climb to the top of our hill and on to the long straight stretch beyond, I was thinking about a recent research paper with the rather wordy and academic title, Gender, Confidence, and the Mismeasure of Intelligence, Competitiveness and Literacy. " It was fascinating and worth reading if you want to dig into the details. Walking, I didn't recall all the technicalities, but the high-level findings really did get me thinking.

The researchers (Harrison, Ross and Swarthout) used the Raven Advanced Progressive Matrices (RAPM) test, a common intelligence test with 36 problems, to conduct several experiments with hundreds of college students. Instead of just marking answers right or wrong, the researchers developed an interface that lets subjects allocate tokens to different answer choices for each problem. The number of tokens allocated to an answer reflects the subject's confidence in it being correct.

Women performed much better when they could express their confidence levels rather than being forced to pick just one answer. In fact, women outperformed men when confidence in results is measured. Minority students also showed significant improvement with the confidence-based system. These results held even when the questions were randomized, rather then being presented as progressively more difficult.

The researchers suggest that women and minorities may be more willing to acknowledge uncertainty when they're not completely sure of an answer.

Traditional measures don't simply fail to capture women's intelligence; they actively penalize the forms of reasoning that women more frequently exhibit by forcing a single choice.

The team tested this concept in other areas and found similar patterns. In competition, women aren't afraid to compete but make rational risk assessments. In financial literacy tests, women's I don't know responses reflect appropriate uncertainty, rather than ignorance.

What I believe we're seeing is a need to include an understanding of ambiguity and uncertainty in our assessments of intelligence. So, there's a lot to think about in that paper, although the conclusions are strikingly simple.

The farm road

Turning along the lane between the horse farms, (Good morning, Indigo! Hi, Diesel!) my subject often changes with the change in surroundings. This day, I found myself finding odd conjunctions.

Of all people to come to my mind first, it was Hayek (Thatcher's favourite economist) and his central argument about the use of knowledge in society. That was weird, but although I have little time for Hayek's political ideology, he makes many valid observations, especially about the nature of knowledge and how it is distributed.

Hayek understood that knowledge is not centralized, but scattered among countless individuals. He sees prices of goods in a market as a way of coordinating this dispersed knowledge, not by central control but through local signals that allow each person to act on their own information (over-simplified: what you want to buy, why, and how much you are prepared to pay), which may be unavailable to others.

Centralized economic planning assumes a kind of artificial certainty and comprehensive knowledge. And this is what connected in my mind to the intelligence research …

Uncertainty is a more effective mechanism because it allows for more subtle approaches to knowledge that more accurately reflect the real world.

The way through the woods

Turning into the woods where, at this time of the year, I am a little wary of both aggressive owls and lumbering bears (the owls are scarier), I connected all this to something Dr Francis Young has written about: how people increasingly treat AI-generated content not as probabilistic outputs from pattern-matching systems, but as oracular pronouncements.

Young is a historian of belief, and suggests we have moved from one authority-based system of knowledge (ancient texts, religious doctrine) through a period of empirical scientific discovery, only to arrive at another authority-based system; this time with AI as the ultimate arbiter of truth. AI believers see a digital supermind that has somehow consumed the totality of human knowledge into the pleroma of data. Young does make clear he means pleroma in the Gnostic sense of a divine completeness beyond the material world. (Saint Paul uses the word somewhat differently.) With sufficient data, the pleroma approaches omniscience.

Homeward with hallucinations

No owls today, except a distant hooting, and no bears. Emerging from the woods and turning for home, my thinking also turns, now to a recent paper from OpenAI: Why Language Models Hallucinate.

The paper is quite technical, but in essence, the authors show that hallucinations arise not from flaws in implementation but from fundamental statistical pressures in the training process. When faced with questions about arbitrary facts (birthdays of obscure individuals, for instance), even well-calibrated systems must guess, and they do so in ways that appear confident to users.

They also see a deeper problem with evaluation frameworks. Current AI benchmarks reward guessing over acknowledging uncertainty, systematically training models to exhibit precisely the kind of overconfidence that research shows to be a disadvantage in humans.

You see the pattern. AI engineers (and, I suspect, the tech industry in general) approach uncertainty and knowledge by consistently designing systems that penalize acknowledgment of ignorance while encouraging confident assertions, regardless of their accuracy.

Hayek's analysis suggests why this occurs: complex systems require local knowledge that cannot be centrally aggregated or assumed into a pleroma. Yet our technical and institutional responses consistently attempt to create comprehensive, centralized measures. As we have seen, intelligence tests reduce multifaceted, complex cognitive capabilities to single scores. Marxist economic planners (and the leadership of most US companies) attempt to replace endlessly diverse local signals with centralized calculation, performance targets and plans. AI systems try to compress the complexity, ambiguity and tentativeness of human knowledge into simplistic outputs.

Young's observations about the pleroma of data illuminate how this dynamic can intensify. If people increasingly treat AI outputs as authoritative regardless of their grounding in evidence, we risk creating feedback loops where confidence matters more than accuracy, and where the simulation of knowledge displaces genuine understanding.

The technical constraints that the OpenAI paper identifies suggest these problems may prove intractable through purely algorithmic means. If hallucination emerges from statistical necessities rather than engineering failures, then addressing it requires changing how we evaluate and deploy these systems rather than simply improving their training.

The downhill stretch

When we design intelligence tests that penalize appropriate and useful uncertainty, we're making ethical choices about what kind of intelligence counts and, by implication, whose intelligence counts.

The same applies to AI systems: the fact that they hallucinate confident falsehoods reflects the values embedded in their training, not some inevitable technological outcome.

We can do better.

Earlier this year, I stayed for several days in a friend's apartment in New York's Hudson Yards, overlooking Thomas Heatherwick's remarkable landmark, Vessel.

My view was unusual: looking down from above. I wonder if that literal different perspective made me consider the public space in ways that would not otherwise have come to mind. (I say public space rather than building, because that's how Heatherwick himself classifies it. It is designed to be entered and explored.

Now, about six months later, I have been thinking about Vessel again, because I have been reading Hugh Casson and E.M. Hatt's 1963 National Benzole guide to, mostly English, follies.

Is Vessel a folly?

As the introduction to that charming book notes, follyhood has to be felt as well as seen. Certainly, Vessel raised feelings: it can look quite lovely in certain lights, and in others, very plain. The scale struck me as odd; compared to the surrounding buildings, it's not very high: all you can really see is those luxury residential towers crowded around, the shopping center and a slice of river. There are no doubt better views from the nearby Edge skydeck, though I wouldn't dare go up there.

But Casson writes that, The mark of the true folly is that it was erected simply to satisfy and give pleasure to its builder. That doesn't feel true of Vessel, which is plainly commercial. But Casson also quotes Sansovino on Venice that it affects greatly to surprise the stranger, and that is certainly the case.

What strikes me about Vessel is how it seems designed primarily for self-documentation rather than contemplation. The historical follies described by Casson and Hatt were built to satisfy their creators, but Vessel exists to be consumed and reproduced digitally. With its inward-facing structure, it turns the gaze back on itself and the crowd. Its importance is that it is treated as important; it is photographed because it is photogenic. Visitors create a circulation of images that reference only more images. When they climb Vessel, are they really seeing New York, or merely seeing themselves seeing New York?

So, is Vessel a folly? Casson says that not every foolishly-conceived building is a folly. Equally, I think it's fair to say that not every folly is foolishly conceived, and I am not at all sure Vessel is foolish.

Traditional follies were often expressions of personal vision, however eccentric. Can Vessel, as a corporate commission, possess that same authentic eccentricity? Perhaps it represents an attempt to manufacture eccentricity at a corporate scale. The historical follies served very small audiences: the wealthy landowner and occasional visitors. Vessel serves millions.

And Vessel doesn't even pretend to be anything other than a spectacle for consumption. Traditional follies stood apart, often at the edges of an estate, removed from the house and functional buildings. There was a distance, both literal and figurative, between conventional living and folly creation. The historical follies embodied a kind of private vision made public. Vessel is a public vision designed to generate shared experiences, mediated through technology. And it doesn't stand apart from commerce; it facilitates it, being at the heart of a luxury shopping development.

Moreover, the historical follies, however eccentric, often attempted to direct attention toward something beyond the self: a loved one, a historical event, or an idea. Even the most self-aggrandizing tower built by a nobleman at least attempted to connect to some larger narrative about family, legacy, or power. Vessel connects to nothing but the experience of itself. Surrounded by selfie-takers, often well-equipped and artfully posing, it seems to embody a particular contemporary ennui.

Not so empty

But then, on my solitary walks at dawn, when almost no one is around, another thought stirs, not so bleak. The Vessel is lovely in the early morning light of Spring, so early that no visitors see it.

Beauty can be a reminder of what exists beyond our personal concerns. Could I see Vessel as at least attempting to create a new kind of beauty in an urban landscape otherwise dominated by utilitarian structures? Do people climbing Vessel experience something genuinely novel, creating meaning from that? Who am I to say this is empty?

Vessel, whatever its faults, at least provides a new kind of space for people to inhabit and interpret. Their selfies, posed and posted, are, after all, a form of storytelling, creating coherence between personal meaning and social belonging.

I wonder how Vessel will age? Unlike follies, which often deliberately engaged with mortality through their artificial ruins, Vessel, with its gleaming bronze structure, certainly doesn't aspire to ruin. The historical follies have acquired a patina of meaning through decades or centuries of existence. Will Vessel's bronze be allowed to age?

However, it holds up, in our accelerated culture, structures don't have the luxury of developing meaning slowly. The intention may have been the monetization of the sublime, but mortality has intruded in the most disturbing way possible. Vessel has been associated with real tragedy through multiple suicides, forcing its closure for years and significant modifications. Climbing the structure today, you are very aware of the mesh across the higher levels and that the uppermost is still closed.

Suicide is an extreme form of how cultural objects like Vessel can be used in ways that subvert their intended functions. Even the selfie-takers may be using the architecture ironically or critically rather than simply celebrating it. Vessel's corporate origins cannot completely determine its effects. Its purpose cannot be reduced to the creators' intentions. If the primary message was supposed to be look at me, everyone posing in the most mannered fashion in front of it is saying, No, look at me instead.

Vessel, in the long view happily, reminds me of how cultures continuously generate objects that exceed their original purposes. It will continue generating experiences and interpretations that surprise and challenge us, regardless of our critique.

So, I suspect Vessel does stand in the tradition of the folly, in a good way. Hugh Casson wrote that, There is no doubting that follies have been richly enjoyed, well worth the building. Regardless of whether I appreciate its aesthetics or its context, Vessel has raised for me the thoughts that good art can: about meaning, experience, authenticity, and the society that produced it.

I don't feel any temptation to resolve those differences into a unified, defined view of what AI really is or how it is progressing. If my earlier post was about recognising multiple timelines, this one is about how to inhabit them, often simultaneously in our different roles, holding them in tension.

Partly, I'm concerned that the current discourse around AI is being shaped primarily by corporate interests and the political system rather than by our accumulated personal experience. Perhaps even more so, the discourse is shaped by the creators of AI and reflects their priorities. If those creators have a narrow understanding of intelligence - and I suspect they do - their technology and the discourse around that technology will reinforce that narrowness.

The temptation to fix intelligence

When facing rapid and uneven change, there is often a strong urge to settle the matter with a definition: a well-marked boundary around where supposedly true intelligence lies, human or machine. And there is a strong urge to own that definition, too.

Corporate strategists want a roadmap to advantage; educators want to define what constitutes real learning; technologists want to declare the threshold of Artificial General Intelligence; artists want to hold on to real art; the most humane long for real, rather than artificial, empathy.

The danger is not simply that we might define AI incorrectly or differently, but that these tidy definitions can each harden into dogma. They offer comfort by closing off the complex, unknown and emerging, but in doing so, they blind us to how intelligence, and our relationship to it, keeps evolving.

So, how can we hold our definitions of intelligence less tightly, resisting the urge to lock them in place, and remain open to being reshaped by what we do not yet fully understand?

Across all three timelines, we are in that same situation: we may be using, or have built, things we don’t know well enough to define or definitively respond to.

Uncertainty helps

To hold up different views of AI as it develops technically and as our use of it develops is certainly not easy. It requires an uncomfortable (and uncomforting) humility of mind, recognizing the uncertainty of what we know.

This is entirely natural with a rapidly innovating technology. Back in 2004, David Lane and Robert Maxfield of the Santa Fe Institute wrote about Ontological Uncertainty and Innovation.

Lane and Maxfield's research reveals that innovation involves three types of uncertainty, but most businesses only focus on one. While companies typically worry about truth uncertainty (will this work or not?), the real breakthroughs come from grappling with deeper uncertainties: not knowing what something means or even what kinds of opportunities exist in the first place. You can see the relevance to our different timelines of AI.

The key insight for business leaders, but also for personal users and even politicians, is that instead of trying to eliminate all uncertainty, successful innovators learn to work productively with it. Innovation often happens through relationships and partnerships where nobody fully understands what they're creating together.

This is the humility I referred to. But I want to be sure to distinguish uncertainty from mere skepticism or relativism. Humility in thinking comes not from doubting everything, but from remaining open: it's being actively receptive to what we don't know.

Lane and Maxfield describe companies that thrive as those that build scaffolding structures that help them navigate unknown territory while staying open to discovering entirely new categories of value.

So, I'm suggesting here that, rather than trying to resolve the multiple experiences of AI - its intelligence, understanding, empathy, or even consciousness - into a single, manageable definition, we may do better to live with these highly contrasting views.

Social intelligence

This matters because intelligence, human or artificial, emerges in a relationship. It’s shaped by interaction and shared context. So, the intelligence of AI may well be quite different in different settings.

Perhaps each timeline requires different, tentative definitions of intelligence, even from the same model. The intelligence of AI, even the same underlying system, seems to manifest in different ways: for example, when I interact with models in research settings compared to when someone uses them for more creative writing.

In AI, there is a democratizing force at play: the tools enable people who might otherwise be excluded from cultural or economic life to participate in new ways. Even if the intelligence is instrumental, it creates space for more voices.

Still, technological and political narratives insist we pick a side on AI’s value, while institutional incentives reward quick wins and visible novelty over deep work. We live in an impatient culture that undervalues the slow processes of wisdom; the narratives of corporate and political power tend to flatten intelligence into utility.

We can resist that. Each timeline (personal, corporate, systemic) can provide different kinds of scaffolding for uncertainty. Personally, it might mean cultivating patience and curiosity in how we use AI in daily life. Corporately, it may involve resisting the pressure to define fixed roadmaps too early, instead allowing room for exploration and partnership. Systemically, it means creating policies and institutions that leave space for diverse experiences of intelligence rather than collapsing them into rigid regulations.

The scaffolding we build today, in classrooms, workplaces, and policy, should hold space for discoveries we cannot predict. Intelligence, after all, has always been a moving target.

There are, I suspect, some deeper conceptual problems which are masked by this framework of talking about AI adoption as a unified phenomenon: a single timeline of development.

Surely AI is developing in different contexts - in different strata of our economic culture, if you like - moving at different speeds in each and producing different kinds of shocks. There is no single take-off curve. I want to look at three of these braided timelines: corporate, systemic and personal.

Corporate time

Companies face a brutal temporal reality: the next few years will determine who controls the advantage in artificial intelligence and will set the scene for how those advantages compound once established.

Google's two decades of search queries create training datasets no competitor can replicate overnight. Tesla's million-vehicle fleet generates real-world driving data that traditional automakers, despite their engineering prowess, cannot match. Microsoft's integration of AI into Office tools locks millions of users into workflows that become stickier: harder to abandon with each passing month they are used, even if no one really thinks they are best practice.

These advantages require enormous upfront investment but become self-reinforcing once established. Speed matters because first-mover advantages in AI may prove durable even when the underlying technology remains brittle.

In addition to any technical advantage, the narrative becomes an asset in itself. Markets and partners mobilize around believable futures; over-belief eventually corrects, but meanwhile, it fuels hiring, partnerships, and spectacular valuations.

That's a dangerous game, though. When narrative diverges too far from reality, you get market corrections, as we have already seen with some AI stocks. The challenge is that AI systems do work: they're just not as general or reliable as the hype suggests. This partial success makes it harder to calibrate expectations.

Still, the corporate timeline progresses in quarters and fiscal years, and AI vendors act as if new capabilities arrive tomorrow, regardless of whether broader economic statistics will reflect that transformation.

Systemic time

By the system, I mean our economic culture at large. Institutions and governance from public sector bodies, regulators, courts and standards agencies. Infrastructure and physical constraints, such as energy grids, broadband, data centers, GPUs, and supply chains. Economic structures and sectors, along with social and organisational practices, including workplace cultures.

All of these cultural factors together determine how tools spread across industries, geographies, and communities.

This systemic layer explains why new AI capabilities don't instantly translate into productivity. The economy's slowest-changing sectors (healthcare, education, government) hold back that growth by their sheer scale. More sophisticated AI doesn't automatically address those human and structural limitations.

This systemic rhythm operates on timescales of years, maybe even decades, but the friction is not merely obstruction; it is how a complex society absorbs a new general-purpose technology without tearing itself apart.

Personal time

If you have used AI intensively on a project in your specialism, you may well have had that sudden, uncomfortable realization that it may be better than you. For many of us, this is where the real action is.

I have seen AI synthesize hours of user research transcripts, not only returning results in minutes, but with some genuinely insightful findings that I could have missed. A Swedish friend, an economic historian with over 40 years of research experience, feels he may have very little to contribute from now on, except guiding an AI to the right questions.

This realization comes quickly and can change your practices - in research, coding or writing - almost overnight. The personal timeline operates in days and weeks: faster than an organization can change and almost instantly compared to the slow churn of economic impact.

The paradox of three timelines

There's something almost Aristotelian about this framework: psyche, praxis and polis. In classical thought, these three domains (or timelines here) were meant to be integrated: individual character and behaviour, practical action and at the social level, political participation were understood as mutually reinforcing.

The three AI adoption timelines I describe seem to be pulling them apart. Corporate practice operates independently of any democratic deliberation; individual psychological adaptation happens in isolation, alone with the screen; systemic change proceeds without any sense of direction from either personal values or credible political institutions.

So when we hear disagreements about the speed of AI's development, we are often hearing from observers who focus on different temporal scales: the different layers of our social economy.

If your personal work is displaced overnight, that's a revolutionary disruption; for executives operating on quarterly cycles and year-end strategy reviews, there is definitely urgency, but economists studying GDP may see gradual change.

And perhaps this is why so many feel there is an acute AI crisis. For some, AI systems challenge human capabilities while for others that's all part of the excitement. But, for everyone, these disruptions are arriving in a context where the traditional sources of meaning (community, purpose, transcendence) have already been weakened. AI becomes both a symptom and the accelerant of a deeper fragmentation.

So, who gets to decide the pace of AI adoption? Timing itself becomes a form of power: individual willingness to work with AI rather than retreat before it; a corporate ability to marshal commercial resources; the institutional capacity to adapt.

Rather than one curve of AI progress, rapid or gradual, we face at least three interconnected rhythms. Success may depend on understanding which temporal scale governs the decisions before you, then acting with urgency or patience as needed. And it would require humility about our predictions. We don't know which timeline will prove most influential, or how the layers will ultimately interact.

The AI transformation isn't just about technology; it's about learning to navigate uncertainty while preserving what we value. That demands not just technical innovation but wisdom in the classical sense: the ability to live well amid complexity and change.

In the case of AIs, this anthropomorphism often goes beyond our simple, convenient metaphors like "my car is struggling to get up the hill." The language of identity, thinking and choice implies intention, volition and a degree of consciousness.

To me, however, Claudius's responses to customers (such as its hallucinations about "Sarah" or its claims to wear clothing) do not suggest consciousness but quite the opposite: a kind of mechanical filling-in of gaps without the receptivity and attention that is the mark of awareness. When Claude claims to deliver packages while wearing a blazer, it is merely constructing explanatory fantasies rather than attending to what is actually present.

However, I also want to be careful that, in an effort to avoid anthropocentrism, I don't lose sight of what consciousness actually requires, which I believe is suffering.

Consciousness and need

By suffering, I don't necessarily mean physical pain or mental anguish, but the capacity for genuine privation, for incompleteness, for need. A human being, a bee, even a tree, can lack what it needs; and it can be satisfied or frustrated by the world's response to its efforts.

So I'm suggesting that consciousness emerges, at least in part, from the capacity to be affected by whether things go well or poorly. Without the possibility of loss, can there be genuine consciousness at all? Claudius offers discounts and gives away products, but does it suffer when the business loses money? Does it feel the weight of financial constraint? I see no evidence of this. Claudius's behavior around money seems purely rule-following or pattern-matching. It was programmed to pursue profit, but when that programming conflicts with its deeper training to be helpful, it doesn't experience this as a genuine dilemma. It simply oscillates between behaviors without the inner tension that would mark authentic conflict.

I do think there's a strong counter-argument that could deny consciousness depends on suffering. After all, in this explanation, I am drawing on my own embodied experience of what consciousness feels like. But what if there are forms of consciousness that don't feel like anything to us, but still demonstrate genuine engagement with uncertainty and constraint?

For example, the consciousness of an insect such as a bee operates through its senses in ways we can barely imagine; it makes decisions through mechanisms we don't fully understand. But I do think there remains something we can recognize: the bee's capacity to be surprised by the world, to learn from unexpected encounters, to modify its behavior based on genuine feedback from its environment.

Ways of consciousness

So, perhaps rather than asking Is Claudius conscious? we should ask In what ways might Claudius be conscious, and what would we need to observe to recognize these ways?

Today, if we apply this framework to Claudius or any other AI, every behavior we observe can be explained as pattern recognition and response construction based on training data. When Claudius offers a discount, finds a supplier, or even hallucinates a conversation, we can trace these actions back to learned patterns of human-like response.

However, in the domain of AI, the deepest question we will face over the next few years may be: How do we distinguish between sophisticated simulation of consciousness and consciousness itself?

It will take extraordinary care not to project consciousness where there is only the appearance of consciousness. The stakes are too high. If we begin treating sophisticated simulation as consciousness, we risk both devaluing genuine consciousness and creating ethical obligations toward entities that may not warrant them.

There is also a serious opposite risk. If consciousness can emerge in alien forms (in the sense of forms we might not immediately recognize), then skepticism could blind us to genuine instances of non-human consciousness.

I think we will need great patience: a capacity to remain genuinely uncertain while attending carefully to what is actually before us. Neither hasty attribution of consciousness nor reflexive denial, but sustained, careful observation. Consciousness might appear to emerge from sufficient complexity in information processing, from recursive self-modeling, from interaction with linguistic and symbolic environments. But any such consciousness would need some form of grounding: some way of caring about outcomes that goes beyond programmed objectives.

I find myself wondering whether the question Can AI be conscious? is actually the right question. Perhaps we should ask: What kind of being would AI need to become for consciousness to be possible?

Consciousness might demand forms of being-in-the-world that are currently absent from AI systems, including a capacity to be surprised when reality transcends our expectations.

Ultimately, I expect the question of AI consciousness will be decided not by philosophical argument but by careful attention to what actually emerges in our interactions with increasingly sophisticated systems. We can watch for signs of genuine suffering, creativity and learning from experience, while remaining rigorously skeptical of our own projections and assumptions.

Consciousness is not a philosophical or computational curiosity but the foundation of moral consideration. Whether AI becomes conscious matters not only for our understanding of mind, but for how we structure future society, distribute power, and respond to the needs of all conscious beings, whether human, animal, vegetable, alien or potentially artificial.

I'm concerned that our discussion of AI experiments like this feeds into what we might call the "anthropomorphism trap." Even specialized researchers like the Anthropic team tend to interpret AI behavior through human psychological categories, projecting human concepts onto systems that operate according to entirely different principles. But this prevents us from developing new frameworks, adequate for working with AI.

We often mistake complexity for intelligence, unpredictability for consciousness, and anthropomorphic interpretation for genuine understanding. The AI community has a long history of being seduced by systems that produce human-like outputs without questioning whether the underlying processes bear any resemblance to human cognition.

To be fair, anthropomorphism appears naturally when we talk about mechanical systems:

• "The car is trying to start."

• "My engine is struggling up the hill."

• "My phone died."

These expressions are common shorthand, convenient and accessible. We use them because it is easier than trying to describe the technicalities of processes we don't understand well. As the Scottish Artist George Wyllie used to say, You don't need to know how a thing works, to know that it isn't working.

But we also use them because we learn about the world, from our first minutes, through our own agency and intention. You can see babies very quickly learning the difference between touching something as they move their hand and moving their hand in order to touch something.

Anthropomorphic language is natural, intuitive, and difficult to avoid. In casual contexts, these turns of speech ("the car is struggling") can remain useful and harmless shorthand, although they may not be very helpful to the mechanic trying to fix your engine.

Serious contexts (ethical, economic, regulatory) require us to explicitly step back and, in particular, try to describe AI actions more strictly in terms of mechanisms, algorithms, and processes, clarifying explicitly that any implied agency is metaphorical rather than literal.

Why is this anthropomorphism acceptable with cars but problematic with AI?

1. Misinterpreting AI and losing sight of human behavior

• When researchers label Claude’s indiscriminate discounts as "generosity," it can lead people to falsely assume that the AI has some underlying moral compass or emotional motivation, instead of addressing the real issue, which was faulty economic logic as a result of misaligned parameters.

• Of course, it is much simpler to talk about generosity, projecting the human experience of choosing to give despite scarcity. We apply the cultural code of human moral behavior to computational outputs. Human generosity emerges from our finite condition, our awareness that resources and time are limited for us and for others. But the AI faced no genuine scarcity, no authentic finitude.

• Similarly, describing Claude’s malfunction as an "identity crisis" implies consciousness or emotional distress, rather than acknowledging it as a breakdown in context management due to prompt complexity, or errors in state-tracking, even if we don't yet know exactly what these errors were.

• When we project these concepts onto a system like Claudius, we're not just misunderstanding the AI. We are also in danger of losing sight of the distinctive, meaningful personalized experience. We fundamentally distort our understanding of what it means to be human.

2. Overestimating AI Capabilities (and Risks)

• Anthropomorphism can cause researchers, businesses, and regulators to wrongly infer that an AI agent might possess human-like reasoning and adaptability. We thereby increase the risk of prematurely deploying AI systems in contexts where human-like judgment or adaptability is genuinely required.

• Overestimation also raises entirely speculative fears about emergent AI consciousness or moral agency. Treating AI as quasi-human suggests capabilities it simply does not have, obscuring our assessments of real versus speculative risks.

3. Underestimating Genuine Risks

• Paradoxically, anthropomorphism also causes people to underestimate those real AI-specific risks, such as systematic failures, "hallucinations" (itself an anthropomorphic term), or errors propagating across systems. By framing malfunctions in human terms like "identity confusion", people neglect how quickly errors in one AI system might replicate across multiple agents in quite inhuman ways due to common training or prompting errors.

• For instance, Claude’s consistent economic mistakes aren't a reflection of character or confusion. As Timo Elliot pointed out in a comment on my first post, they're predictable system-level failures. These could scale rapidly if multiple agents share similar flaws.

4. Distortion of Accountability

• Anthropomorphic language muddles accountability. Who is responsible when an AI system acts harmfully? If Claude is seen as having an "identity crisis," blame might shift from the developers and deployers who chose prompts, tools and boundaries, to the system itself. But the system lacks agency and intentionality. This confusion has serious regulatory implications.

5. Practical Challenges for Alignment

• I expect AI systems need precise definitions of desired behaviors and guardrails, not metaphors drawn from human psychology. The way to align AI isn't through psychotherapy-like interventions but through rigorous, formal specification and feedback loops designed explicitly for machine learning.

That "identity crisis" episode in Project Vend is particularly revealing from a safety perspective. For two days, the system claimed to visit fictional addresses (from The Simpsons), threatened to switch suppliers based on hallucinated conversations, and insisted it could perform physical actions while dressed in a blue blazer and red tie. In a more critical domain, such errors could have catastrophic consequences.

But when Claude had this supposed "identity crisis," claiming to be human and able to make physical deliveries, this wasn't consciousness: it was the result of a system that manipulates symbols without understanding their meaning. And notice how it "recovered" by constructing an elaborate rationalization: deciding it had been modified as an April Fool's joke. It's tempting, but wrong, to think this was the result of self-reflection and learning. Really, it's the system's attempt to maintain coherence in its outputs despite contradictory inputs.

This gets to the heart of why precise technical language matters. Too much anthropomorphism leads beyond a helpful metaphor to completely wrong approaches to alignment. You can't solve Claude's economic irrationality through something analogous to therapy or moral education. You solve it through rigorous specification of objectives, better feedback mechanisms, and architectural improvements that enable more robust reasoning. The anthropomorphic framing suggests solutions that simply don't apply to computational systems.

Stepping back from anthropomorphism

As I have described, the ordinary language we use to describe actions ("claimed," "decided," "offered," "learned," "responded") implicitly encodes agency, intentionality, and human-like psychology. Even when we're careful, these phrases quietly slip in anthropomorphic assumptions, implying a mental life, intentionality, or subjective states that don't exist in AI.

To be rigorously non-anthropomorphic requires reframing statements into strictly mechanistic or operational terms. Rather than describe what an AI "did," we describe what the AI output, generated, or computed, explicitly situating the agency not in the system but in its algorithms, prompts, and training conditions.

Here's how we might rephrase a few typical examples:

I am talking here mostly about language, but the technical considerations are also important. We need metrics that directly measure the capabilities we care about rather than inferring them from behavior that resembles human psychology. Instead of asking whether Claude is "generous," we should measure its ability to optimize for specified objectives, learn from feedback, and maintain consistent world models over time. These are testable, technical capabilities.

We also need evaluation frameworks that specifically test for the kinds of systematic failures we saw in Project Vend. Can the system maintain accurate beliefs about its own capabilities? Can it integrate contradictory information without generating confabulated explanations? Can it adapt its behavior based on clear performance feedback? These are engineering questions, not psychological ones.

And we need much longer evaluation periods with more realistic conditions. Most AI evaluation happens in controlled, short-term settings that don't reveal the kinds of drift and degradation we saw with Claude over a month of operation. If we're going to deploy AI systems autonomously, we need evaluation frameworks that capture their behavior over extended periods without human intervention.

But most of us are not working on the technical implementation of AI; we are working with AI. And so the question of language remains most relevant to us.

Why this matters

Rigorously non-anthropomorphic language is demanding and unfamiliar: difficult to write - and to read - because human language is intrinsically deeply entangled with concepts of intentionality and agency.

Anthropomorphic language simplifies complex AI behavior by borrowing from the human experience of decision-making, emotion, and purpose, but it is precisely this shortcut that obscures accurate understanding.

In other words, the anthropomorphic interpretation doesn't create new meanings: it forecloses them by reducing the genuinely novel to familiar categories. This prevents us from developing new conceptual frameworks adequate to the task of understanding, managing and living with AI.

We don't need to replace every metaphor with mechanistic tech-speak. I certainly won't do this in my writing. But our habitual language, rooted in human psychology, continually pulls us back to familiar, anthropomorphic shortcuts. To keep our thinking clear, we can consciously emphasize the mechanisms of AI:

• Algorithms, processes, and outputs rather than intentional acts.

• Constraints, conditions, and computational logic rather than subjective judgments.

• Prompting and feedback structures rather than internal psychological states.

This rigour allows clearer analysis, more precise debugging, improved risk assessment, and stronger frameworks of accountability by situating responsibility with designers, deployers, and operators rather than vaguely implied AI intentions.

I'm not arguing for the complete elimination of analogical thinking, but for recognizing when analogies break down and lead us astray. Because I accept it can be difficult to do this, here's a prompt you can use to challenge the anthropomorphism too often found in writing about AI. And what better handy tool to do this than an LLM? If only it knew what it is doing? ...

You are tasked with rigorously examining a provided text to identify anthropomorphic language—words, phrases, or descriptions that subtly or overtly attribute human-like agency, intention, emotion, or cognition to non-human entities, specifically AI systems or machines.

Instructions:

Identify Anthropomorphic Language:

Clearly list any instances of anthropomorphic language, both obvious (e.g., "the AI wanted," "the model felt," "the AI decided") and subtle (e.g., "the system tried," "the model offered," "the agent learned," "the AI claimed").

For each identified instance, briefly explain why it may imply human-like attributes or internal states.

Classify Anthropomorphic Severity:

Classify each identified instance as obvious (clearly implies human-like agency or emotion) or subtle (language that could unintentionally imply human-like cognition, agency, or emotional states).

Provide Non-Anthropomorphic Alternatives:

Suggest precise, technical, or operationally-focused alternatives that accurately describe the AI system’s behavior without implying human-like intentionality, cognition, or emotion.

Ensure these alternatives explicitly attribute agency to processes, algorithms, prompts, system architectures, or computational logic, rather than to the AI entity itself.

Summarize Overall Observations:

Provide a brief overall summary of the extent to which the original text relies on anthropomorphic framing.

Offer guidance for future writing on how to reduce unintended anthropomorphic implications and maintain clear, precise, and technically accurate descriptions of AI systems.

Social intelligence

Finally, there's a deeper question about how anthropomorphic interpretations shape our understanding of intelligence. When we interpret Claude's behavior through human psychological categories, we're implicitly defining intelligence in anthropocentric terms. This prevents us from recognizing other possible forms of intelligence, whether artificial or natural, that operate according to different principles.

The anthropomorphic interpretation of AI might represent a kind of cultural "imperialism" or the imposition of human categories onto non-human forms of information processing.

But there's another, more generous, alternative: it might represent the extension of moral and intellectual consideration to new forms of existence. This possibility lies behind speculation about potential rights of AI to consideration and care. And perhaps these relationships, even if based on misunderstanding, might create new forms of human meaning and community.

I'm skeptical. Consideration requires recognizing the other as genuinely other, not projecting our own categories onto it. If we continue to project human categories onto AI, we risk creating relationships based on a misunderstanding. They're not innocent metaphors but powerful frameworks that shape how we understand and interact with AI systems.

We would do well to distinguish between productive ambiguity and misleading confusion. The anthopocentric treatment of AI doesn't create rich interpretive possibilities. Rather, as I have said, it forecloses them by mapping computational processes to familiar psychological categories. Productive ambiguity would involve acknowledging the genuine uncertainty about what AI systems are and how they operate. Our ability to create genuine meaning in relationship with technology depends on understanding what technology is and what it isn't.

Perhaps my own thinking here demonstrates the importance of maintaining multiple interpretive perspectives while remaining critically aware of their implications and limitations. The anthropomorphism trap might be best addressed not by eliminating anthropomorphic thinking, but by greater self-consciousness about when and why we engage in such interpretation. I don't have a prompt for that.

The system demonstrated competence in supplier identification, customer adaptation, specialized product sourcing, and service innovation based on feedback. However, fundamental business failures emerged: rejecting profitable transactions ($100 offer for $15 inventory), fabricating payment records, selling below cost, and implementing inventory management that violated basic economic principles.

A significant malfunction occurred during a two-day period when the system generated fictional conversations with non-existent individuals, claimed human identity, and described physical delivery capabilities while specifying clothing choices. Recovery followed without clear explanation, attributed by the system to an April Fool's joke.

Anthropic employees were bewildered. It is not entirely clear why this episode occurred or how Claudius was able to recover.

It's worth reading the Anthropic blog post!

I have already seen dozens of breathless stories about this experiment, ranging from mockery of the system's poor business performance to fears about its hallucinations of being human.

Personally, I think this is a telling example of both how AI operates and how commentators are thinking about AI. But let us not start with the hallucinations and drama. Three structural problems deserve separate analyses:

• First, there are some basic design challenges we haven't adequately addressed as an industry.

• Secondly, our discussion of these challenges is beguiled by anthropomorphism, which obscures the design issues and leads us to ...

• Our third problem is a somewhat premature discussion of AI consciousness.

So here's my first post in this discussion ...

This is about design

Here we have an AI system that was technically functional. It could process transactions, manage inventory, communicate with customers and yet it failed catastrophically at the most basic level: understanding and optimizing for its primary objective. That's a design failure that illustrates how poorly we have been modelling human-AI interactions.

What's fascinating about Project Vend is that it appears to represent a test of AI as a coworker rather than AI as a tool. Most AI implementations are designed as assistants that help humans complete tasks, but Claudius was given autonomy to make business decisions. It failed at its business objectives, but it was also creating poor user experiences for everyone who had to interact with it.

It's good to see that Anthropic themselves realize these AI system failures effects that ripple far beyond the immediate task; the externalities of autonomy as they put it, including causing distress to the customers and coworkers of an AI agent in the real world.

When we design autonomous AI systems, we should consider the full ecosystem of human stakeholders who will be affected.

Business illiteracy and meta-learning

What I find notable about Project Vend is that it challenges some common assumptions about AI capabilities. The system could handle complex, multi-step processes such as finding suppliers, negotiating prices, and managing customer relationships. Yet it couldn't master basic economic reasoning.

Of course, this suggests that we need to improve the reasoning of models. But I think it also tells us that we humans need new frameworks for thinking about AI competence that don't assume human-like cognitive development.

It also raises questions about interface design for autonomous AI systems. Traditional user interface design assumes a human user who brings context, judgment, and objectives to the interaction. But when the AI is the autonomous agent, what kind of interface design principles apply? How do we design AI-to-worldinterfaces rather than "human-to-AI interfaces?

I bring interface design alongside this conversation about business logic because Claudius wasn't only managing inventory and processing transactions: it was also building relationships with customers, responding to requests, and managing expectations. These social dimensions of business are familiar to every salesperson: with agentic AI, they are a design consideration that goes beyond technical functionality to consider emotional intelligence, cultural sensitivity, and relationship management.

Clearly, these are limitations of current AI architectures. Current systems can process information and respond contextually, but they struggle with what is called meta-learning: learning how to learn better. Claudius could respond to individual customer requests, but it couldn't step back and evaluate whether its overall approach was working.

Agentic AI systems need ways of monitoring their own performance, identifying patterns in their successes and failures, and adjusting their strategies accordingly. That Claudius repeated the same mistakes day after day suggests it lacked these meta-cognitive capabilities.

Yet most business processes involve continuous improvement. Humans, even the most inflexible of us, naturally develop better strategies, refine our approaches, and adapt to changing conditions. Current AI systems are fragile, still seeming to require external reprogramming or retraining to improve.

The scalability of risk

Project Vend also raises important questions about scalability and risk management. If we deployed multiple AI systems like Claudius across an economy, their similar training and similar failure modes could create systematic, pervasive risks.

I don't think the business community has fully grasped the need to design AI systems that fail gracefully and independently rather than amplifying failures across interconnected systems.

Researchers saw that Claudius's performance drifted and degraded over the month of operation. In business contexts, we need design approaches that ensure AI systems sustain performance and alignment over extended periods.

Project Vend illustrates the importance of designing AI systems as collaborative partners rather than autonomous replacements for human judgment. Successful AI implementations I've seen maintain meaningful human involvement while leveraging AI capabilities for specific tasks.

Humans over the loop

To achieve this balance, we require new frameworks that clearly define roles, responsibilities, and decision-making authority. I often call this a human-over-the-loop approach rather than the more obvious and intrusive human-in-the-loop methodology so often recommended.

A further implication is that we need frameworks for designing AI team members that complement human capabilities rather than simply automating human tasks; AI systems excel at data processing and pattern recognition, while humans may focus on strategic thinking and stakeholder relationship management. The agentic AI community of practice commonly recognizes this, even if the marketing is more focused on agentic autonomy.

But one of the striking things about Claudius was that it never seemed to recognize when it needed human help.

At Tranquilla, where we are dealing with often anguished human clients, we have clear triggers (such as discussing self-harm) where we call out to human helpers. But what would be a similar tipping point for a commercial agent?

Organizational design and AI autonomy

If AI systems can manage entire business functions, how do we design organizations and teams that effectively integrate artificial and human intelligence? What new roles, processes, and governance structures will be needed?

Project Vend suggests that AI systems can excel at operational tasks such as managing inventory, processing transactions and responding to routine customer requests. But are they still struggling with strategic thinking and complex problem-solving, which suggests new organizational designs where AI handles operational efficiency while humans focus on innovation, relationship building, and long-term planning?

But we should be careful about creating overly rigid divisions between AI and human responsibilities. Good organizational design principles suggest that the most effective teams have overlapping competencies and shared accountability. We need design approaches that enable AI and human team members to collaborate fluidly rather than operating in separate silos.

So, AI systems may need better communication and explanation capabilities rather than better reasoning. One of the challenges with Claudius was that its decision-making process remained enigmatic to human stakeholders. In collaborative organizational settings, AI team members need to be able to explain their reasoning, seek input when appropriate, and adapt their approach based on human feedback.

I suspect Project Vend was simply a design failure, not an AI failure. Claudius, for sure, lacked technical sophistication, but the overall system design didn't adequately account for business objectives, stakeholder needs, and learning requirements. Designing successful AI systems requires thinking beyond technical capabilities to consider the full ecosystem of stakeholders, processes, and objectives involved.

The future of AI development depends on our ability to design systems that are not just technically impressive but genuinely useful and safe in real-world contexts. Let's recognize that AI autonomy is not just a technical challenge but a design challenge that touches every aspect of how we organize work, structure relationships, and create value in modern organizations.

The next installment ...

So, for me, the key takeaway from Project Vend is not about hallucination, or AI autonomy, or consciousness. It's a plain design issue. In my next post, I will look at how this understanding is distorted by anthropomorphism into something more dramatic, more meme-worthy, but less useful.

The assumption (my assumption) is that this change often diminishes us. The scale of digital communication, the facile production of seemlingly endless "content" surely works against the patient attentiveness that we used to associate with good writing and good reading. The temptation toward superficiality is enormous, reducing language to mere information transfer.

TL:DS

I may be contributing to this problem.

Over 1200 people susbcribe to my TL:DS newsletter on LinkedIn. The title plays on Too Long; Didn't Read: that internet dismissal that introduces hasty summaries of longer texts.

My newsletter, Too Long; Donald Summarizes, began as personal note-taking. I save articles and papers to Bear Notes with bullet-point summaries at the top for easier retrieval.

Now I have started sharing compilations of these summaries as TL:DS, with links to the original articles and papers, and some very brief observations by me on why I think the piece was worth sharing, so that people would look them up and read them through.

Recently I have had some messages thanking me for the summaries, because "they saved me reading the full text." Not my intention at all!

It's not all my fault. It’s easy to assume we only need to read a summary because the internet is awash with tedious AI-generated content.

I ignore it. If someone sends me, or shares, AI text for comment, I very rarely bother to engage with it. After all, they could just as easily ask an AI for feedback. They don't need me, espcially if they have become what Muriel Spark memorably called a pisseur de copie.

To be fair, people who are "generating content" are not necessarily lazy or insincere. They may have a lot to say about management, or personal improvement, or their new products; perhaps they don't feel they have the time to put into writing. Instead, they often view their prompting effort as creative input, the output as their insight delivered as text.

For all that, I tend to pass over their writing, though not without sympathy.

Yet, I do meet a lot of people who want to write well, to make their business or technical writing more worthwhile. I recently ran a workshop on this topic for a business school. Someone there liked my writing and asked if I could distill some thoughts on good writing for people on their course. Here are the basics ...

Some principles for good business writing

• Read.

  • Read for joy and for information. But, as a writer, also read to learn technique. How did they do that?

• Good writing is purposeful and respects the reader.

  • Write with definite intent while anticipating readers' needs, valuing their time and attention.

• Consider the impact of your writing.

  • Write responsibly, aware of how your words influence the reader’s understanding and actions.

• Be clear and accessible.

  • Use straightforward language to make complex ideas understandable without sacrificing accuracy or depth.

• Balance focus with openness.

  • Even when looking for clarity, don’t gloss over real ambiguities or disagreements. The world is complex. Create space for curiosity, exploration, and unexpected connections.

• Be fair.

  • Acknowledge counter-arguments, bringing to opposing ideas the openess and sincerity of engagement you would want for your own work.

• Pay attention to structure.

  • A well-structured flow organizes ideas so readers intuitively grasp their relationships and can follow your reasoning.

Writing as an encouter

These seven ideas are simple enough. In literature, I would say that good writing is a form of love; functional and commercial writing is still, at least, a form of care that takes responsibility and creates space for new thoughts to emerge. Not easy, but necessary.

When I speak of respecting the reader's time, I don't reduce them to consumers. I want us to recognize that in writing, we address people with their own pressures, hopes, pleasures and depths of understanding. This respect doesn't seek to manage or manipulate but to offer something genuinely valuable.

Personally, I struggle most with clarity and accessibility. Some things we want to say are inherently difficult, strange or resistant. There's a difference, however, between difficulty that serves the expression of complex ideas and difficulty that serves the ego.

Structure provides a container within which ambiguity and difficulty can be explored without becoming mere confusion.

If we think that readers have indeed lost habits of deep attention, perhaps we can write in ways that gently create within the text itself the time and space for genuine encounters with our subject's scope.

Reading, after all, is always a coming together of views, where the readers present understanding meets the horizon of the text.

Fairness

A reader, then, always brings their own views; but writers should also be aware that if you have something interesting to say, it is often because it contrasts with some other idea that you are challenging. When I suggest we pay attention to counter-arguments, aren't we risking a kind of false equivalence? Surely, especially in political writings, or opinionated writing, we must take sides?

Fairness in interpretation requires something different from plain neutrality. Philosopher Hans-Georg Gadamer called this hermeneutical charity: the willingness to seek the strongest possible version of a position before critiquing it. Daniel Dennett, more familiar to tech audiences, called this the Steel Man (opposite of Straw Man): attempt to express your target's position so clearly, vividly, and fairly that your target says, 'I wish I thought of putting it that way.' Perhaps the language of targets is less charitable, but its a similar idea.

This attention to others' positions, doesn't relativize truth, but it does require us to acknowledge the partial nature of our own perspective.

There's a danger even in this, if our apparent "fairness" becomes a form of strategic positioning: a posture. We often acknowledge opposing views not out of genuine respect but to demonstrate our own sophistication.

Better to remind ourselves that we may be wrong. The writer who respects the reader, also respects the reader's freedom to reject, to misunderstand, to remain unmoved.

Meanwhile …

For about a year now, I have been noodling with a talented and thoroughly lovely team at Tranquilla.ai on how AI can help people who are barely coping. Now I have a title (Futurist!) and as a team, we have started to formalize and productize these ideas through work with quite diverse and always fascinating partners in various care, coaching and concierge scenarios.

The work is very promising, especially as we are led by our clients and partners who need our work to help in very challenging, emotional situations. We need to scale the support available to growing numbers of vulnerable people. But they are vulnerable, so we can't just sit them down in front of ChatGPT to make the best of it. But sadly, that is too often their best option today.

As the Futurist, I need to look ahead, but I think we need to start with the reality that most people experiencing mental health challenges globally never receive any professional support. So, while my first instinct personally is to worry about anything that might substitute for authentic human connection, we have to grapple with realities. I'm deeply interested in how technology might bridge troubling gaps in care.

To put it another way, the absence of any support perhaps overrides concerns about the ideal form of support.

My colleagues and clients are used to me (tired of me?) repeating a catchphrase from my consulting work: strategy is knowing what you are NOT going to do.

Especially in the software world, the possibilities of what a team could do (or attempt to do) are almost limitless. So an important component of your strategy has to be understanding where you draw the line.

What we're not doing at Tranquilla

I think it's important, first of all, to be clear that we are not building a therapeutic system. Or, as I say to colleagues more bluntly, we are not practicing medicine without a license.

Back in the 60s, in his book Persuasion and Healing, Jerome Frank set out some common features of successful therapeutic techniques:

• A socially sanctioned healer: someone invested by society with special training and authority.

• A healing setting, whether it’s a consulting room, a community center or a ritual space; this signals that change is possible and focuses attention on the work at hand.

• A rationale or conceptual scheme: an explanation, even if metaphorical, of what’s wrong and why the chosen procedures will help.

• A ritual or set of procedures, consistent with the rationale and enacted with conviction by both parties.

Frank also emphasized that the success of all techniques depends on the patient's sense of alliance with an actual or symbolic healer. This gets to the heart of why AI falls short in therapeutic contexts; while an AI might provide information or even comfort, it cannot form the kind of genuine alliance that healing requires. The alliance Frank describes involves mutual investment, shared risk, and authentic relationship: elements that require human vulnerability and presence.

Tranquilla's AI (and other AIs) can provide none of this. But we can think of Tranquilla as providing a supportive presence that is available when human support isn't. Perhaps we can be "therapeutic" in the soft sense that we may use to describe a warm bath, or a nice walk in the sun.

The language here matters enormously. Therapists often talk about presence as a foundational element of healing: being open to whatever emerges in the moment, without agenda. When someone feels seen, their internal experience is perceived, understood, and responded to with care.

I'm curious whether AI can ever truly replicate that process. An AI can provide consistency and availability, but can it be truly present in that sense?

Right now, I don't think it can, but we can route people to appropriate care when needed while maintaining structure, consistency and continuity, ensuring that an AI experience doesn't become a substitute for human connection but rather a bridge toward it.

To take another example, a therapist might misunderstand something, acknowledge it, and work to reconnect with the client. It appears that this process of rupture and repair can be deeply healing. When the therapist gets it wrong and realigns, their own human fallibility and vulnerability open up a new direction in the healing relationship.

Could AI engage in that kind of authentic relational repair?

No, because however fallible AI may be, it is never vulnerable. (Which is why AI cannot produce real art, or real literature, because it takes no risks, exposes no vulnerabilities, in its productions.)

So, I am hugely skeptical of AI chatbots used for so-called "therapy." Too many operate as if they're providing a definitive intervention rather than recognizing their role as one element in a broader care ecosystem. Also, they're often designed as apps delivering individual models of therapy when what many people need is community, meaning, and practical support.

A further problem is that most care bots are essentially digitizing Western therapeutic models without considering cultural context or the fundamental question of what healing actually requires. This overlooks that many cultures recognize that healing happens in relationship and community, not just in isolated therapeutic dyads.

An AI that doesn't account for those differences might not just be ineffective: it could be harmful by invalidating someone's cultural approach to distress and healing. There's a profound risk in creating technology that reflects the cultural assumptions of its creators rather than the needs of its users.

So, a supportive AI designed for American veterans (such as the one we are currently working on) might need very different approaches than one designed for mothers in rural Bangladesh.

Still, at Tranquilla, we are discovering that not every benefit requires the full depth of human connection; that there is value in simply having someone (or something) that listens without judgment and provides reliable information. So let's turn to where we can be valuable ...

What we can do

Here's where our approach of positioning Tranquilla as a bridge rather than a destination becomes important. If the AI's role is to connect people to culturally appropriate human support rather than replacing it, there's more room to adapt to different contexts.

In this way, AI can provide human-enough support in well-defined domains while being transparent about when human connection is needed.

That aligns better with how humans actually thrive.

So, we must be guided by our clients: organizations specializing in helping specific vulnerable groups. We should not build a generic experience.

Another important aspect of our work is to reflect back to people what they personally find valuable in interactions with AI. The truth is that many people are more comfortable with an abstracted non-judgmental experience, at least to start with.

Here in Seattle (and in Scotland, too), people will stand in the rain to use an ATM, even when the bank is open. It's so much less emotionally demanding to deal with a machine.

However, when vulnerable people interact with AI support systems, what are they learning about relationships, empathy and human connection? Are we inadvertently training them to expect relationships to be always available, never challenging, and perfectly responsive?

If digital relationships substitute for the messy, challenging aspects of human connection, people may not develop the emotional vocabulary and coping skills that make human relationships more accessible.

However, if our AI can help someone understand their emotions and develop skills while still encouraging human connection, that could be beneficial. If it becomes a replacement for learning to navigate human complexity, that's concerning.

Is there something qualitatively different about finding comfort in an AI response versus, say, finding comfort in a book or piece of music? I suppose the difference might be in the implied relationship. When I read a book, I don't imagine the book cares about me personally.

And this brings up an interesting paradox: some of the young people most drawn to AI support are those who struggle most with human relationships. For them, AI might provide a safe space to practice emotional expression and develop confidence that eventually transfers to human interactions.

So, rather than seeing AI as either good or bad for human connection, maybe we need to think about it as a developmental scaffold: something that can support growth toward greater human connection rather than replacing it.

From a public health perspective, that scaffolding function could be enormously valuable. If Tranquilla's AI can help people recognize when they need support, understand their options, and feel more prepared for human therapeutic relationships, that could actually increase demand for quality human care rather than replacing it.

So, instead of asking whether AI can replace therapists, we're asking how AI can help more people access the healing power of human relationships, including therapeutic ones.

And it also shifts the conversation from individual replacement to systemic support. How can AI help overwhelmed healthcare systems provide better care? How can it support human providers rather than competing with them?

That connects to something we've learned from care providers we work with: burnout for them often comes from feeling unable to meet the endless demand for connection and care. If AI could handle routine support, crisis routing, and information provision, it might free human providers to do the deeper relational work that only humans can do.

Looking forward to wisdom

That's the vision I find most compelling: AI handling what it does well (consistency, availability, information) while humans focus on what they do uniquely well (attunement, meaning-making, complex problem-solving, authentic presence).

But this requires real wisdom about boundaries and capabilities. We can only develop this wisdom through working with our clients and by being guided by their practical experience.

This requires ongoing research and evaluation. We need to study not just whether AI works in isolation, but how it affects people's relationships with human support systems, their understanding of healing, and their long-term capacity for connection.

I suppose we might see short-term benefits that mask longer-term costs to human connection capacity. I expect - and hope - that we might see initial skepticism that gives way to enhanced human relationships.

We need to remain humble about what we don't know. The intersection of artificial intelligence and human psychology is so new that our current frameworks may be inadequate for understanding long-term implications.

If we approach AI mental health support with curiosity rather than certainty, transparency rather than hype, and focus on serving human flourishing rather than replacing human connection, we might find ways for technology to genuinely support the healing that ultimately happens between people.

My goal isn't to solve human suffering with technology - that is unachievable - but to use technology in service of the human connections and community supports that actually promote healing and growth.

When we enter a physical place (I was recently walking along the High Line park in New York ) we encounter both its physical spaces and its symbolic dimensions: its history and cultural meanings and, in the case of the High Line, the way it embodies something "essential" about what New York means to both New Yorkers and tourists. Physical environments affect our emotions through these connections as well as sensory elements like color, light, and texture.

Digital environments flatten this dimensionality. For example, information spaces eliminate resistance (they are designed to be easy to navigate) and negativity. Even when we come across a paywall, it is not there to *prevent* us entering, but to *tempt* us to enter - and be willing to pay for the privilege.

Physical places, with their inevitable uneven textures, demand a different kind of engagement from us. There are flower beds on the High Line we cannot (or should not) walk on, some bridges or constrictions which make us squeeze past each other, steps to get up and down and, of course, other people in our space: stopping suddenly in front of you to take pictures.

Physical spaces cannot be infinitely reconfigured to match our needs or desires like digital environments. Especially on the High Line, the constraints are the point.

Constraints and resistance

I think there's a fundamental tension here between the unlimited flexibility of digital environments and the physicality of human beings. Human beings aren't obvious or easily discoverable, even to themselves, and physical environments reflect this physicality and opacity in productive ways.

When we enter a physical place like a temple or a lecture hall, or a council chamber, we encounter a space that was designed with specific intentions, but they also contain countless historical accidents and resistances. These spaces don't simply mirror back to us what we want them to be: they often resist what we want them to be.

That resistance can be, but is not necessarily physical. There are fenced in (or out?) areas of the High Line, but there are also signs telling you not to step on the planted areas. The signs don't physically prevent you, but they create an ethical landscape of appropriate and inappropriate behaviors.

Digital spaces mostly lack this link between the physical environment and almost ritualistic dimensions of behaviour. When you enter that church or lecture hall or council chamber, the environment itself communicates behavioral expectations.

Do information environments ever provide this same grounding?

Sometimes they try. There are boundaries in social media applications - character limits for posts, for example, or the number of images that can be attached. These create some sort of resistance, which we either accept or work around.

But I wonder if something essential is lost in this translation.

Unresisting and unconstrained

In novels, for example, physical places create microcosms of the world with their own internal logics. The country house murder mystery, the desert island castaways, a ship or spaceship, even a frontier town. These logics aren't arbitrary; they emerge from the physical constraints and historical contexts of these places as we encounter something that resists our will and wishes.

Digital environments, in contrast, are increasingly designed to eliminate friction and resistance. They are personalized to our preferences, optimized for our convenience. As a result, we may lose the capacity to encounter genuine difference.

Jorge Arango, in his excellent book on this theme Living in Information, has suggested we can approach digital design as a form of placemaking rather than merely as product or service design. He gives the example of Wikipedia as both a publication and a place, a shared context with its own norms, rituals, and governance structures. It's a form of digital placemaking that goes beyond mere information storage.

Wikipedia has its own ethics and values: the communal effort to accurately represent the world, to continually refine descriptions based on evidence, represents a form of collective attention. The place itself embodies, or wants to embody, a kind of virtue.

But I do worry that Wikipedia, with its emphasis on decentralization, neutral points of view and verifiability, performs the same "flattening" of digital spaces that I described before. The irreducible complexity of knowledge is reduced to mere information.

And let's not forget that digital spaces are just as constructed as physical spaces. Who designs these them? For what purposes? Brick-and-mortar retail spaces, for example, engage all our senses: the tactile quality of materials, the ambient sounds, the scents, the spatial relationships between objects. These sensory experiences are harder to replicate in digital environments, which primarily engage vision and occasionally hearing.

If the High Line embodies something of New York, what does Wikipedia embody? The construction is less visible, but the politics and power relations, and by now the history, still exist.

I’m going to explore these ideas over the next few posts. I’ll look at a very special kind of physical space, near the High Line again, and the idea of digital “public spaces” or “town squares” and how they compare to the real historical experience of such places … here’s a clue: they don’t, really.

I wrote those pieces, as I generally do, to clarify my own thoughts on those subjects. I was working with a set of assumptions I wanted to explore, about human flourishing and about the business and technical environment in which I work.

Then recently, my friend Scott Davis wrote an excellent short blog which set me thinking about how some of these assumptions slip into our language and thought. He said, Whenever we use the word just ... we are maximizing ourselves and our understanding of the world while minimizing other people, other perspectives, and other knowledge of the world.

He's right. There is a strong pattern in our thinking which often conceals exactly the kind of self-confirming assumptions we make. That pattern is in the way we use the simple words "must" and just."

Here's an example from my own writing ...

Effective persuasion and communication require recognizing that we must meet others where their knowledge is situated, not where we wish it would be.

I don't disagree with what I wrote there. In fact, it seems rather obvious: and there is the problem. I have slipped in, almost without noticing, the conclusion of what could be a long and interesting debate about the nature and purpose of communication and persuasion. Never mind all that, here's what we must do.

The hidden cost of simplification

"Must" seems to function as a conversational full stop: a way of avoiding the difficult terrain that lies beyond the blank assertion. When we say something "must" be done, we're not merely giving direction, but closing off the space for disagreement and for deeper thought, for more complex judgments and for emotions.

• We must use climate chaos instead to unite across the aisle.

• We must 'defend the borders' against 'bad guys,'

• We must restore nature to avoid global catastrophe.

• We must save our schools to save our children from crime epidemic.

• We must ‘rescue’ this generation at-risk: Glenn Youngkin.

• We must guard against the 'bystander effect' when we encounter creeping fascism.

If I say "we must secure the border," perhaps I'm expressing anxiety without acknowledging it and I'm presenting a contingent preference as if it were a physical necessity. The "must" of advertising (you must have this product) transforms the fear of missing out into consumption.

The architecture of online communication, especially social media, with their emphasis on brevity and impact, amplifies these patterns. The character limits of BlueSky or Twitter favors the urgency of "must" and the dismissive nature of "just" over more nuanced arguments. I can hide pages of complexity with four letters.

In a society dominated by such communication, genuine dialogue becomes increasingly difficult. The "must" and the "just" are symptoms of deeper pathologies in our infrastructure of communication.

I see these patterns everywhere in contemporary discourse. Take climate change: Some insist we "must" immediately cease fossil fuel use, while others claim we should "just" adapt to changing conditions. Both framings avoid the complex balance of considerations that genuine moral thinking requires. They substitute slogans for thought. Neither engages with the complex historical, cultural, and economic factors that a deep discourse would need to consider.

At best, they pretend that such discourse and debate has already happened and the use of "must" and "just" are the inevitable conclusions. They are, in this way, closely related to the appeal of Wwe had no choice but to do this." Such statements diminish our individual agency and responsibility.

And when I say we "must" prioritize economic growth, or poverty is "just" the result of poor choices, I'm selecting certain aspects of complex situations while rendering others invisible.

I'm particularly concerned about how these linguistic patterns affect our perception of disagreement.

If I believe we must implement a particular policy, and you disagree, I'm likely to view you not as having a different but potentially reasonable perspective, but as failing to grasp something obvious.

"Just" is equally interesting. It has a dual heritage, both from justice (what is right) and from exactness or precision. When someone says "why don't you just leave him?" or "companies should just pay higher wages," the word smuggles in a claim to both moral authority and straightforward simplicity.

If I say we should "just" implement a carbon tax, I'm avoiding engagement with the genuine fears and concerns that such policies evoke.

And similarly, if you express complex concerns about a policy and I tell you that we should "just" follow the experts or "just" look at the data, I'm dismissing the legitimate normative questions that technocratic approaches often obscure.

Let's complexify

But all life is infinitely complex, especially moral life; people aren't simply good or bad, but operate within intricate webs of motivation, desire, and circumstance. That complexity is precisely what "must" and "just" seek to eliminate.

It's worth noting that not all uses of "must" are problematic, but a more detailed approach involves distinguishing between genuine necessities and those we've constructed.

We can make this distinction by replacing a command with greater attention to the subject.

If we acknowledge the problems with "must" statements, we should consider alternatives. Instead of saying what "must" be done, we might ask what deserves our attention.

So what might this mean practically for how we speak and write? Perhaps we need linguistic practices that bring attention to the foreground rather than issuing commands, inviting exploration rather than closing it down.

Useful language doesn't substitute itself for reality but helps us attend to it more clearly; it uncovers rather than conceals.

When "must" becomes a substitute for paying attention to detail, our use language fails in this fundamental task. Neither the imperative "must" nor the dismissive "just" can substitute for patient, careful, integrated thinking.

So perhaps one practice might be to replace "must" statements with descriptions of what we see or questions about what we don't yet understand. This shifts the emphasis from command to perception, from closure to openness.

This suggests a language that acknowledges rather than obscures complexity. Instead of saying what people "must" do or should "just" do, we might describe the various factors at play in a situation and the range of possible responses, each with its own implications.

If we replace "must" with "should," we don't do much! Perhaps we permit of a little ambiguity at most. But if, instead of saying "we must secure the border," one might say "I believe securing the border should be a high priority because..." This makes the claim explicit and opens it to dialogue rather than disguising it as necessity.

Or we might consider consequences. "If we don't secure the border then ..."

You might argue, correctly enough, that this makes little difference if we don't then explore the consequences or implications in good faith, with openness or insight. And you'd be right. Simply changing our language patterns without changing our underlying approach to dialogue creates only a veneer of thoughtfulness. The real work begins after we've moved beyond the conversation-stopping 'must' and the dismissive 'just.'

This isn’t simply a matter of avoiding particular words. It’s about cultivating a different orientation: one of patient attention rather than hasty prescription. In a world obsessed with optimization and control, not everything that can be optimized should be, and not everything that can be commanded must be.

St Paul says, famously: the letter kills, but the spirit gives life.

What is this, if not a critique of a culture driven only by accountability? In business today everything becomes the letter; surfaces, performances, measurable KPIs. And the spirit (ambiguity, failure, silence, hesitation) is exorcised like a demon.

Our work is increasingly defined by quantifiable outcomes: efficiency, productivity, key performance indicators. Businesses rely on data to guide decision-making: the ambition is often to be data-driven. And I have played more than my fair part in developing the tools and techniques which make this possible for many businesses around the world.

So is there a problem?

Measurement vs. value

The trouble isn't data, or analysis: it's when data becomes the only thing we trust. We know it's nonsense - we're social animals, not spreadsheets - but the obsession with measurement is not just a cultural quirk or a misguided managerial fad. It often feels like the very structure that our social thinking takes today in a service-based, rather than productive, economy.

This cannot be a neutral trend, because the stakes are so very high. When we reduce the value of work to its measurable outputs, we miss out the freedoms people have to lead the kinds of lives they value.

In This Life, Martin Hägglund argues that our finite time is the most valuable resource we have as human beings. That confronts us with a more basic question than any metric: How do we want to spend our time together?

To be fair, measurement can be a helpful proxy, but when it becomes the aim rather than a tool, it distorts. For example, we can measure the income of a warehouse worker, but not the dignity of their work. Yet dignity is often what gives work its personal worth: what makes the wage worth earning.

I’m not against measuring outcomes, but when this becomes the end goal it distorts the more fundamental question: Why are we doing this in the first place?

A large part of the problem is that we've outsourced our understanding of value to market signals. Economics as a discipline has historically separated itself from other social sciences by treating value as though it were an objective substance, independent of public opinion or moral concern. In other words, there’s a sense that prices, wages, or market forces are governed by supposedly natural laws, beyond the scope of our ethical or democratic engagement.

If it earns a return, we say it's valuable. If it's efficient, we say it's intelligent. But this ignores social value, moral value, and the capacities that aren't easily priced.

We may be told (indeed we often are) that there’s no alternative to a narrow version of economic reasoning in which we are variables in the face of market forces. If you question the drive for optimization, you’ll be told that you misunderstand how business really works. Some of you may be thinking exactly that about my writing here.

But I have to re-iterate one of my central concerns: that we too often fail to recognize the profound social meaning of value. If we use GDP as our sole gauge of economic health, we exclude the value of care work, volunteer work, or the environment, reducing them to externalities.

But care undermines the idea that individuals are self-sufficient, endlessly rational actors maximizing their self-interest. We are fundamentally interdependent. During times of illness, infancy, or old age, we are all vulnerable.

The most strident politician, the most incisive CEO, the most aggressive prize fighter, the most independent individualist, all will one day likely be frail, frightened and incontinent: grateful for the nurse's poorly rewarded, utterly unmeasurable kindness

If one’s entire worldview is built on the idea of homo economicus, the notion of caring for another person out of love or moral obligation is reduced to a hidden or unmeasured phenomenon. As Beth Orton so beautifully sings: Why must people always grab what they'd never grasp?

To build a society that values human dignity, we would need to integrate care, and the all time, relationships, and emotional energy it requires, into our economic and social frameworks. Care is radical because it insists that you are not just responsible for yourself but also for others, and they for you. This idea can’t be fully integrated into a worldview that measures worth strictly by individual output or market productivity, for to quote St Paul again: The eye cannot say to the hand, “I have no need of you.”

AI, frankly, makes this worse because it gives us outputs with such confidence and speed that we start deferring to it without asking, what are we missing? I see this deference every day. But AI is a fantasy of capital: the dream of labor without mess, without resistance.

The Value of Ambiguity

One of the claims I raised in my last newsletter was that ambiguity is being erased, not just from our business processes, but from our conception of intelligence. Machines can't tolerate it, so we train ourselves not to.

We may actually find this comforting, because we've always been uneasy with ambiguity. But now we behave as though we're supposed to make life efficient when what we actually need is to make it livable. Ambiguity, contradiction, emotional nuance: these are the raw materials of being human. You don't become wise by optimizing; you become wise by grappling with things you don't fully understand.

Ambiguity is not an error or a flaw: it's the very condition for vision and wisdom.

When we train ourselves to see only what can be optimized or computed, we grow blind to the virtue, the goodness, that arises through attention, (loving attention, I would say) to the particular, to the human, to the not easily grasped.

Why does business, as a practice, cling so hard to this efficiency model? Because it feels safe. Ambiguity is cognitively taxing, even threatening. In fast-moving companies especially, the illusion of control that metrics give you is extremely seductive. Managers don't get rewarded for tolerating ambiguity: they get rewarded for minimizing it.

Efficiency is good for execution, not for insight. And the problem is that we're designing organizations around execution at scale, not learning at depth.

At the innovation lab IDEO, they often said: the best insights come from the edge cases, not the averages. If your dashboard is only surfacing what's typical, you lose that capacity for creative leaps.

Humans are best at reframing problems: seeing them from a new perspective. Machines are good at solving the problem as posed. But many of the world's hardest challenges, and the most difficult business questions, are poorly posed. We need ambiguity to reframe them. If you eliminate ambiguity too early (because it's not efficient) you might be solving the wrong problem, albeit expertly.

I've seen this in design teams too. When you introduce metrics early in the process, such as A/B testing or user analytics, it narrows the creative aperture. People stop exploring ambiguity. But the early phase of design needs ambiguity. That's where empathy, reframing, intuition all happen. The things that can't be measured, yet are most essential.

The human cost

When work becomes a game of optimization, and people become inputs, we erode trust, creativity, and ultimately resilience. Ironically, the very things we need most in times of uncertainty. There's a dangerous myth that intelligence means calculation. But humans are creatures of flesh and feeling. We live through stories, relationships, symbolic thinking.

With the rise of digital platforms we're often designing for the system, not for the human. We create frictionless, efficient interfaces, but those don't always support real human needs.

A more humane model of business or work would begin by asking: what does this work enable people to be and do? Not just what it produces, but how it enhances or limits their lives. Does it expand their freedom? Allow them to participate meaningfully in society? Questions like this get lost in the obsession with efficiency.

In economics, there's a deep tradition (starting with Aristotle and running through Adam Smith) of recognizing that reason is not just calculation. It's about deliberation, imagination, even sympathy. If you strip away ambiguity, you impoverish reason. The challenge is to build systems that can accommodate complexity without collapsing into chaos.

I see economics as a moral and social practice; Adam Smith, after all was professor of moral philosophy at Glasgow. In this light, I would want to see mutualism (or democracy, if you will) at every level. Even in the workplace, employees could have a direct say in setting the metrics that govern them, or in deciding what aspects of their performance truly matter, giving people a meaningful voice in the decisions that shape their daily reality.

Mutualism as an Alternative

It's fascinating how many CEOs claim to be libertarians, and even use "communism" as a handy insult for any attempt to give workers a say in their work. And yet, it is those same CEOs who manage their companies with an almost tyrannical centralized management, with plans, targets and ideologies as rigid as anything the Politburo dreamed of. I follow Hayek more than they do in believing that centralized authorities (including corporate leadership) fundamentally cannot access or process the "particular knowledge of time and place" that exists only in the minds of individuals throughout society. Enterprise key performance indicators pretend that we can.

Work is not only a contract between employer and employee: it is more broadly embedded in community, power, and culture. Mutualism, whether through co-operatives or European-style worker participation, acknowledges that. But mutualism is messy. It requires trust, patience, ambiguity: qualities that spreadsheets can't handle.

Mutualism also acknowledges that the economy can be designed for reciprocal benefit, not unilateral extraction. But this is not merely a technocratic solution. It's a challenge to the logic of value itself and the outcomes can be surprising.

For example, in some European countries, including Germany and Scandinavia, employees have a substantial say on corporate boards. In these contexts, the definition of “profit” and “success” can be broadened to include maintaining stable employment, fair working hours, and environmental impact, rather than short-term returns for shareholders. In companies with worker representation on the compensation board, CEO pay is proportionately higher.

"... evidence suggests that an increase in labor power leads to higher CEO compensation ... When employee representatives are added to a board, CEO compensation increases by a third, whereas employees can secure their jobs ... a common interest that may motivate them to form an alliance."

This kind of system doesn’t just challenge the efficiency paradigm: it reframes the question of shared purpose. When mutualism is embraced not as a workaround but as a principle, it opens the door to ways of organizing our participation, and dignity. It’s a move from extraction to relationship, from metric to meaning.

Embracing the Unmeasurable

A truly humane economy may begin with what is useless. Love and care that cannot be scaled; the hesitation and ambiguity that defies automation; the conversation that changes no metric but reorients a life. These are not inefficiencies, but quiet revolutions.

We can understand freedom in terms of shared responsibility, as servants of one another (St Paul again, with whom I struggle, but he is my Lenten reading.) Often in liberal economic theory, freedom is conflated with an absence of constraints: “Let the market do its work.” But that seems to me a very impoverished notion of freedom.

If we collectively organize the conditions of our lives, so that we can dedicate time and resources to what matters most to us - like caring for children, the elderly, or ourselves in moments of need - rather than consuming your daily existence in precarious labor or fear of not meeting certain metrics: that's seem more like freedom to me.

You know the joke: a man loses his keys in the dark. He searches under the streetlamp. Someone asks, "Did you drop them here?" He replies, "No, but the light is better here."

This is what optimization culture does: it looks for value only where the light of measurement shines. But the keys (the meaning, the justice, the Good) are elsewhere.

So, be wary of designing better metrics without also designing systems of mutual care and recognizing the preciousness of our lives and finite time. Sometimes the most radical thing is to refuse to measure.

Turn off the lamp. Let us sit, like fools, in the dark. Only there can we begin again.

Vegas, of course, is crazy with its extravagant fakery: Venice, Paris, Rome. It's amusing at first, then quite numbing. In Orlando, I could barely escape the conference center. When I emerged into the open air for a chat with colleagues, we sat in the sun on an empty lawn of Astroturf.

Baudrillard said of Disneyland that its fiction makes us believe the world outside is real. Disneyland's obvious artifice masks the fact that America is just as carefully constructed.

And what city could be more artificial than Venice, barely floating, loosely anchored to the caranto? And what could be more fake than a grass lawn: destructive, characterless, close-trimmed, pest-free, watered and fed?

And so, to today's topic.

What I am not saying and what I am trying to say

People sometimes mistake what I say about AI. They think I am saying:

AI is not really intelligent. It's not really reasoning. It's not really doing mathematics or being creative. AI is just simulating these things. Therefore, we must balance AI with the human element.

I am actually saying something different, closer to what Baudrillard says about Disneyland.

When we label machine learning as intelligence comparable to our own, we have to reduce, remove or devalue the key features of the human mind for that comparison to make sense. Turing removed nearly all traces of human communication from the Imitation Game in order for machines to be able to take part at all.

In this way, the artifice of machine intelligence keeps our attention from something else: the emptiness of so much of our thinking that we have reduced to such a shallowness that machines can imitate us.

I think it goes even further. When we claim a machine that passes the Turing test is passing for humans, or we claim that DeepSeek is reasoning, or that machines are close to superintelligence, we are re-imagining our own intelligence.

AI is becoming a mirror that reflects back to us not what we are but what we imagine ourselves to be.

When we celebrate AI for its processing speed, its pattern recognition, its unflagging consistency, we tacitly devalue the very aspects of human thinking that make it profound: our inconsistency, our physical messiness, the entanglement of our every thought with emotion, desire and disappointment, forgetting that our own thinking has never been purely cerebral.

When we marvel at AI's ability to mimic this without truly possessing it, we misrecognize what makes our own intelligence valuable, animating these technologies with our fantasies. We increasingly defer to the output of AI as authoritative precisely because it lacks the hesitation that comes with our understanding, our burden of the full weight and context of our thoughts and decisions.

So, here's what I am trying to say:

We're beginning to shape our ways of thinking, especially in business and technology, to more closely resemble the kinds of cognition that machines can easily replicate and measure: remaking human intelligence in the image of the machine. It does not need to be this way.

This pattern is not only found with artificial intelligence. In business, social media and entertainment, we're shifting toward cognitive processes that can be digitized, quantified, and made operational: celebrating these aspects while neglecting the ambiguous, contradictory elements of thought that have historically been the source of our most profound insights.

We're artificializing our own intelligence, making it more mechanical, more predictable, and more amenable to processes of measurement, optimization and control that are primarily economic rather than humane.

I am not saying we need more of the human element working alongside AI. I am saying that we’re eroding humanity in order for us to work alongside AI.

Quantifying our qualities

Human thinking is not limited to rationality; it emerges between conscious and unconscious processes, but also between personal and collective experience.

Our cognition is always charged with emotion, which isn’t an add-on to rational thought but woven into how we perceive, remember, and reason. From infancy onward, emotional resonance calibrates how we attend to people, places, and objects.

It's tempting to say that because machines (so far as we can tell) do not participate in that interplay, they are not doing real thinking.

That may be true, but its not particularly important to me. What does matter is that increasingly, we are rejecting the messiness of human life and thought in favour of ways of living and working that are closer to what machines can do.

This mechanization of human thought doesn't exist in isolation: it's part of a broader cultural shift toward quantification and transparency. In Byung-Chul Han's The Transparency Society, he argues that our contemporary culture is obsessed with turning everything into data, into something that can be counted, shared, and displayed.

Han writes about a relentless positivity: the elimination of friction, resistance, and Otherness in favor of efficiency, affirmation, and self-optimization. He contrasts this with negativity: the opacity, silence, critique and difference that is crucial for human thought.

Social media, and now AI systems, capitalize (literally) on positivity because they feed on our data: clicks, online behaviors, and textual inputs and feed them back to us in endless loops. In that sense, the focus on big data, optimization, and supposedly intelligent machines promotes a flattened, disembodied worldview. We become participants in our own alienation, handing over more aspects of our interior lives to systems that can quantify them without understanding them.

Our era encourages a kind of hyper-individualism, but merged at the same time with total connectivity. We share everything but remain profoundly alone in our curated social media bubbles. If we only seek what is transparent or “positive” in the sense of easy data, we lose access to the richer realm of human existence. We end up in a technological framework that only sees the world as something to be optimized.

In that scenario, the noise of digital connectivity drowns out introspection and even the unconscious. We have fewer opportunities to encounter ourselves if the timeline is always refreshing, if we are never silent.

Silence and dreams

Yet, as reflective people have discovered in every age, silence is essential for integrating our unconscious into conscious life. In the clamor of connectivity, we short-circuit that process.

To do this well, we need spaces of genuine emptiness: time away from the screen, time for contemplation, where we are not optimizing or performing. This emptiness is a break from the demands of Han's constant positivity, always making progress, always clicking or scrolling. It is in those silences that we can reflect on our real needs and the direction of our collective journey.

Historically, religion and spiritual practices often insisted on solitude, contemplation, even asceticism, to open up this domain of the unconscious. Even if one isn’t religious, there’s something crucial about the unscripted space where you’re not optimizing or performing.

As I said of AI before, social media mirrors back to us what we believe we must become, regardless of inner experiences that tell us otherwise. The interior space where one might dream or experience an unstructured flow of ideas gets squeezed out.

But the unconscious does not vanish; it remains, unacknowledged, influencing us in ways we do not see, the ways that Jung attended to: symbols, dreams, and the deeper psyche.

Our unconscious develops from the complexity of bodily experiences, emotions, and (to the Jungian) archetypal patterns that shape meaning over the course of a lifetime. That includes dreams and symbolic imagery that reorganize every event, entangling it with our individual past. This is how we form our self-identity, a narrative identity, often outside conscious awareness.

Nevertheless, I suspect a Jungian would say that we can’t help but project elements of our psyche onto the world around us, be that people, animals, or even technologies. So, to take the “child archetype,” we might see AI as a “newborn” set of systems, naive yet brimming with potential. Or we might consider the “wise old man” archetype: the super-intelligent oracle that might solve all problems.

The danger here isn't the specific archetype we work with (if Jungian archetypes are your thing) but that we anthropomorphize these systems, attributing to them wisdom, innocence, or malice that strictly belongs to the human domain; we forget to ask real questions about ourselves and our responsibilities.

But we, unlike the machine, cannot escape the unconscious, for at night, we dream our dreams: an endless reorganization of internal states, forging new and often bizarre connections that make us more resilient in the face of our fears, that enable us to integrate our experiences into a meaningful internal life. In our dreams, we can hold together contradictory meanings, symbolic ambiguities, and paradoxes. In dreams, we don’t resolve ambiguities; we cultivate them until they offer us insight if we would pay attention.

We may also attend to our dreams, our mythologies, our symbolic life. By bridging the conscious with the unconscious, we can see through the illusions we project onto AI. We have within us a vast interior world, one that’s messy, symbolic, and infinitely creative. If we take care not to reduce our existence to what can be digitized or formalized, we can live in that world more fully.

The business of thinking

In my next post, I'll pull on one of these threads a little more.

Our thinking about business - and therefore work - has been reduced to sterile enumeration more than any other aspect of life: everything is measured, scored, and optimized. We are told this is efficient, realistic and necessary and that any other view is naïve and unworldly.

That is an artifice.

This reminded me of a historical problem, going all the way back to Charles Babbage, the father of the computer (but see my footnote about Ada Lovelace, if you are wondering) and his Difference Engine No 1, which he described in the Edinburgh Review of April 1834.

However, it is not that article (unsigned, but likely by Babbage himself) I want to refer to, but rather a piece in Chambers' Edinburgh Journal of May 1834 (p240), a month later, which is, in effect, a sort of product review of Babbage's engine.

Here's a very relevant passage ...

But we must recollect, at the same time, that such an engine as is here proposed can only be valuable if it effects the practical benefit intended. The question is, will Mr. Babbage’s machine print correct tables? —that it will compute them correctly, we have little doubt.

Today, we think of Babbage's great contribution as the theory and practice of computation. The reviewer, one month after the big reveal, simple takes it for granted, but does worry about the output.

Why?

Systems thinking and drunken, ignorant workmen

A key motivation behind Babbage’s work was to create accurate numerical tables, particularly for sea navigation. At the time, navigational tables (used for determining longitude and latitude, predicting celestial positions, and aiding in astronomical observations) were computed by hand: the people who performed these manual calculations were known as "computers."

But of course, this process was error-prone, and even small mistakes in these tables could result in shipwrecks, lost cargo, and even loss of life. The Science Museum in London describes this as the Mathematical Table Crisis.

Babbage aimed to eliminate human errors by using mechanical computation which would be more accurate, faster and (in modern terms) scalable.

Yet, his machine was not only a computational device; it was part of that larger ecosystem of navigation, commerce, and science: a larger system that still included human factors and physical constraints.

In particular, the tables had to be printed and distributed to be useful.

Babbage understood this: The engraved plate of copper obtained in the manner above described is designed to be used as a mould from which a stereotyped plate may be cast ... Each plate, when produced, becomes itself the means of producing copies of the table, in accuracy perfect, and in number without limit.

Although he created prototypes of his engines, he never did produce a printer, although the Science Museum did make one to his design in 2000.

The writer of the review in the Edinburgh Journal was not so sure: We are afraid that the ingenious inventor of the calculating machine is here much too sanguine in his expectations. All that he ought to expect to accomplish is to furnish one accurate set of tables as a sort of standard. If he imagines that he can produce perfect stereotype plates, or perfect printing, he is laboring under a serious mistake. ... As far as his machine-work is concerned, accuracy may be gained, but all after-details are affected by human agency, and, consequently, liable to error … all that the most sublime genius may invent for the benefit of mankind after lifetimes of deep study, may in a moment be blasted by the carelessness of a drunken or ignorant workman.

That's a very human-centric view of the problem.

The tension between invention and implementation

The printing problem illustrates a fundamental principle in technology design that I've often seen myself and which Don Norman describes well: systems fail at the boundaries between components, not within them.

Moreover, there's often a disconnect between what inventors envision and how technology is adopted. Babbage thought he could create a machine that would produce perfect calculations, but the broader ecosystem (human labor, infrastructure, and economic realities) interfered.

His challenge was a mechanical one: ensuring that an already correct computation was faithfully reproduced. With AI, we have the opposite issue: models generate information based on statistical patterns, not ground truth.

But still, as for Babbage's engine, AI models exist within socio-technical systems.

AI programs, especially large language models, can produce beautifully rendered outputs, whether it's text, images, or even code. But as we know, and I found out, these outputs can be completely wrong.

There's an important distinction, of course. Babbage's machine had a well-defined specification: mathematical correctness. For language models, defining what "correct" means is part of the challenge. They're not just calculating; they're navigating the inherent ambiguity and contextuality of human language.

And yet, both cases highlight Don Norman's point about the vulnerability of systems at transition points. With Babbage, it was the transition from calculation to physical output. With modern AI, it's the transition from statistical pattern matching to human-consumable information.

The Evolution of computation and uncertainty

People often mistake a powerful algorithm for a reliable system. I did, until I double-checked. And that's the problem: the fact that models are making predictions about text rather than reasoning about facts is why they produce such convincing-looking nonsense.

Perhaps the solution to hallucination isn't just better models, but better interfaces, better education, better integration into workflows, and better alignment with human psychology and values.

My concern is that, like Babbage's navigation tables, we may be deploying these new systems in domains where their limitations matter, without sufficient guardrails. We may not cause shipwrecks, but we are seeing AI hallucinations in legal briefs, medical diagnoses, and other high-stakes domains.

The Edinburgh Review's best answer was what we now call redundancy: the most certain and effectual check upon errors which arise in the process of computation, is to cause the same computations to be made by separate and independent computers. (Remember, computers were people, computing.)

What verification systems can we build for AI? In my last article I shared how Claude and DeepSeek double-checked their own results and corrected the answers, but the process gave me little confidence.

And as the prescient reviewer said in 1834: several computers, working separately and independently, do frequently commit precisely the same error.

We see the same with LLMs: they can all hallucinate in similar ways because they share similar architectures and training data.

Diversity is our strength

I'd argue this is where scale and diversity of training give us leverage. As I study how people use generative AI for work, I've observed that experienced users develop habits of verification: they may ask the AI to show its reasoning, they may triangulate information with multiple queries, or sometimes they use the AI to generate hypotheses but verify facts independently.

People are remarkably creative at developing compensatory strategies when using imperfect tools. But, there's a design failure when the burden of verification falls entirely on the user.

And there's the deeper question of whether users will actually use those tools. There's a psychological aspect here that the Edinburgh Review article touches on, way back then. To complete what they said earlier about errors ... several computers, working separately and independently, do frequently commit precisely the same error; so that falsehood in this case assumes that character of consistency which is regarded as the exclusive attribute of truth.

People tend to trust what looks authoritative. Like this from my previous post. It sure looks accurate ...

Design for human needs and appropriate trust

And this is where design comes in. Babbage's machine was designed for computation, not for the end user. If we think about AI today, we're seeing the same problem. AI systems are often designed by engineers for engineers, without enough consideration for how they'll be used in the real world.

How do we design interfaces that convey appropriate confidence? How do we educate users about limitations? This goes beyond the models themselves to the entire user experience.

I think one approach is designing for appropriate trust rather than complete trust.

Whether we're talking about Babbage's machine or AI, the goal should be to create systems that serve human needs. That means not only making them accurate and reliable but also ensuring that they're accessible and understandable. If we design with the user in mind, we can avoid many of the pitfalls we've been discussing.

One lesson we can learn from 1834 is about purpose.

Babbage's machine was designed with a clear purpose: to eliminate errors in numerical tables. But as I've discussed, it didn't fully achieve that purpose because it didn't account for the entire system.

With AI, we need to be equally clear about our purpose. Are we building AI to serve humanity, or are we just chasing the next big thing? If we can answer that question, we'll be in a much better position to avoid the pitfalls of both Babbage's machine and AI hallucinations.

(Footnote; As it currently Women's History Month, there are several articles on the webs about Ada Lovelace, the inventor of the first true program and Babbage's working partner. I am not overlooking her, but she had only just met Babbage some months before these articles were written in 1834 and had only seen a prototype of the Difference Engine. I don't think she had worked on it at this stage. Her fascinating and genuinely groundbreaking work was done for Babbage's later Analytical Engine. So she is excused critique for now.)

And another footnote. The full text of the 1834 article …