Artificial Intelligence

TL;DR The Trust series is my answer to one question: what has to be true before you can hand a non-deterministic system a real job and walk away? Read in this order: research map → evals → security → world models → trajectory evaluation → interpretability Supporting posts cover reliability, context engineering, and safety foundations Full series index: /series/trust/ Start here What I’m Researching in AI Right Now - the research map and trust through-line AI Evals Are Broken - why public benchmarks stopped measuring real capability Securing AI Agents - MCP hardening, confused deputy, and what I run on my home stack World Models: What Comes After the Language-Only Era - when text-only agents hit their ceiling Evaluating Agents in Production: Trajectory Metrics - step-level scoring, not just final answers Mechanistic Interpretability: Reading the Mind of a Model - the inside-out complement to behavioural safety Supporting reading AI Agents That Actually Work - patterns from real projects The Agent Reliability Problem - debugging non-deterministic systems Context Engineering - curating the window across a whole agent run AI Reliability Is Weird - why testing LLMs breaks familiar QA AI Safety From First Principles - engineering safety vs speculative scenarios Related paths Home Agent Stack - build the stack these defenses protect AI Dev Tooling - the coding-agent side of the same problem Related Reading AI Economics and Hardware: A Reading Path - cost and infrastructure decisions that constrain what you can actually deploy Expertise and Work in the Age of AI - how trust and accountability reshape what human expertise is for Agent Protocols in 2026: MCP, A2A, and ACP - the protocol layer where many trust boundaries live Structured Outputs and Schema Design for LLMs - making agent behaviour predictable enough to evaluate

TL;DR Recursive self-improvement (RSI) is the idea of an AI that improves its own ability to improve - each round producing a smarter system that does the next round better. It is the engine behind every “intelligence explosion” story since I.J. Good described it in 1965 The narrow version is already real. Systems like AlphaEvolve and the AI Scientist measurably improve algorithms, code, and even research output - including, in AlphaEvolve’s case, the infrastructure that trains the models themselves The leap people fear is different: improving an algorithm is not the same as improving general intelligence. Nothing in 2026 has crossed that line, and the gap is structural, not just a matter of scale Four bottlenecks decide whether RSI runs away or fizzles: compute, data, verification, and diminishing returns. Each is a hard physical or informational limit, not a temporary engineering nuisance The realistic picture is steady, human-paced acceleration - AI assisting AI research - not an overnight takeoff. METR’s time-horizon data shows fast but smooth exponential progress, which is exactly what a bottlenecked process looks like In May 2026 Anthropic put numbers on this from inside a frontier lab. Its essay When AI Builds Itself reports that over 80% of the code it merges is now written by Claude, that task horizons are doubling every roughly four months rather than seven, and lays out a candid three-way bet on where this ends. None of it overturns the bottlenecked-flywheel picture - but it sharpens it It still deserves serious safety attention, because a slow takeoff is the one we can actually govern There is a particular shape of argument that has haunted artificial intelligence since before the field had a settled name. It goes like this: build a machine slightly better than humans at designing machines, and it will design a machine better than itself. That machine designs a better one. The loop tightens, each turn faster than the last, and intelligence runs away from us in an afternoon. ...

When I wrote about Donald Hoffman, I was working through one half of a question I keep saying I have not settled: whether consciousness is fundamental, there from the start as part of the floor of reality, or computational, something that switches on once a physical process organises information in the right way. Hoffman is the most serious case I have found for the fundamental side, and I gave it a fair hearing because I genuinely find it compelling. ...

When I wrote about Yampolskiy’s Personal Universes recently, I left a thread hanging. The question underneath that whole post - the one I said I genuinely had not settled - was whether consciousness is fundamental, there first with the universe as something it experiences, or whether it is computational, something that switches on once a process gets complex enough. I said I had only recently started reading my way into the fundamental side, mostly through Donald Hoffman. This is me pulling on that thread properly. ...

First, the thing this is all in service of. The AI alignment problem is the challenge of making a powerful AI system reliably pursue what we actually want it to pursue - getting its goals, values, and behaviour to line up with human intentions, and to stay lined up even as the system becomes more capable than the people supervising it. It sounds simple and is not: we struggle to state our own values precisely, those values conflict between people, and an AI optimising hard for a slightly-wrong objective can produce outcomes nobody asked for. The multi-agent version - aligning one system with all of humanity at once, rather than a single person - is harder still, and it is the specific version Personal Universes is trying to dodge. ...

The “is the software engineer dead” genre has been running long enough that you can predict most of the takes before you read them. The interesting interviews are the ones where the person being interviewed is in a position to know something the rest of us do not. Boris Cherny, the creator of Claude Code at Anthropic, is one of those people. Sky News got him in front of three charts and asked him to react. ...

For most of the last few years, the “AI and cybersecurity” conversation has been a vibes argument. One side said the models would soon write novel exploits at scale. The other side said the models were still tripping over basic shell commands and could not be trusted to hack anything more dangerous than a CTF box. The honest answer was that nobody had hard numbers, so the debate stayed stuck on intuition. ...

TL;DR Prompt engineering optimised the wording of a single human-written request. Context engineering optimises the entire set of tokens in the model’s window across a whole run - system prompt, tool definitions, retrieved documents, tool results, conversation history, and memory The shift happened because of agents. The window is no longer one prompt you wrote - it is an accumulation that grows on every step, and most of it is produced by the system, not by you More context is not better context. Research on “context rot” and the older lost-in-the-middle effect show model accuracy degrades as the window fills, even well below the advertised limit The four levers are retrieval (what you pull in), memory (what persists across runs), tool results (what tools dump back), and compaction (what you summarise and discard) Treat the window as a budget. Measure its token composition, design tools to return terse output, curate rather than accumulate, and keep the static prefix stable so prompt caching still works For a few years, “prompt engineering” was the named skill of working with language models. It meant finding the wording, the framing, the few-shot examples, and the role instructions that coaxed the best answer out of a single request. It produced a small industry of prompt libraries, prompt marketplaces, and job titles. And in 2026 it is mostly gone, absorbed into something larger and harder. ...

TL;DR Start with What Actually Belongs in My AI Dev Stack in 2026 - the canonical stack essay Then An AI Tooling Learning Path - phased skill-building order Deep dives below cover comparisons and spec-driven workflows; single-tool posts are briefs, not entry points Canonical essays What Actually Belongs in My AI Dev Stack in 2026 An AI Tooling Learning Path: Logical Phases for 2026 Context Engineering - the production skill behind reliable coding agents Spec-Driven Development - when the brief becomes the product Deep dives Claude Code vs Cursor: A 6-Month Comparison GitHub Spec Kit and Spec-Driven Development GitHub Spec Kit in 2026: SDD Goes Mainstream My AI-Augmented Design Workflow When to Fine-Tune vs When to RAG Briefs (moment-in-time) These are useful snapshots, not the starting point: ...

TL;DR Cost is a design constraint, not an afterthought - model tier, context size, and deployment location are economic decisions Read the essays below in any order; start with Token Economics if you only have time for one Pairs with open-weight models and local inference guides Core essays Token Economics: Why the Cost of AI Isn’t Going Down GPU Servers vs AI API Credits: The Real Cost Breakdown Local AI vs Cloud AI: The Tradeoff Landscape in 2026 The AI Energy Crisis: Why Data Center Power Will Define the Next Decade Cerebras, Groq, SambaNova: The Inference Hardware Insurgents Adjacent The State of Open-Weight Models in 2026 - when open weights beat closed APIs on price Prompt Caching - the quiet latency and cost win The Token Efficiency Mindset - curating spend per conversation Is the $20 AI Subscription Era Over? We Are Learning to Buy Intelligence Related Reading AI Dev Tooling: A Reading Path for 2026 - canonical path for coding agents and stack decisions that depend on these cost constraints Home Agent Stack: From Mac Studio to Secured MCP Tools - building the hardware and software layer these economics govern Reasoning Models in 2026: What Changed and What Didn’t - why reasoning models carry a different cost profile than base models The Free Intelligence Era - the macro argument for where intelligence costs are headed