Unity Catalog in Practice

Unity Catalog in Practice: Lessons From the Field

The views in this post are my own personal reflections on industry patterns, written in my own time. They are not about any specific employer, team, or colleague, past or present, and do not draw on any non-public information. TL;DR Unity Catalog is a unified access-control and metadata layer for tables, volumes, models, and notebooks - it is not a data-quality tool, a discovery engine, or a masking system, and teams expecting those will be disappointed Migrating from Hive metastore remains the biggest operational challenge in 2026; the hybrid path (migrate reference data first, stage the rest) is the most common in practice Design catalogs around medallion layers (bronze/silver/gold), not per-environment schema sprawl, and grant permissions only through roles, never directly to users Budget realistically: $50k-$200k of engineering time for large-organisation migrations, roughly a third to half of one engineer’s time ongoing, and under 5% query overhead Skip UC for single-team startups, sandbox data, and some streaming workloads; full adoption typically takes 6-12 months Unity Catalog sounds straightforward: “one governance layer for all your data and AI assets.” In theory, it’s elegant. In practice, you’ll run into gotchas that docs don’t prepare you for. ...

April 3, 2026 · 11 min · James M
What the Amiga got right that we are still copying in modern computing

What the Amiga Got Right (That We're Still Copying)

TL;DR The Amiga, launched in 1985 and dead by 1994, was a commercial failure - and almost every good idea in modern computing traces back to it Preemptive multitasking, graphics compositing, hardware-accelerated audio and video, plug-and-play expansion, and system-wide scripting all shipped on the Amiga while IBM PCs were still effectively 8-bit Jay Miner’s radical design used multiple custom processors, each with its own job - the same philosophy behind today’s GPUs and specialised silicon What killed it was not the technology but Commodore’s management collapse The lesson: deep architectural insight can put a machine a decade ahead, and still lose to distribution and business execution The Commodore Amiga was not the most successful computer. It was not the fastest. It was not the cheapest. It was introduced in 1985, bought by Commodore in a panic, and discontinued by 1994 as the company collapsed. By most commercial metrics, it was a failure. ...

April 3, 2026 · 11 min · James M
Compunet - Britain's forgotten pre-internet online community

Compunet: Britain's Forgotten Pre-Internet Community

TL;DR Compunet, started in 1982 from a flat in Islington on the Post Office’s Prestel videotex network, became one of the world’s first genuine online communities - 15 years before most people had an email address It had everything the internet later promised: real-time messaging, online games, pseudonymous identities, digital art, and a culture of people who knew they were part of something new Prestel was the world’s first public videotex system (1979), accessed by modem and paid by the minute through modified televisions and home computers By the 1990s it had vanished - Prestel shut down, servers went offline, and the community was almost erased from computing history Its story matters because it shows online culture was invented by hobbyists on the margins, not delivered by the internet’s arrival Long before Facebook, Twitter, or Reddit, there was Compunet. In 1982, in a small flat in Islington, London, two teenagers set up a computer bulletin board system on a network called Prestel. Within a few years, it had become one of the world’s first genuine online communities - thousands of people meeting in cyberspace, exchanging messages, playing games, and falling in love, all before the internet existed in public consciousness. ...

April 3, 2026 · 10 min · James M
AI subscription pricing illustration

Is the $20 AI Subscription Era Over?

TL;DR The $20/month subscription tier is not disappearing, but what you get for it is quietly shrinking - agent features are being capped or metered while the price holds The Claude Code episode (briefly paywalled for Pro users) was a deliberate A/B test, not a glitch - a signal that Anthropic is steering heavy users toward the Max tier at $100 - $200/month Agent workflows like Claude Code consume 50 - 500x more tokens than a chat session, making flat all-you-can-eat pricing economically unsustainable for power users Most major providers (Anthropic, OpenAI, Google, Cursor) are projected to raise consumer tiers by $5 - $10 by end of 2026, with sharper increases at the enterprise level If you are a chat-only user the $20 plan remains a good deal; if you are running agents daily, budget for a higher tier or pay-as-you-go API access instead For the last three years, $20 a month has been the magic number. Claude Pro, ChatGPT Plus, Gemini Advanced, Copilot Pro, Cursor Pro - all twenty dollars, all clearly priced to anchor against Netflix rather than against enterprise software. That anchor is cracking. The labs are burning cash on inference for power users, the frontier models cost more per token than they did a year ago, and agent tools like Claude Code and Codex are consuming ten to a hundred times the compute a chat session does. Something has to give. ...

April 3, 2026 · 10 min · James M
Abstract illustration of a person sitting with a tool laid down beside them

The Meaning of Work in an Age of Abundance: Finding Purpose When Agents Do the Heavy Lifting

TL;DR Modern knowledge work has quietly built identity on producing things - and AI pressure makes that fragility visible without you having to lose your job to feel it History (Keynes’ 1930 prediction) suggests freed-up capacity defaults to “more work”, not leisure - the shift to meaningful work has to be chosen deliberately What stays valuable when execution gets cheap: deciding what is worth doing, taking responsibility, sitting with other humans, craft for its own sake, and growing other people The “everyone will do deeper work” narrative ignores the dignity problem - for many people, work is structure and belonging, not just a vehicle for meaning Put your meaning somewhere that does not depend on being the cheapest producer of an artefact - it was never a secure place to put it, and agents are just making that clearer This is another “thinking out loud” post, in the same spirit as the agent-first architecture piece. I do not know how any of this is going to land. I am writing it partly because the question has been rattling around in my head for months, and partly because I suspect a lot of people in and around software are quietly wondering the same thing without quite wanting to say it out loud. ...

April 2, 2026 · 13 min · James M
SpaceX Cursor Deal Banner

SpaceX Buys the Right to Buy Cursor for $60 Billion

TL;DR SpaceX has signed an option to acquire Cursor (made by Anysphere) for $60 billion, or pay $10 billion for the joint work if it walks away Cursor’s valuation has risen 24x in fifteen months - from $2.5 billion in January 2025 to a $60 billion option price in April 2026 The deal sits under SpaceX rather than xAI directly, because SpaceX holds the balance sheet after the SpaceX - xAI merger valued at $1.25 trillion For xAI, buying Cursor is a faster route to developer relevance than out-marketing OpenAI’s Codex or Anthropic’s Claude Code If the acquisition closes, three of the main AI coding interfaces will sit inside three frontier labs - raising questions about model neutrality and pricing pressure on independent tools It’s rare to see an option contract make the front page, but that is what landed on 21 April 2026. SpaceX disclosed that it has signed a deal with Cursor - the AI coding tool made by Anysphere - giving it the right to buy the startup outright for $60 billion later this year, or to walk away with a $10 billion payment for the joint work the two teams are doing in the meantime. ...

April 2, 2026 · 6 min · James M
The demoscene - where art met assembly language on 1980s home computers

The Demoscene: Where Art Met Assembly

TL;DR A demo is a real-time audiovisual production built from nothing but code and mathematics - a 4-minute music video that weighs 64 kilobytes and runs on a Commodore 64 The scene evolved from the cracking scene in the early 1980s: groups added intros to cracked games, then realised the intros were the interesting part and dropped the games Size limits (4KB, 64KB) and weak hardware were the point - constraint was the art form Demoscene alumni went on to shape the games industry, graphics programming, and real-time rendering techniques still in use It remains the strongest example of a creative community that valued technical mastery as an aesthetic in itself The demoscene wasn’t about games. It wasn’t about productivity software or killer apps. It was about taking a computer that wasn’t designed for art, stripping away the operating system, and hand-crafting something beautiful in 512 bytes of RAM. ...

April 2, 2026 · 7 min · James M
From BASIC in 1981 to Claude Code in 2026

From BASIC in 1981 to Claude Code in 2026: What Programming Has Always Been About

TL;DR 45 years separate typing BASIC on a ZX Spectrum and using Claude Code on a MacBook - the abstraction rose, but the core loop stayed the same Programming has always been: figure out what you want, type, run, fail, fix AI agents compress the feedback loop and raise the abstraction - they do not replace the act of directing the machine The tools changed from line numbers and 16KB RAM to natural language and cloud models; the discipline did not I suspect that loop will still be true in another 45 years I’m sitting at a desk with two machines. ...

April 2, 2026 · 9 min · James M
Claude Code source leak - Anthropic 2000 file exposure

Claude Code Source Leak: Anthropic's 2,000-File Exposure and What It Means

TL;DR An internal debugging file was accidentally included in a public package update, exposing a compressed archive of roughly 500,000 lines of code across around 2,000 files - not a breach, but a packaging mistake The leaked material revealed unreleased features including persistent memory, an always-on autonomous background assistant, and multi-device remote access Competitors gained rare visibility into Anthropic’s development pipeline and longer-term product direction, which is the primary competitive damage The incident undermines Anthropic’s safety-first positioning, particularly because it was the second such exposure in just over a year The broader lesson for the AI industry: internal operational security is becoming as critical as defending against external threats, especially as AI tools target enterprise customers Anthropic’s Claude Code has been making waves as one of the most capable AI coding assistants available, but a significant internal leak has exposed the underlying technology behind the platform for the second time in just over a year. The incident raised fresh concerns about how the company handles sensitive internal information and operational security. ...

April 1, 2026 · 4 min · James M
Meta employee tracking banner

Meta Is Tracking Its Own Employees to Train AI Agents

TL;DR Meta’s Model Capability Initiative installs software on US employee laptops that captures keystrokes, mouse movements, and screenshots to train AI agents - there is no opt-out The program is US-only because EU and UK employees are protected by GDPR; the scope of the tracking maps directly onto the absence of legal protection Meta CTO Andrew Bosworth openly framed the end state: agents do the work, humans direct and review - the surveillance and the automation plan are the same story The irony is deliberate: Meta’s defence of the program - narrow purpose, safeguards, not used against the person - echoes its long-standing defences of consumer data collection This is a signal about where the agent-training bottleneck actually sits: not reasoning or context windows, but the long tail of real software interactions that only real employees can provide Meta has started installing tracking software on the work laptops of its US-based employees. It captures keystrokes, mouse movements, clicks, and occasional screenshots. The captured activity is fed back into training data for AI agents. There is no opt-out. The program was disclosed to staff in an internal memo in April 2026, and the response from inside the company has been about what you would expect. ...

April 1, 2026 · 8 min · James M