The Intelligence Bottleneck

Amazon can get an air fryer to my front door tomorrow. Getting electricity to some of its new AI equipment appears to be proving rather more troublesome.

Business Insider reported in October that AWS was taking early ownership of billions of dollars’ worth of Nvidia systems, with some equipment waiting in warehouses because the infrastructure to run it wasn’t ready. Amazon says staging equipment ahead of deployment is normal practice. Fair enough. Securing scarce kit before somebody else buys it makes sense.[1]

Still, there’s something magnificent about the image. Machines that could help invent the future, waiting for a sparky to get the main consumer unit installed before they knock off just after lunch, why do they do that?

I’m sure Jeff could shift a few on Prime if it all goes wrong. You can see it now, the similar items bar, “Customers who bought this, also bought a nuclear power station.”

But behind the joke is a question I keep coming back to: how much of the future we’re being promised can we actually build?

My Aunt Silvia once thought cloud computing happened in actual clouds. When I explained that it all happened in big buildings on the ground, she asked why they called it the cloud, then. Honestly, Silvia, it’s possibly the biggest trade descriptions case in history.

We’ve made computing sound weightless. Now we’re discovering quite how much concrete, copper and electricity intelligence requires. Therefore it weighs quite a lot.

This has become oddly personal for me. Our home in Pennsylvania is about four miles, line of sight, from Three Mile Island. Yes the famous one, the closest the US came to its own Chernobyl, that said, it only affected one reactor, the undamaged reactor there operated until 2019. The energy infra company, Constellation, is now working to restart it, supported by a twenty-year power purchase agreement with Microsoft, adding roughly 835 megawatts back to the grid.[2]

You spend your working life watching computers get smaller, then discover that the next upgrade involves restarting the local nuclear power station.

That gives the phrase “system requirements” a slightly different flavour.

And generating the electricity is only part of it. You have to get it to the right building. The International Energy Agency estimates that grid constraints could delay around a fifth of global data-centre capacity planned for construction by 2030.[3]

Having power somewhere else is like having a full pint at the other end of the pub. Encouraging, but it doesn’t resolve the immediate problem.

Follow the supply chain backwards and the same awkwardness appears. Advanced chips depend on specialist factories, memory and packaging. ASML is the world’s only manufacturer of EUV lithography systems, which print some of the most intricate chip layers. TSMC’s advanced packaging connects processors with high-bandwidth memory.[4][5] You need the whole procession to arrive before the thing can do useful work.

You can announce another hundred billion dollars on a Tuesday. You cannot announce another decade of manufacturing experience into existence by Friday.

And while you wait, the equipment keeps ageing. Imagine buying the latest laptop, leaving it boxed for two years, then finally opening it as the manufacturer unveils something twice as good. Now add enough zeroes to make the finance director need a little lie-down.

This is where I think the AI conversation gets interesting. We spend so much time arguing about whose model is cleverest that we can overlook who can actually deliver intelligence in useful quantities.

A brilliant model still needs somewhere to run. Its customers need capacity they can afford and rely on. The companies that can bring the chips, power and infrastructure together have an opportunity to turn extraordinary technology into something millions of people can actually use.

And I think tomorrow’s winners are already showing us how they intend to win: by owning and coordinating whole swathes of that value chain.

It’s also why Musk’s Terafab ambition interests me. Tesla and SpaceX want to bring chip manufacturing, memory and advanced packaging together.[6] The appeal is understandable: if your future depends on somebody else’s factory schedule, eventually you start wondering whether you should own the factory.

Making that work will take an absolutely heroic amount of engineering. The ambition still has to become a working factory. But the logic is compelling: bring more of the process together and you have a better chance of getting design, production and deployment moving at the same speed.

The opportunity is bigger than Elon. Owning more of the chain can give a business more control over costs, priorities and delivery. If the engineers designing the product can work directly with the people making it, there’s a chance to solve problems faster and carry the improvements all the way through.

As someone who runs businesses, that makes perfect sense to me. Every dependency is another phone call, another timetable and another opportunity to hear that somebody is “chasing it”. Own more of the process and you can do more than chase. We applied this same thinking to Airebo, removing all the chasing from key parts of the building compliance process.

That’s what makes this exciting. Better research, useful personal assistants, small businesses doing things that once required whole departments. Delivering those possibilities at scale is an enormous commercial opportunity for the businesses that can build the machinery underneath them.

Efficiency will help. Better software can squeeze more useful work from existing hardware, and new capacity will arrive. The uncertainty is how quickly that happens compared with how quickly we find more things to ask AI to do.

For now, I’d pay rather more attention to what has actually been connected and switched on. An artist’s impression of a data centre has remarkably low operating costs, however, its productivity will disappoint. I will also add this, I, along with anyone who spends a lot of time working with AI, know when a provider is hitting their compute limits, the models get lazier, the quality of performance drops and stuff takes longer, we’ve already moved from token maxing to optimising, the available compute is already biting.

AI could make intelligence far more abundant. Getting there will reward the businesses that can turn clever ideas into dependable capacity, bringing more of the value chain under their own roof and getting the whole thing working together.

So alongside “How clever is it?”, I’d ask: “Who can build it, who can power it, and who can deliver it at scale?”

Tomorrow’s winners are already giving us clues. They’re buying the machinery, securing the power and building the factories. If you believe intelligence will be on tap, owning the plumbing looks like a bloody good business.