AI-infused software is eating up the world

Ever since Marc Andreessen coined the now-famous quote in 2011, millions of software companies have been established. Low interest rates led to an excess of available capital, and the relatively low cost of building software compared with most other industries drove that growth for the most part. But what enabled this growth in the first place was the fact that computing was, at its core, cheap and that costs were predictable. With AI, this has changed.

LLMs, that started as chat companion, are now ubiquitous. Decisions are made based on their output, user-facing features rely on them, and even entire companies are built on top of those tools. You name it, and someone has probably already created a SaaS for it. But these computations are expensive, and they cost a lot more than the industry is used to paying. For example, inference on cheaper models can cost anywhere from $0.08 to $0.60 per million tokens, while mid-range reasoning models can cost between $2 and $15, and if you've ever interacted with an LLM, you know that 1 million tokens are basically nothing. Compare that with roughly $0.000001 per request that CPU-based computations cost on most cloud services, and you get the picture. A power user, or someone abusing your infra, depending on how you frame it, can now skyrocket operating costs in no time. Keep in mind that the prices being charged by LLM providers are subsidized. And yet, nobody seems willing to back down on AI adoption.

We are heading in a direction where, if something isn't AI-infused, it won't be trusted. Traditional apps doing repetitive tasks reliably for years will no longer be a thing, and the fuzziness that comes with AI models will become a risk accounted for. When this paradigm shift is complete, there won't be a way back to traditional software, and no matter what the running cost of it is going to be, people and companies will just accept it. And this realization makes losing money now worth it for large LLM companies. In the unlikely event that inference costs aren't reduced by a single dime in the years to come, they can just stop subsidizing and pass the costs on to end users without much of a problem.

As Anthropic and OpenAI are heading towards historic IPOs at valuations most of us can't even fathom, it seems almost certain that their best days are still ahead of them. With the huge influx of capital they will see and their access to ample resources and highly skilled people, they are on their way to becoming the most valuable companies the world has ever seen, dwarfing all of their counterparts, since they will effectively be the world's sole intelligence providers. Imagine a world where every computation run everywhere is owned by them. What a nightmare that would be.