Is the Tokenpocalypse upon us?

Microsoft recently unveiled significant adjustments to GitHub Copilot's pricing model, changes so impactful that one Reddit user humorously dubbed the situation the "Tokenpocalypse."

On a recent episode of TechCrunch’s Equity podcast, Kirsten Korosec, Sean O’Kane, and I delved into the potential ramifications of these shifts for the broader artificial intelligence ecosystem. As major AI firms like Anthropic prepare for public offerings, facing scrutiny over profitability, it's highly probable that similar price escalations will affect other AI products.

Sean questioned the core challenge, asking, “Can these AI labs collapse that cost [and] progress the tech enough in a way that it eventually meets in the middle with customers’ appetite for spending?”

Kirsten, for her part, posited that these developments also underscore “how quickly things are moving.” She observed that within a mere few months, companies became fixated on “tokenmaxxxing,” only to pivot against it due to the exorbitant costs. This rapid evolution, she noted, poses a unique challenge for AI companies drafting their IPO filings: “How do you even write these risks in, because they are evolving before our eyes?”

Our conversation continued with a deeper dive into these critical issues.

I initiated the discussion by recalling Sean's "Tokenpocalypse" moniker during our planning, highlighting Microsoft’s move to charge per token for GitHub Copilot, rather than a flat rate. This entire AI ecosystem, I pointed out, is heavily reliant on investor subsidies, meaning that what appears to be a cost-free service is, in reality, incredibly expensive. We are now reaching a juncture where these substantial costs will increasingly be transferred to the end consumer. The impact on user behavior remains uncertain, but I anticipate considerable friction.

Sean expressed concerns about the numerous token-related risk factors expected in Anthropic’s S-1 filing, a recurring theme on the podcast. He cited Uber as a prime example, noting their rapid progression over a month and a half from realizing they had "blew through our budget on this stuff way quicker than we thought this year" to implementing usage caps and limits within the company. This swift reversal at a major user like Uber is disquieting, reinforcing his earlier question: "Can these AI labs collapse that cost [and] progress the tech enough in a way that it eventually meets in the middle with customers’ appetite for spending?"

Reflecting on ChatGPT Plus's initial $20 monthly charge, Sean speculated that "there was really any strategy involved," suggesting it was more of an arbitrary figure. While users now pay more for advanced models, he believes even these higher prices are insufficient to bridge the gap to the "true cost," underscoring this as the paramount challenge facing the industry.

Kirsten emphasized that these developments vividly illustrate the unprecedented speed of change. She noted that "the whole tokenmaxxxing thing has become a thing, peaked, and now is seen disfavorably, within six months." She pointed out that the current pricing mechanisms were established before robust business models for AI labs had fully materialized. Concurrently, governments are striving to keep pace; President Trump recently signed a narrow executive order aimed at allowing federal review of powerful AI models. This rapid confluence of technological, economic, and regulatory shifts, she stated, is unlike anything she has previously witnessed. Consequently, she is eager to review upcoming S-1 IPO registration statements, particularly the risk factors, given the daily evolution of these challenges: “How do you even write these risks in, because they are evolving before our eyes, and day by day?”

I further explored the Uber analogy, Sean, acknowledging its eventual profitability through scale. However, I stressed that achieving this required Uber to undergo profound transformations, expanding into new business areas and, in many ways, "squeezing" both customers and drivers. These often-painful adjustments were essential for Uber to reach profitability. I concluded that many AI companies will likely need to enact similar radical transformations if they are to survive and thrive.

Sean pondered whether AI labs possess the same avenues to "squeeze pennies" as Uber did with its drivers. He questioned if there's enough "squishy" flexibility in their cost structures to allow for such measures, acknowledging that AI costs appear "harder, more straightforward in a lot of ways," making the path to profitability a compelling watch.