From GPUs to Inference: Why Early Investors Are Betting $400M

General Compute, an emerging AI inference cloud startup, has successfully secured a substantial $400 million loan from Upper90, a prominent tech investment firm. This transaction is particularly noteworthy as it may represent the first instance of using inference-specific chips as collateral—hardware meticulously engineered to execute already-trained AI models with speed and efficiency, in contrast to the more costly chips typically employed for model development.

This significant financing serves as a clear indication that financial markets are actively addressing concerns regarding the expense of AI tools and tokens. The prevailing trend suggests a pivot towards infrastructure solutions capable of running open-source models more economically than the cutting-edge large language models (LLMs) developed by leading AI research labs.

Founded by CEO Finn Puklowski, General Compute previously raised a $15 million seed round in May, earmarked for constructing an inference-focused "neocloud." This specialized infrastructure leverages silicon from SambaNova, an Intel-backed chipmaker. Unlike the general-purpose cloud services offered by hyperscalers such as AWS or Azure, neoclouds are purpose-built and optimized for demanding AI workloads.

The company's proprietary SN50 chips are specifically engineered for inference tasks, boasting remarkable power efficiency and negating the need for expensive water-cooling systems. This design facilitates quicker deployment across a wider array of data centers compared to traditional GPUs. General Compute asserts that these new chips will deliver inference speeds up to 16 times faster than current GPU-based cloud offerings.

However, a considerable challenge for any nascent company lies in securing a substantial quantity of these advanced chips to meet demand.

Upper90 co-founder and CEO Billy Libby, a former quantitative trader at Goldman Sachs, brings a proven strategy to this endeavor. In 2021, his firm pioneered financing for GPU acquisitions by Crusoe, an energy-focused data center startup. Libby believes this marked the first loan ever extended against the intrinsic value of advanced chips.

At that time, traditional lenders largely shied away from such deals, apprehensive about the risks associated with GPU depreciation and market uncertainties. Yet, with CoreWeave successfully transforming chip-backed loans into a viable business model and subsequently the foundation for a blockbuster IPO, this form of asset-backed financing has become increasingly mainstream.

“When we financed Nvidia GPUs as the first group to do that, the market was inefficient,” Libby shared with TechCrunch. He added, “We could really put together something as an early participant, and kind of get compensated for the risk.”

With GPUs now being comparatively well-understood and potentially over-purchased, Upper90 is strategically shifting its focus to companies like General Compute, aiming to capitalize on the next evolutionary phase of the AI boom. Libby elaborated on this strategy, stating, “We think open source models are going to be important, and we went and looked for a player last year that was in inference. Everyone doesn’t need a supercomputer, but they do need inference and AI.”

This investment thesis has gained significant traction, evidenced by the soaring valuations of companies providing access to open models, such as OpenRouter and Fireworks, in recent funding rounds. Furthermore, new models like Kimi’s K3 have recently demonstrated competitive performance against the latest releases from industry leaders Anthropic and OpenAI on crucial coding benchmarks. Concurrently, innovative chipmakers like Groq and Cerebras are attracting considerable interest from both potential acquirers and public markets.

General Compute's strategic ability to access chips outside of Nvidia’s dominant ecosystem is critically important for these very reasons. Similarly, TensorWave, another AI infrastructure firm, is pursuing a comparable strategy through a partnership with AMD. As a broader spectrum of alternatives to Nvidia emerges, compute providers not exclusively tied to Nvidia deals may gain a significant competitive edge in delivering more cost-efficient inference capabilities.

“There are a bunch of chips that are starting to scale that have amazing [total cost of ownership], or that can operate much faster than Nvidia, but there’s not too many buyers for them,” Puklowski observed. He further emphasized the broader implications of the Upper90 partnership, stating, “By getting together with Upper90, this is not just, ‘a cool startup got some money to buy some compute.’ Like, this is the first signal of capital organizing itself and the fragmenting of Nvidia’s monopolistic dominance.”