Hot French Startup ZML Offers Free AI Inference Scaling

While Nvidia continues to hold a significant position in the market, a growing landscape of alternatives and competitors is beginning to emerge.

ZML, a promising French AI startup backed by Turing Award laureate Yann LeCun, has unveiled inference-performance software designed to facilitate the execution of diverse open-source large language models across a wide array of hardware. This includes chips from Nvidia, AMD, Google's TPUs, Apple Metal, and Intel Arc.

Steeve Morin, founder of ZML, informed TechCrunch that the company's newly introduced LLM inference server, ZML/LLMD, seeks to dismantle current technological barriers. Its objective is to unlock the full potential of various chips for AI applications, often enabling them to operate at or even beyond their advertised maximum speeds.

Morin highlighted that as AI increasingly integrates into professional and personal spheres, the optimization of inference — the process of handling prompts — has surpassed model training in criticality. However, this crucial area often suffers from underlying inconsistencies due to software and architectural limitations, frequently resulting in vendor lock-in.

This capability to achieve optimal performance across a multitude of chip architectures represents a significant technological advancement. Furthermore, it holds the potential to disrupt the market, particularly given growing concerns surrounding the costs associated with AI deployment.

ZML's vision is to empower enterprises and cloud providers with the flexibility to utilize a diverse combination of chips, including those that may offer cost savings or reduced energy consumption. Morin emphasized, “The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated.”

Morin noted that this software-driven assistance could significantly benefit emerging AI chip manufacturers, many of whom are based in Europe, such as Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. He stressed that beyond their geographical origin, his primary focus is ZML's ability to collaborate with these companies on “things that haven’t been done before anywhere in the world.”

Despite these developments, Morin is not pessimistic about Nvidia's prospects, partly due to its established supply chain. He informed TechCrunch that ZML maintains a positive relationship with the AI chip behemoth, which has been actively preparing for the escalating demand in inference processing.

The domain of inference has garnered such substantial investment that it has been dubbed the “inference gold rush.” Consequently, ZML faces competition from players like Baseten, recently valued at $13 billion; Inferact, developed by the creators of the open-source vLLM project; and RadixArk, the commercial entity behind SGLang.

While vLLM and SGLang present partial competition to LLMD, Morin's aspirations for ZML extend significantly further. He stated, “We have reached the point where we are co-designing silicon,” attributing the Parisian startup's rapid progress and planned future releases to its agile team of just 20 individuals.

This compact team also benefits from substantial funding relative to its size. Leveraging his prior success as VP of Engineering at Zenly, a company acquired by Snapchat for a nine-figure sum in 2017, Morin secured $20 million in funding from prominent venture firms, including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.

In contrast to ZML’s initial public offering—an inference-focused ML framework launched in 2024 and updated in March—ZML/LLMD is not open source. However, it is being launched as a free product with the primary objective of gathering user insights. Morin explained his strategy: “I’d rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go.”

While the timeline for ZML/LLMD’s transition to a paid offering and its market adoption remains to be seen, the startup’s impressive cap table signals significant interest from influential figures. These include Solomon Hykes, founder of Dagger and Docker; Clément Delangue and Julien Chaumond of Hugging Face; and LeCun, currently affiliated with AMI Labs. This also reinforces the growing narrative that European AI startups are increasingly capable of thriving domestically. Morin affirmed, “I couldn’t do ZML anywhere but in Paris.”