Reface, Prisma Creators Unite to Turbocharge On-Device AI with Mirai

The contemporary discussion surrounding artificial intelligence predominantly centers on expanding cloud infrastructure and establishing colossal data centers to execute AI models. Concurrently, major technology firms like Apple and Qualcomm are in the nascent stages of enhancing on-device AI utility. Amidst these developments, a 14-person technical team at London-based Mirai is committed to optimizing the performance of AI models on personal devices such as smartphones and laptops.

Mirai, founded last year by Dima Shvets and Alexey Moiseenkov, recently secured a $10 million seed funding round spearheaded by Uncork Capital. Both founders possess extensive experience in developing scalable consumer applications. Shvets notably co-founded Reface, a face-swapping application backed by a16z, and subsequently served as a scout for the venture capital firm. Moiseenkov was the CEO and co-founder of Prisma, a widely popular AI filter application from the previous decade.

Shvets explained that, as consumer developers, both had been contemplating the integration of AI and machine learning on devices even prior to the widespread popularity of generative AI.

"When we met together in London, we started to chat about technology, and we realized that within the hype of gen AI and more AI adoption, everybody speaks about cloud, about servers, about AGI coming. But the missing piece is on-device [AI] for consumer hardware," he conveyed to TechCrunch.

The impetus for Shvets and Moiseenkov to establish Mirai stemmed from their ambition to leverage AI in creating a pipeline capable of enabling complex tasks directly on phones. Their inquiries among other consumer app developers revealed a collective demand for improved cost optimization and better margins per token usage.

Currently, Mirai is actively developing a framework designed to enhance the on-device performance of AI models. The company has engineered an inference engine specifically for Apple Silicon, which significantly optimizes on-device throughput. Mirai states that its forthcoming SDK will allow developers to seamlessly integrate this runtime into their applications with just a few lines of code.

Shvets articulated their core vision, stating, "One of the visions why we started the company was that we wanted to give developers, like this Stripe-like, eight lines of code [integration] experience…you basically go to our platform, integrate the key, and start working with summarization, classification, or whatever your use case is."

The startup claims that its engine, developed using Rust, can accelerate a model’s generation speed by up to 37%. Furthermore, the company assures that when tuning models for a specific platform, it avoids altering model weights, thereby preserving the quality of the output.

Mirai’s current technology stack primarily focuses on enhancing text and voice modalities, with future plans to incorporate vision support. The team has initiated collaborations with frontier model providers to optimize their models for edge deployment and is in discussions with various chipmakers. The company also intends to extend its engine’s compatibility to Android devices in the future.

Additionally, Mirai aims to release on-device benchmarks to enable model makers to rigorously test performance directly on hardware. Shvets acknowledges that not all AI workloads are suitable for on-device execution. To facilitate a hybrid operational mode, the team is developing an orchestration layer designed to route requests that cannot be processed locally to the cloud.

While the startup is not yet directly partnering with specific applications, its engine holds the potential to power a diverse range of on-device functionalities, including intelligent assistants, transcribers, translators, and chat applications.

Andy McLoughlin, managing partner at Uncork Capital, reflected on his investment in an edge machine learning company a decade ago, noting that the venture was ahead of its time and was ultimately acquired by Spotify. He believes the present landscape offers a distinct opportunity.

"Given the cost of cloud inference, something has to change… For now, VCs are happy to continue funding the rocketship companies, spending inordinate sums on cloud inference. But that won’t last — at some point, people will focus on the underlying economics of these businesses and realize that something has to change," he asserted. McLoughlin added, "It feels like every model maker will want to run part of their inference workloads at the edge, and Mirai feels very well positioned to capture this demand."

Mirai’s seed funding round also garnered participation from a distinguished group of individual investors, including David Singleton (Dreamer CEO), Francois Chaubard (YC Partner), Marcin Żukowski (Snowflake co-founder), Mati Staniszewski (ElevenLabs co-founder), Gokul Rajaram (former Google AdSense product manager and Coinbase board member), Scooter Braun (Groq investor), Vijay Krishnan (Turing.com CTO), Ben Parr and Matt Schlicht (Theory Forge Ventures), and Aditya Jami (former Netflix technical leader).