Ex-Meta Scientists Unveil Isaac 0.5 to Bridge Visual AI and Physical Factory Work

While artificial intelligence has revolutionized the digital world, it has largely remained confined to screens. Startups are now increasingly focused on bringing this technology into the physical realm to interact with the real world.

Perceptron, co-founded by two former Meta researchers, is at the forefront of this movement. Established in November 2024, the startup specializes in developing advanced vision models designed to enhance how machines interact with physical environments.

This week, the company unveiled Isaac 0.5, a model engineered to enable machines to "perceive, reason and act" within industrial settings. The software is specifically designed to assist vision-guided robots in traversing complex environments such as warehouses and factory floors, while also allowing companies to extract valuable visual intelligence from bot recordings.

Isaac 0.5 is also being released as an open-weight model, allowing anyone to inspect its parameters and training materials.

Backed by a recent $21 million funding round led by Bessemer Venture Partners, the startup was co-founded by Armen Aghajanyan and Akshat Shrivastava, both of whom previously worked at Meta’s Fundamental AI Research (FAIR). The founders view their technology as the future of automated industrial deployment.

"Physical AI today forces a false choice: generalist foundation models that need multiple dedicated cloud GPUs for every instance, or narrow models that handle perception or control, but never both," the company states.

Aghajanyan and Shrivastava argue that their tool differs from existing models because it is general-purpose. Rather than being built for a single, repetitive task, the model is designed to be flexible and adaptable to specific environments or situations.

In an interview, Aghajanyan and Shrivastava highlighted the complexity of simple physical tasks. Shrivastava asked, "Imagine there’s a robot being deployed to sort packages right now. What are the tasks it would need to do?" This task involves numerous steps, such as reading package labels, performing spatial analysis to locate boxes, and deciding which one to pick up. If handling multiple boxes, the robot must also plan the order of operations.

Perceptron’s software aims to assist robots through every stage of this process. While the industry has tools for many of these individual steps, few programs are designed to handle them with the necessary flexibility.

So, where does the data for this advanced algorithm come from?

Models like Isaac 0.5 acquire operational skills by ingesting massive amounts of video training data. The company claims its model was trained on one million hours of general video to teach the algorithm to identify specific settings and scenarios. Additionally, Perceptron relied heavily on ego video—recorded from a person's perspective using cameras like GoPros—and UMI video, which records repetitive human actions to teach AI systems movements.

Although Perceptron has not disclosed the specific sources of its training data, Shrivastava revealed that the company has internally constructed petabyte-scale datasets spanning multiple modalities, including images, text, video, and robotic trajectories.

The utility of software capable of enabling robots to operate competently in warehouses is immense, and Perceptron believes it is well-positioned to lead this wave of automation. The startup is preparing to market its software to a variety of vendors, aiming to integrate its intelligence layer across a broad spectrum of industries.

Target industries for this technology include manufacturing, logistics and warehousing, security, mobility, and media and entertainment.

"Nothing like this really exists out there," Aghajanyan remarked. "We’re really excited about it."