AI's Looping Dilemma

During a recent appearance at Meta’s @Scale conference on Friday, Boris Cherny, the visionary behind Claude Code, was met with an unexpected opening question from the audience: a query about the role of "loops" in the future of AI development.

The questioner probed, “Are loops the next hype cycle, or are they for real?”

Cherny's response was unequivocal and enthusiastic: “Yes, they’re for real,” he affirmed.

He elaborated on the rapid evolution of code generation, stating, “Two years ago, we wrote source code by hand. We started to transition so agents write the code. And now we’re transitioning to the point where agents are prompting agents that then write the code.” Cherny emphasized the monumental impact of this shift, adding, “As big as the step from source code to agents was, loops are just as important and as big a step.”

Further into his presentation (approximately 32 minutes into the accompanying YouTube video), Cherny provided concrete examples of these continuous loops in his own workflow. He described one agent persistently seeking architectural improvements in code, while another actively identifies and unifies duplicated abstractions. These agents operate autonomously, submitting pull requests akin to human developers, and remain in constant operation due to the perpetually evolving codebase.

This concept is particularly potent, especially with an influential figure like Cherny championing it. As AI transitions towards agentic models, the primary focus for most users has been on meticulous agent management: defining clear objectives, monitoring progress in discrete segments, and ensuring they adhere strictly to prompts. However, the introduction of loops elevates this further, empowering a collective of agents to work continuously and indefinitely in the background. While this necessitates a significant degree of trust in AI, the rapid advancements in model capabilities suggest it could be the pivotal next step in enabling AI to undertake substantial, real-world tasks.

It's important to recognize that this concept isn't entirely novel. Recursive loops—functions designed to call themselves repeatedly until a specific stopping condition is met—are a fundamental element of introductory computer science curricula. While these agentic loops operate with a non-deterministic logic, where a sub-agent determines the termination point rather than a fixed condition, the underlying recursive principle remains consistent. It was inevitable that some form of recursive loop, with AI overseeing AI, would emerge as programmers began leveraging AI for task completion.

In contrast to conventional computing, agentic loops can possess surprising simplicity. A popular technique, known as the "Ralph Loop" (named after Ralph Wiggum), exemplifies this by summarizing the model's work and querying if its objective has been achieved. This method effectively prevents AI models from becoming disoriented during prolonged operations, essentially "bouncing" the model back and forth until the task is successfully completed.

Another perspective views loops as integral to the broader drive for increased "test-time compute." As OpenAI researcher Noam Brown noted recently, contemporary models can resolve nearly any problem given sufficient computational resources. This implies that one strategy for problem-solving is simply to continuously apply compute until a solution is found. This approach is especially effective for "hill-climbing" problems, such as refining a codebase, where the model can make iterative improvements until a desired threshold is met. Or, as demonstrated in Cherny’s scenario, it can continue making incremental enhancements for as long as computational resources are available.

Such continuous operation naturally raises concerns about cost. Similar to agentic AI, AI loops consume tokens at a significantly higher rate than basic Q&A chatbots. Furthermore, given their design for perpetual operation, there is no inherent limit to the expenditure. While this model may be advantageous for companies like Anthropic, which are inherently in the business of selling tokens, it could prove to be a costly operational approach for most other users.

Nevertheless, depending on the specific problem an agentic loop aims to address and with the implementation of robust oversight mechanisms for token expenditure, drift, and other common AI challenges, the potential benefits could be profound enough to substantially outweigh the associated costs.