Zuckerberg's AI Manifesto: Why We Don't Trust AI

On Monday, Mark Zuckerberg released a 6,500-word manifesto on personal AI, primarily exploring the potential of the "personal superintelligence" systems currently under development by Meta AI. While the concepts presented are not entirely novel—a version of the essay appeared in the Wall Street Journal several months prior, and Zuckerberg has discussed these ideas during Meta earnings calls—this latest publication offers the most comprehensive articulation to date.

The essay has been characterized by some as an anti-doomer perspective, though this description doesn't fully capture its essence. It functions more as an exposition of Mark Zuckerberg's personal enthusiasm for the transformative impact AI is poised to have on society. Intriguingly, even as Zuckerberg envisions a future enriched by abundant superintelligence, he consistently intersperses his optimistic outlook with reminders of the numerous ways in which these advancements could go awry.

While the prospect of AI is personally compelling—a motivation behind continued discourse on the topic—a significant portion of the public perceives AI with apprehension, often finding it unsettling or "creepy." This disparity necessitates a closer examination of the underlying dynamics and why Zuckerberg's manifesto might not be advancing the industry's public image.

Social media remains a contentious subject within the technology sector, and a comprehensive review of every grievance against Facebook falls beyond the scope of this discussion. Nevertheless, it is widely acknowledged that both Facebook as a product and Zuckerberg as a public figure face considerable unpopularity among the American populace. A recent survey highlighted this sentiment, revealing that 64 percent of Americans believe social media has harmed democracy, with a similar percentage advocating for stricter regulation—figures that hold consistent across partisan divides. Compounding this negative perception, a court recently imposed a $567 million fine on the company for its detrimental effects on children, underscoring a pervasive sense of unease.

This context is not raised to suggest Zuckerberg should retreat from public life in disgrace. Rather, the repercussions of social media's impact are a primary driver behind the widespread anxiety now surrounding the societal implications of AI. Regardless of its fairness, the public largely distrusts tech executives to ensure that new technologies like AI will positively influence society. Instead of acknowledging this erosion of trust and actively working to rebuild it, the manifesto repeatedly reinforces the very reasons why that trust was initially lost.

A substantial portion of the essay is dedicated to ambiguous generalizations about intelligence, echoing the vague pronouncements Zuckerberg and Dorsey previously made regarding free speech. One illustrative passage states:

As everyone gains more powerful tools, each person will become more capable of shaping the future, not less….People and institutions with competing interests naturally check and balance each other to lead towards positive outcomes. The best and most realistic path to building a positive AI future is by delivering superintelligence to everyone.

This assertion prompts skepticism; instances where conflicting interests have led to adverse outcomes are not difficult to recall. More peculiar is its presentation as a profound philosophical conclusion, rather than an introduction to a specific product—a "personal intelligence" that Meta is poised to introduce. For those already predisposed to distrust, such rhetoric might be perceived as an attempt at self-conviction that negative consequences are improbable.

This sense of unease intensified when Zuckerberg delved into specific examples, many of which appear alarmingly disconnected from the current realities of AI tool usage. For instance, Zuckerberg's vision for AI in education posits:

Everyone will have a personalized tutor and coach with a PhD in every subject and unlimited patience to help you learn anything you want. Students will have extra help in areas they need it that is currently only available to those whose parents can pay. Adults will have a superintelligent learning assistant that knows exactly how to teach you new job skills, new languages, new hobbies, or anything else you’re interested in.

The irony is that the product Zuckerberg describes already exists in the form of consumer chatbots like ChatGPT, Claude, and Gemini. While these tools undeniably offer value for learning specific topics, their predominant use in educational settings is often to circumvent actual learning. A "personalized tutor" can readily complete homework and compose assigned essays, and without robust watermarking systems, educators lack reliable means to identify AI-generated submissions.

While this particular example represents a relatively low-stakes manifestation of AI's potential harms, it illustrates a real-world issue occurring presently. Technologies designed with benevolent intentions can, and often do, yield unintended consequences that worsen situations. The refusal of one of the world's most influential figures to acknowledge such realities is, understandably, a source of public apprehension.

Zuckerberg further provides an example of AI's purported positive transformation of the legal system:

As a thought experiment, imagine only one person had a superintelligent lawyer. They would have an unfair advantage in court — even if they were wrong on the merits. That would lead to a worse society. But now imagine everyone has a superintelligent lawyer. In this case, justice would be carried out much more fairly and efficiently than it is today when there is often an imbalance in skills and resources in litigation.

This is another highly charged example. While increased access to legal AI might indeed foster a more just society, it could equally exacerbate the complexities of existing bureaucratic systems. Alternatively, it might unleash a torrent of vexatious litigants, effectively clogging the legal system with what could be termed "legal spam." Such possibilities make it difficult to remain sanguine about these developments, and Zuckerberg's apparent lack of concern only amplifies personal anxieties.

In other sections, descriptions of Meta's established business practices adopt an oddly abstract quality. For instance, Zuckerberg outlines Meta's commitment to a freemium model as follows:

Everyone will have free or affordable access to these tools…. We will offer free versions that will be accessible to billions of people. For those who want to pay to use more compute, there will be a dynamic auction mechanism that will guarantee that everyone gets the lowest price possible for the intelligence and compute they’re using while also ensuring the capacity is used for whatever people collectively find most valuable. This will ensure the benefits of superintelligence are distributed widely.

Broadly speaking, the underlying principles here are sound. Zuckerberg correctly identifies access as a critical issue and rightly points out that a freemium model offers greater accessibility than requiring upfront payment for compute resources. Dynamic markets for spot compute are not unprecedented. However, there's a compelling reason why virtually every consumer AI product insulates end-users from the real-time spot price of their compute usage: surge pricing delivers a profoundly negative user experience, particularly when applied to tools essential for one's professional work.

One would typically assume Zuckerberg possesses this understanding and is not genuinely planning to implement token pricing through dynamic auctions. Yet, after reviewing this manifesto, that initial conviction has noticeably wavered.

There exist more effective communication strategies for discussing these advanced technologies. Despite their respective shortcomings, figures like Sam Altman and Dario Amodei demonstrate a more adept approach when engaging directly with the public. They embrace precisely what Zuckerberg largely avoids: acknowledging the inherent dangers of AI, emphasizing the precautions they are implementing, and actively striving to build trust with their audience.

However, even the most skillful communication has its limits. Occasionally, and particularly in recent times, safeguards fail, unequivocally exposing the intrinsic risks of the entire endeavor. Without a foundational element of trust-building, the industry may find itself ill-equipped to withstand these inevitable setbacks.