AI Creators: Blurring the Lines

Social media platforms find themselves in a state of perplexity.

This analysis is presented by The Stepback, a weekly newsletter dedicated to dissecting crucial developments within the technology sector. For deeper insights into the complexities of AI, readers are encouraged to follow Robert Hart. The Stepback is delivered to subscribers' inboxes every Thursday at 8 AM ET.

Initially, discerning AI-generated influencers proved a straightforward task, and their impact on the broader social media landscape appeared minimal, beyond sporadic surges of promotional activity. Early virtual personalities such as Lil Miquela, characterized by her distinct fringe and freckles, Imma with her vibrant pink bob, and Shudu Gram, renowned for her impeccable skin, were unequivocally digital creations. Their commercial partnerships were often publicized with considerable fanfare, and their content production demanded significant investment in studios, finances, meticulous coordination, and extensive post-production refinement.

Subsequently, a noticeable shift occurred as these artificial personas increasingly mirrored human users on social media feeds. Figures such as Emily Pellegrini and Aitana Lopez began to embody a greater sense of realism – or, at minimum, they evoked the archetype of a privileged, well-traveled acquaintance from one's past, consistently sharing updates from luxurious dining venues, picturesque destinations, or high-profile events like Coachella and Wimbledon. While not inherently relatable, this characteristic is often shared by many established human influencers.

Despite their evolving realism, many of these accounts operate outside conventional influencer models. For instance, Lopez is a creation of The Clueless, a Spanish creative agency specializing in managing a portfolio of AI influencers. Similarly, Pellegrini's originator, known pseudonymously as Professor EP, revealed a background in managing OnlyFans creators, having since transitioned to offering courses that instruct individuals on how to develop their own AI influencers.

This emerging trend is now being widely embraced by a growing number of individuals.

The initial novelty has dissipated. Where early AI influencers once stood out due to their scarcity, they now blend into a vast and often chaotic deluge of AI-generated content flooding social platforms. This includes uninspired text generated by chatbots, poorly rendered images and videos, and even viral phenomena like the memorable Lord of the Rings disco track that dominated TikTok feeds for a period.

These artificial personas are now ubiquitous, engaged in activities ranging from promoting low-quality drop-shipped merchandise and defrauding individuals with fabricated images, to disseminating misinformation and prejudiced narratives. They also frequently cater to specialized, often sexually suggestive, niches, including a considerable volume of 'thirst trap' content. Concurrently, a substantial amount of mundane content exists, where avatars merely replicate popular trends initiated by human creators, often by simply superimposing their synthetic visages.

Consequently, assessing the full scope of AI content creator influence presents a significant challenge. Social media platforms do not release data on the proportion of their user base comprising artificial entities. Furthermore, the majority of AI avatars fail to achieve the level of popularity or influence that garnered media attention for their predecessors. While databases such as Virtual Humans monitor hundreds of prominent avatars, these represent only a fraction – those sufficiently unusual, distinctive, or large-scale to attract notice. Beneath this visible layer lies an immeasurable number of accounts operating entirely undetected.

A key factor contributing to the stealth of these accounts is the substantial advancement in the underlying AI generation technology. A static image of a synthetic individual can now convincingly pass as authentic upon a cursory glance, particularly within feeds populated by human influencers who extensively utilize staging, filters, and editing techniques. Video and audio capabilities are rapidly advancing, endowing virtual personas with voices and movements capable of deceiving casual viewers. Moreover, the tools required are no longer specialized or prohibitively costly. Mainstream offerings from entities like Google and OpenAI now complement dedicated services from companies such as Higgsfield, HeyGen, and ElevenLabs. This accessibility means that with minimal effort, nearly anyone can create an AI influencer, or even a collection of them, without requiring professional studios, specialized gear, or significant financial investment.

This complex situation presents social media platforms with a challenge they appear reluctant to confront directly. Following years of contending with AI-generated visual and auditory content, most prominent platforms have established policies addressing synthetic media. However, beyond mandating labels for AI-generated material, these regulations often merely attempt to force such content into pre-existing categories like scams, spam, impersonation, or graphic content. AI personas, particularly those meticulously crafted to emulate real individuals, do not align well with these classifications. They are not inherently engaged in fraudulent activity, disseminating explicit material, or impersonating a specific person – given they are not real, whom would they impersonate? Furthermore, if they openly declare their content as AI-generated, the specific rules they might be violating remain ambiguous.

Presently, platforms seem to embrace a state of ambiguity, neither fully endorsing nor outright rejecting AI creators. They maintain a contradictory stance, simultaneously championing AI as an innovative creative instrument while striving to prevent an overwhelming influx of low-quality content from inundating their services. Platforms such as YouTube, TikTok, and Instagram have instituted guidelines for labeling synthetic media, particularly highly realistic forms, even as they promote their proprietary AI toolkits, some of which are capable of cloning or simulating users. Nevertheless, these regulations typically target individual pieces of content rather than the accounts and underlying personas, thus positioning AI influencers within a regulatory 'gray area'.

Amidst this regulatory uncertainty, the AI influencer ecosystem is flourishing. Projections from certain market research firms suggest the virtual influencer market could exceed $60 billion by 2030, a substantial increase from approximately $12 billion this year. Their cultural influence is also expanding, evidenced by the emergence of AI influencer awards, beauty pageants, and specialized talent agencies dedicated to representing synthetic creators. Furthermore, a burgeoning market exists where synthetic creators offer courses and tools, promising to equip individuals with the means to develop and manage their own artificial personas, often with the allure of generating passive income without direct personal involvement. This environment bears the faint resemblance of an online gold rush, characterized by a handful of prominent success stories amidst a multitude of vendors selling the tools for participation.

It is plausible that a significant confrontation is imminent. The proliferation of low-quality AI content is already a source of frustration, and platforms can only sustain a finite amount before becoming effectively unusable, particularly given their consistent reluctance to provide users with filtering options. Artificial personas masquerading as real individuals represent an even more insidious manifestation of this problem. Yet, beyond merely applying labels and enforcing existing regulations, platforms largely appear content to observe developments. For these platforms, engagement remains the primary metric, irrespective of whether it originates from a synthetic or human creator. Consequently, as long as AI creators continue to post and adhere to current guidelines, there appears to be scant motivation for a stricter crackdown.

Furthermore, the long-term viability of widespread AI avatar presence online raises questions. If a significant proportion are designed solely to monetize human users, what transpires when the human user base diminishes? For instance, there is a finite number of individuals willing to invest in courses and tools to develop their own influencers. This premise also assumes that social media itself can withstand the influx of AI personas. By its very nature, social interaction online necessitates a critical mass of human participation. If left unregulated, these networks risk collapsing under the sheer volume of artificial entities, as human users are invariably driven away.

This dynamic could shift if public discontent intensifies. Growing backlash concerning deepfakes, impersonation, and synthetic spam is already compelling legislators and regulatory bodies to increase scrutiny, especially following incidents involving non-consensual sexual deepfakes created with tools like Grok. Europe’s AI Act may serve as a catalyst, particularly once its transparency requirements for AI-generated content become effective. These regulations will mandate that deployers of generative AI systems explicitly disclose AI-generated or manipulated material, potentially pressuring companies to enhance their flagging of AI content or face substantial penalties. However, even with these measures, the primary focus remains on the content itself, rather than on whether the posting account represents a genuine human being.

As is frequently the case across social media, the onus of responsibility often reverts to the users. Numerous platforms have effectively outsourced the moderation of AI content, depending on their user base to identify and report dubious profiles. Yet, self-moderation proves an inadequate and unsustainable solution for a problem specifically engineered to elude detection. A burgeoning demand for AI-free online environments is already evident. Should platforms decline to establish clear distinctions between genuine and artificial, it is anticipated that users themselves will ultimately define these boundaries.