Fanfiction's AI War: A Community Divided

Within the realm of fanfiction, readers are actively seeking methods to identify whether generative AI contributed to the creation of fanworks, though the efficacy of these detection techniques remains highly debatable.

A recent movement has emerged in fan communities over the last week, focused on identifying and addressing the use of generative AI by authors. However, the detection strategies employed are proving to be unreliable, raising concerns that legitimate fanfiction writers could be unfairly implicated.

Widespread disapproval of AI tools like Claude and ChatGPT has been a long-standing sentiment within creative communities, including fanfiction. Enthusiasts have shared various informal tips for identifying AI-generated content, ranging from specific punctuation like em dashes to the broader characteristic of "purple prose." This changed on June 29th, when an anonymous X (formerly Twitter) account, @heatedrivalryai, introduced what appeared to be a more robust solution. This account released a user-installable skin—analogous to a browser extension—for the prominent fanfiction platform Archive of Our Own (AO3), claiming it could detect unique coding artifacts embedded by Anthropic’s Claude bot.

The @heatedrivalryai account explained its mechanism: "When a Claude-generated response is pasted directly into AO3 from Claude, the text is wrapped by a Claude-injected code ‘font-claude-response-body.’ Its presence indicates the use of Claude definitively." If a user accesses an AO3 page, such as a fanfic, containing this specific code, the installed skin automatically changes the entire background to red.

To verify its functionality, several test posts were uploaded to AO3. Upon personal testing, the background promptly turned red when viewing these examples. Further experimentation involved publishing a Claude-generated short story; the red screen appeared consistently when the text was pasted directly from the chatbot into the AO3 editor, but disappeared if the same text, or any other, was pasted from an intermediary source rather than straight from Claude.

Alongside the Claude detector, examples of fanfiction containing these artifacts were provided. The anonymous creator asserted that these examples were intended solely to validate the system's functionality, not to "create an environment of mistrust or accuse particular users." Despite this, fanfiction communities rapidly began to publicly identify and criticize writers whose works were flagged by the tool. The creator's stance on AI is clearly negative, stating, "Fandom is a uniquely connective, collaborative space. It thrives on the human element and the creative spark which drives it and feeds off it. If we unknowingly allow AI to corrupt these spaces, what will be left of them?"

Anthropic did not respond to inquiries regarding the accuracy of this fan-developed Claude detector. Nevertheless, the underlying methodology appears robust, and independent testing corroborates its claims. The presence of Claude-specific code within a text offers strong evidence of the bot's involvement. However, the system carries a significant risk of both false negatives and misinterpretations.

Crucially, the embedded code is only retained if text is copied directly from Claude and pasted into AO3’s editor. This means the detector will not identify content that has been edited in external applications like Google Docs or Microsoft Word before being transferred to AO3. As a professional writer, I can attest to the inherent risks of composing directly within a Content Management System. Already, some writers whose works were flagged have updated them to remove these artifacts, indicating that future works can readily circumvent this detection method.

Conversely, the presence of the detection tag does not quantify the extent of Claude's usage within a work. The striking red background could signify a story entirely generated by AI, or it could simply indicate that an author used Claude for minor tasks such as spell-checking or translation of a few human-written sentences, subsequently moving them back to AO3.

For some members of the fandom, this distinction is irrelevant, as they perceive any application of generative AI as an unacceptable breach of trust within the broader creative community. Common concerns cited include the technology's environmental footprint and its training methodology, which involves scraping vast amounts of data from the open web, potentially including fanworks hosted on platforms such as AO3.

The utility of this specific tool is inherently limited, given that AO3 is not the sole platform for fanwork publication, and Claude represents only one of numerous AI models available. While at least one individual claims to have developed separate code capable of detecting usage from “Claude, Deepseek, and some ChatGPT,” this solution has not been publicly released or detailed. Inquiries to Google and OpenAI regarding traceable artifacts in their text generation models, detectable by similar methods, have not yet received a response.

Indeed, the existence of a universally reliable AI detection system would be highly unexpected. Having covered issues related to AI detection for several years, I am unaware of any current technological solution that can dependably differentiate between AI-generated text and human-authored content. While initiatives like C2PA Content Credentials and Google’s SynthID show promise in identifying generative AI in visual and auditory media through invisible watermarks and metadata, these mechanisms do not translate effectively to copy-pasted text.

AI development companies, in particular, possess strong incentives to address and resolve this detection challenge internally.

This situation may evolve, as AI companies are strongly motivated to find internal solutions. Initial AI models were trained on vast amounts of text indiscriminately gathered from the internet. Should human-generated content become overshadowed by synthetic counterparts, there's a risk of "model collapse," which would severely compromise the accuracy and quality of future AI outputs.

Presently, however, fandom communities largely continue to rely on subjective impressions or "vibes." The majority of fanfiction is not evaluated by tools such as the AO3 skin, but rather by perceived "tells," which can range from distinctive sentence structures—like the often-criticized "it’s not X, it’s Y" phrasing—to an excessive use of elaborate metaphors. (Fortunately, the more bizarre AI hallucinations, such as "benches becoming men," have not yet permeated fandom.) It is crucial to remember that AI frequently adopts these stylistic traits precisely because it is trained on human-written content, essentially attempting to emulate human expression. While I refrain from sharing my personal AO3 bookmarks, I recall encountering numerous overly verbose fanfics in the pre-ChatGPT era that would undoubtedly fail current, often dubious, AI detection criteria.

The most effective solution for differentiating AI-generated works on AO3 is already integrated into the platform: its comprehensive tagging system. A specific "Created Using Generative AI" tag is available, and many authors conscientiously utilize it to disclose their use of tools such as Claude. This, however, necessitates genuine transparency, an attribute that currently lacks strong incentive given the prevailing negative sentiment. Furthermore, it's important to recall that fanfiction is fundamentally a hobby, not a regulated industry.

In the fervent pursuit to safeguard human creativity from the perceived encroachment of AI, authors whose writing styles do not align with current notions of "acceptable quality" risk becoming unwitting casualties of this burgeoning "witch hunt." Already, at least one writer has been implicated after a trusted editor utilized Claude during the editing process of their fanfiction. Therefore, should a future fanfic strike you as somewhat robotic, remember that its origin may not, in fact, be entirely artificial.