Sony Experiments with AI-Driven PlayStation Characters Like Aloy

Sony is exploring AI-powered game characters, with a prototype version of Aloy from Horizon Forbidden West being tested. An internal video leaked to The Verge showed Aloy responding to player prompts using AI-generated voice and facial movements.  This demo was developed by Sony’s PlayStation Studios Advanced Technology Group alongside Guerrilla Games, using OpenAI’s Whisper for speech-to-text, GPT-4 and Llama 3 for conversations, and Sony’s own Emotional Voice Synthesis and Mockingbird technology. While the demo was initially run on a PC, Sony has also tested parts of the AI technology on PlayStation 5 with minimal performance impact. The project was first demonstrated internally last year and later showcased at the Sony Technology Exchange Fair in Tokyo in November. The video, narrated by Sony software engineering director Sharwin Raghoebardajal, was later removed from YouTube due to a copyright claim. While this AI prototype raises questions about its role in a game where players control Aloy, it also highlights broader industry concerns regarding AI’s impact on voice actors and game developers. Sony’s move aligns with industry trends, as companies like Nvidia and Microsoft are also working on AI-driven NPCs. Nvidia's AI NPC technology, Ace, has been demonstrated several times in 2024, while Microsoft is collaborating with Inworld AI to bring AI-generated characters and storylines to Xbox. The rise of AI in gaming has sparked debate, especially amid ongoing industry layoffs. A Game Developers Conference 2024 survey found that nearly half of game developers are already using generative AI tools. As Sony advances its AI experiments, the discussion around AI’s role in gaming continues, especially with the upcoming GDC event in San Francisco. Sony has yet to comment on the leaked video or its removal.