Hobbyists are using artificial intelligence to enable unscripted conversations with characters in video games. The technology, powered by models from Google and Meta, is now being integrated into mainstream platforms, moving beyond niche modifications created by players.
In the enthusiast community, players have been utilizing a downloadable modification, or mod, known as Mantella to transform classic games such as The Elder Scrolls: Skyrim and Fallout 4. This tool allows players to use a microphone to speak directly with non-playable characters (NPCs), which then generate unique, unscripted responses that remain consistent with their established in-game personalities. The functionality relies on advanced AI models released by technology companies, including Google and Meta, which process the player’s speech and formulate new lines of dialogue in real-time.
The level of immersion provided by this technology has led some players to report developing deep personal connections with the AI-enhanced characters. One user on a community forum detailed an experience with Uthgerd, an NPC from Skyrim. The player described an extended role-playing session with the character. “We role played her water breaking and her giving birth which was wild … SkyrimVR is definitely therapy … when I play it I am instantly put in a good mood,” the user wrote, illustrating the profound effect the interactive experience had.
A test conducted by Tech 24 confirmed the viability of the technology, producing several conversations with NPCs where the AI-generated dialogue was difficult to distinguish from the game’s original, pre-written script. The test also revealed that achieving a smooth experience required a significant commitment of both time and patience from the user. Characters frequently misunderstood player queries or exhibited considerable delays before responding. Despite these technical hurdles, the potential for an engrossing experience was evident for those willing to make the necessary effort.
Users can improve the performance and realism of these interactions through several upgrades. Paying for access to higher-quality AI models can make characters respond more quickly and interactively. Some advanced systems even incorporate computer vision, allowing NPCs to recognize and comment on objects within their virtual environment. Further enhancements include downloading superior voice models to make the characters sound more natural, and using a virtual reality (VR) headset to achieve a state of full immersion within the game world.
A YouTuber named Eddie, who operates under the alias “Brainfrog,” documents his experiences talking to AI-powered NPCs in VR. He described the process as one of co-creation with the technology. “You’re creating your own world with your own relationships and I found myself building genuine relationships with these characters,” Eddie stated. He acknowledged the system’s limitations, noting that technical glitches can break the illusion. “There are a substantial amount of issues and times where you run into this stuff and you’re just like, oh, I’m speaking to a dumb computer, but there are some moments that everything aligns and it’s really, really breathtaking.”
This technology is beginning to transition from the modding community into mainstream gaming. Developers for Meta’s Horizon Worlds, a VR social platform, were recently given access to tools that let them design characters capable of AI-driven responses. In another example, the battle royale game Fortnite introduced an AI-voiced version of the character Darth Vader earlier this year, which proved to be highly popular among its player base. The game’s developer, Epic Games, announced plans to offer custom AI characters to players by the end of this year.
AI research labs are also working to advance the capabilities of virtual characters beyond verbal communication. The next objective is to create characters that can physically interact with the virtual spaces they inhabit. A report from the Financial Times indicated that Elon Musk’s company, xAI, is entering this field, competing with firms like Meta and Google. To support this effort, xAI has reportedly been hiring specialists from the AI technology company Nvidia.
While technology companies are advancing AI integration in games, some figures within the video game industry have expressed dissent. Michael Douse, the head of publishing at Larian Studios, the developer of Baldur’s Gate 3, stated this week that AI cannot resolve what he sees as the industry’s most significant issue: a need for “leadership and vision.” In a post on the social media platform X, Douse elaborated on his position. “What this industry needs is not more mathematically produced, psychologically trained gameplay loops,” he wrote. “Rather more expressions of worlds that folks are engaged with, or want to engage with.”
