124 MCP Tools, Virtual XInput, and Background Driving: How a Developer Got Claude AI to Play GTA 4 Ahead of GTA 6
Just ten days ahead of the global launch of Grand Theft Auto VI on November 19, 2026, anticipation around Rockstar Games' flagship series is fueling some remarkable technical experiments. On Reddit's r/ClaudeAI community, developer Wonderful-Ad-8579 unveiled and open-sourced LLM Game Automation MCP on GitLab. The project connects Anthropic's Claude Code to a custom Model Context Protocol (MCP) server capable of launching and playing Grand Theft Auto IV in the background on Windows 10 and Windows 11 without ever stealing window focus or locking up the user's physical mouse and keyboard.
In-Process Swapchain Capture and Virtual XInput Per Game
Most existing setups designed to let artificial intelligence play PC games rely on the Windows SendInput API paired with full-desktop screenshots. That approach requires keeping the game window in the foreground, hijacks the user's physical cursor, and restricts testing to a single game at a time. To eliminate that bottleneck, the project's low-level C++ layer, named vinput, operates strictly per process.
When attached to a target executable, the command-line tool (vinput.exe) injects a custom dynamic link library (vinput64.dll or vinput32.dll, powered by MinHook) directly into the game process and communicates over a dedicated named pipe. Instead of capturing the Windows desktop, vinput grabs frames straight from the game's own rendering swapchain via a Present hook across Direct3D 9, Direct3D 11, Direct3D 12, OpenGL, Vulkan, and DirectDraw, automatically converting 10-bit and 16-bit HDR buffers into 8-bit images. Even when GTA IV is completely covered by a web browser or code editor, the AI agent receives clean frames directly from the GPU.
On the input side, vinput spoofs active focus inside the game, swallows Windows deactivation messages, and delivers commands across every channel modern and legacy engines poll: window messages (WM_KEYDOWN and WM_CHAR), Raw Input, DirectInput 8 (using a virtual cursor guard that bypasses GTA IV's notoriously aggressive mouse capture trap), and a virtual XInput pad 0 controller (XInputGetState). While Anthropic recently added a server-side rule blocking Claude Code from cloning commercial video games, this MCP bridge turns the coding assistant into an autonomous playtester capable of driving multiple games in parallel. Built strictly for offline single-player titles, the injector also includes a safety check that automatically refuses to load whenever it detects anti-cheat modules such as EasyAntiCheat or BattlEye.
From the Liberty City Pier to Helicopter Flight: Closed-Loop Self-Debugging
In the unedited, real-time footage shared by the creator, Claude carefully drives Niko Bellic through busy Liberty City intersections toward a map waypoint, flies a helicopter, and walks to an objective marker. Running on top of the C++ injector, a TypeScript MCP server (which reached 124 tools in version v0.5.3 on November 9, including nav_go, nav_perceive, click_text via native Windows.Media.Ocr, and aim_shoot) translates screen state into structured actions.
Rather than hardcoding scripts for each game, the framework uses declarative JSON navigation profiles (nav-profile@1) that define HUD geometry, radar crop coordinates, route colors, and arrival conditions. The repository ships with 21 built-in profiles: alongside GTA IV Complete Edition and GTA V Enhanced, it supports GTA: The Trilogy - The Definitive Edition, Red Dead Redemption (on horseback and on foot), Sleeping Dogs, Watch_Dogs, Far Cry 3, Far Cry 5, Dishonored, Hogwarts Legacy, The Elder Scrolls III: Morrowind, Kingdom Come: Deliverance, Dragon Age: Inquisition, and Halo Wars 2, with additional field tests recorded in Detroit: Become Human, Cities: Skylines, and Totally Accurate Battle Simulator.
Even more fascinating is how the toolset was built: according to Wonderful-Ad-8579, Claude Code planned the architecture, wrote the vast majority of the codebase (with occasional help from Codex), and debugged the navigation loops by playing the games itself. Each sub-agent is assigned a single game as a test bed and a specific failure case, such as a car stalling at a junction. The agent plays through the MCP server, diagnoses what its perception tools misread, patches the TypeScript or C++ code, and measures before-and-after arrival rates on the exact same route. During GTA IV testing, for instance, the agent discovered that holding the brake key reverses a stationary car, updating its driving logic to apply brakes only while rolling forward and achieving 3 out of 3 confirmed arrivals from the starting pier to Roman's apartment.
Blind Indoors Without a Mini-Map and Controller Lock-On Fixes
Despite the impressive engineering under the hood, the developer is candid about where the model still struggles. Because the navigation loop relies heavily on reading the GPS route and radar blips, Claude loses its sense of direction indoors or in any sequence without a mini-map. On the streets of Liberty City, the AI also drives deliberately slowly, taking cautious turns to avoid overshooting its radar heading calibration.
Combat has proven equally challenging. Using pure computer-vision mouse aiming, Claude struggled to track moving targets in GTA IV, scoring zero kills in initial interleaved trials. To solve that bottleneck in the v0.5.3 release published on November 9, the combat_compass tool introduced a pad_assist mode: the agent holds the left trigger on its virtual XInput controller to engage Rockstar's native target lock-on and fires only when it visually confirms the target lock ring on screen, jumping from 0 to 16 kills across test runs. Coming just days after former Rockstar technical director Obbe Vermeij discussed how AI tools could accelerate future PC ports after seeing GTA IV decompiled, the complete LLM Game Automation MCP source code and prebuilt Windows x64 binaries are available now on GitLab.
My Take on the Spec Sheet
What stands out to me in the vinput architecture isn't just watching Niko Bellic cautiously obey Liberty City's road layout, but the closed-loop self-evaluation built around the Model Context Protocol. By isolating Direct3D frame capture and virtual XInput inside a per-process pipe, this developer solved the biggest practical headache in PC game AI research: running hours of automated playtesting in the background without hijacking the host machine. Watching Claude Code diagnose why its own braking logic triggered reverse gear in GTA IV, rewrite the controller code, and verify the fix against a 325-file test suite shows just how fast autonomous QA tooling is maturing.
Grand Theft Auto 6 (GTA 6)
Explore official trailers, technical specs and all latest coverage for Grand Theft Auto 6 (GTA 6).
PC hardware enthusiast and competitive gamer, Arthur 'Heimdall' splits his time between France and the UK working in enterprise B2B software solutions. A business school graduate passionate about component benchmarking, he has been building, modding, and tuning PCs since childhood, optimizing every thermal degree and millisecond of display latency. An avid gamer across Assassin's Creed and sandbox titles, he forged his alliance with Grimtag and Chaya during late-night World of Warcraft raids before joining Pro-Gamer.
