A well-known developer gave a neural network a challenging task in a popular game. Yuyan Zhou assigned the GPT Astra model to mine a valuable diamond. The programmer started a screen recording and calmly went to sleep for the night. By morning, the precious stone was already sitting in the game character's inventory. The experiment's authors released a special toolkit into full open access. The key detail of the experiment lies in the method of computer control. The agent operates entirely without special mods or a programming interface. The model simply looks at the screen and moves a regular mouse. The artificial intelligence independently presses the necessary keys on the keyboard. The toolkit for the Windows operating system can do only three things. It passes regular screenshots to the agent and sends input events. The toolkit optionally allows visual verification of the result obtained. The model itself and the popular game's client are not included in the package. World saving and an internal memory interface are also completely absent.
Setting up the project takes just a few simple steps for the user. The folder needs to be opened as a local project on a Windows computer. The user must enter the world and leave the game window visible. Then the repository along with the in-game goal must be passed to the agent. Yuyan Zhou separately noted a complete absence of prior knowledge about the game. The agent built all necessary skills on the fly after receiving the task. The source code was released specifically to simplify future complex runs. The developer gave no specific instructions such as digging long mine shafts. The model independently chose a strategy of digging long parallel tunnels. In another run, the agent found a far more interesting solution to the task. After two fatal falls, the model placed an ordinary wooden boat. This trick helped completely negate the damage from a severe fall. The character successfully reached the very bottom of a deep cave alive.
The comments gathered no less skepticism than genuine admiration. The run took place in a special peaceful mode without mob spawning. The task essentially boiled down to a straightforward search for the needed resource. At the three-minute mark of the recording, the agent walked past a diamond ore vein. The model simply failed to notice the valuable resource during the search. A human can miss it too, but the probability is significantly lower. The model repeatedly turned the debug screen on and off without apparent reason. The agent broke a regular crafting table with a pickaxe instead of the correct axe. Torches were placed in spots where lighting was not needed at all. The developer did not publicly disclose the exact token consumption at all. Zhou asked an OpenAI employee to reset his usage limit. This step was needed to enable longer gaming sessions. The team plans to distill the actions obtained into a local Qwen model. The new model has twenty-seven billion useful parameters.
Zhou built the toolkit together with his colleague Tianyu Wei. The repository is distributed under the Apache 2.0 license. The diamond scenario itself has long served as a standard quality benchmark. The Minecraft Diamond environment is built around twelve sequential stages. The path to mining a diamond requires progressing through a technology tree. The player needs to master more than three hundred and seventy-nine crafting recipes. Computer control became the primary focus of the new Astra model. This neural network officially launched on September 3 of the current calendar year. On OSWorld benchmark simulations, the model demonstrates excellent results. The new score reaches seventy-two percent compared to sixty-five. The previous version, Sol, spent seventy-five minutes on the task. The new model handles an equivalent task in forty minutes.