OpenAI has detailed its efforts to teach a neural network how to play Minecraft.
The organization says(Opens in a new window) he used a “massive untagged human game Minecraft video dataset”, along with a “small amount of labeled contractor data”, with a technique called Video PreTraining (VPT) as he trained the neural network to playing the popular block-based title.
But the process wasn’t as simple as having a computer watch a bunch of Minecraft videos on YouTube. OpenAI says it first trained an Inverse Dynamics Model (IDM) with 2,000 hours of footage that showed which keys a player pressed when a certain action was performed.
This IDM was then used to help train the basic VPT model, as shown here:
“Trained on 70,000 hours of IDM-labeled online video,” says OpenAI, “our behavioral cloning model (the “VPT Core Model”) accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning at It learns to cut down trees to collect logs, turn those logs into planks, and then turn those planks into a crafting table; this sequence takes about 50 seconds or 1,000 consecutive gameplay actions for a human mastering Minecraft .”
Recommended by our editors
OpenAI didn’t stop there. His model also learned to perform other actions, including “swimming, chasing animals for food, and eating that food”, as well as a technique called “pillar jumping” which involves “repeatedly jumping and placing a block under yourself”.
The organization says it’s “open access to our contractor data, Minecraft environment, model code and model weights” so others can explore the possibilities. by VTP. He also published an article(Opens in a new window) with additional information on the results of this experiment.
Receive our best stories!
Register for What’s up now to get our top stories delivered to your inbox every morning.