Google's SIMA can play video games like a human, follow native language orders
How a lizard-like robot could help the Navy 'prevent catastrophes': decorated veteran
Doug Philippone, a venture capitalist, touted three devices companies in his portfolio have developed. He says they could provide significant benefits to the military.
Google has developed an artificial intelligence system that can play video games like a human and take orders from players and could eventually even have real-world implications down the line.
"This work isn't about achieving high game scores," the SIMA research team wrote in a Google DeepMind post earlier this month. "Learning to play even one video game is a technical feat for an AI system, but learning to follow instructions in a variety of game settings could unlock more helpful AI agents for any environment."
SIMA, which stands for Scalable Instructable Multiworld Agent, isn’t like a typical computer player that’s built into a specific game. Rather, the AI agent plays alongside and learns like a human — through image recognition and from native language commands — and plays with keyboard and mouse outputs.
WHAT IS ARTIFICIAL INTELLIGENCE (AI)?
"SIMA needs only the images provided by the 3D environment and natural-language instructions given by the user," according to the DeepMind post.
For now, the AI agent is only a research project. It’s meant to serve as a companion for human players that can carry out tasks.
"SIMA isn’t trained to win a game; it’s trained to run it and do what it’s told," the Google researchers wrote.
Google worked with eight game developers, including Hello Games and Embracer, to help train and test SIMA. As part of their goal to have the program task oriented, the researchers primarily trained SIMA on open-play environments, like No Man’s Sky and Goat Simulator 3, a bizarre game in which players control a goat that causes chaos.
NOW THAT ARTIFICIAL INTELLIGENCE CAN CLONE VOICES, VIDEO GAME ACTORS MUST WEIGH THEIR OPTIONS
"This research marks the first time an agent has demonstrated it can understand a broad range of gaming worlds, and follow natural-language instructions to carry out tasks within them, as a human might," the Google team wrote.
So far, SIMA has learned about 600 basic skills, like turning left and climbing a ladder. The Google researchers eventually want the AI agent to be able to adapt to games it’s never played and without specific training.
Nvidia is developing real-world robots equipped with artificial intelligence capabilities. (Justin Sullivan/Getty Images)
But, ultimately, Google hopes the SIMA research will build toward "more general AI systems and agents that can understand and safely carry out a wide range of tasks in a way that is helpful to people online and in the real world," the researchers wrote in the blog post.
"Our research shows how we can translate the capabilities of advanced AI models into useful, real-world actions through a language interface," the researchers added.
But it’s still a long way off before SIMA is ready for even basic video game play, having, for example, only succeeded in one-third of its tasks in No Man’s Sky.
"We hope that SIMA and other agent research can use video games as sandboxes to better understand how AI systems may become more helpful," the SIMA researchers wrote.
Ethan Barton is a producer/reporter for Digital Originals. You can reach him at
