DeepMind and Blizzard to release StarCraft II as an AI research environment

An agent that can play StarCraft will need to demonstrate effective use of memory, an ability to plan over a long time, and the capacity to adapt plans based on new information. Computers are capable of extremely fast control, but that doesn’t necessarily demonstrate intelligence, so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”. StarCraft’s high-dimensional action space is quite different from those previously investigated in reinforcement learning research; to execute something as simple as “expand your base to some location”, one must coordinate mouse clicks, camera, and available resources.  This makes actions and planning hierarchical, which is a challenging aspect of Reinforcement Learning.

We’re particularly pleased that the environment we’ve worked with Blizzard to construct will be open and available to all researchers  next year. We recognise the efforts of the developers and researchers from the Brood War community in recent years, and hope that this new, modern and flexible environment – supported directly by the team at Blizzard – will be widely used to advance the state-of-the-art.

We’ve worked closely with the StarCraft II team to develop an API that supports something similar to previous bots written with a “scripted” interface, allowing programmatic control of individual units and access to the full game state (with some new options as well).  Ultimately agents will play directly from pixels, so to get us there, we’ve developed a new image-based interface that outputs a simplified low resolution RGB image data for map & minimap, and the option to break out features into separate “layers”, like terrain heightfield, unit type, unit health etc. Below is an example of what the feature layer API will look like.

StarCraft II DeepMind feature layer API

We are also working with Blizzard to create “curriculum” scenarios, which present increasingly complex tasks to allow researchers of any level to get an agent up and running, and benchmark different algorithms and advances. Researchers will also have full flexibility and control to create their own tasks using the existing StarCraft II editing tools.

We’re really excited to see where our collaboration with Blizzard will take us. While we’re still a long way from being able to challenge a professional human player at the game of StarCraft II, we hope that the work we have done with Blizzard will serve as a useful testing platform for the wider AI research community.