For the past few months, we here at Strife AI have been developing brand new ways to engage with video games: through machine learning. As a starting point, we decided to develop a game of our own. Our goal with this project is to figure out how to make machine learning engaging and interesting in a video game setting. The project, with the working title "Strife", has taken on various iterations and has experienced certain challenges. This article is all about where our project has been and the challenges we have encountered thus far.
Strife - The wave-based arena shooter game
The very first iteration started off being a wave-based shooter in which the player must train his fellow allies (green squares and blue squares) how to defend (or how to attack). Enemies (red squares) came from various corners of the map and make their way to the castle in the center of the map. They learned about their environment by casting 32 rays around them and gathering data such as what object hit the ray, and how far away said object is.
We found that once the allies had an optimal strategy, the game would no longer be an engaging experience. Leading to a relatively short game.
After further discussion, we decided to take the game down a different path.
Strife - The puzzler adventure game
Our game went down a different path over the course of the Holiday season. We decided to try and make our game a bit like "Pikmin" but instead training the allies to instead do certain actions in order to accomplish a task. We even built our first vertical slice out of this game concept. The goal was to bring a coin back to the castle. But a series of puzzles prevent the player from simply picking the coin up. The player could spawn in three different types of allies, each one trained to do a separate task.
Our thought was that machine learning can help the player do multiple things at once without the player needing to tend to them all the time. So managing and switching between the allies was essential.
This version of the game used a neural network module called LSTM (Long Short-Term Memory), which provided the allies with some basic memory capabilities. The memory of recent actions can allow the allies to more easily navigate the world; however, this comes at a cost of longer training time. LSTM networks that were deep enough to provide the benefits we wanted caused the training time to become too long for any player to enjoy.
After this, we decided to go a little simpler...
Strife - The Infinite Runner Game
We were getting a little closer, but our game designs weren't exactly meeting our expectations. So we went ahead and took a step back and decided to make our game as simple as possible. In a couple of days, we switched from a top-down perspective to a side scroller. The gameplay is simple: navigate the map by jumping, double-jumping, and sliding. The bot would collect coins along with you.
We found that machine learning in this instance works really well when performing discrete actions. That is, deciding to jump on a single frame. Machine Learning no longer became a problem: it became a matter of how to make a simple game fun with machine learning.
This project has existed in several different forms and has jumped between codebases, but still seeks one goal: develop fun games that utilize machine learning. Our latest iteration is still in the works at the moment. Though we are hoping to have something in the hands of players by mid 2020.