TensorFlow brings self-driving to Mario Kart

While many automotive companies are striving to impress customers with their self-driving concepts, the open source software library for machine intelligence, TensorFlow, is bringing the advanced driving model even to video games like Mario Kart.

Open source developer, Kevin Hughes, has deployed Google’s TensorFlow to build a self-driving Mario Kart model, which he has called TensorKart. The developer has used the open source N64 emulator mupen64plus to play Mario Kart on his desktop computer, and has taken development support from TensorFlow and cuDNN. Also, he has written a Python program using open source libraries such as wxPython, Pygame and Matplotlib to continuously capture screenshots from the emulator, synchronised with a joystick input. This is the basis of the training set.

“The idea of exploring AI techniques in video games is not new, but what motivated me to do this project was to showcase the complete pipeline of a

machine learning system. I wanted to pick a popular game because I thought it might interest more people and expose them to how machine learning works,” revealed Hughes. The combination of TensorFlow, cuDNN and some open source libraries has enabled Hughes to develop TensorKart’s gameplay.

Hughes did face initial obstacles when he set out on his journey of providing a self-driving model in a video game. “I had to dust off my C programming skills and spend some time remembering all the nuances of C build systems and Makefiles,” Hughes told Open Source For You, adding that he also faced initial issues in debugging the originally trained model when it was not moving the car correctly. However, Hughes took support from Stack Overflow and persevered after taking a day off in the middle of the project.

“When I first hit the issue of how to send input back to the emulator, I did not think it would be difficult. I needed to take a day off and appreciate this part of the problem and then approach it with the energy required to fix it properly,” stated Hughes.

Hughes believes open source is the key to run innovations like TensorKart around the machine learning area. “I have been a believer of open source pretty much since. It became apparent early on how much more efficient open source was; it just felt right. I have benefited a lot from what others have shared and feel good to share my own work,” he said.

Going forward, Hughes plans to expand his TensorKart with new developments. He also expects it to attract more people from the community.

Meanwhile, you can visit GitHub to download the same input plugin that has been enabling self-driving on Mario Kart and build your own advanced gaming experience.