Pushing Buttons
Inside the minds of the control freaks breaking the rules of how we play games
BY JEN SIMPKINS
There’s a universally popular appeal to somebody playing a videogame with an improbable device. It could be running
Dark Souls with the Donkey Konga bongos, using a MIDI keyboard to grind XP in
World Of Warcraft or piloting an interactive cardboard box by pressing hand-drawn buttons in Space Box – it is all but guaranteed to spawn news articles, clicks and views galore.
It’s a watchable phenomenon, as Twitch streamers everywhere will attest, with curious viewers waiting to see whether their champion can really win a round of team deathmatch using only voice commands. It takes a certain kind of person to think beyond the standard input devices, and an even more determined one to engineer the feat. The question is – given the time, expense and technical frustrations – why they feel compelled to do it at all.
Some have no choice but to adapt: born with a condition that affects his muscle growth, Mike ‘BrolyLegs’ Begum learned to play Street Fighter at a competitive level by using his mouth and face to manipulate a gamepad. Others feel compelled to push themselves beyond the limits of regular controllers not out of necessity, but out of a deep fascination with how we interface with games.
For some, using the ‘wrong’ controller sits somewhere between showing off and testing themselves, becoming equally addicted to the game and the bizarre input method they’ve chosen to use. What starts as a challenge or simple curiosity can often transform into something deeper: a belief that a different kind of controller can change the way we think and feel about what’s on a screen. Then there are the artists who’ve discovered the value in forgoing the very notion of a display, who’ve whittled down a game to the unique dynamic created between controller and player.
Our conversations with some of the industry’s most inventive controller enthusiasts turns the spotlight on individuals who have made a niche interest their livelihood, who revel in subverting expectations, and who know there’s an irresistible new aspect of videogames to be explored. Why do they do it? Well, why does anyone do it? Because pushing buttons makes things happen. Pushing buttons feels good.
Twitch streamer Dylan Beck, better known by his online handle Rudeism, delights in playing wellknown games the wrong way. Dark
Souls III on a dance mat, Rocket League with a Guitar Hero controller, Overwatch using flight sticks – you name it, he’s probably tried it. His antics regularly attract thousands of viewers: the prospect of seeing somebody tapdance Iudex Gundyr into submission is an intriguing, irresistible one, and Beck is happy to oblige his audience. His appetite for controller-based challenges has continued to grow, and when he learned of the Makey Makey, a USB-based apparatus that can send keyboard and mouse signals to a computer via almost any input device imaginable, he realised it was time for something new.
It started with bananas – 15 of them, purchased at Beck’s local supermarket. The idea was to create a thematically appropriate controller to play as Winston,
Overwatch’s gorilla scientist, for one of his streams. “I wired up each of them,” Beck says. “It was pretty simple in terms of planning and execution: each banana sends either one button press or one mouse movement. WASD takes four bananas, each mouse direction needs one banana, and so on.” With an earthing cable clipped to his person, touching each piece of fruit conducted an electric charge that allowed him to control the game, with bananas programmed to jump, shoot and activate Winston’s shield and Ultimate.
Viewers ate up the novelty, and it attracted attention from several gamingnews sites. Beck was pleased with the resulting controller and the widespread reaction it prompted. The Overwatch experiments continued: using a baguette to work Widowmaker’s sniper rifle; playing Symmetra via a microwave; and drinking from teacups to control Ana. Creating his own unusual controllers became a new source of entertainment outside of playing games, and he began to document and explain the process behind making each one. “I find tinkering away to be a really relaxing kind of fun,” Beck says. “I know what I need to do, there’s no rush to do it, and if I’m streaming my build, it’s an opportunity to have a good conversation with my chat.” Talking to his viewers while actually playing something like Playerunknown’s Battlegrounds with a wired-up frying pan, meanwhile, is more of a struggle: “It can be a little hard to form sentences when you’re using some weird contraption to play at the same time.”
His viewers are not always there for the commentary, and his performance leaves something to be desired when he’s playing with, say, two tennis balls and a balance board. Ultimately, people flock to Beck’s streams to see the inadvisable made briefly, hilariously viable through nothing more than a few wires and some determination. “Whenever I do something remotely skilful, it’s always a huge deal,” Beck says. “Chat gets really into it if I get even a single kill.” There’s something grimly fascinating about seeing someone spend six hours gluing wires to a broomstick to perform an in-game task normally accomplished by holding left click. It’s Beck versus the improbable, and although his creations often fail, the satisfaction when a controller works as imagined makes victory all the sweeter. A previously world-class Guitar Hero player, it’s perhaps unsurprising that Beck’s controller-crafting motivations stem from a deeply ingrained competitive spirit. “At first, it was the challenge of competing against players who are using the ‘correct’ control schemes that pulled me in,” he says. “I wanted to prove that any game can be playable in any way you can think of, and I still believe that – but I think my reasons for doing this have definitely grown a lot.
“I really enjoy building more and more complex controllers, because it’s fun to use over-the-top contraptions, but also because it’s a great challenge just to build these things. I never saw myself as a hardware engineer of any variety, but it’s really addictive work.”
CREATING HIS OWN UNUSUAL CONTROLLERS BECAME A NEW SOURCE OF ENTERTAINMENT
“I T WAS EXACTLY WHAT I WAS LOOKING FOR: BRIGHT, GLOWING BUTTONS THAT YOU CAN MASH”
Similarly, if you had told a young TJ Hughes that he would one day build a game to be played with an electronic music controller, he might not have believed you. Nour originated from a series of 3D models he’d sculpted in Blender. “I had boba tea, and I loved the drink: the flavour, the aesthetics of it,” he says. “So I modelled a little boba tea for fun, and as a sort of artistic challenge. I wanted to see if I could get the shader looking right, the milkiness of it.” Following another good meal, a bowl of ramen was his next muse: “I uploaded it to Twitter, and it blew up.”
Hughes’ appealing pastel creations were stimulating more than just interest. “I’d hear a lot of, ‘Oh man, I’m hungry now’,” he says. “And that’s what interested me. I thought: ‘Okay, I can make people feel hungry, convey a flavour with colour and geometry – I wonder how much further I could go with this idea.’” The result was Nour, a game that encourages you to play with your food by popping popcorn, swirling boba pearls and twirling noodles. An early demo at a local art event proved successful, but Hughes felt there was still something missing. Then, a friend introduced him to one of Shawn Wasabi’s music videos, showing the electronic artist using a MIDI Fighter. “I was so entranced by the controller,” Hughes says. “It was exactly what I was looking for: bright, glowing buttons that you can mash. It’s so satisfying. That was the perfect experience to line up with the game itself. I was like, ‘How can I use this device, even though I’m not a music producer?’”
The MIDI Fighter, designed as a DJ controller, allows the MIDI-mapping of audio samples to its buttons. When pressed, buttons launch audio clips (and programmed ‘light shows’ via LED rings) in realtime, meaning users can perform stunning live sets. The buttons that give the controller its name are made by the legendary Japanese company Sanwa: the kind found on arcade fighting game cabinets, hard-wearing, offering ultra-low latency and, importantly for Hughes, producing a wonderfully tactile click when pushed.
While Wasabi favours a custom-made 64-button version, Nour uses the 16-button MIDI Fighter 3D, which also contains a motion-tracking sensor. Hughes has incorporated this function into the game. “Making the popcorn scene was the first time that I realised how amazing this controller was: tilting it, and seeing the platform respond,” he says. Watching convention-goers feel that same joy of discovery is incredibly rewarding, he says. “People approach, press one button, see something happen and then press another – and then they start going faster and faster, mashing buttons and laughing. It’s amazing to see the transition from curiosity to mindless fun.”
There is no timer, score or objective in Nour: rather, Hughes’ goal is to foster experimentation. Players often invent their own tasks: plopping a pair of chopsticks into the ramen using a magnet function on one button, or having toasters pop out slices in perfect synchronicity. Its concept may be simple, but Nour’s allure is already proven, having raised close to $30,000 on Kickstarter from a thronging crowd of evidently hungry backers.
When Nour releases in April, most buyers won’t be able to experience the game as fully intended: after all, not everyone has a spare $200 to blow on a specialist music controller for a single game that can just as easily be played on a keyboard. Hughes is programming Nour so that players will be able to plug in all manner of MIDI controllers, and hopes for even more creative solutions. It’s a bittersweet trade-off: not everybody will be able to experience Nour as its creator truly intended, but there’s no controller out there more suited to its premise than the MIDI Fighter 3D – and with that knowledge, Hughes must surely be satisfied.
“IF I COULD MAKE MY GA ME AVAILABLE TO EVERYONE, I WOULD TOTALLY DO THAT”
For experimental hardware game developer Robin Baumgarten, the controller comes first. He is perhaps best known for Line
Wobbler, a one-dimensional dungeon crawler communicated by a long strip of LEDs and played with a door-stopper-spring joystick. For Baumgarten, it’s mechanics that lead the way: not in a gameplay sense, but in a physical sense. He builds a controller, then designs a game for it. “When I start from scratch, I take a sensor, or an interaction, and I’ll try to build something around it, and see if there’s an emergent gameplay that comes from this interaction,” he says. “For Line
Wobbler, that worked out super well: when we put together the spring and the LED strip, the game was there.”
A sensor attached to the spring measured the joystick’s movement back and forth, while a debug output to the LED strip had a little green light wobble in tandem with the spring. “I was like, ‘Oh shit, that looks kind of violent’,” Baumgarten says. “So if it looks violent, it might be an attack, and if it’s an attack we have enemies.” The dungeon-crawling element quickly took shape, Baumgarten working within the limits of a single dimension to add conveyors that push the player along, dangerous lava that requires careful timing to cross and even tricky slopes that can be modelled according to how the LED strip is placed in a reallife environment.
It’s not exactly a simple game to explain, or visualise: even videos of Line Wobbler aren’t particularly useful, its bright LEDs washing out the picture. It has to be played to be understood. “The first big show I took it to was the Alt.Ctrl exhibition at GDC,” Baumgarten says. “Tim Schafer was there, and he was my childhood hero – Monkey Island was my thing growing up. He really liked it, and said ‘Can I buy one?’ The fanboy in me was like, ‘What?! Tim Schafer wants to buy one, oh my god!’ I thought, maybe I really have something here, and I’m going to try to push it as far as I can.”
Like most of Baumgarten’s creations, Line Wobbler is now exhibited in museums, game festivals and other public spaces (most recently, King’s Cross station in London) for people to play, with Baumgarten receiving an artist’s fee. It’s not a big income, but alongside a Patreon that allows fans to support his general tinkerings, he is now able to work on his experimental hardware, and games for it, full-time. “There’s certainly a niche for this product to exist, but it’s definitely not a mass-market thing,” Baumgarten says. “If I made a Kickstarter, I would run into problems. People who haven’t seen Line Wobbler in person might say, ‘It’s an LED strip on a spring, how good can it be? How expensive can it be?’ I’d have a lot of trouble selling it online, because the price point right now is fairly high.”
It takes Baumgarten two days to make each copy of the game. The LED strip alone costs around $150, specially chosen for its high framerate and low latency, which pushes the sale price up to around $1,000. “For museums, on the other hand, that’s almost small change,” he says. “This exclusivity is a problem in that my game isn’t as accessible.” Yet that’s precisely what makes Baumgarten’s one-of-a-kind creations so appealing to museums and festivals looking to offer something unique. “But it’s definitely not by design. If I could make my game available to everyone, I would totally do that.”
Perhaps the closest Baumgarten has come to designing a mass-market product is his slider controller, the first creation he publicly showed. The idea came from his time working on mobile games. “I wanted to make a game where people used sliders on a touchscreen. Back then, I was just getting into Arduino, with controllers and faders. I thought ‘Maybe I can make this in the real world’.” The motorised sliders were fairly cheap, and all Baumgarten had to do was build a container for the mechanism. The difficult part, as always, was finding the right kind of minigames to fit the controller. A reversed Flappy Bird- style
game, where the bird flew automatically and players used the slider to move the gates, worked well as a one-to-one, physical experience. It even looked exciting, thanks to the motorised sliders playing the game automatically whenever they were left untouched.
But the game was neither deep nor fun enough for Baumgarten. “Starting with the mechanic – building a game from the bottom up, basically – is a challenge that doesn’t always have a good result,” he says. “A few of my controllers have had this issue, where I find a controller, but the application isn’t obvious. And then the question is, ‘Do I put more time into this trying to find this game? Or do I just make the next thing?’”
It seems Baumgarten is increasingly determined to have both game and controller exist in such perfect symbiosis that the combination becomes something else entirely. His latest piece, Wobble Garden, is one such creation. “There’s an interesting, fuzzy barrier between what is a controller and what is an alternative-hardware game,” Baumgarten says. “Line Wobbler definitely fell on the game side, but Wobble Garden is a weird overlap between a toy, an interactive art piece and a videogame.”
The hardware itself, a grid of 36 sensor-enabled springs and reactive LED lighting hooked up to two tiny Arduino computers, was made over the course of three weeks at the London Hackspace. Baumgarten took the completed controller to his next game jam – Splash Jam, which takes place aboard a cruise ship in Norway. Just as TJ Hughes’ public demos have heavily influenced Nour’s development (seeing people intuitively trying to tilt the controller spurred him on to incorporate the MIDI Fighter 3D’s motion-sensitivity as a game feature), Baumgarten’s observations of others interacting with Wobble Garden informed the type of game he would make for it. “At Splash Jam, we did a little Whac-A-Mole prototype,” Baumgarten says. “But I didn’t enjoy playing it. It felt very hectic; like you were forced to touch the springs, rather than doing it because you wanted to see what would happen. People were drawn to this weird controller, so I thought, ‘Maybe it doesn’t matter so much if the game is super tight.’” A slower pace was required, and Rainbow Frog was born, a game where the garden’s coloured lights melt into different seasons and players touch springs to interact with skittering LED creatures.
It’s the simple thrill of new challenges and experiences that first inspired Baumgarten to teach himself circuitry and soldering and begin his hardware game experiments. It’s why he continues to dedicate his time to them, too. “When you only work with computers on software games it gets very abstract and theoretical. But I really enjoy this combination of not only staring at a screen and changing little numbers, but also making hands-on things. It’s a continuous discovery process.” And the more he discovers, the more boundaries at which he finds himself compelled to push. “I feel like custom controllers can help make clear to people that games can be more than sitting with a standard gamepad on a standard videogame,” he says. “This opening up of interfaces is what I’m going for.”
Baumgarten has high hopes for the future of his hardware experiments: giant installations on the sides of towers, or even interactive furniture. Whatever he makes, however, the focus is always on developing interactions between player and controller. His latest concept is a button that, depending on how hard it is pressed, actively changes its resistance. “There’s this weird internet subculture about keys and keyboards, like the Cherry MX and whatnot. There are all these graphs about pressure and when you press it, whether there’s a click, or no click – or it’s dampened, or not dampened. There’s a whole other world there in this one question: what happens if you push things, and they push back?”
“STARTING WITH THE MECHANIC IS A CHALLENGE THAT DOESN’T ALWAYS HAVE A GOOD RESULT”