Push­ing But­tons

Inside the minds of the con­trol freaks break­ing the rules of how we play games



There’s a uni­ver­sally pop­u­lar ap­peal to some­body play­ing a videogame with an im­prob­a­ble de­vice. It could be run­ning

Dark Souls with the Don­key Konga bon­gos, us­ing a MIDI key­board to grind XP in

World Of War­craft or pi­lot­ing an in­ter­ac­tive card­board box by press­ing hand-drawn but­tons in Space Box – it is all but guar­an­teed to spawn news ar­ti­cles, clicks and views ga­lore.

It’s a watch­able phe­nom­e­non, as Twitch stream­ers ev­ery­where will at­test, with cu­ri­ous view­ers wait­ing to see whether their cham­pion can re­ally win a round of team death­match us­ing only voice com­mands. It takes a cer­tain kind of per­son to think be­yond the stan­dard in­put de­vices, and an even more de­ter­mined one to engi­neer the feat. The ques­tion is – given the time, ex­pense and tech­ni­cal frus­tra­tions – why they feel com­pelled to do it at all.

Some have no choice but to adapt: born with a con­di­tion that af­fects his mus­cle growth, Mike ‘BrolyLegs’ Begum learned to play Street Fighter at a com­pet­i­tive level by us­ing his mouth and face to ma­nip­u­late a gamepad. Oth­ers feel com­pelled to push them­selves be­yond the lim­its of reg­u­lar con­trollers not out of ne­ces­sity, but out of a deep fas­ci­na­tion with how we in­ter­face with games.

For some, us­ing the ‘wrong’ con­troller sits some­where be­tween show­ing off and test­ing them­selves, be­com­ing equally ad­dicted to the game and the bizarre in­put method they’ve cho­sen to use. What starts as a chal­lenge or sim­ple cu­rios­ity can of­ten trans­form into some­thing deeper: a be­lief that a dif­fer­ent kind of con­troller can change the way we think and feel about what’s on a screen. Then there are the artists who’ve dis­cov­ered the value in for­go­ing the very no­tion of a dis­play, who’ve whit­tled down a game to the unique dy­namic created be­tween con­troller and player.

Our con­ver­sa­tions with some of the in­dus­try’s most in­ven­tive con­troller en­thu­si­asts turns the spot­light on in­di­vid­u­als who have made a niche in­ter­est their liveli­hood, who revel in sub­vert­ing ex­pec­ta­tions, and who know there’s an ir­re­sistible new as­pect of videogames to be ex­plored. Why do they do it? Well, why does any­one do it? Be­cause push­ing but­tons makes things hap­pen. Push­ing but­tons feels good.

Twitch streamer Dy­lan Beck, bet­ter known by his on­line han­dle Rudeism, de­lights in play­ing well­known games the wrong way. Dark

Souls III on a dance mat, Rocket League with a Gui­tar Hero con­troller, Over­watch us­ing flight sticks – you name it, he’s prob­a­bly tried it. His an­tics reg­u­larly at­tract thou­sands of view­ers: the prospect of see­ing some­body tap­dance Iudex Gundyr into sub­mis­sion is an in­trigu­ing, ir­re­sistible one, and Beck is happy to oblige his au­di­ence. His ap­petite for con­troller-based chal­lenges has con­tin­ued to grow, and when he learned of the Makey Makey, a USB-based ap­pa­ra­tus that can send key­board and mouse sig­nals to a com­puter via al­most any in­put de­vice imag­in­able, he re­alised it was time for some­thing new.

It started with ba­nanas – 15 of them, pur­chased at Beck’s lo­cal su­per­mar­ket. The idea was to cre­ate a the­mat­i­cally ap­pro­pri­ate con­troller to play as Win­ston,

Over­watch’s go­rilla sci­en­tist, for one of his streams. “I wired up each of them,” Beck says. “It was pretty sim­ple in terms of plan­ning and ex­e­cu­tion: each ba­nana sends ei­ther one but­ton press or one mouse move­ment. WASD takes four ba­nanas, each mouse di­rec­tion needs one ba­nana, and so on.” With an earth­ing cable clipped to his per­son, touch­ing each piece of fruit con­ducted an elec­tric charge that al­lowed him to con­trol the game, with ba­nanas pro­grammed to jump, shoot and ac­ti­vate Win­ston’s shield and Ul­ti­mate.

View­ers ate up the nov­elty, and it at­tracted at­ten­tion from sev­eral gam­ingnews sites. Beck was pleased with the re­sult­ing con­troller and the wide­spread re­ac­tion it prompted. The Over­watch ex­per­i­ments con­tin­ued: us­ing a baguette to work Wi­d­ow­maker’s sniper ri­fle; play­ing Sym­me­tra via a mi­crowave; and drinking from teacups to con­trol Ana. Creat­ing his own un­usual con­trollers be­came a new source of en­ter­tain­ment out­side of play­ing games, and he be­gan to doc­u­ment and ex­plain the process be­hind mak­ing each one. “I find tin­ker­ing away to be a re­ally re­lax­ing kind of fun,” Beck says. “I know what I need to do, there’s no rush to do it, and if I’m stream­ing my build, it’s an op­por­tu­nity to have a good con­ver­sa­tion with my chat.” Talk­ing to his view­ers while ac­tu­ally play­ing some­thing like Playerunknown’s Battlegrounds with a wired-up fry­ing pan, mean­while, is more of a strug­gle: “It can be a lit­tle hard to form sen­tences when you’re us­ing some weird con­trap­tion to play at the same time.”

His view­ers are not al­ways there for the com­men­tary, and his per­for­mance leaves some­thing to be de­sired when he’s play­ing with, say, two tennis balls and a bal­ance board. Ul­ti­mately, peo­ple flock to Beck’s streams to see the in­ad­vis­able made briefly, hi­lar­i­ously vi­able through noth­ing more than a few wires and some de­ter­mi­na­tion. “When­ever I do some­thing re­motely skil­ful, it’s al­ways a huge deal,” Beck says. “Chat gets re­ally into it if I get even a sin­gle kill.” There’s some­thing grimly fas­ci­nat­ing about see­ing some­one spend six hours glu­ing wires to a broom­stick to per­form an in-game task nor­mally ac­com­plished by hold­ing left click. It’s Beck ver­sus the im­prob­a­ble, and al­though his cre­ations of­ten fail, the sat­is­fac­tion when a con­troller works as imag­ined makes vic­tory all the sweeter. A pre­vi­ously world-class Gui­tar Hero player, it’s per­haps un­sur­pris­ing that Beck’s con­troller-craft­ing mo­ti­va­tions stem from a deeply in­grained com­pet­i­tive spirit. “At first, it was the chal­lenge of com­pet­ing against play­ers who are us­ing the ‘cor­rect’ con­trol schemes that pulled me in,” he says. “I wanted to prove that any game can be playable in any way you can think of, and I still be­lieve that – but I think my rea­sons for do­ing this have def­i­nitely grown a lot.

“I re­ally en­joy build­ing more and more com­plex con­trollers, be­cause it’s fun to use over-the-top con­trap­tions, but also be­cause it’s a great chal­lenge just to build these things. I never saw my­self as a hard­ware engi­neer of any va­ri­ety, but it’s re­ally ad­dic­tive work.”



Sim­i­larly, if you had told a young TJ Hughes that he would one day build a game to be played with an elec­tronic mu­sic con­troller, he might not have be­lieved you. Nour orig­i­nated from a se­ries of 3D mod­els he’d sculpted in Blender. “I had boba tea, and I loved the drink: the flavour, the aes­thet­ics of it,” he says. “So I mod­elled a lit­tle boba tea for fun, and as a sort of artis­tic chal­lenge. I wanted to see if I could get the shader look­ing right, the milk­i­ness of it.” Fol­low­ing an­other good meal, a bowl of ra­men was his next muse: “I up­loaded it to Twit­ter, and it blew up.”

Hughes’ ap­peal­ing pas­tel cre­ations were stim­u­lat­ing more than just in­ter­est. “I’d hear a lot of, ‘Oh man, I’m hun­gry now’,” he says. “And that’s what in­ter­ested me. I thought: ‘Okay, I can make peo­ple feel hun­gry, con­vey a flavour with colour and ge­om­e­try – I won­der how much fur­ther I could go with this idea.’” The re­sult was Nour, a game that en­cour­ages you to play with your food by pop­ping pop­corn, swirling boba pearls and twirling noo­dles. An early demo at a lo­cal art event proved suc­cess­ful, but Hughes felt there was still some­thing miss­ing. Then, a friend in­tro­duced him to one of Shawn Wasabi’s mu­sic videos, show­ing the elec­tronic artist us­ing a MIDI Fighter. “I was so en­tranced by the con­troller,” Hughes says. “It was ex­actly what I was look­ing for: bright, glow­ing but­tons that you can mash. It’s so sat­is­fy­ing. That was the per­fect experience to line up with the game it­self. I was like, ‘How can I use this de­vice, even though I’m not a mu­sic pro­ducer?’”

The MIDI Fighter, de­signed as a DJ con­troller, al­lows the MIDI-map­ping of au­dio sam­ples to its but­tons. When pressed, but­tons launch au­dio clips (and pro­grammed ‘light shows’ via LED rings) in re­al­time, mean­ing users can per­form stun­ning live sets. The but­tons that give the con­troller its name are made by the leg­endary Ja­panese com­pany Sanwa: the kind found on ar­cade fight­ing game cab­i­nets, hard-wear­ing, of­fer­ing ultra-low la­tency and, im­por­tantly for Hughes, pro­duc­ing a won­der­fully tac­tile click when pushed.

While Wasabi favours a cus­tom-made 64-but­ton ver­sion, Nour uses the 16-but­ton MIDI Fighter 3D, which also con­tains a mo­tion-track­ing sen­sor. Hughes has in­cor­po­rated this func­tion into the game. “Mak­ing the pop­corn scene was the first time that I re­alised how amaz­ing this con­troller was: tilt­ing it, and see­ing the plat­form re­spond,” he says. Watch­ing con­ven­tion-go­ers feel that same joy of dis­cov­ery is in­cred­i­bly re­ward­ing, he says. “Peo­ple ap­proach, press one but­ton, see some­thing hap­pen and then press an­other – and then they start go­ing faster and faster, mash­ing but­tons and laugh­ing. It’s amaz­ing to see the tran­si­tion from cu­rios­ity to mind­less fun.”

There is no timer, score or ob­jec­tive in Nour: rather, Hughes’ goal is to fos­ter ex­per­i­men­ta­tion. Play­ers of­ten in­vent their own tasks: plop­ping a pair of chop­sticks into the ra­men us­ing a mag­net func­tion on one but­ton, or hav­ing toast­ers pop out slices in per­fect syn­chronic­ity. Its con­cept may be sim­ple, but Nour’s al­lure is al­ready proven, hav­ing raised close to $30,000 on Kick­starter from a throng­ing crowd of ev­i­dently hun­gry back­ers.

When Nour re­leases in April, most buy­ers won’t be able to experience the game as fully in­tended: af­ter all, not ev­ery­one has a spare $200 to blow on a spe­cial­ist mu­sic con­troller for a sin­gle game that can just as eas­ily be played on a key­board. Hughes is pro­gram­ming Nour so that play­ers will be able to plug in all man­ner of MIDI con­trollers, and hopes for even more creative so­lu­tions. It’s a bit­ter­sweet trade-off: not ev­ery­body will be able to experience Nour as its cre­ator truly in­tended, but there’s no con­troller out there more suited to its premise than the MIDI Fighter 3D – and with that knowl­edge, Hughes must surely be sat­is­fied.


For ex­per­i­men­tal hard­ware game de­vel­oper Robin Baum­garten, the con­troller comes first. He is per­haps best known for Line

Wob­bler, a one-di­men­sional dun­geon crawler com­mu­ni­cated by a long strip of LEDs and played with a door-stop­per-spring joy­stick. For Baum­garten, it’s me­chan­ics that lead the way: not in a gameplay sense, but in a phys­i­cal sense. He builds a con­troller, then de­signs a game for it. “When I start from scratch, I take a sen­sor, or an in­ter­ac­tion, and I’ll try to build some­thing around it, and see if there’s an emer­gent gameplay that comes from this in­ter­ac­tion,” he says. “For Line

Wob­bler, that worked out su­per well: when we put to­gether the spring and the LED strip, the game was there.”

A sen­sor at­tached to the spring mea­sured the joy­stick’s move­ment back and forth, while a de­bug out­put to the LED strip had a lit­tle green light wob­ble in tan­dem with the spring. “I was like, ‘Oh shit, that looks kind of vi­o­lent’,” Baum­garten says. “So if it looks vi­o­lent, it might be an at­tack, and if it’s an at­tack we have en­e­mies.” The dun­geon-crawl­ing el­e­ment quickly took shape, Baum­garten work­ing within the lim­its of a sin­gle di­men­sion to add con­vey­ors that push the player along, dan­ger­ous lava that re­quires care­ful tim­ing to cross and even tricky slopes that can be mod­elled ac­cord­ing to how the LED strip is placed in a re­al­life en­vi­ron­ment.

It’s not ex­actly a sim­ple game to ex­plain, or vi­su­alise: even videos of Line Wob­bler aren’t par­tic­u­larly use­ful, its bright LEDs wash­ing out the pic­ture. It has to be played to be un­der­stood. “The first big show I took it to was the Alt.Ctrl ex­hi­bi­tion at GDC,” Baum­garten says. “Tim Schafer was there, and he was my child­hood hero – Mon­key Is­land was my thing grow­ing up. He re­ally liked it, and said ‘Can I buy one?’ The fan­boy in me was like, ‘What?! Tim Schafer wants to buy one, oh my god!’ I thought, maybe I re­ally have some­thing here, and I’m go­ing to try to push it as far as I can.”

Like most of Baum­garten’s cre­ations, Line Wob­bler is now ex­hib­ited in mu­se­ums, game fes­ti­vals and other pub­lic spa­ces (most re­cently, King’s Cross sta­tion in Lon­don) for peo­ple to play, with Baum­garten re­ceiv­ing an artist’s fee. It’s not a big in­come, but along­side a Pa­treon that al­lows fans to sup­port his gen­eral tin­ker­ings, he is now able to work on his ex­per­i­men­tal hard­ware, and games for it, full-time. “There’s cer­tainly a niche for this prod­uct to ex­ist, but it’s def­i­nitely not a mass-mar­ket thing,” Baum­garten says. “If I made a Kick­starter, I would run into prob­lems. Peo­ple who haven’t seen Line Wob­bler in per­son might say, ‘It’s an LED strip on a spring, how good can it be? How ex­pen­sive can it be?’ I’d have a lot of trou­ble selling it on­line, be­cause the price point right now is fairly high.”

It takes Baum­garten two days to make each copy of the game. The LED strip alone costs around $150, spe­cially cho­sen for its high fram­er­ate and low la­tency, which pushes the sale price up to around $1,000. “For mu­se­ums, on the other hand, that’s al­most small change,” he says. “This ex­clu­siv­ity is a prob­lem in that my game isn’t as ac­ces­si­ble.” Yet that’s pre­cisely what makes Baum­garten’s one-of-a-kind cre­ations so ap­peal­ing to mu­se­ums and fes­ti­vals look­ing to of­fer some­thing unique. “But it’s def­i­nitely not by de­sign. If I could make my game avail­able to ev­ery­one, I would to­tally do that.”

Per­haps the clos­est Baum­garten has come to de­sign­ing a mass-mar­ket prod­uct is his slider con­troller, the first cre­ation he pub­licly showed. The idea came from his time work­ing on mo­bile games. “I wanted to make a game where peo­ple used slid­ers on a touch­screen. Back then, I was just get­ting into Ar­duino, with con­trollers and faders. I thought ‘Maybe I can make this in the real world’.” The mo­torised slid­ers were fairly cheap, and all Baum­garten had to do was build a con­tainer for the mech­a­nism. The dif­fi­cult part, as al­ways, was find­ing the right kind of minigames to fit the con­troller. A re­versed Flappy Bird- style

game, where the bird flew au­to­mat­i­cally and play­ers used the slider to move the gates, worked well as a one-to-one, phys­i­cal experience. It even looked ex­cit­ing, thanks to the mo­torised slid­ers play­ing the game au­to­mat­i­cally when­ever they were left un­touched.

But the game was nei­ther deep nor fun enough for Baum­garten. “Start­ing with the me­chanic – build­ing a game from the bot­tom up, ba­si­cally – is a chal­lenge that doesn’t al­ways have a good re­sult,” he says. “A few of my con­trollers have had this is­sue, where I find a con­troller, but the ap­pli­ca­tion isn’t ob­vi­ous. And then the ques­tion is, ‘Do I put more time into this try­ing to find this game? Or do I just make the next thing?’”

It seems Baum­garten is in­creas­ingly de­ter­mined to have both game and con­troller ex­ist in such per­fect sym­bio­sis that the com­bi­na­tion be­comes some­thing else en­tirely. His lat­est piece, Wob­ble Gar­den, is one such cre­ation. “There’s an in­ter­est­ing, fuzzy bar­rier be­tween what is a con­troller and what is an al­ter­na­tive-hard­ware game,” Baum­garten says. “Line Wob­bler def­i­nitely fell on the game side, but Wob­ble Gar­den is a weird over­lap be­tween a toy, an in­ter­ac­tive art piece and a videogame.”

The hard­ware it­self, a grid of 36 sen­sor-en­abled springs and re­ac­tive LED light­ing hooked up to two tiny Ar­duino computers, was made over the course of three weeks at the Lon­don Hackspace. Baum­garten took the com­pleted con­troller to his next game jam – Splash Jam, which takes place aboard a cruise ship in Nor­way. Just as TJ Hughes’ pub­lic demos have heav­ily in­flu­enced Nour’s de­vel­op­ment (see­ing peo­ple in­tu­itively try­ing to tilt the con­troller spurred him on to in­cor­po­rate the MIDI Fighter 3D’s mo­tion-sen­si­tiv­ity as a game fea­ture), Baum­garten’s ob­ser­va­tions of oth­ers in­ter­act­ing with Wob­ble Gar­den in­formed the type of game he would make for it. “At Splash Jam, we did a lit­tle Whac-A-Mole pro­to­type,” Baum­garten says. “But I didn’t en­joy play­ing it. It felt very hec­tic; like you were forced to touch the springs, rather than do­ing it be­cause you wanted to see what would hap­pen. Peo­ple were drawn to this weird con­troller, so I thought, ‘Maybe it doesn’t mat­ter so much if the game is su­per tight.’” A slower pace was re­quired, and Rain­bow Frog was born, a game where the gar­den’s coloured lights melt into dif­fer­ent sea­sons and play­ers touch springs to in­ter­act with skit­ter­ing LED crea­tures.

It’s the sim­ple thrill of new chal­lenges and ex­pe­ri­ences that first in­spired Baum­garten to teach him­self cir­cuitry and sol­der­ing and be­gin his hard­ware game ex­per­i­ments. It’s why he con­tin­ues to ded­i­cate his time to them, too. “When you only work with computers on soft­ware games it gets very ab­stract and the­o­ret­i­cal. But I re­ally en­joy this com­bi­na­tion of not only star­ing at a screen and chang­ing lit­tle num­bers, but also mak­ing hands-on things. It’s a con­tin­u­ous dis­cov­ery process.” And the more he dis­cov­ers, the more bound­aries at which he finds him­self com­pelled to push. “I feel like cus­tom con­trollers can help make clear to peo­ple that games can be more than sit­ting with a stan­dard gamepad on a stan­dard videogame,” he says. “This open­ing up of in­ter­faces is what I’m go­ing for.”

Baum­garten has high hopes for the fu­ture of his hard­ware ex­per­i­ments: gi­ant in­stal­la­tions on the sides of tow­ers, or even in­ter­ac­tive fur­ni­ture. What­ever he makes, how­ever, the fo­cus is al­ways on de­vel­op­ing in­ter­ac­tions be­tween player and con­troller. His lat­est con­cept is a but­ton that, de­pend­ing on how hard it is pressed, ac­tively changes its re­sis­tance. “There’s this weird in­ter­net sub­cul­ture about keys and key­boards, like the Cherry MX and what­not. There are all these graphs about pres­sure and when you press it, whether there’s a click, or no click – or it’s damp­ened, or not damp­ened. There’s a whole other world there in this one ques­tion: what hap­pens if you push things, and they push back?”


When not creat­ing con­trollers, Dy­lan ‘Rudeism’ Beck makes mo­bile games at New Zealand stu­dio Run­away

Beck builds most of his con­trollers to play Over­watch, but says that his PUBG fry­ing pan (which al­lows for both shoot­ing and melee at­tacks) is his favourite

Hughes hopes to de­velop a plugin that lets the MIDI Fighter’s LEDs re­spond to chang­ing scenes: “Maybe a flavour of the boba tea would de­ter­mine their colour”

TJ Hughes, aka Ter­ri­fy­ing Jellyfish, created sur­vival sim Feesh and pro­vided tech­ni­cal art for Smug­gleCraft

Baum­garten’s slider con­troller al­lows for mul­ti­player minigames. The faders lend them­selves to a va­ri­ety of me­chan­ics, such as weigh­ing, aim­ing or bal­anc­ing

Robin Baum­garten trained in pro­gram­ming, but later re­vis­ited his child­hood love of wood­work and sol­der­ing

Baum­garten is keen to push the bound­aries of how we in­ter­face with games. “Pain hasn’t been ex­plored much as a con­cept,”he says. His blade-avoid­ance game, Knife ToMeetYou, ex­am­ines dan­ger, risk and com­pet­i­tive­ness

Su­per Light Com­bat, Baum­garten’s fight­ing game, ended up show­ing him the lim­i­ta­tions of 1D. The LED strip’s 100fps fram­er­ate made truly re­ac­tive play nearim­pos­si­ble, and so fos­ter­ing a metagame was dif­fi­cult

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.