See­ing is be­liev­ing

Idealog - - CONTENTS -

Elly Strang talks with Real­i­tyVir­tual's Si­mon Che de Boer about cre­at­ing slice-of-life, hy­per-re­al­is­tic VR ex­pe­ri­ences

How did one New Zealan­der go from see­ing his house and all his prized pos­ses­sions go up i n fl ames, to be­com­ing one of the world’s most soughtafter vir­tual reality di­rec­tors and cin­e­matog­ra­phers?

Elly Strang sits down with real­i­tyvir­tual. co founder Si­mon Che de Boer to get to know the man who says he’s i n the busi­ness of ‘slice- ofl i fe’ vir­tual ex­pe­ri­ences that are hard to dis­tin­guish from the real thing.

With a shock of white hair tou­sled around his face like a lion’s mane, black cloth­ing from head to toe and a pair of rare Dil­lon Op­tics dif­fused avi­a­tors perched on his head, Che de Boer is a strik­ing in­di­vid­ual to first lay eyes on. Guess­ing what ca­reer he’s in is an in­ter­est­ing chal­lenge for strangers to grap­ple with: when he’s trav­el­ing through Arab coun­tries, they flock to him and ask for his au­to­graph, sus­pect­ing he’s a Tar­garyen from Game of Thrones. When ob­serv­ing the large, heavy black cases he’s lug­ging around be­hind him, you might think he’s a rocker in a band. But what’s in­side the case is ac­tu­ally tens of thou­sands of dol­lars’ worth of heav­ily mod­i­fied photography and vir­tual reality equip­ment.

Che de Boer’s hy­per-re­al­is­tic VR cin­e­matog­ra­phy has ac­crued rave re­views from his in­dus­try coun­ter­parts around the world, from be­ing called “some of the most ex­cit­ing work I’ve ever seen” by Duned­in­based An­i­ma­tion Re­search’s Ian Tay­lor to Santa Clara based NVIDIA Rick Cham­pagne stat­ing “The re­al­ism and de­tail in Si­mon's work helps us see the beauty in the world we live in again”.

But what might as­ton­ish view­ers of his cre­ations is Che de Boer is ac­tu­ally con­sid­ered legally blind un­der New Zealand law.

He took an Uber to and from our in­ter­view as he can’t op­er­ate a car and when the avi­a­tors come off, his eyes are no­tice­ably crin­kled as they swivel around, strain­ing to take in the el­e­ments of the world around him. If you ob­serve him edit­ing his work on a com­puter, his nose is al­most brush­ing against the screen as he takes in ev­ery in­fin­i­tes­i­mal de­tail.

But it’s this so-called dis­abil­ity that, if viewed in a dif­fer­ent light, has led to a su­per-abil­ity.

Che de Boer says his vi­sion has made him de­velop an ex­tremely good en­vi­ron­men­tal spa­tial aware­ness and mem­ory, as he maps out places in his mind care­fully be­fore be­com­ing con­fi­dent in them, and it’s this skill that helps him pro­duce such de­tailed work.

“So un­less some­one de­cides to put 5000-yearold gold statue some­where in the mid­dle of the ground with­out telling me, I'm fine,” he says.

This spe­cific sce­nario didn’t ac­tu­ally hap­pen, but a re­cent trip to Egypt to shoot Ne­fer­tari: Jour­ney to Eter­nity – an ed­u­ca­tional VR ex­pe­ri­ence that lets view­ers ex­plore the an­cient Egyp­tian queen’s tomb – did get Che de Boer’s heart rac­ing when he found out the team who hired him didn’t take out li­a­bil­ity in­sur­ance.

“You're work­ing with these price­less ob­jects and you know, you're freak­ing out,” he laughs.

He says the other ad­van­tage of this con­di­tion is he doesn’t get lost in the de­tails of the re­al­life en­vi­ron­ment he’s doc­u­ment­ing, as he sim­ply doesn’t have that lux­ury.

“Where oth­ers are bom­barded with so much de­tail, they might have a nar­row vi­sion of what they're ac­tu­ally do­ing. The rea­son the de­tail is so good in our ex­pe­ri­ences is be­cause in the real world when we're scan­ning them, I can never quite get close enough to truly ap­pre­ci­ate it. ” he says.

“If I was to lean into the Mona Lisa the way I'd want to, I’d be right up against that mas­ter­piece, mois­ture breath­ing all over the damned thing and you know, I’d be ar­rested. With vir­tual reality, the level of in­ti­macy is com­pelling as these lim­i­ta­tions are not in play any­more.”

Ris­ing from the ashes

Unique vi­sion aside, Che de Boer’s deep dive into the vir­tual reality world was re­ally spurred from pure tragedy. Prior to Real­i­tyVir­tual, Che de Boer says he was a 31-year-old in a happy place, liv­ing in an apart­ment in the heart of Auck­land city for eight years for which he had ex­ten­sively ren­o­vated. He had his then four-year-old daugh­ter and his part­ner Chantelle with him. He played drums, sung in a band and had a few “geeky” hob­bies on the side that in­volved play­ing around with com­put­ers.

On the morn­ing af­ter Si­mon’s 31st birth­day, an Airbnb guest staying in the apart­ment spot­ted smoke and flames start­ing to take hold of the place. “Is this normal?” he asked Che de Boer, who replied no, and swiftly went into fight-or-flight mode, climb­ing half-naked up a lad­der and flee­ing to a spot on the roof that was safe from the flames. He and fel­low ten­ants were later res­cued by fire­fight­ers, but his home – and all of his pos­ses­sions – were not so lucky.

The fire was classed as an ar­son, and the tim­ing of events was sus­pi­cious con­sid­er­ing a five-year-long cus­tody bat­tle over Che de Boer’s daugh­ter was about to come to a close. Four days later, the court threw out the case due to him not hav­ing a home and he lost cus­tody.

It was af­ter this that the great per­sonal losses kept com­ing. Che de Boer and his part­ner Chantelle’s re­la­tion­ship de­te­ri­o­rated due to the mul­ti­tude of stresses, and later that year she took her own life.

“I went through hell. Lit­er­ally ev­ery­thing in my life went from on top of the world to ut­ter shit. You know, I lost my daugh­ter, lost my part­ner, lost my house in a pe­riod, in just a short pe­riod of time,” he says.

“It al­most broke me. Well, it prob­a­bly did break me and made me like some sort of weird mad genius, as I had to ab­sorb my­self in some­thing. It was a cat­a­lyst to force me to just re­ally put all my men­tal ef­fort into some­thing.”

That some­thing, it turns out, was vir­tual reality and visual ef­fects. When re­turn­ing to his beloved cre­ative medium of mu­sic re­minded him too much of what he’d lost, he de­cided to pivot into an­other sec­tor and leave that chap­ter of his life be­hind him.

The rea­son the de­tail i s so good i n our ex­pe­ri­ences i s be­cause i n the real world when we're scan­ning them, I can never quite get close enough to truly ap­pre­ci­ate i t. At l east i n vir­tual reality, one can. If I was to l ean i nto the Mona Lisa the way I'd want to, I’d be right up against that mas­ter­piece, mois­ture breath­ing all over the damned thing and you know, I’d be ar­rested. With vir­tual reality, the l evel of i nti­macy i s com­pelling as these l i mi­ta­tions are not i n play any­more.

While suf­fer­ing from post-trau­matic stress dis­or­der, Che de Boer came up with the idea of cre­at­ing a vir­tual replica of his house so he could re­gain what he’d lost and re­turn home. He locked him­self away from the world with a com­puter and a first gen­er­a­tion Ocu­lus headset and be­gan re­search­ing how to recre­ate real world en­vi­ron­ments in vir­tual re­al­i­ties.

One talk he had come across prior that proved in­flu­en­tial was a 2008 TEDTalk show­cas­ing Mi­crosoft Pho­to­synth, a now de­funct app that al­lowed you to crudely nav­i­gate through your pho­to­graph col­lec­tions in 3D space.

“This was the first time I’d seen such magic, I just didn’t re­al­ize at the time what it was,” he says. “I re­mem­ber all that time back then want­ing to take hun­dreds of pho­tos of my house, I just couldn’t af­ford a cam­era.”

This in­tro­duced Che de Boer to the magic that can be cre­ated with pho­togram­me­try – the 3D re­con­struc­tion of real-life as­sets by us­ing pho­tos – and down the rab­bit hole he went.

While at first he was re­search­ing whether he could use old fam­ily pho­to­graphs to re­build his burnt-down home, he wound up spend­ing 18 months ab­sorb­ing all the in­for­ma­tion he could on how to go be­yond photo-re­al­ism of lo­ca­tions to cre­at­ing a sense of ac­tual re­al­ism, us­ing pho­togram­me­try, visual ef­fects sig­nal pro­cess­ing, vir­tual reality and game en­gines.

It turns out he was onto a good thing. Af­ter post­ing early works of his pho­togram­me­try-to-VR pipe­lines on YouTube, calls be­gan to come in. In­vest­ment giants like Chi­nese-based Shanda Group were all of a sud­den en­quir­ing about his in­tel­lec­tual prop­erty. Che de Boer re­alised he needed to take his dis­cov­er­ies more se­ri­ously and be­gun ex­ten­sively doc­u­ment­ing some more real-life lo­ca­tions in even more de­tail.

But it was not with­out trial-and-er­ror. He says in the ini­tial re­search stages, he crashed, in­vested or lost ap­prox­i­mately $20,000 worth of drones while shoot­ing lo­ca­tions. But he was un­de­terred, as the tim­ing was align­ing for ev­ery­thing else.

“The com­put­ers had just got fast enough, while the al­go­rithms that other peo­ple were in­vent­ing for the ini­tial point cloud gath­er­ing had just kind of hit a holy grail of be­ing able to do larger scenes,” he says. “So that's when it was like, how cool would it be to just do whole en­vi­ron­ments?”

Given his past tin­ker­ing, it’s lit­tle won­der Che de Boer fell into such a high-tech space. Back be­fore Alexa

and Siri were a phe­nom­e­non, around 2008, Che de Boer had hacked to­gether dif­fer­ent bits of soft­ware to cre­ate a fully nat­u­ral lan­guage, home au­to­mated PC. The talk­ing com­puter was called Brian – with a posh Bri­tish ac­cent and all – and he’d con­verse and de­liver any whim of in­for­ma­tion to him and his amused daugh­ter. A stripped down ver­sion proved a hit on TradeMe, so he sold a cou­ple off to fill the cof­fers.

But still, what he wanted to achieve with VR still seemed like a few years away, tech­nol­ogy-wise, un­til he was in­tro­duced to a piece of soft­ware called Real­i­tyCap­ture. This proved in­stru­men­tal in help­ing him cre­ate truly large scale en­vi­ron­men­tal scans.

Com­bin­ing these dif­fer­ent, in­cred­i­bly com­plex tech­ni­cal el­e­ments – pho­togram­me­try, visual ef­fects sig­nal pro­cess­ing, au­to­matic ex­trap­o­la­tion and game en­gine soft­ware – is ul­ti­mately the se­cret sauce be­hind Real­i­tyVir­tual. Che de Boer says Real­i­tyVir­tual has utilised these dif­fer­ent tech­niques and “put them on steroids”.

“We have de­rived de­light­ing tech­niques, deep learn­ing tech­niques for su­per-en­hance­ment and much more. Our ex­pe­ri­ences are a mag­ni­tude higher in de­tail than that of the near­est com­pe­ti­tion due to our clever data and mem­ory man­age­ment method­olo­gies.”

Ever­coast and 8i co-founder Se­bas­tian Marino, who has worked on the pro­duc­tion of the likes of Star Wars to Avatar and won an Academy Award for tech­ni­cal achieve­ment, says Che de Boer is work­ing a very nascent field with tech­nol­ogy that is not yet ma­ture.

“What he does is not easy,” Marino says. “He has the vi­sion and con­fi­dence to bring to­gether dis­parate ideas such as pho­togram­me­try and ma­chine learn­ing, which are in­di­vid­u­ally com­pli­cated enough to make most peo­ple's heads ex­plode, and he's able to use these as tools to cre­ate beau­ti­ful art that looks ef­fort­less in its re­al­ism. What a great thing to give to the world.”

Che de Boer puts this down to his mind be­ing a sponge for dif­fer­ent ideas.

“I rou­tinely de­scribe English as my sec­ond lan­guage, and ab­stract thought as my first,” he says.

“The first TEDTalk I ever watched was this great TEDTalk called ‘When ideas have sex’, and that’s kind of the phi­los­o­phy be­hind it – mix­ing things to­gether, try­ing to find some­one else you can talk your mad­ness to and hope­fully they grasp the con­cept,” he says.

As well as this, he says that part of the rea­son he broke new ground in the VR in­dus­try is be­cause he had no legacy struc­ture to fol­low, so he didn’t think what he was try­ing to achieve in terms of hy­per­re­al­ism was crazy.

“I found so­lu­tions to prob­lems that peo­ple had been hav­ing in re­gards to man­ag­ing such large datasets. It was my thing. You get all these bil­lions of points and you’re like, ‘I want to have that in real time’, and peo­ple are like, ‘No, you have to scale it down to 1/1000th of that’. I didn't know that be­cause I had never been clas­si­cally trained in the visual ef­fects in­dus­try.

“In VR, your mind ex­pects to see ev­ery blade of grass, ev­ery bit of etch­ing in the wood and all the im­per­fec­tions and all the rest of it. And so, I just ba­si­cally ran into a prob­lem and just re­fused to have it be a prob­lem. And as a re­sult, I found clever ways of data man­age­ment and mem­ory man­age­ment, which I can't talk about in de­tail be­cause that would be giv­ing away a mas­sive chunk of our se­cret sauce. That's one of our many se­cret sauces, and that's why we're in­ter­est­ing, we're con­stantly try­ing to make our own pipe­line ob­so­lete.”

He ex­pects other play­ers will even­tu­ally catch on to Real­i­tyVir­tual’s IP.

“They’re go­ing to work it out very soon I think, but hon­estly – we don't care, be­cause we will be do­ing the next thing,” he ex­plains.

It al­most broke me. Well, i t prob­a­bly did break me and made me l i ke some sort of weird, mad genius, as I had to ab­sorb my­self i n some­thing. It was a cat­a­lyst to force me to j ust re­ally put all my men­tal ef­fort i nto some­thing.

Come near 2019, Real­i­tyVir­tual has sev­eral high pro­file jobs be­hind it. Some have been pub­li­cised, and some are still sub­ject to dis­cre­tion, but Real­i­tyVir­tual has found its sweet spot: doc­u­ment­ing cul­tural her­itage sites that are un­der threat so they’re pre­served dig­i­tally for fu­ture gen­er­a­tions.

One of its re­cent projects, Ne­fer­tari: Jour­ney to

Eter­nity, takes view­ers up close-and-per­sonal in VR to an an­cient civil­i­sa­tion that’s not ex­pe­ri­enced in many peo­ple’s life­times: Queen Ne­fer­tari’s tomb in Egypt. Doc­u­ment­ing Ne­fer­tari took Che de Boer 36 hours of trav­el­ling to Egypt, and about six to eight hours to pho­to­graph 6.4 bil­lion points of colour.

He says the project has achieved some­thing in­cred­i­bly im­por­tant: col­lect­ing a re­al­is­tic data-set of a price­less her­itage site that has pre­vi­ously been closed to the pub­lic due to degra­da­tion, so ‘vir­tual tours’ can fill in when the site gets closed off to the pub­lic in fu­ture.

Most of the re­views on the STEAM site are in awe of the re­al­ism of the ex­pe­ri­ence, with one of the high­est rated re­views say­ing: “It looks ab­so­lutely amaz­ing, re­ally good graph­ics. It al­most feels like you are there. I can just imag­ine a few more years down the line where you won't be able to vis­ually tell the dif­fer­ence. This is an­other step to­wards that re­al­iza­tion.”

For the month of Au­gust and July, Che de Boer says Ne­fer­tari was the 11th high­est rank­ing STEAM game based on re­views and user feed­back.

“Not the 11th high­est VR game or ed­u­ca­tional app, or sub­cat­e­gory, lit­er­ally the 11th high­est ex­pe­ri­ence full stop on STEAM, and that’s a mas­sive plat­form,” he says.

“It shows an ex­tremely strong de­mand for ‘sliceof-life’ ex­pe­ri­ences, and for the abil­ity to be some­where and ex­pe­ri­ence some­thing that very few peo­ple get the priv­i­lege of ac­tu­ally ex­pe­ri­enc­ing. That’s ex­tremely im­por­tant, as it shows we can move away from first per­son shoot-them-ups and get ex­cited about ‘be­ing there’ sen­sa­tions. The in­dus­try is fi­nally see­ing VR as more than an ex­ten­sion to gam­ing. We have to treat VR like its own medium, it needs its own rules.”

While not speak­ing of spe­cific projects, Che de Boer says each in­ter­na­tional gig that Real­i­tyVir­tual picks up can pay any­thing be­tween US$50,000 to $250,000, and the jobs are be­com­ing far more fre­quent now per an­num – with more lined up for 2019 than any pre­vi­ous year.

But he says the com­pany still lives from gig-to-gig – or in other words, “ex­tremely Dutch”.

“I just don't know what would hap­pen if we found our­selves in the po­si­tion of re­ally hav­ing $20 mil­lion in the cof­fers. Like, that could just be dis­as­trous or ex­tremely fun,” he laughs.

It also still op­er­ates like a start-up: the team, which has now ex­panded to in­clude cre­ative tech­nol­o­gist Dan Mon­aghan and ma­chine learn­ing ex­pert Miles Thomp­son, all still work out of Mon­aghan’s lounge to­gether in Welling­ton. A stint at work­ing in an ac­tual of­fice proved ill-fated when Che de Boer was caught try­ing to have a cig­a­rette out of the win­dow.

“I just can't do of­fices, and es­pe­cially a shared of­fice,” he says. “I pre­fer some­where cosy, cre­ative, where one can chill and have a red wine. Also, as we are de­vel­op­ing IP, we re­alised that this way was safer.”

Real­i­tyVir­tual is con­sid­er­ing float­ing an ICO or in­vest­ment to scale up when the tim­ing’s right. Cur­rently, its op­er­a­tions have been run­ning strictly on cash-flow fund­ing re­search, but many of its core tech­nolo­gies are break­away prod­ucts and could be con­sid­ered sep­a­rate en­ti­ties for in­vest­ment.

Chas­ing a small pot of gold

A re­port by the New Zealand VR/AR As­so­ci­a­tion last year fore­casted that New Zealand’s cross reality sec­tor (VR and AR) will reach over $320 mil­lion in an­nual rev­enue and dou­ble the num­ber of peo­ple em­ployed in the sec­tor cur­rently to 2200 peo­ple by 2019.

Yet de­spite the groundswell of in­ter­est for this new medium – and Real­i­tyVir­tual now get­ting global recog­ni­tion for its ground­break­ing work (as this is­sue goes to print, Che de Boer is keynot­ing at the Cairo In­ter­na­tional Film Fes­ti­val) – in 2017, Che de Boer was al­most ready to call it quits in the VR in­dus­try. He was the pri­mary care­giver of his daugh­ter Anika, sup­port­ing them both with a ben­e­fit, and had burned through a lot of cash try­ing to make it work. He’d knocked on the door of gov­ern­ment or­gan­i­sa­tions many times, ask­ing for fund­ing and sup­port, to no avail.

In a last-ditch at­tempt, he de­cided to get on a plane to Los An­ge­les and try make a name for him­self through rub­bing shoul­ders with Hol­ly­wood’s elite. Sur­pris­ingly, it wasn’t too in­tim­i­dat­ing a prospect for him, ei­ther. Che de Boer says he feels less awk­ward mix­ing with the peo­ple in the film in­dus­try in Los An­ge­les than he does with the peo­ple liv­ing in his home­town in New Zealand.

“It was like re­turn­ing home to the moth­er­ship,” he says.

It proved to be a good call, as this led to his first big break. US video game gi­ant Epic Games – mak­ers of the now wildly suc­cess­ful game Fort­nite – gave him a cheque to go make a demo of what­ever he wanted to do.

But Che de Boer says his con­cern is an in­ter­na­tional gam­ing jug­ger­naut helped the com­pany get off the ground, rather than a New Zealand or­gan­i­sa­tion.

“We're re­ally sti­fling a hell of a lot of in­no­va­tion here, in my opin­ion,” he says, “If it wasn't for Epic Games who lit­er­ally just gave us, I think it was NZ$35,000 – no con­tract, no noth­ing, just, ‘What's your bank ac­count num­ber?’ then we would’ve gone bank­rupt a long time ago. Ev­ery­one here’s just chas­ing this lit­tle tiny pot of gold and get­ting quite bitchy with each other. So it's a lot of un­der­selling each other and as a re­sult, then ev­ery­one gets paid way less.”

Real­i­tyVir­tual was re­cently hired to scan the New Zealand House of Par­lia­ment in ad­di­tion to in­di­rectly be­ing hired to doc­u­ment the Treaty of Wai­tangi, but he still feels as though the com­pany still isn’t be­ing pri­ori­tised for lo­cal gigs.

“We find our­selves lo­cally be­ing put third or fourth in line, while the com­pa­nies that spend all their money on mar­ket­ing dol­lars get all the praise. It seems around here it’s not about the qual­ity of con­tent, but more who you know,” he says.

Now that it has re­ceived world­wide ac­claim, fund­ing has

proven a lit­tle eas­ier to come by – but this irks Che de Boer.

“This is when I go into my plug about how de­lighted I am, but hon­estly, fuck off, you took four years. And it's pid­dly,” Che de Boer laughs. “No, look, I mean I'm happy that New Zealand Film Com­mis­sion re­cently got be­hind us. Good on them.”

“But at the start – Callaghan In­no­va­tion, Te Papa, Her­itage New Zealand, NZTE, ATEED – the list goes on and on, we tried for years and years lo­cally to get some kind of ini­tial fund­ing of any na­ture and got no favours what­so­ever,” he says.

“It feels like in­cu­ba­tors are just set­ting up in­cu­ba­tors. We need to take a leaf out of Epic Games' book and just fund start-ups di­rectly with­out the bureaucratic bull­shit.

“It amazes me, the at­ti­tude to lo­cal tal­ent. A great ex­am­ple is Air New Zealand hir­ing CyArk – a San Fran­cisco-based com­pany

– to scan our na­tion's marae, yet not ap­proach­ing New Zealand com­pa­nies, de­spite the fact we had al­ready done two amaz­ing ex­pe­ri­ences with in­dige­nous Māori that had world rap­port.”

While fund­ing for VR projects in New Zealand is slim pick­ings, the New Zealand Film Com­mis­sion In­ter­ac­tive Fund grant gives a grant of up to $25,000 for the con­cept de­vel­op­ment of orig­i­nal nar­ra­tive-based in­ter­ac­tive and games con­tent, or up to $50,000 in ex­cep­tional cir­cum­stances.

I j ust don't know what would hap­pen i f we found our­selves i n the po­si­tion of re­ally hav­ing $ 20 mil­lion i n the cof­fers. Like, that could j ust be dis­as­trous or ex­tremely fun.

Reality check

While cul­tural preser­va­tion is the fo­cus of Real­i­tyVir­tual, it has a promis­ing fu­ture ahead if it gets a spin-out com­pany off the ground that is called deep­PBR (Deep Learn­ing Phys­i­cally Based Ren­der­ing).

Che de Boer says through his work, he’s been un­der­tak­ing a bold new task: ex­trap­o­lat­ing the fun­da­men­tal prin­ci­ples of reality. Us­ing ma­chine learn­ing, he’s trained a neu­ral net­work to learn the con­cept of light, its ma­te­rial prop­er­ties and its 3D ge­om­e­try all from one sin­gle im­age, which could be the holy grail for the visual ef­fects in­dus­try.

“In pho­togram­me­try, when you go out and do a scene, you're stuck with the en­vi­ron­ment that you're given and how the light and the weather was that day,” he says. “That's a bit of a prob­lem, be­cause if you want to re­light it from day to night, or if you want to, you know, cre­ate vari­a­tions in weather, you ba­si­cally can’t. So what we've been do­ing is we taught a gen­er­a­tive ad­ver­sar­ial neu­ral net­work to break down the fun­da­men­tal prin­ci­ples of reality just by look­ing at the im­age in it­self, such as how it would look like if there were in­fi­nite light­ing vari­ables: what is shiny, what is matte.”

He says the ma­chine has ac­quired enough data through Real­i­tyVir­tual’s work over the years to now have learnt the fun­da­men­tal be­hav­iour of light and ma­te­rial prop­er­ties, and has the po­ten­tial to do what would take a team of 100 VFX artists and pho­tog­ra­phers months, with a sin­gle in­di­vid­ual and a DSLR cam­era in a mat­ter of hours.

“You've got the, you know, in­cred­i­ble re­al­ism of cap­tur­ing per­fec­tion in the real world and then you've got to ben­e­fit of be­ing able to re-light it, which has not been the case pre­vi­ously. Not eas­ily, any­way,” he says.

“We've done a thing called hy­brid light­ing over the years, and it's kind of like a pseu­do­light­ing, but it's not based on true phys­i­cal val­ues. So the new tech­nique, which we are kind of pig­gy­back­ing on, they call it phys­i­cal-based ren­der­ing, and it’s ba­si­cally the game en­gines try­ing re­ally hard to ba­si­cally base all ma­te­ri­als on reality.”

This means de­tails such as rough­ness, smooth­ness and whether an ob­ject is shiny or matte can also be re­mod­elled in the en­vi­ron­ment, as can tak­ing a scene from day to night. Or­gan­i­sa­tions like Weta Dig­i­tal, Ubisoft, Au­todesk, Unicef and Unesco have al­ready shown in­ter­est.

But Che de Boer says the only prob­lem with head­ing in this di­rec­tion is that it's go­ing to make all of their other work po­ten­tially de­funct within the next few years.

“So that's quite crazy, you know, we’re lit­er­ally, po­ten­tially killing off our old busi­ness model, but hey, wel­come to progress.”

And progress is para­mount for some­one like Che de Boer, whose short at­ten­tion span and in­sa­tiable hunger for in­for­ma­tion means he

flips ca­reers of­ten. The everde­vel­op­ing VR in­dus­try and his de­vel­op­ments in the deep learn­ing space holds enough ex­cite­ment to keep him sta­tion­ary in one role for now.

How­ever, he says he wants to keep in­no­vat­ing un­til he makes him­self ob­so­lete.

“It's an in­ter­est­ing idea that you're only as good as your last project, so ev­ery time we do a new project, we are try­ing to see if there's new ways to do things,” he says.

To con­tinue on his quest to dig­i­tally pre­serve her­itage sites for years to come, Che de Boer re­cently formed a strate­gic part­ner­ship with fel­low Kiwi abroad Pro­fes­sor Sarah Ken­der­dine, head of dig­i­tal muse­ol­ogy at EPFL, a pres­ti­gious re­search uni­ver­sity in Switzer­land.

To­gether, they’re look­ing to vir­tu­ally re-cre­ate the Christchurch Cathe­dral in its pre-quake state, in ad­di­tion to sev­eral other cul­tur­ally sig­nif­i­cant lo­ca­tions.

“We are build­ing the frame­work and au­to­ma­tion tools to es­sen­tially democra­tise the process of back­ing up the planet,” he says. "That is the holy grail.”

If all goes well, Che de Boer says the en­tire Real­i­tyVir­tual en­ter­prise will be up­rooted and taken to Europe.

But for now, Che de Boer says the fu­ture is bright – and ex­tremely de­tailed.

We are build­ing the frame­work and au­to­ma­tion tools to es­sen­tially democra­tise the process of back­ing up the planet. That i s the holy grail.

Newspapers in English

Newspapers from New Zealand

© PressReader. All rights reserved.