A string of re­cent VFX block­busters have gone ‘frac­tal-crazy’ – Ian Failes ex­plores the art and sci­ence of these math­e­mat­i­cal func­tions

3D World - - CON­TENTS -

Ian Failes ex­plores the world of com­plex frac­tals, from Man­del­brots to Man­del­bulbs, speak­ing to VFX stu­dios about their block­buster hits

Plenty of film­mak­ers are al­ways look­ing for ‘or­ganic’ forms to rep­re­sent alien worlds or mag­i­cal mo­ments in their movies. So it’s per­haps un­sur­pris­ing that they would seek to em­brace frac­tals. Af­ter all, frac­tals tend to look like nat­u­rally oc­cur­ring and in­fin­itely re­peat­able objects, yet can of­ten be sim­u­lated with math­e­mat­ics.

And so it is that sev­eral re­cent

films, in­clud­ing Doc­tor Strange, Sui­cide Squad, Guardians of the

Galaxy Vol. 2, Lucy and An­ni­hi­la­tion have adopted frac­tals – es­pe­cially three-di­men­sional ones – to help tell their sto­ries. And they’ve seen use in im­mer­sive projects too, where frac­tal sim­u­la­tions can help re­alise com­plex forms for users to ex­plore.

3D World asked some of the vis­ual ef­fects stu­dios tasked with mak­ing com­plex frac­tals – par­tic­u­larly Man­del­brot and Man­del­bulb sets – how they went about tack­ling these tasks for some im­pres­sive VFX.

Frac­tals of the galaxy

“A frac­tal is Amaz­ing in the sense that it’s just A tiny piece of sim­ple code” Kevin Smith, vis­ual ef­fects su­per­vi­sor, Weta Dig­i­tal

Put sim­ply, frac­tals are com­plex, which is ex­actly why Weta Dig­i­tal looked to them as in­spi­ra­tion for the Planet Ego se­quences in

Guardians of the Galaxy Vol. 2. The stu­dio car­ried out some early tests em­bed­ding im­plicit func­tions into its pro­pri­etary Manuka path­trace ren­derer via plug­ins, with promis­ing re­sults. It matched the client con­cept art – which specif­i­cally fea­tured 3D Apol­lo­nian gas­ket-like shapes – but it was soon re­alised that re­ly­ing on purely

math­e­mat­i­cal func­tions could be lim­it­ing.

“A frac­tal is amaz­ing in the sense that it’s just a tiny piece of sim­ple code, but very small changes to the in­puts of frac­tals tend to re­sult in un­pre­dictable, large-scale changes in the out­put,” out­lines Weta Dig­i­tal vis­ual ef­fects su­per­vi­sor Kevin Smith. “They’re very chaotic. This was prob­lem­atic for us as I couldn’t sit in a re­view with a client and ask them to give notes on a piece of code, much less one that’s com­pletely un­pre­dictable. We knew that what­ever method­ol­ogy we chose needed to be art di­rectable.”

So Weta Dig­i­tal con­sid­ered mod­el­ling by hand, but again aban­doned this ap­proach due to the in­fi­nite de­tail required (“Also, the mod­els su­per­vi­sor yelled at me when I brought it up,” says Smith). That left the stu­dio with new re­quire­ments: defin­ing an ar­bi­trary shape that was art di­rectable but could still match those gas­ket shapes, and achiev­ing a dig­i­tal en­vi­ron­ment that felt like it had in­fi­nite prac­ti­cal de­tail.

The R&D team de­vised a method that let artists use curves to de­fine an axis and a pro­file in Maya, and then code that would use a cus­tom sphere-packing al­go­rithm to bool­ean out spheres from the

“Vis­ual ef­fects for me has Al­ways been About the com­bi­na­tion of Art And sci­ence” Kevin Smith, vis­ual ef­fects su­per­vi­sor, Weta Dig­i­tal

first shape, to give the ap­pear­ance of an Apol­lo­nian frac­tal in a user­con­trolled vol­ume. “For the de­tail,” ex­plains Smith, “in­stead of try­ing to add in­fi­nite frac­tal minu­tia, we used an in-house piece of soft­ware called Gen­e­sis to es­sen­tially spray­paint lit­tle in­stances of frac­tal ge­om­e­try all over the re­sult­ing shapes pro­duced by the first tool. The lay­out depart­ment came up with Gen­e­sis brush pre­sets that used com­bi­na­tions of the tiny in­stances with dif­fer­ent scales to es­sen­tially make pseudo-frac­tals. This let us add a lot of de­tail with­out in­cur­ring an in­fi­nite cost. It also helped us age the main sec­tions of the en­vi­ron­ment, since the Planet Ego in the film was very old.”

Then, a stum­bling block. The Maya plugin could not quite achieve all the shapes in the con­cept art. Weta Dig­i­tal needed a way to gen­er­ate some of the more es­o­teric forms from the art that had orig­i­nally come from Man­del­bulb soft­ware, but at­tempts thus far had required pro­hib­i­tive amounts of me­mory with­out the required res­o­lu­tion. The so­lu­tion, de­vised by se­nior mod­eller Pas­cal Raim­bault, was to gen­er­ate a 4K turntable of the rel­e­vant ar­eas in the Man­del­bulb soft­ware – in­stead of ge­om­e­try – and then feed those ren­ders to Weta Dig­i­tal’s pho­togram­me­try soft­ware.

“It to­tally worked,” ex­claims Smith. “It pro­duced sharper, cleaner, higher-res­o­lu­tion images than we were get­ting with vox­eli­sa­tion, and al­lowed us to build a li­brary of shapes we could use to dress in de­tail that was not just close to the con­cept but ex­actly matched it.”

“Vis­ual ef­fects for me has al­ways been about the com­bi­na­tion of art and sci­ence,” adds Smith, “and it was great to be able to take a purely math­e­mat­i­cal con­cept like frac­tals and not only make some­thing new and dif­fer­ent, but to use it to help drive the nar­ra­tive of an awe­some movie like Guardians of the Galaxy Vol. 2.”

Frac­tals as Weapons: doc­tor strange

An ear­lier Mar­vel film, Doc­tor

Strange, was also one that adopted Man­del­brots and frac­tal shapes in its art di­rec­tion. In­deed, the Man­del­brot was one of the key weapons used by the film’s vil­lains to re-shape the world and re­con­fig­ure lo­ca­tions. Among other VFX stu­dios, Frame­store was called upon to craft such lo­ca­tions with Man­del­brot prop­er­ties, in­clud­ing for a fight in the Sanc­tum Foyer where a cor­ri­dor is made in­fin­itely long.

To do that, artists first mod­elled each set. They then used Sidefx’s Hou­dini to ‘slice up’ each set into much smaller objects. “Rig­ging cre­ated a new work­flow for the an­i­ma­tors to rig ev­ery ob­ject them­selves,” ex­plains Frame­store vis­ual ef­fects su­per­vi­sor Alexis Wa­js­brot. “It was im­por­tant that they were able to an­i­mate and have the power to move, pivot or even du­pli­cate the ge­om­e­try if needed. We also cre­ated a tool­box for the an­i­ma­tor, to be able to an­i­mate a large num­ber of objects – some­times up to 5,000 – with a math­e­mat­i­cal func­tion to cre­ate waves and any cool mo­tion we felt was ap­pro­pri­ate.”

The next step was to ap­ply the Man­del­brot noise to the CG set. “We tried to im­ple­ment mul­ti­ple Man­del­brot func­tions in­clud­ing the ‘Man­del­box’ and ‘Man­del­bulb’,” says Wa­js­brot, “but we de­cided to only use the ‘Man­del­sponge’ which was less or­ganic, more geo­met­ric.

“Af­ter try­ing dif­fer­ent tech­niques, we de­cided to ap­ply the Man­del­brot at ren­der time us­ing an Arnold shader we de­vel­oped to get the max­i­mum amount of de­tail. To drive the shader, FX TDS used Hou­dini where they had a vol­ume rep­re­sen­ta­tion of the Man­del­brot. When we were happy with the

pa­ram­e­ters, we baked at­tributes into the ge­om­e­try it­self so that the shader was read­ing at ren­der time.”

Wa­js­brot says the level of de­tail that could be achieved us­ing the Man­del­brot func­tion was high, but fur­ther art di­rec­tion was nec­es­sary to make it work for the film. “It’s very easy to make some­thing that looks com­plex and cool, but it’s a lot harder to nail a spe­cific look or be­hav­iour. In the fi­nal shots, I would say the an­i­ma­tion is [the] main com­po­nent, and the Man­del­brot is only the last 20 per cent of the im­age that com­pletes the magic.”

im­merse your­self

Away from feature films, Frame­store has also been ex­plor­ing how frac­tals can form part of im­mer­sive en­ter­tain­ment, specif­i­cally for vir­tual and aug­mented re­al­ity ex­pe­ri­ences. Their CORAL app, which is the brain­child of ex­ec­u­tive cre­ative di­rec­tor Aron Hjar­tar­son and se­nior cre­ative de­vel­oper Jo­hannes Saam, is one such project.

The ex­pe­ri­ence is es­sen­tially a jour­ney into frac­tal shapes (in­clud­ing coral-like struc­tures), con­trolled by the user. It was made in Unity and sup­ports Vive and Ocu­lus head­sets as well as a nonvr desk­top mode. “We’ve played with scale by shrink­ing the users to rel­a­tively microscopic sizes and back de­pend­ing on how close they get to each frac­tal,” says Saam. “With the right con­troller the user can morph cer­tain pa­ram­e­ters be­spoke to each frac­tal, cre­at­ing a mes­meris­ing kalei­do­scope of trans­form­ing shapes.”

A lot of R&D went into mak­ing this kind of frac­tal ex­plo­ration pos­si­ble in a VR/AR ex­pe­ri­ence. “The base ren­der­ing sys­tem uses ray­march­ing as its core con­cept,” de­scribes cre­ative de­vel­oper Patrick Beavers. “How­ever, it uses mul­ti­ple hi­er­ar­chi­cal passes to be able to con­verge to a so­lu­tion faster, adding more de­tail by in­creas­ing the res­o­lu­tion it­er­a­tively. Ad­di­tion­ally, given the fact that this ex­pe­ri­ence is de­signed for VR, we de­vel­oped a tech­nique that re­pro­jects in­ter­me­di­ate re­sults from one eye to the other, thus re­duc­ing the amount of cal­cu­la­tions that are needed to ren­der the shape of each frac­tal.

“Other tech­niques we use to ac­cel­er­ate the ren­der­ing are fixed foveated ren­der­ing, in which the screen has more res­o­lu­tion closer to the centre of the eye, and vi­gnetting, which helps us di­rectly dis­card pix­els that are on the pe­riph­ery. As for ren­der­ing each frac­tal, we tried to stay away from con­ven­tional ren­der­ing tech­niques and used dif­fer­ent tricks that we de­vel­oped to make in­ter­est­ing vi­su­als while keep­ing good per­for­mance.”

The im­mer­sive ex­pe­ri­ence is still un­der de­vel­op­ment. Frame­store has also con­sid­ered turn­ing CORAL into a mu­sic vi­su­aliser or po­ten­tially us­ing the frac­tal landscapes as a foun­da­tion for a rhythm game. “We would love to al­low users to ex­plore with their friends in mul­ti­player, and it will of course con­tinue to be an out­let for real-time ren­der­ing re­search at Frame­store,” notes cre­ative de­vel­oper Mar­i­ano Mer­chante, “but ul­ti­mately CORAL will al­ways first and fore­most be a play­ground of dis­cov­ery, and we want to make sure we main­tain the spirit of that no mat­ter what we do go­ing for­ward.”

“coral will Al­ways first And fore­most be A play­ground of dis­cov­ery” Mar­i­ano Mer­chante, cre­ative de­vel­oper, Frame­store

above: com­plex frac­tal sim­u­la­tions can serve to add a mes­meris­ing dream-like ef­fect to cg scenes

left: ini­tial de­vel­op­ment for the frac­tal pieces went through Weta dig­i­tal’s pro­pri­etary manuka ren­derer

in order to ren­der the fi­nal images, Weta dig­i­tal de­vised a clever tech­nique that com­bined man­del­bulb ren­ders and pho­togram­me­try

Be­low: Weta dig­i­tal needed to match spe­cific frac­tal­look­ing client con­cept art for the in­te­rior of Planet ego for guardians of the galaxy vol. 2

ge­om­e­try pro­duced for the se­quence in­cluded these 3d apol­lo­nian gas­ket-like shapes

a pro­mo­tional im­age for Frame­store’s coral app, which in­cor­po­rates the clas­sic man­del­bulb frac­tal shape

screen­shots from a trial ver­sion of Frame­store’s coral app show how the man­del­bulb frac­tal could ap­pear in ar

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.