Desk­top apps 360 VR videos

Dis­cover why Adam Ox­ford thinks 360 video is fi­nally start­ing to find its feet.

Linux Format - - CONTENTS -

It’s a nascent art form, but Adam Ox­ford thinks 360 video is start­ing to find its feet, so rush out and cap­ture your own.

Look­ing back it seems strange, but some of the first “killer apps” for smart­phones were aug­mented and vir­tual re­al­ity (AR and VR) in na­ture. As early as Jan­uary 2009, just 18 months af­ter the orig­i­nal iPhone ap­peared, there were al­ready sev­eral star chart maps that used the built-in GPS and ac­celerom­e­ter to track the night sky us­ing the phone screen.

It seems strange be­cause ever since then, AR and VR have been the Next Big Thing. Al­ways ex­cit­ing, al­ways promis­ing… but never quite com­pelling enough to hit the big time. Poke­monGo! came and went, leav­ing a mil­lion te­dious “vi­ral” mar­ket­ing im­i­ta­tors in its wake. To­gether, AR and VR have had so many false starts that not even Mark Zucker­berg’s pre­pos­ter­ous en­trance to Mo­bile World Congress last year, in which 1,000 jour­nal­ists were con­vinced to don VR specs so that they could pic­ture them­selves sit­ting in the seat they were al­ready sit­ting in could con­vince the world it was ready for VR again.

So AR and VR still feel like nov­elty sub­jects, de­spite pre­dat­ing the smart­phone era. Which is a shame, be­cause they’re a lot of fun to play with. Es­pe­cially in their most ba­sic form, the 360-de­gree video.

Gen­er­ally speak­ing, we can di­vide the world of im­mer­sive video into three dis­tinct ar­eas. Aug­mented Re­al­ity (AR) is when vir­tual in­for­ma­tion is lay­ered on top of the real world. Like Poke­monGo! this is typ­i­cally done in real-time us­ing the cam­era on your phone and its screen. Ap­ple and Google are putting huge ef­forts into this area, and Ap­ple’s ARKit – part of the lat­est iOS tool­kit – is de­scribed by de­vel­op­ers as some­thing of a break­through in ease of use.

Vir­tual re­al­ity, mean­while, is a com­pletely com­puter gen­er­ated world pro­jected onto the screen of your phone or into a head­set. Not only is it stereo­scopic, so that it ap­pears 3D, it’s also fully in­ter­ac­tive. Think games like Elite:Dan­ger­ous played with an Oc­cu­lus Rift head­set.

Im­mer­sive video

What we’re in­ter­ested in, how­ever, is 360-de­gree video. Like VR, this is com­pletely im­mer­sive, but it’s not in­ter­ac­tive. You can watch it on a com­puter screen, with a viewer that en­ables you to move the cam­era an­gle around, or you can use a head­set or Google Card­board-style viewer for full im­mer­sion. It might also be fully 3D if it’s recorded with two cam­eras for stereo­scopic ef­fects, but over­all it’s the least ex­plored of the three cat­e­gories.

There have been some in­ter­est­ing ex­per­i­ments with 360-de­gree video: from ac­tion cams strapped to the heads of moun­tain bik­ers or down­hill skiers, to a se­ries of films shot for be­hind-the-scenes glimpses into BBC doc­u­men­tary mak­ing. For the most part, the aim of these videos is to give the viewer an im­me­di­ate feel­ing of do­ing some­thing else – they’re rough, unedited and don’t even at­tempt to por­tray some­thing as nu­anced as a story.

There are rare ex­cep­tions. The cre­ators of Mr.Ro­bot edited to­gether a se­quence of scenes from the first se­ries as

360-video, for ex­am­ple. Un­like most 360 videos, the cam­era doesn’t al­ways put the au­di­ence in a first-per­son view – but rather en­ables them to ex­plore a scene while it’s play­ing out. It’s an in­trigu­ing – if un­com­mon – ex­pe­ri­ence.

There’s clearly po­ten­tial for 360-de­gree video – it’s a log­i­cal pro­gres­sion for film­mak­ers who want to con­trol what their au­di­ence sees – but it’s still more-or-less un­ex­plored ter­ri­tory and ripe for an au­teur to show what can be done.

Cap­tur­ing 360-video

There are three dif­fer­ent ways to cap­ture 360-de­gree scenes. Early pi­o­neers built rigs upon which they mounted mul­ti­ple cam­eras and stitched the re­sults to­gether in desk­top edit­ing suites, but things have pro­gressed since then.

The eas­i­est way to record 360-de­gree video is with a ded­i­cated cam­era such as the Ri­coh Theta S or Sam­sung Gear 360. These have a wide-an­gle lens on the front and back that gives an al­most-but-not-quite 180-de­gree from either side of the cam­era, and the two video feeds are stitched to­gether into the spher­i­cal view on the cam­era it­self. Pro­fes­sional 360-de­gree cam­eras often have a third lens point­ing up­wards to elim­i­nate blind spots.

Ded­i­cated de­vices are still quite pricey, so cam­eras that work with your phone are a good sec­ond best. We used an In­sta360 Air, which plugs into the USB port on a smart­phone. It’s a bit more cum­ber­some to use than a ded­i­cated cam­era, but al­most twothirds less ex­pen­sive. The end re­sults are ex­cel­lent and the phone app stitches the video to­gether, en­ables you to watch other 360-de­gree videos and ex­ports the fi­nal movie clip in a for­mat we can use on the desk­top.

Fi­nally, you can shoot 360-de­gree still images us­ing your phone or cam­era by tak­ing lots of still pics from all around you and stitch­ing them to­gether us­ing imag­ing soft­ware. It’s a great (and free – see box­out) way to get started, but it’s not video and so is no good for our pur­poses. There are barely any ap­pli­ca­tions for Linux that can han­dle 360-video prop­erly. Blender, for ex­am­ple, can cre­ate 3D worlds and ex­port fly-throughs as 360-video, but it’s not so handy for ad­ding el­e­ments to films you’ve al­ready shot. For­tu­nately, 3D video can be edited and cut to­gether in al­most any reg­u­lar non-lin­ear video edit­ing suite. How­ever, you’ll need to in­stall sep­a­rate soft­ware to preview your ed­its, and there’s a few choices you can opt for. The open source Valiant360 project ( https://github. com/flimshaw/Valiant360) can turn your browser into a re­spon­sive 3D video viewer if you’re keen on cus­tomis­ing it to use as a lo­cal player. For some­thing eas­ier and more off-theshelf, the folks at GoPro have a .deb pack­age of GoProVRPlayer (, which works flaw­lessly on most De­bian de­riv­a­tives. For an open source player, how­ever, Ver­sion 3 of VLC player does sup­port 360-de­gree video, al­though it doesn’t have the same view­ing op­tions as GoProVRPlayer. It’s not in­cluded in dis­tri­bu­tion re­pos yet, though. To get the lat­est build ( VLC 3.0.0-git Veti­nari) in Ubuntu, for ex­am­ple, you’ll need to add the “nightly” PPA: sudo add-ap­trepos­i­tory ppa:videolan/master-daily && apt in­stall vlc . Over­all, GoProVRPlayer gave us the best play­back qual­ity and in-video con­trols. Once you’ve cut your video, if you want to share it with the world you’ll want to up­load it to Google, Face­book or Vimeo et al. All sup­port shar­ing and view­ing 360-de­gree videos with their own in-browser play­ers.

As­tral pro­jec­tions

To start edit­ing your video you’ll need to get it off of your phone and onto the desk­top. This will dif­fer from cam­era to cam­era, but the prin­ci­ple is usu­ally the same. In the In­sta360 app, for ex­am­ple, you’ll need to ex­port the video to your Videos gallery us­ing the in-app conversion tool un­der “Share”. You may find that there are a few

op­tions for the ex­ported file – par­tic­u­larly re­lat­ing to the view­point that might be “Tiny Earth”, “Spher­i­cal”, “Fish-eye” or “Equirect­anglar”.

If these are op­tions, it’s the last that you want to se­lect. This is the for­mat that’s most com­mon and eas­i­est to work with, and ex­ports the 3D video as a reg­u­lar 2D MP4 file. It’ll be in a 2:1 su­per-widescreen ra­tio and you want to se­lect the high­est res­o­lu­tion you can (4K is usu­ally an op­tion).

On the desk­top

Equirect­an­gu­lar pro­jec­tions flat­ten spheres into a rec­tan­gle, keep­ing rel­a­tive di­men­sions ac­cu­rate. So when viewed, the world they por­tray looks like two fish-eye images stitched to­gether. When curled into a ball ev­ery­thing re­gains its proper pro­por­tions. The best ex­am­ples of equirect­an­gu­lar ver­sus other pro­jec­tions in­volves maps of the Earth.

Think of the way the globe is di­vided up into lines of lon­gi­tude and lat­i­tude. As you get closer to the poles, the phys­i­cal size of each box bounded by a line of lon­gi­tude and lat­i­tude gets smaller, be­cause the curve of the Earth makes lon­gi­tu­di­nal lines con­verge and means there’s less dis­tance for each line of lat­i­tude to cover.

This presents a chal­lenge to car­tog­ra­phers. How do you rep­re­sent a curv­ing line as a rec­tan­gle on a flat map? Some maps – such as the fa­mous Mer­ca­tor pro­jec­tion – do this by draw­ing each “box” from the globe as an equally sized box on a map. In do­ing so, they ex­ag­ger­ate the size of land­masses, so that tiny Green­land can ap­pear ge­o­graph­i­cally sim­i­lar to Africa, which is big enough to con­sume China, the USA and all of Europe.

Equirect­an­gu­lar maps, mean­while, try to com­pen­sate by shrink­ing the size of the lon­gi­tude/lat­i­tude boxes ap­pro­pri­ately. They have a more com­pressed look at the top and bot­tom and make land­masses near the poles look smaller than they might on a globe. If you’ve ex­ported your video cor­rectly, it should look a bit like that.

The edit suite

With your 360-de­gree videos in a rec­tan­gu­lar as­pect ra­tion and com­mon dig­i­tal for­mat, you can start ac­tu­ally cut­ting them to­gether in any video edit­ing suite. It’s re­ally that straight­for­ward – the only thing you’ll need to make sure of be­fore you start is that the cam­era’s start­ing ori­en­ta­tion is the same in every clip you edit. View­ers can look up and down once the video be­gins, but switch­ing the ceil­ing and the floor half­way through will be very dis­ori­ent­ing.

Once you’ve put to­gether the movie you want, though, things get a lit­tle bit more tricky. It’s best to ren­der the fi­nal cut as an MKV or MP4 file, be­cause they’re widely sup­ported in terms of soft­ware and cloud shar­ing ser­vices. In­deed, some on­line ser­vices such as YouTube will only ac­cept 360-de­gree videos with the MP4 file ex­ten­sion.

Be­cause the 360-de­gree for­mat is new, how­ever, and not re­ally sup­ported by any desk­top edi­tors in Linux, your fi­nal file will be miss­ing some meta­data that iden­ti­fies it as a 360-de­gree film.

The GoProVRPlayer doesn’t care – it’ll twist any file that you give it into a 3D view­point – but YouTube and other on­line ser­vices do. If you don’t add the right meta­data (sim­i­lar to EXIF in­for­ma­tion in a still im­age), they’ll only play the file back in its flat equirect­an­gu­lar for­mat if you don’t iden­tify it cor­rectly.

Thankfully, there’s an easy way to do this: Google’s engi­neers have cre­ated a tool called the Spa­tialMe­dia Me­ta­dataIn­jec­tor. Grab the file from its Github archive (­tial-me­dia/tree/master/ spa­tialme­dia), and ex­tract it to a suitable lo­ca­tion in your Home folder. Say ~/In­jec­tor.

You’ll also need to in­stall Python from your dis­tri­bu­tion re­pos if you haven’t done so al­ready, with the com­mand sudo apt in­stall python .

Once you’ve ex­tracted the Spa­tialMe­di­aMe­ta­data In­jec­tor, us­ing it is sim­ple. Open a ter­mi­nal and type the fol­low­ing com­mand: python ~/In­jec­tor/spa­tialme­dia -i ~/ pathto/in­put.mp4 ~/pathto/out­put.mp4 where in­put.mp4 is your edited video and out­put.mp4 is the cor­rected ver­sion with the new meta­data added.

And that’s more or less it. Mak­ing a 360-de­gree video is much the same as mak­ing a reg­u­lar 2D video, with just a few things to watch for be­fore and af­ter the edit­ing process. The next step is to think about how to cap­ture and map the sound with mul­ti­ple mics in 360-de­grees as well. But that would be a whole other walk­through…

We spin you right round, right round.

The equirect­an­gu­lar pro­jec­tion is the eas­i­est one to work with when edit­ing 360-de­gree video. It’s ba­si­cally like edit­ing a reg­u­lar movie.

The ex­cel­lent In­sta360 Air comes with a rub­ber case to pro­tect the lenses while it’s in your bag.

It’s not open source, but the GoProVRPlayer pro­vides the best-qual­ity 360-de­gree video in our ex­pe­ri­ence.

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.