What the pixel did next

Is the digital revolution that began with Toy Story stifling art – or saving it?

In 2011 the Westfield shopping mall in Stratford, east London, acquired a new public artwork: a digital waterfall by the Shoreditch-based Jason Bruges Studio. The LCD screens of the 12m high sculpture form a subtle semirandom flickering display, as though water were pouring down its sides. Depending on the shopper’s mood, this either slakes their visual appetite, or leaves them gasping for a glimpse of real rocks, real water, real life.

Over its 10-year life, Bruges’s piece has gone from being a comment about natural processes (so soothing, so various, so predictable!) to one about digital images, a nagging reminder that underneath the apparent smoothness of our media lurks the jagged line and the stair-stepped edge, the grid, the square: in other words, the pixel.

We suspect that the digital world is grainier than the real, coarser, more constricted, and stubbornly rectilinear. But this is a prejudice, one that’s neatly punctured in A Biography of the Pixel, a new book by electrical engineer Alvy Ray Smith, co-founder of US computer animation studio Pixar. This eccentric work traces the intellectual genealogy of Toy Story (Pixar’s first feature-length computer animation in 1995) over bump-maps and around occlusions, through endless samples, computations and transformations, back to the mathematics of the 18th century.

Smith’s Whig history is a little hard to take – as though, say, Joseph Fourier’s efforts in 1822 to visualise how heat passed through solids were merely a way station on the way to Buzz Lightyear’s calamitous launch from the banister rail – but it’s a superb shorthand in which to explain the science.

We can use Fourier’s mathematics to record an image as a series of waves (visual patterns, patterns of light and shade and movement, “can be represented by the voltage patterns in a machine,” Smith explains). And we can recreate these waves, and the image they represent, with perfect fidelity, so long as we have a record of the points at the crests and troughs of each wave.

The locations of these high- and low-points, recorded as numerical coordinates, are pixels. (The little dots you see if you stare far too closely at your computer screen are not pixels; strictly speaking, they’re “display elements”.)

Digital media do not cut up the world into little squares (only crappy screens do that). They don’t paint by numbers. On the contrary, they faithfully mimic patterns in the real world.

This leads Smith to his wonderfully upside-down-sounding catchline: “Reality,” he says, “is just a convenient measure of complexity.”

Once pixels are converted to images on a screen, they can be used to create any world, rooted in any geometry, and obeying any physics. And yet these possibilities remain largely unexplored. Almost every computer animation is shot through a fictitious “camera lens”, faithfully recording a Euclidean landscape. Why are digital animations so conservative?

I think this is the wrong question: its assumptions are faulty. The ability to ape reality at such high fidelity creates compelling and radical possibilities of its own.

I discussed some of these possibilities with Paul Franklin, cofounder of the SFX company DNEG, who won Oscars for his work on

Christopher Nolan’s sci-fi blockbusters Interstellar (2014) and Inception (2010). Franklin says the digital technologies appearing on film sets in the past decade – from lighter cameras and cooler lights to 3D printed props and LED frontprojection screens – are positively disrupting the way films are made. They are making film sets creative spaces once again, and giving the director and camera crew more opportunities for on-the-fly creative decision-making. “We used a front-projection screen on the film Interstellar, so that the actors could see what visual effects they were supposed to be responding to,” he remembers. “The actors loved being able to see the supermassive black hole they were supposed to be hurtling towards. Then we realised that we could capture an image of the rotating black hole’s disc reflecting in Matthew McConaughey’s helmet: now that’s not the sort of shot you plan.”

Now those projection screens are interactive. Franklin explains: “Say I’m looking down a big corridor. As I move the camera across the screen, instead of it flattening off and giving away the fact that it’s actually just a scenic backing, the corridor moves with the correct perspective, creating the illusion of a huge volume of space beyond the screen itself.”

Effects can be added to a shot in real-time, and in full view of cast and crew. More to the point, what the director sees through their viewfinder is what the audience gets. This encourages the sort of disciplined and creative filmmaking Méliès and Chaplin would recognise, and spells an end to the deplorable industry habit of kicking important creative decisions into the long grass of post-production.

What’s taking shape here isn’t a “good enough for TV” reality. This is a “good enough to reveal truths” reality. (Gargantua, the spinning black hole at Interstellar’s climax, was calculated and rendered so meticulously, it ended up in a paper for the journal Classical and Quantum Gravity.) In some settings, digital facsimile is becoming, literally, a replacement reality.

In 2012 the EU high representative Baroness Ashton gave a physical facsimile of the burial chamber of Tutankhamun to the people of Egypt. The digital studio responsible for its creation, Factum Foundation, has been working in the Valley of the Kings since 2001, creating ever-more faithful copies of places that were never meant to be visited. They also print paintings (by Velázquez, by Murillo, by Raphael...) that are indistinguishable from the originals.

From the perspective of this burgeoning replacement reality, much that is currently considered radical in the art world appears no more than a frantic shoring-up of old ideas and exhausted values. Earlier this month Damien Hirst launched The Currency, a physical set of dot paintings, the digitally tokenised images of which can be purchased, traded and exchanged for the real paintings.

Eventually the purchaser has to choose whether to retain the token or trade it in for the physical picture. They can’t own both. This, says Hirst, is supposed to challenge the concept of value through money and art. Every participant is confronted with their perception of value and how it influences their decision.

But hang on: doesn’t money already do this? Isn’t this what money actually is?

It can be no accident that nonfungible tokens, which make bits of the internet ownable, have emerged even as the same digital technologies are actually erasing the value of provenance in the real world. There is nothing sillier, or more dated looking, than the Neues Museum’s scan of its iconic bust of Nefertiti, released free to the public after a complex three-year legal battle. It comes complete with a copyright license in the bottom of the bust itself – a copyright claim to the scan of a 3,000-yearold sculpture created 3,000 miles away.

Digital technologies will not destroy art, but they will erode and ultimately extinguish the value of an artwork’s physical provenance. Once facsimiles become indistinguishable from originals, then originals will be considered mere “first editions”.

Of course literature has thrived for many centuries in such an environment; why should the same environment damage art? That would happen only if art had somehow already been reduced to a mere vehicle for financial speculation. As if!

If copies are identical to the original work, the original becomes merely a ‘first edition’

g Giant leap: Pixar’s Toy Story (1995) was the first featurelength computer animation