From The Archives: Digital Pictures
Lodewijk Coen and Mark Klein reveal how they gamified full-motion video
Back in March 1995, in an article penned for Next Generation magazine, Tom Zito expressed his feelings about television. He said it was “no secret to anyone” that he hated the medium but he also acknowledged the irony. That’s because he was the president of Digital Pictures – a developer obsessed with marrying filmed footage and games. And it was he who led the FMV craze of the Nineties thanks to a desire “to talk back to your TV”. Digital Pictures was born in 1991, but Tom’s interest in creating interactive television emerged six years earlier when he was employed at Axlon, one of Atari founders Nolan Bushnell’s growing number of companies.
He was working on the AG Bear, a talking toy that responded to human voice and he liked the interactively. When the project was done, he approached Nolan with a new idea to combine interactive images with a video stream. Nolan gave him the green light to develop a prototype.
Tom gathered a formidable team. There was Rob Fulop, creator of the Atari 2600
classics Missile Command, Demon Attack and Cosmic Ark, David Crane, cofounder of Activision and Pitfall! developer, graphics artist and games designer Michael Becker, and Steve Russell, the American computer scientist who coded Spacewar! in 1962. They examined the Colecovision console’s graphics chip and discovered it could display videogame images over a clear background.
It was the breakthrough they needed, but Nolan baulked at the $7 million Tom estimated to get such a machine ready. Tom sought an alternative financial backer and found a willing partner in the toy company Hasbro. More staff was brought on board and the team worked on the console, codenamed Never Ever Mention Outside (or NEMO), under the watchful eye of Axlon. When Hasbro wanted development to quicken, however, Tom faced a tough choice: to leave Axlon and Nolan or risk losing Hasbro’s financial backing. Tom promptly left and he created a new company called Isix.
NEMO was an interesting console as it utilised VHS tapes rather than cartridges. This allowed computer data to sit alongside video and audio tracks, all of which could be switched to allow for interaction, branching and switching. To show off the new tech, three demos were created: a baseball game called Bottom Of The Ninth Inning, an interactive music video for The Cars’ You Might Think I’m Crazy, and Scene Of The Crime which dabbled with the idea of letting players view the gaming action through a series of surveillance cameras.
Among those who assisted with the creation of Scene Of The Crime was technical wizard Mark Klein. “I was hired by Tom as a consultant to design the operating system for NEMO,” Mark tells us. “And I developed the scripting language INTERVAL that was used to implement the gameplay of Scene Of The Crime. The titles were unique as the first appearance of interactive video at the frame-by-frame granularity. We were pioneering a genre.”
Scene Of The Crime was the highlight of Tom’s successful NEMO pitch to 22 executives at Hasbro’s headquarters in Pawtucket, Rhode Island in December 1986. Having been granted more funding, Tom looked to take on more staff. His roster would eventually include writer Ken Melville, graphics artist Lodewijk Coen, business development expert Anne Flaut-reed and Kevin Welsh, who Lodewijk dubs an “interactive video and technical wizard”.
Tom also threw himself into two key productions. The first was Night Trap which grew from the concept of Scene Of The Crime. Shot like a Hollywood production in Culver City, California using actors over 16 days in 1987, it cost a staggering $1.5 million. Jaws dropped further when Tom spent $3 million creating Sewer Shark, a first person, on-rails shooter written by Ken Melville. All was going well. “We finished the hardware and software development and the system was almost ready for large-scale manufacturing,” Mark Klein says.
But then, in late 1988, two months before the console was due to be released, Hasbro axed NEMO – or the Control-vision, as it was to be called. Executives feared the $299 price tag would cause it to tank in a market dominated by the cheaper NES. Tom’s team was also hit by the illness of Hasbro’s chief executive officer, Stephen D Hassenfeld, who died in 1989 of pneumonia and cardiac arrest aged 47. Stephen was a big supporter of the NEMO project.
“I was surprised the Control-vision was cancelled,” says Mark. “There had been other milestones that we barely made or barely missed and yet the project had not been cancelled then. Only once we had virtually completed everything was the product shelved.” The Isix team was disbanded and the employees went their separate ways.
Tom bought the rights to the games and stored the assets, including the filmed footage, in a warehouse in Rhode Island. There they languished, gathering dust before discussions got underway with Nintendo in 1991 to port Sewer Shark to its proposed new Cd-based Play Station. Tom created a new company called Smart TV – swiftly changed to Digital Pictures – and he approached Mark, Ken, Lodewijk, Anne and Kevin to help him.
“I think Sony was interested in using its massive entertainment business assets in interactive projects, especially music videos, which were still huge in those days of MTV,” recalls Lodewijk. “The hybrid nature of the medium – of watching a movie passively and playing a game interactively – posed some challenges but we really did some pioneering work in game design.”
Unfortunately, the plans hit the buffers when Nintendo fell out with Sony and the Nintendo Play Station was scrapped. But, as luck would have it, Sega was working with CD-ROM for a Mega Drive add-on, convinced the format was the future of gaming media. Needing unique content, Sega and Tom began talking. An agreement to port the unreleased FMV games to the proposed Mega-cd was struck.
One of the first tasks was figuring out how to work with CD-ROM and this was, as Mark admits, a difficult task. “We developed a video compression algorithm tailored to the needs of full-motion video,” he tells us. “We digitised and compressed all the footage, frame-by-frame but, because CD-ROMS were in their infancy at the time, we faced and surmounted many technical challenges.” One issue was that the Sega CD wasn’t designed to display video (“It was far from high definition,” Mark says). Making life easier, however, was that footage for the two games was already filmed.
“Kevin Welsh did an awesome job rebirthing Night Trap for the Mega-cd,” says Lodewijk. Based on the original concept by Rob Fulop and James Riley, Kevin produced the game along with Ric Lacivita. Meanwhile James directed it and Lodewijk created the computer graphics. Extra footage was shot for the introduction which referenced Sega’s products and the game was dedicated to Stephen Hassenfeld. “We had to juggle the trade-offs relating to the length of video, the quality of the image, the user interface, audio and gameplay on this tiny CD and that was the main challenge we were facing,” says Lodewijk. “Initially, we basically blew our competitors out of the water with that technique.”
Sewer Shark was also ported. Produced by Joanne Michels-bennet and Amanda Lathroum it picked up on Melville’s work and Fulop’s original concept. “Amanda Lathroum was a Harvard PHD and she started out as our receptionist when Digital Pictures was based on Hamilton Avenue in Palo Alto,” recalls Lodewijk, whose own team was responsible for the graphical user interface of the games. “She’s incredibly smart and she
was the one who rebirthed Sewer Shark, working day and night to redesign the game on a new platform from raw movie footage. The video had to be converted to a very limited color palette, resulting in heavy pixels and banding. I think a palette of 16 colours was set aside for the UI and 200 or so for the video.”
Both games sold well and Sewer Shark, with
100,000 sales, was bundled with the Mega CD shortly after its North American release in October 1992. It was a mega triumph, grossing about $18 million at retail, and a vindication of Tom’s determination to make a success of FMV. Yet trouble wasn’t too far away.
Night Trap caught the attention of US Senators. The game was deemed to be ultraviolent: “In the scene played at the news conference, the attackers get their screaming victim and attach the blood-draining device to her neck with a high-pitched drilling noise,” a newspaper reported on 2 December 1993.
Senator Joseph Lierberman led the issue. “We’re not talking Pac-man or Space Invaders any more,” he said, adding that he would, if possible, ban the likes of Night Trap and Mortal Kombat from being sold. Yet Night Trap wasn’t all that violent and neither did it contain sexual content and nudity despite such claims. “It was political grandstanding,” says Mark. “Most of us thought they were comical – the game and the politicians who called it out as an example of a violent videogame.”
Even so, retailers such as Toys ‘R’ Us and the Kaybee chain took it off the shelves and Sega decided to
the video had to be converted to a very limited color palette, resulting in heavy pixels and banding Lodewijk Coen
pull the game itself in January 1994. The game was later rereleased with new box art and it was also ported to the Sega CD 32X, 3DO, PC and Mac. But the furore had led to the creation of the Entertainment Software Rating Board in September 1994 and a mass realisation that games were not always aimed at children.
In the meantime, Digital Pictures had ploughed on developing new titles, among them a gaming series called Make My Video which let players take a song by INXS, Kris Kross, or Marky Mark and the Funky Bunch and produce a new music video for it using preshot clips and ‘snazzy’ special effects. None of the games sold or indeed reviewed very well, leaving Digital Pictures out of pocket. Lodewijk believes the problem was down
to the limitation of the medium. “Too low-res video, too low-res interface, an editing timeline with tiny video screens, low-res audio – it was a tour de force that we even did this,” he says.
Better was Digital Pictures’ second wave of games, among them Ground Zero: Texas and Double Switch. Flush with cash from Night Trap and Sewer Shark
(“Our funding came from a combination of revenues and investment,” reveals Mark), Digital Pictures spent $2 million making Ground Zero: Texas, using a full Hollywood film crew (the game also starred Steve Eastin, who had appeared in many TV shows including The A-team and TJ Hooker). “Producing video, including hiring talent, was a major component of the cost of our games,” Mark says.
Despite the large investments needed, the company was growing fast. It moved to larger premises on Page Mill Road,palo Alto. “We also worked on Sand Hill Road in Menlo Park, in the Quadrus campus, surrounded by beautiful nature in the foothills and amazing art,” Lodewijk adds. Digital Pictures had hired around 80 people at its peak in 1993. “We kept on growing, developing new concepts and titles,” Lodewijk says.
The company approached its games in much the same way. “Usually we’d start with off-site meetings – gatherings in nice remote settings that were conducive to creativity,” Lodewijk remembers. “Everyone from the receptionist to chairman would actively participate and we’d filter the best concepts in a democratic way.” They would then create concept boards (“I sometimes hired story board artists especially for those jobs,”) and these would be laminated “after they were colour printed on incredibly slow inkjet printers”.
“Sometimes movie snippets were shot, anything to make a concept as clear as possible,” Lodewijk continues. “They were then either taken to possible investors or tested on future audiences via focus groups.” Some of this was done in-house. “We had great talent on staff like Cuyler Gee who I collaborated with for so many years,” Lodewijk remembers. Steve Russell also took a job at Digital Pictures. “He had a great game sense and he was a good development manager,” Mark explains.
That said, some of the games retrod themes from Digital Pictures’ games. Double Switch was very similar to Night Trap, for instance, having players study cameras to avoid or set traps (I briefly met Deborah Harry on the set,” Lodewijk says). But others were rather unique. “Prize Fighter was revolutionary,” argues Mark of a black and white FMV game that had gamers playing as The Kid from a first-person perspective, fighting in the ring against various opponents.
“I just remember focusing on how the graphic overlays, sprites and interface would work with the filming to get a ‘natural’ effect,” says Lodewijk. “For Prize Fighter, I had to go to the movie set in order to help film the boxing gloves for the first person pointof-view boxing moves. Those were shot separately on a green screen, to be extracted later and turned in to sprite overlays for the POV fighting scenes. The idea was for the player to feel that he was really fighting the opponent which was different to, say, Street Fighter where you fought sideways.”
» [Mega-cd] Midnight Raiders was as Nineties Hollywood as it could be, complete with exploding choppers.
» [Mega-cd] The key to beating Prize Fighter was pressing the right button at the right time.
» Stick the Mega-cd disc of a Digital Pictures track two to hear a recording game into an audio player and select of a zombie-like chant. Then play it backwards.
» [Mega-cd] The violence in Night Trap was not as bad as some people made out.
demo of Scene Of executives watching the NEMO » This video clip of Hasbro Trap as an Easter egg. The Crime appeared in Night
» [Mega-cd] Sewer Shark was a rail shooter that was bundled with the Mega-cd and later ported to the 3DO.
» [Saturn] Jeremiah Birkett played Winston in Corpse Killer. in movies including LA Confidential. He’s since starred
» [Mega-cd] The writing and acting in Night Trap was rather cheesy but the game was innovative to a point.