Better Shots
Clippers owner Steve Ballmer and Second Spectrum want to change the way we watch basketball when it's back
STEVE BALLMER PRESSED THE HEELS OF HIS HANDS AGAINST HIS THIGHS AND LEANED FORWARD. HE WAS SITTING IN HIS
customary spot at Staples Center, along the baseline in the arena’s north end. Most NBA owners sit at mid-court, and dress like the corporate executives with billiondollar fortunes that they are. Ballmer was wearing a plaid shirt and casual pants. “I can’t wear a sport coat to a basketball game,” he’d said a few minutes earlier.
In front of him, L.A. Clippers forward Paul George dribbled without moving. Then he passed the ball along the perimeter. Eventually it reached teammate Kawhi Leonard, who was waiting at the top of the circle, behind the three-point line. Leonard, who last season led the Toronto Raptors to the NBA title, flicked a shot with a flattened arc that swished through the hoop. Ballmer gave a quick twitch of satisfaction and sat back in his chair. But soon he was leaning forward again, arms tensed.
The Clippers’ opponents that February evening were the San Antonio Spurs, who have won five championships in the past 21 seasons. In the Clippers’ 50year history, spread across three cities, they’ve never even appeared in a final. When Ballmer bought them six years ago, they were widely regarded as the worst franchise in sports.
A few weeks later, the NBA would suspend its season to help mitigate the spread of COVID-19. But it had already become apparent that the Clippers and Spurs had switched places. For the first time in a generation, the Spurs were losing more than they were winning. They seemed almost certain to miss the playoffs. And the Clippers, who on a single day last summer traded five first-round draft picks to San Antonio for George and then signed Leonard to a three-year, $103 million contract, had emerged as one of the NBA’S best teams.
The Clippers’ evolution into a championship contender ranks as Ballmer’s most visible achievement since leaving Microsoft, where he served as CEO from 2000 to early 2014. But what could become his greatest contribution to sports is run out of a nondescript suite of offices in L.A.’S Little Tokyo neighborhood. While Ballmer watched his team rally to beat the Spurs, two employees of Second Spectrum, a company that he advises and owns a significant stake in, were laying the foundation to change the way sports is consumed on television.
They didn’t seem to be doing much of anything, actually, other than glancing occasionally at the video screens in front of them. The technology they were monitoring, which relies on artificial intelligence, is almost entirely automated. “Someone has to push the ‘Go’ button,” explained cofounder and CEO Rajiv Maheswaran.
Second Spectrum has the official contract to supply the NBA with player-tracking statistics based on spatial data that the company culls from video cameras planted throughout the league’s arenas. (Recently, it started doing the same for Major League Soccer and England’s Premier League.) In addition, the company creates and sells proprietary metrics to the vast majority of the NBA’S
teams. But its partnership with Ballmer and the Clippers has resulted in a transformative new technology.
Most of the 18 screens mounted on the wall of Second Spectrum’s small control room were showing versions of that same Clippers game. In one of them, the Xs and Os of the offensive plays that the teams were running were superimposed over the action. On another, the chance that each offensive player had of making a shot, recalculated in real time, was displayed beneath him as he moved around the court. Whenever a player on either team scored or grabbed a rebound, his updated statistics flashed on the screen. A third variation overlaid plays with explosions, shaking baskets, and licks of fire, as in a video game. A fourth combined elements of the first three. It was as if a producer were able to see a minute or two into the future and continuously layer in relevant graphic elements as the game unfolded. No live producer is capable of that, of course. “But
AI is,” Maheswaran said.
Since the start of the 2018–19 season, a group of Clippers fans has been able to toggle between these views in Second Spectrum’s Courtvision app as they stream games on their smartphones, tablets, and laptops. The number of users—1,500—was specified by the agreement the
Clippers made with Fox Sports
Prime Ticket, the regional sports network that owns the rights to games in the Los Angeles area until 2022. “We said, ‘Give us unlimited rights,’ ” Ballmer says.
“They said, ‘B.S.’ So we negotiated ourselves to this position.”
The same Courtvision technology is also incorporated into a limited number of games broadcast on the Fox Sports app. Occasionally, it appears on national networks. In March 2019, it debuted on ESPN with a Lakers-bucks game and a panel of talking heads—literally disembodied heads, floating just above the crawl—adding commentary.
The technology has come at a propitious time. Before play was suspended in March, NBA ratings were down roughly 12% across all networks from last season. Other sports are experiencing similar declines. The vast array of entertainment options available at the touch of a screen has promoted a more casual consumption of sports events: a few minutes now and then, maybe a highlight package on demand. “There’s no question that consumption patterns among young people are changing,” says
Doug Perlman, founder and CEO of Sports Media Advisors, a consultancy that works with pro franchises and leagues. “They’re used to a high degree of personalization.” That’s something traditional telecasts simply don’t deliver. With sports potentially suspended into the summer, even devoted fans may find themselves seduced by competing options—and spending potentially even less time watching their favorite teams when they do return. That will ratchet up the pressure on the leagues to create more enticing broadcast experiences.
Like every content provider, the NBA sees its future on phones and tablets. It would surprise nobody if Amazon Prime and Netflix, and perhaps even Twitter and Instagram, participate in the bidding for the next round of national NBA rights when the current nine-year deals expire in 2023. NBA commissioner Adam Silver is looking even further ahead. “The next generation not only expects the product to conform to their schedule,” he says. “They want to mold the product to conform to what it is they want to consume.”
In that sense, Courtvision isn’t just an enticing new way to watch a basketball game. It actually may be instrumental in the sport’s survival.
ALLMER BOUGHT
Bthe Clippers in 2014 in what should have been a fire sale. Donald Sterling, who had owned the team since 1981, had been caught on tape making disparaging remarks about African Americans. Silver had become the NBA’S commissioner only a few weeks before. In a bold move, he insisted that Sterling sell the team.
But Ballmer didn’t get a fire-sale price. His $2 billion investment was nearly four times as much as anyone had ever paid for an NBA franchise. It was almost as much as the $2.15 billion baseball’s Los Angeles Dodgers had sold for in 2012—and that deal included Dodger Stadium and the right to develop nearby parking lots. But Ballmer had missed the chance to buy the Sacramento Kings in 2013 and move them to Seattle. He wasn’t going to blow this one. As it happens, Ballmer is exactly the kind of multibillionaire that the NBA wants to own its teams. Half of the NFL’S owners inherited their franchises from relatives, but today’s NBA is filled with highachieving innovators, including venture capitalist Joe Lacob of the Golden State Warriors; the Houston Rockets’ Tilman Fertitta, who made a billion dollars in restaurants and another billion in casinos; and Alibaba cofounder Joe Tsai, of the Brooklyn Nets. Silver is tapping their skills to keep the league relevant as the entertainment landscape evolves. Ballmer is a vital addition. “His voice carries enormous respect in the room,” Silver says.
During Ballmer’s 34-year career at Microsoft, a large part of his job was to take stock of the ideas being generated around him and
which to fund. He often cared less about the immediate return of a particular innovation—such as Xbox, which Microsoft developed during his tenure, or Skype, which it bought from ebay in 2011 for $8.5 billion—than its long-term prospects. “What Steve loves is big ideas,” says Lisa Brummel, a former Microsoft executive vice president who left the company in 2014 and is now part owner of the WNBA’S Seattle Storm. “If you’re not thinking big enough, that’s almost as bad as not having the proper analytical thought behind your idea. And if you are thinking big, think even bigger.”
Ballmer likes to say that he’s retired. What he means is, he no longer works for Microsoft, so he can spend time doing what he wants. That includes wading deep into the tall grass of running a sports team. When Ballmer took over the Clippers, in 2014, Second Spectrum had recently started selling its proprietary enhancements of player-tracking data to several teams, including the Clippers. Ballmer requested a login to the software. “I’m a tech guy,” he says. “I thought, Okay, I’ll take a look at this analytic stuff.” Maheswaran learned of Ballmer’s interest and reached out to offer a demonstration of how the data is compiled. “I never thought I’d hear back,” Maheswaran says. But a few weeks later, Ballmer sent a note: He was in L.A. and wanted to come by.
Ballmer was impressed by the presentation. Beyond that, he was impressed with Maheswaran and his team. “He basically said, ‘What are you guys about?’ ” Maheswaran says. “‘What do you want to do in the world?’ ”
Maheswaran had an answer. In a follow-up meeting, he explained to Ballmer that apart from instant replays and graphic adornments such as first-down lines, the experience of watching sports hadn’t materially changed since the first live event was televised, the 1936 Berlin Olympics. Sure, sports viewership has advanced to digital channels. But unlike other forms of content— music, television, podcasts—those channels aren’t personalized or responsive. Watching games on television, Maheswaran inevitably had questions: Who is that player? Is he playing well? Was that a good shot he’d just taken? He fantasized about teaching a machine to figure out how to answer those questions, then show the results on a screen as the game unfolded. Theoretically, at least, he believed such a feat was possible: “People should basically be able to create their own channels. They should be able to visualize a game in any way they want.”
Though Ballmer famously missed out on smartphones—as Microsoft CEO, he laughed off the iphone’s launch in 2007—he has a better track record with video innovation. Shortly before leaving Microsoft, he’d worked a $400 million deal with the NFL that made the Surface tablets the league’s official sideline technology sponsor. That partnership, which was renewed and expanded in March, was supposed to start Microsoft down the road toward interactive in-game programming, though Ballmer left before he could begin that process.
When he heard Maheswaran’s pitch, Ballmer was smitten. He agreed to buy 8% of the company and meet with Maheswaran and Second Spectrum executives monthly to offer what help he could. And if Maheswaran and his partners believed that the technology existed to do what they were proposing in real time, he told them, they should figure out how to do it immediately, even if it wasn’t clear how they’d make money on it.
AHESWARAN’S FATHER,
MMurugesapillai, left Sri Lanka to study astrophysics at Cambridge alongside Stephen Hawking. He returned home, but when civil war broke out in the mid-1980s, he immigrated with his family to the U.S. He eventually landed at the University of Wisconsin– Stevens Point at Wausau. He taught math and spent his free time watching sports on TV. “He went from rugby and cricket,” Maheswaran says, “to basketball and football.”
Maheswaran watched with him. He enjoyed sports because of the competition, but also because of the numbers that explained how well the players were performing. He graduated from the University of Wisconsin–madison, majoring in applied math, engidecide
neering, and physics, then earned a master’s and doctorate in electrical and computer engineering from the University of Illinois. By the time he landed a job in the computer science department at the University of Southern California—specializing in spatiotemporal pattern recognition, the science of moving dots— Michael Lewis had written about Billy Beane in Moneyball and the sports analytics revolution had begun.
Soon after, the NBA hired a company to mount sensors inside its arenas and track where players moved during every game. Instantly, the capability existed to create an exponential number of new metrics from the raw information, though very few teams were actually doing that. “Tracking data was like grain,” Maheswaran says. “Instead of foraging for fruits and nuts, someone gives you grain. ‘Here, this will change your life! You make stuff out of it!’ But nobody knew how to do it. So what do you do? You put it in a closet.”
What were basketball players on a court if not moving dots? In 2011, Maheswaran attended the annual MIT Sloan Sports Analytics Conference, in Boston, and managed to procure some of the tracking data. Then he and Yu-han Chang, a colleague of his at USC, applied their work to basketball. They were able to determine the correlation between a shooter’s position on the court and the likelihood of his team getting an offensive rebound. “We discovered something about the sport that no one else knew,” Maheswaran says. At the following year’s conference, they won the award for the best paper. It earned them enough attention that they decided to start a company on the side.
Second Spectrum launched its first product in 2013: a software platform that used machine learning to translate the tracking data into insights relevant to NBA coaches and executives. In effect, it
was teaching them how to cook the grain. The software could identify patterns, such as pick and rolls or flare screens. It could also create new metrics, such as an assessment of the quality of a shot. The idea was that teams would give the company their data and it would send them back information they didn’t already know. That summer, Dallas Mavericks owner Mark Cuban sent Second Spectrum its first check.
As Second Spectrum grew, Maheswaran filled as many positions as possible with former athletes. About a dozen were former MIT basketball players, a group that includes captains or co-captains from the past four seasons. Maheswaran dispatched the basketball players to the NBA’S summer league in Las Vegas, where they sought out coaches and general managers. They were trying to find out what would be useful for someone inside the sport to know. “If they said, ‘We’d like to see information on off-ball runs,’ we’d go learn how to do that,” Maheswaran says. They usually needed no more than two iterations for their software to describe the activity in any piece of video as well as the experts they were consulting could.
In 2016, Second Spectrum won the NBA’S contract as its official provider of tracking data. By then, Maheswaran was immersed in the alpha version of what eventually would become Courtvision. The program accessed the data to identify where the ball and the players were located at all times. It taught the software to recognize patterns, and matched those patterns with the game’s particular nomenclature—all those weak-side traps and off-ball runs. It used mathematical monitoring to continuously calculate shot probabilities. And it empowered the AI to choose what content to show on a screen at any given time—when to display how many points a certain player has, for example, or when to set a net on fire. “Literally for every play that happens, it knows, like, 100 things,” Maheswaran explains. “It picks some subset of them and thinks, This is the stuff I’m going to draw, this is the stuff I’m going to put on text.”
Clippers Courtvision launched to its select group of subscribers in the fall of 2018. Ballmer likes to say that no product ever truly works until its third iteration, but the initial augmented broadcast that Second Spectrum sent out into the world looked a whole lot like the fantasy Maheswaran had pitched to Ballmer during that 2014 presentation. The problem is, at least until the NBA’S contracts with broadcasters expire, only 1,500 fans have been allowed to access it. “The way rights are chopped up,” Ballmer says, “nobody has an economic incentive to put this in the market.” But he remains committed, especially since the recent upheavals to the NBA season have offered a glimpse of a virtual reality future in which live attendance at games becomes only nominally relevant.
Maheswaran, too, has his sights set further down the road. “We’re more concerned about, ‘What are the right environments for this to progress the fastest?’ ” he says. The answer isn’t necessarily to work with the most teams and make the most money. Instead, he has moved into soccer. He has tracked data for the Premier League this current season and will debut augmentations of that data for Major League Soccer when that season resumes. And once soccer and basketball, the most popular global sports, are covered, Second Spectrum can start picking off regional sports with high levels of passionate interest. The NFL? Cricket? Australian Rules Football? Why not?
FOR ALL OF THE CLIPPERS’ RECENT
success, their games at Staples Center still don’t have the energy of Lakers games, which are played in the same arena. The A-list celebrities largely stay home for the Clippers, other than Billy Crystal, who has been a loyalist since the terrible years. Empty seats are too easy to spot.
Ballmer expresses frustration that the culture change he has tried to implement hasn’t happened rapidly enough. Microsoft had been geared toward inventing not just new products but entire categories, but the Clippers are merely one outpost of a 30-team enterprise. They’re locked into long-term contracts with players, sponsors, and content providers, like the one that limits the distribution of Courtvision.
By design, NBA games feel interchangeable, whether they’re played in Staples Center, Madison Square Garden, or Salt Lake City. The league has instituted best practices in everything from the pregame warm-up schedule to the volume of the simulated clapping on the video board. “We have very little latitude,” Ballmer says. “We have a playlist of 50 songs that we can use during games, for example. That’s it.”
The Clippers are in the process of designing a new arena near the site of the old Hollywood Park racetrack. Ballmer is hoping to subtly incorporate some new technologies, perhaps including Courtvision, through a nontraditional scoreboard. He’s also pushing new ideas through the NBA pipeline. This includes heading the league’s effort to implement some form of Courtvision leaguewide as soon as the current broadcast agreements expire. “Steve is one of the leaders,” Silver says. “I’m learning an enormous amount from him.”
One morning last September, three Turner Broadcasting executives sat at the table in the Second Spectrum conference room. Several more were connected by videophone. So were representatives of Amazon Web Services (AWS), which supports Courtvision’s machine-learning technology. The purpose of the meeting was to brainstorm ways that TNT might use Courtvision with AWS as a sponsor.
Maheswaran drew boxes representing AWS, TNT, the NBA, and Second Spectrum on a virtual whiteboard. Then he drew lines connecting them. It seemed simple: AWS wanted to tell its story to NBA fans, Turner could deliver the league and its audience, Second Spectrum had the technology.
But what Maheswaran actually had to offer was real estate. The explosion that fills the screen on Mascot Mode when someone slams home a dunk? That can be branded. So can the shooting stats on Player Mode and even the complex play diagrams on Coach Mode. Star players could eventually have their own unique features that showed up when they hit a three-pointer, say, or blocked a shot. Individual broadcasters could be given a signature graphic that accompanied a certain type of play. And the sponsorships for all of them were up for sale.
The meeting ended with an agreement to keep talking. In the months that followed, Turner lost more than 10% of its game audience from last season. It started experimenting with subscription-based streaming to reach fans who no longer have cable packages—laying the foundation, perhaps, for Maheswaran’s vision.
“Turner and ESPN and everyone else do a fantastic job with a linear broadcast,” Maheswaran said after the room had emptied. “There’s not much more juice to squeeze from that orange.” Like Silver, he believes that a new approach to televised sports is not only possible but necessary. Eventually, he predicted, Courtvision will involve touchscreen technology: Viewers could get whatever information they wanted by simply hovering a finger over a particular player.
“Once you have the ability to touch the screen and find out stuff—who’s that guy? how’s he doing?” Maheswaran said, “you won’t want to watch sports any other way.”
Insightful owners have always realized that sports teams were competing against movie theaters and rock concerts and even the family Monopoly night, not other teams. But as all forms of entertainment have started to coalesce on a single device, the terms of that competition have become clear. “Everyone’s fighting for your attention to be part of their content universe,” Maheswaran says. “The NBA is asking you to be part of its content universe. Beyoncé is asking you. Fortnite is asking you. And when everyone can access everything, you need to bring better stuff.”
He paused to emphasize what would come next. “This is it,” he proclaimed, jabbing a finger at the empty whiteboard. “This is the better stuff.”