This fore­cast brought to you by math

Popular Science - - FEATURES - By An­drew Blum

How com­puter-driven weather pre­dic­tion can help us bet­ter pre­pare for ex­treme storms.

ON A LATE WIN­TER AF­TER­NOON, IN A WHITE and glass of­fice build­ing 30 miles north of Bos­ton, Jim Lidr­bauch, a soft­ware en­gi­neer at the Weather Com­pany, plugs his lap­top into a gi­ant mon­i­tor at the head of a con­fer­ence table and fires up the pro­gram they call HOTL—for “hu­man over the loop.” It’s half Google Earth, half time ma­chine. A red slider at the bot­tom of the screen lets me­te­o­rol­o­gists fast-for­ward the weather like a movie. As Lidr­bauch demon­strates, the map be­comes pocked with poly­gons, as if a 5-year-old went nuts with a draw­ing tool. Each shape rep­re­sents a change made by a hu­man.

Since 2015, the Weather Com­pany’s computerized sys­tem has run au­tonomously, gen­er­at­ing weather fore­casts on de­mand some 25 bil­lion times a day for peo­ple around the world. The poly­gons rep­re­sent the me­te­o­rol­o­gists’ last ves­tige of ac­tive in­volve­ment. One spec­i­fies “no ice,” telling the fore­cast­ing en­gine to out­put ei­ther rain or snow, but not sleet. An­other blob, over Ge­or­gia, is the work of some­one in At­lanta. It in­structs the sys­tem to in­crease the cloud cover by 5 per­cent and de­crease the tem­per­a­ture by 1 de­gree. “A tweak,” Lidr­bauch says.

At the far end of the table, Lidr­bauch’s boss, Peter Neil­ley, squints over the top of his lap­top. “We have an ac­tive fil­ter that says de­crease the tem­per­a­ture?” he asks. “Where is that ap­plied to? What pe­riod of time does that cover?”

“This af­ter­noon,” Lidr­bauch says. He clicks around to see who made the change: a me­te­o­rol­o­gist named Juan. Had he looked out the win­dow and not liked the way the sys­tem agreed with the sky? Was he bored?

Neil­ley sighs. “Sub­tract­ing 1 de­gree Fahren­heit from At­lanta for the rest of the af­ter­noon: Was that a good use of time? Hu­mans by na­ture will fill their time with work.” THE WORK OF ME­TE­O­ROL­OGY IS CHANG­ING. A fore­cast­ing of­fice once needed a good view of the sky, but most of the weather you’ll see from inside this build­ing is frozen in plas­tic lo­gos above the door: the iconog­ra­phy of the Weather Com­pany and three of its af­fil­i­ates and di­vi­sions. There’s the Weather Chan­nel; WSI, the brand­ing at­tached to its weather “busi­ness so­lu­tions”; and Weather Un­der­ground, the In­ter­net’s first weather site, which the Weather Chan­nel pur­chased in 2012, right be­fore swap­ping out “Chan­nel” for “Com­pany” in its name.

As when Ap­ple dropped “Com­puter,” the move re­flected a revo­lu­tion in the firm’s busi­ness. We are all more likely than ever to get our weather re­ports from an app or a web­site rather than from a hu­man me­te­o­rol­o­gist on tele­vi­sion. That shift in our con­sump­tion has co­in­cided with a qui­eter one in pre­dic­tion. Since the 1980s, me­te­o­rol­o­gists have reg­u­larly used com­puter mod­els—based on the laws of physics and run on pow­er­ful su­per­com­put­ers by govern­ment agen­cies—that spit out de­tails pro­ject­ing the at­mos­phere of the fu­ture. It used to be left to the me­te­o­rol­o­gists to sift this raw data into weather fore­casts. But over the past decade, the mod­els have im­proved to the point that they are more than mere “guid­ance,” as me­te­o­rol­o­gists like to say. The data they re­turn now is an al­most user-ready fore­cast. The al­go­rithms are tak­ing over. An­other logo above the door punc­tu­ates that fact: IBM, which, in 2016, pur­chased the Weather Com­pany (but not the Weather Chan­nel, which op­er­ates in­de­pen­dently while li­cens­ing the Weather Com­pany’s data).

But al­go­rithms don’t write them­selves. The sys­tem that gen­er­ates the fore­casts has been cre­ated over the past 20 years by a team led by Neil­ley, direc­tor of weather fore­cast­ing sciences and tech­nolo­gies at the Weather Com­pany. He over­sees the de­vel­op­ment of the back-end en­gine, doggedly mak­ing sure that im­prove­ments in the big weather mod­els show up in the weather re­ports we see on our screens. That work serves not only the Weather Chan­nel web­site and app, but also the bil­lions of fore­casts viewed on Google, Ap­ple, Ya­hoo, Face­book, and count­less other web­sites and tele­vi­sion sta­tions.

For a long time that meant bring­ing his hu­man fore­cast­ers the best pos­si­ble data, and leav­ing it to them to push it out to the world. But in July 2015, with­out an­nounce­ment or fan­fare, he ac­ti­vated a new it­er­a­tion of the sys­tem: From then on, the Weather Com­pany’s ma­chine would no longer de­pend on hu­man fore­cast­ers to feed its pre­dic­tions di­rectly to our wid­gets, apps, search re­sults,

and dig­i­tal as­sis­tants. The con­se­quences of that change ex­tend be­yond how the weather re­port is served up each day. It marked the rein­ven­tion of the role of the me­te­o­rol­o­gist.

“I WAS AL­WAYS IN­TER­ESTED IN HOW WE WOULD make a bet­ter fore­cast for to­mor­row,” Neil­ley says, in a con­fer­ence room called Tsunami. As a kid in New Jersey in the 1970s, he fell in love with me­te­o­rol­ogy be­cause he wanted to fore­cast snow so he could go ski­ing. At grad­u­ate school at MIT in the 1980s, Neil­ley had a prag­matic streak. While many of his class­mates pur­sued a more the­o­ret­i­cal un­der­stand­ing of the weather, Neil­ley built his own PC and cus­tom­ized an op­er­at­ing sys­tem to help move the depart­ment’s re­search data from ana­log to dig­i­tal.

His “flap of a but­ter­fly mo­ment,” as he puts it, came in 1997, while he was work­ing as a sci­en­tist at the Na­tional Cen­ter for At­mo­spheric Re­search, in Boul­der, Colorado. A team from the Weather Chan­nel had come look­ing for help. They had bought the do­main but re­al­ized they couldn’t “just have a bunch of TV guys with grease pen­cils pop­u­lat­ing the web­site,” he says. They needed a new sys­tem. Neil­ley, who was work­ing in a lab fo­cused on prac­ti­cal re­search ap­pli­ca­tions, wasn’t in­vited to the meet­ing, but, eaves­drop­ping from his of­fice across the hall, grew ex­as­per­ated enough to burst in and tell them they were think­ing of it all wrong. The sys­tem they wanted to build would turn the meth­ods of hu­man fore­cast­ers from around the world into a pro­gram­mable logic. Neil­ley saw that it wouldn’t scale. The al­ter­na­tive he sketched— which blended out­puts of mul­ti­ple mod­els— got him the job, even­tu­ally bring­ing him here to Mas­sachusetts, and started him on the two-decades-and-count­ing chal­lenge of con­stantly im­prov­ing the re­sults.

For its first 15 years, the fore­cast­ing en­gine worked like a fun­nel. Into the wide end Neil­ley’s team poured a range of in­puts, from real-time ob­ser­va­tions to sta­tis­ti­cally fi­nessed weather-model data. At the nar­row end was a hu­man. A staff me­te­o­rol­o­gist would take the au­to­mat­i­cally gen­er­ated fore­cast as a first guess, make any im­prove­ments they thought nec­es­sary, and send it out to the world. “The hu­man al­ways con­trolled the pub­lish but­ton,” Neil­ley says.

That worked fine for a decade. But by the late aughts, the nar­row end of the fun­nel had stretched un­man­age­ably wide. The mod­els were get­ting more ac­cu­rate, for more days in ad­vance. They were get­ting more pre­cise, with higher spa­cial res­o­lu­tion. And there were more of them to con­sider, with the ad­di­tion of new ones that bet­ter ac­counted for the chaos of the at­mos­phere. The hu­mans sim­ply couldn’t check the com­put­ers’ work quickly enough. Me­te­o­rol­o­gists were adding less to the process. A decade ago, Neil­ley asked his team to stop chang­ing the tem­per­a­ture.

“Look,” he told them, “when you mod­ify the tem­per­a­ture fore­cast, you’re mak­ing it worse as likely as you’re mak­ing it bet­ter. It’s just not a good use of your time!”

Neil­ley was also frus­trated by how much de­tail the hu­man-in-the-loop sys­tem left out. The switch from desk­top to mo­bile meant peo­ple were ac­cess­ing fore­casts more of­ten and from more places. The weather mod­els had the right data, but the hu­man fore­cast­ers couldn’t keep up with it. If Neil­ley could break the log­jam at the end of the process in At­lanta, where most of the Weather Chan­nel’s fore­cast­ers sat, his sys­tem could give more ge­o­graph­i­cal pre­ci­sion (a cooler tem­per­a­ture by the ocean, for ex­am­ple) and do it more fre­quently (pre­dict­ing rain in hour­long in­cre­ments), pro­vid­ing us with up-to-the-minute fore­casts for our ex­act lo­ca­tions.

Neil­ley’s big­gest con­cern was pre­serv­ing “the wis­dom of the fore­caster.” Re­gard­less of the mod­els’ pre­ci­sion, there was still a gap be­tween its met­rics and our re­al­ity. The nu­ance of call­ing rain “show­ers” or “storms” is hard to au­to­mate. The so­lu­tion was to take the fore­cast­ers out of

the end of the loop, where they were a bot­tle­neck, and put them “over the loop”—tun­ing and qual­i­fy­ing the sys­tem’s fore­casts as needed. In ef­fect, they’d be an­other in­put.

“Be­fore, they had to wait un­til the model came out, and then they did their thing and they posted it,” Neil­ley says. “Now the fore­cast is go­ing out whether or not they touch it.”

THOUGH NEIL­LEY IN­SISTS ON CALL­ING THE shift to over the loop an “evo­lu­tion” not a “revo­lu­tion,” it’s re­mark­able none­the­less: Out­side of ex­treme events, the fore­cast­ers are no longer fore­cast­ing. The Weather Com­pany is at the fore­front of this trans­for­ma­tion, but the changes are hap­pen­ing across the field. While the Na­tional Weather Ser­vice has a corps of 2,500 fore­cast­ers tasked, in part, with hand-pub­lish­ing the kinds of fore­casts that the Weather Com­pany man­ages with a staff of 13, it is now test­ing a more au­to­mated sys­tem.

“There’s no ques­tion we’re go­ing to move to a role that’s more about com­mu­ni­ca­tion than ac­tu­ally fig­ur­ing out if, three days from now, it’s go­ing to be 66 or 68 de­grees,” says Ryan Han­ra­han, an on-cam­era me­te­o­rol­o­gist at NBC Con­necti­cut. Yet the recog­ni­tion of this par­a­digm shift in me­te­o­rol­ogy is un­evenly dis­trib­uted. “I think some are in de­nial that com­put­ers can do as good a job as you can,” he says. But that doesn’t mean all the fore­cast­ers are be­ing re­placed by ma­chines. The Weather Com­pany keeps its hu­mans busy, of­fer­ing cus­tom­ized ser­vices to com­pa­nies—such as air­lines and en­ergy traders—af­fected enough by the weather to be will­ing to pay for help judg­ing what its im­pacts will be.

The Na­tional Weather Ser­vice is sim­i­larly shift­ing staff pri­or­i­ties. That means spend­ing more time ex­plain­ing to emer­gency man­agers and public-works of­fi­cials the like­li­hood and sever­ity of the event. “We’re fun­da­men­tally chang­ing where our job ac­tu­ally ends,” says Louis Uc­cellini, the direc­tor of the Na­tional Weather Ser­vice.

Para­dox­i­cally, it’s pre­cisely the im­prove­ments in the au­to­mated fore­casts that make this new em­pha­sis on com­mu­ni­ca­tion im­por­tant. When the fore­cast was wrong half the time, de­ci­sions were harder to make: Flights were can­celed later, schools closed af­ter snow had al­ready fallen. To­day’s fore­casts are ac­tion­able—of­ten sev­eral days in ad­vance. “We can no longer ig­nore how the in­for­ma­tion is com­mu­ni­cated and used,” Neil­ley says.

For Uc­cellini, it means a re­turn to ba­sic prin­ci­ples. “If you look at the Weather Ser­vice mis­sion, the first part is to pro­duce and de­liver ob­ser­va­tions, fore­casts, and warn­ings of weather, wa­ter, and cli­mate,” he says. Then he ref­er­ences the sec­ond part, to “save lives and prop­erty and en­hance the na­tional econ­omy.” That’s more im­por­tant than ever. Whether it comes from a per­son or a ma­chine, a weather fore­cast is only as good as the de­ci­sion you make with it.

An­drew Blum’s book about the weather will be pub­lished in 2018 by Ecco/HarperCollins.

Il­lus­tra­tions by Mike McQuade

Peter Neil­ley

Newspapers in English

Newspapers from USA

© PressReader. All rights reserved.