Len Testa from Tour­ing­plans.com

WDW Magazine - - Content - IN­TER­VIEW BY CARL TRENT

Carl - Let's start off with, tell me just a lit­tle about your­self, your back­ground and how you got started with Tour­ing­plans.

Len - I'm Len Testa, I am the pres­i­dent of Tour­ing­plans.com and also co-au­thored the unofficial guides for Walt Dis­ney World, the Dis­ney Cruise Line, Wash­ing­ton D.C., and I con­trib­ute to Dis­ney­land and Uni­verse of Or­lando as well. I got started, and this is ac­tu­ally a great tech­nol­ogy story. Back in the 90s, mid 90s, right be­fore I started grad­u­ate school I went to Walt Dis­ney

World with my twin sis­ter. We waited in line for 2 hours at the Great Movie Ride. Some­where in that two hours, Carl, be­fore I passed from heat stroke I thought to my­self, "You know, there's got to be a bet­ter way to see these rides". While I was in line I came up with the idea of a com­puter pro­gram that would tell you the or­der in which you should ride the rides to min­i­mize your wait in line. So, es­sen­tially a guide that walks you around the park and told you where to go next. So, I went back to my school, I started school in the fall.

I talked to my the­sis ad­vi­sors about this and I said, "You know, I think I want to study this." They had two ques­tions, and they ac­tu­ally sent me to an ac­tual li­brary, you re­mem­ber li­braries, Carl?

Carl - (Laugh­ing) Yeah, with books.

Len - Ac­tu­ally I used mi­cro­fiche, which, kids to­day, I pretty would be speak­ing French. They had two ques­tions. One, is it, was it hard enough, like was this ques­tion that a stu­dent's go­ing to study was it hard enough to be a the­sis topic? Then two, and this is the funny ques­tion, does any­one but you care about this? So, it turns out that build­ing a sched­ule for Walt Dis­ney World is one of the clas­sic prob­lems in com­puter sci­ence; even has a name, it's called a trav­el­ing sales­man prob­lem. For your read­ers or lis­ten­ers who are tech­nol­ogy minded it's in this class of prob­lems that's called Np-com­plete, and it means that there is, there's no re­ally ef­fi­cient way to solve those kinds of prob­lems with com­put­ers for very big in­stances of it.

So let me give you an ex­am­ple. In the Unofficial Guide to Walt Dis­ney World, I think our stan­dard 1-day tour­ing plan for the Magic King­dom has like 25 rides in it. If you were to ask a com­puter for the ex­act so­lu­tion, the per­fect route for those 25 rides it would prob­a­bly take some­where around 500 mil­lion years for the com­puter to fig­ure that out. The prob­lem is this, there are 25 ways to see the first ride, there's 24 ways to see the sec­ond ride, there's 23 ways

to see the next ride, 22 ways and so on. So, it's 25 times 24 times 23 times 22 and so on pos­si­ble tour­ing plans for that one day in the Magic King­dom and no com­puter is go­ing to be able to solve that. I think it's like 155 sep­til­lion com­bi­na­tions, I mean just a big num­ber.

So, the way that you solve those prob­lems is us­ing these things called heuris­tics or rules of thumb and some of them have been de­vel­oped over, you know, the last 40 or 50 years or so and I still hate those. In­ter­est­ingly, here's the in­ter­est­ing thing about it, the prob­lem of min­i­miz­ing your wait in line at Dis­ney World is ex­actly the same prob­lem that com­pa­nies like Fed-ex and UPS face ev­ery­day when they're try­ing to min­i­mize the cost of de­liv­er­ing pack­ages.

So, imag­ine you're a driver for UPS or Fed-ex, the next place that you visit isn't nec­es­sar­ily the one that's go­ing to be clos­est to you, it's the one with the least amount of travel time. What that means is traf­fic is im­por­tant right? We all know this; you go on the in­ter­state at 5 o'clock in the morn­ing it's vastly dif­fer­ent than at 5 o'clock in the evening right? Yes?

Carl - Yeah, ab­so­lutely, oh sure.

Len - Yeah, so travel time is what's im­por­tant there and that's what UPS and Fed-ex seek to min­i­mize. In fact, my mas­ter's the­sis ex­panded on the PHD dis­ser­ta­tion of UPS'S lead re­search sci­en­tist, that tells you how close these 2 prob­lems are. So, dur­ing grad­u­ate school I worked on this and I emailed, ac­tu­ally I wrote Bob Sehlinger an ac­tual let­ter, like on pa­per with a stamp. And, I said, "Hey I know you wrote this book and you've got these tour­ing plans, and I'm look­ing at this prob­lem too, do you have any data that I could use?" It turns out he was look­ing at the prob­lem a com­pletely dif­fer­ent way. He was work­ing with some guys at M.I.T. to solve the prob­lem, and he was ap­proach­ing it from the point of view of some­body run­ning a theme park. I ap­proached it from the point of view of a customer in a theme park where you have vastly less in­for­ma­tion like Bob needed to know things like how many boats were run­ning at Splash Moun­tain and stuff like that and I just looked at the wait times which is a huge sim­pli­fi­ca­tion.

So it turns out we couldn't share the data but Bob was pro­vid­ing tips along the way of things like how to han­dle restau­rants, how to han­dle things like Tom Sawyer Is­land, how to han­dle things like, do you re­mem­ber the Di­a­mond Horse­shoe Sa­loon used to serve lunch?

Carl - Yes.

Len - So, if you wanted to eat lunch and see the show the most ef­fi­cient way to do that was to visit the Di­a­mond Horse­shoe Sa­loon at noon right?

So he'd pro­vide tips like that. So, I get to grad­u­ate school, the uni­ver­sity patents my re­search,

ob­vi­ously I get through grad­u­ate school, I pub­lish my the­sis and we take the soft­ware and we con­tinue to build on it for like 3 more years. So I grad­u­ate in 2000, in 2003 the soft­ware was ac­tu­ally good enough to start re­plac­ing the tour­ing plans in the book and we've been de­vel­op­ing the tech­nol­ogy ever since then. So in 2008 we pro­vided cus­tom tour­ing plans to peo­ple on­line. In 2011 we launched the app and then we al­lowed peo­ple to op­ti­mize their tour­ing plans while they were in the park. This year we did, we started do­ing crazy things like we'll au­to­mat­i­cally de­tect when rides are down and we'll au­to­mat­i­cally re-route you around those and all kinds of other sup­port for things like the morn­ing magic hours now, the af­ter hours events, all kinds of, the evening sa­fari stuff is now a spe­cial fea­ture.

That's all the tech­nol­ogy go­ing on be­hind the scenes, and be­hind the scenes we're us­ing Ama­zon's cloud.

So this is tech­nol­ogy, so ev­ery­thing's run­ning on Ama­zon's cloud, the en­tire web­site, all of the op­ti­miza­tion stuff hap­pens there and I'm a huge fan of Ama­zon's cloud servers, we use it ex­ten­sively.

Carl - Yeah, they do work al­most all the time, they're re­ally, re­ally good.

Len - So that, that's the in­ter­est­ing thing so there's 2 in­ter­est­ing parts about Ama­zon's cloud. One is you don't know when a server's go­ing to go down so you have to build around that. Are you fa­mil­iar with Ama­zon's queu­ing ser­vices or any­thing like that?

Carl - I am, but go ahead and ex­plain it.

Len - So, mes­sage queues, if you're just us­ing a reg­u­lar web browser and you're com­mu­ni­cat­ing with a server one of the prob­lems with that is that if the com­mu­ni­ca­tion goes down for a sec­ond like if you're in the mid­dle of a call to a web server or if the server's in the mid­dle of de­liv­er­ing

you a re­sponse and for what­ever rea­son the con­nec­tion is broke. Like, you're us­ing your cell phone and you just hap­pen to go out of cell phone range or what­ever. All that com­mu­ni­ca­tion's lost, so in or­der to guar­an­tee the delivery of the tour­ing plans we use these things called mes­sage queues and it's es­sen­tially a pipeline that never goes away even if ei­ther end of the con­nec­tion is bro­ken. What it does is when you make a re­quest for a tour­ing plan we put a re­quest on this mes­sage queue thing and then even­tu­ally a server on Ama­zon reads it and Ama­zon guar­an­tees that mes­sage queues are per­sis­tent and re­li­able. But here's the in­ter­est­ing thing, the way that they guar­an­tee that it's per­sis­tent and re­li­able is by mak­ing mul­ti­ple copies of them, of the mes­sage queue and putting them around the coun­try.

So, whereas you and I look at one mes­sage queue, and by the way this is the most tech­ni­cal con­ver­sa­tion I've had.

Carl - (Laugh­ing) Yeah!

Len - So, you and I look at it as one mes­sage queue but there are ac­tu­ally 6, and the in­ter­est­ing thing about that is when you ask a ques­tion like, "Is there a mes­sage on the queue?" It's only go­ing to send that ques­tion to 1 of the 6 queues and the an­swer you get might be dif­fer­ent if you ask the ques­tion twice be­cause some mes­sage queues might not have all of the mes­sages yet. So you end up with this prob­a­bilis­tic thing where you ask, you ask the mes­sage queue, "Do you have any mes­sages?" "No." Then you ask it again, "Do you have any mes­sages?" "Yes." And you ask it again, "Do you have any mes­sages?" It could say no.

So, you end up with this, we're hav­ing to write code that han­dles not a de­fin­i­tive an­swer but a prob­a­ble an­swer.

And that was fas­ci­nat­ing, that took us like 2 days to fig­ure out. Like, how do you han­dle a prob­a­bilis­tic an­swer of do you have a mes­sage? That was su­per in­ter­est­ing stuff. We fig­ured our way around it and you know, we do load bal­anc­ing across all the servers so ev­ery­thing is ef­fi­cient.

Carl - Just so ev­ery­body un­der­stands, this mes­sage queue thing is hap­pen­ing in tiny frac­tions of a sec­ond.

Len - Oh, mil­lisec­onds… So the time it takes to get a mes­sage, once we get the mes­sage from you, to route it through all the in­fra­struc­ture, to do all the load bal­anc­ing, to get to op­ti­mizer is un­der a tenth of a sec­ond. In fact, from the time that you push the but­ton to op­ti­mize your re­quest usu­ally takes about a minute. Of that minute prob­a­bly half of it is just network la­tency. Like, to get it through all the dif­fer­ent sys­tems, es­pe­cially if you're on your cell phone the vast

ma­jor­ity of that is network la­tency. Carl - Yeah. Okay, enough of the geeky stuff (laugh­ing). Len - (Laugh­ing) It's kind of awe­some though, yeah?

Carl - Geek talk­ing to geek, well we could do this all day, but okay. So, lets talk about your crowd pre­dic­tions. How does that hap­pen?

Len - So, we do this thing where we tell you the or­der in which you should ride the rides to min­i­mize your wait in line, that's our route-plan­ning thing in our lines app. But, in or­der to do that we have make pre­dic­tions about how long you're go­ing to wait in line at every ride so we ac­tu­ally have pro­fes­sional statis­ti­cians. We have a data sci­en­tist, Fred, we have a statis­ti­cian, Steve whose job it is to build mod­els that tell you how long you're go­ing to wait in line at every ride, every day for the next year. So, to tell you the com­mit­ment that we give this, there are lots of web­sites that have crowd coun­ters, we are the only peo­ple that do this sci­en­tif­i­cally. I think ev­ery­thing else, the vast ma­jor­ity of web­sites out there that have crowd coun­ters are do­ing this for like search en­gine op­ti­miza­tion. We spend hun­dreds of thou­sands of dol­lars here on these pre­dic­tions.

So what we do is this, ev­ery­day we col­lect wait times from every ride in every park that Dis­ney op­er­ates, every 5 min­utes.

Carl - Now, how do you do that? Do you do that by look­ing at Dis­ney's app or do you have some­body in a park?

Len - We do 3 dif­fer­ent ways, so one we have staff whose job it is to be in the parks ev­ery­day and our guy in Dis­ney­land for ex­am­ple, his name is Guy and he is tasked with rec­og­niz­ing when he looks at a line whether Dis­ney's app has the right wait time for it. So lets say that you walk into Peter Pan and the posted wait time is 5 min­utes but the line is all the way through the queue, all the way out the door and down Fan­tasy Land. Well some­thing's wrong there right? The wait isn't 5 min­utes so at that point Guy will get in line and start tim­ing his ac­tual wait in line to tell us what is ac­tu­ally cor­rect. So staff, we have peo­ple whose job is to be in the parks ev­ery­day that's their job.

The sec­ond thing that we do is we have users, who through our lines app can tell us both the posted wait time and then their ac­tual wait in line. So we have hun­dreds of thou­sands of users of our lines app and mil­lions of users of our web­site. At any given time, any given day we have be­tween 3 and 5 hun­dred fam­i­lies in the park, gen­er­ally more dur­ing the hol­i­days and they're con­stantly giving us wait times for how long they're wait­ing in the parks.

The third way that we do it is yeah, we look at Dis­ney's wait time app.

So we get any­where be­tween 8 and 10 thou­sand wait times a day. We have a data­base of around 11 mil­lion wait times that goes back a num­ber of years. For each wait time that we col­lect we ap­pend to that wait time any­where from 2 to 6 hun­dred other pieces of in­for­ma­tion. Ev­ery­thing from the ex­tra magic hours sched­ule for all the parks for that day, to the park hours, to the weather, to things like how many other rides were down in the park. For ex­am­ple if Soarin' is down at Ep­cot it's go­ing to af­fect the wait times at Test Track.

So you need to know how much ca­pac­ity is lost. The other things that we look at are even more es­o­teric. We look at things like the con­sumer price in­dex trend over the last 30, 60, 90, and 180 days to see how much the U.S. econ­omy fac­tors into that. We look at unem­ploy­ment rates. We look at cur­rency con­ver­sions for Canada, Mex­ico, Brazil, and the U.K., and lag­ging in­di­ca­tors as well, so what was the econ­omy for those coun­tries like a month ago, 2 months ago, 3 months ago and so on. Be­cause, what's hap­pen­ing to­day in the parks is the re­sult of peo­ple plan­ning trips months ago, not yes­ter­day.

So when you're mak­ing a de­ci­sion you're mak­ing a de­ci­sion to go to the park, you know 4 months from now based on the eco­nomic re­al­ity that to­day is, right?

So we look at all that. Then, we have a re­ally so­phis­ti­cated in­fra­struc­ture that takes all these data and makes pre­dic­tions from one day to 365 days in ad­vance. We do some, we orig­i­nally started off do­ing a sta­tis­ti­cal re­gres­sion, this was back like 2001 to 2012. It turns out that re­gres­sion is not the best way to do that. You tend to run into prob­lems when you have more than like a

dozen or two dozen vari­ables that you need to pre­dict off of. So, we look at, we moved to some ma­chine learn­ing pack­ages in 2012. There's this thing called Treenet, which is a sto­chas­tic gra­di­ent boost­ing tree al­go­rithm.

Carl - Oh now that's a big word. You're go­ing to have to ex­plain that one.

Len - That's why it cost money, yeah so every­one knows what a de­ci­sion tree is right? So, like, imag­ine you're try­ing to make a de­ci­sion about whether you need a jacket and an um­brella to­day right? So the first ques­tion, the first de­ci­sion you might ask is the prob­a­bil­ity of rain greater than 50%? If yes, bring an um­brella, if no don't. Then the next ques­tion might be do you need a jacket, is the tem­per­a­ture go­ing to be more than 70 de­grees, yes or no? So imag­ine a tree where the first layer is ask­ing a ques­tion about whether you need an um­brella and the sec­ond layer is ask­ing a ques­tion about whether you need a jacket. So there are four pos­si­ble answers there right? No jacket, no um­brella. Jacket, no um­brella.

Carl - Right.

Len - Um­brella, no jacket. Um­brella ... Okay and at the end of the day you can go back and tell that tree whether you were right, right? So maybe there was only a 30% chance of rain but it ac­tu­ally rained, right? So at the end of the day you can give credit or pe­nal­ize the de­ci­sion tree based on how suc­cess­ful it was, so you as­sign var­i­ous weights and penalty func­tions to it. Imag­ine now in­stead of one tree you had a thou­sand trees that looked at all of these dif­fer­ent as­pects of Walt Dis­ney World like how many schools are in ses­sion? Is it a hol­i­day? Is it an ex­tra

magic hour morn­ing, right? So each tree is mak­ing one de­ci­sion or maybe two de­ci­sions but you com­bine them all right, so that the out­put of each of those trees, which is es­sen­tially go­ing to be a wait time. You take the out­put and you com­bine it mag­i­cally, we'll skip over how, into a pre­dic­tion about what the wait time is go­ing to be at a par­tic­u­lar ride in a par­tic­u­lar time on a par­tic­u­lar day, that's what that does.

So the sto­chas­tic gra­di­ent boost­ing part, sto­chas­tic means that the pre­dic­tion from one tree de­pends on the pre­dic­tion at a pre­vi­ous point in time, so sto­chas­tic is a time se­ries thing. Gra­di­ent boost­ing, gra­di­ent looks at the curve of the er­ror in the trees and then boost­ing is es­sen­tially how you ad­just the weights. So, yeah, so we've been us­ing that since 2012. Last year I started look­ing at even more so­phis­ti­cated, or sorry we started look­ing at more so­phis­ti­cated ma­chine learn­ing things. So in­stead of hav­ing one set of de­ci­sion trees what if you have like fifty sets of de­ci­sion trees and you use like even more so­phis­ti­cated sta­tis­ti­cal anal­y­sis and you com­bine these trees in dif­fer­ent ways how does that work?

So that's what we're work­ing on now, lit­er­ally like this morn­ing I spent do­ing that. It's, you know I tell friends like, we've gone be­yond ba­sic sta­tis­tics, we've gone be­yond ba­sic ma­chine learn­ing at this point. We're like, the places that we're get­ting ideas from are es­sen­tially peo­ple's PHD dis­ser­ta­tions, which just to put in per­spec­tive, I've seen Dis­ney's in­ter­nal pre­dic­tions for some of these rides. We're more ac­cu­rate than they are.

Carl - Yes.

Len - I'm not say­ing it's per­fect, you know there are things that go wrong. Weather for ex­am­ple is one thing but we are very good. So the in­ter­est­ing thing is we've been look­ing at this, you know for the last year. One of the fun­da­men­tal ques­tions I had for these stats guys is how good or how ac­cu­rate can the pre­dic­tions be, right? Know­ing that I have to make a pre­dic­tion sixty days in ad­vance and I don't know what the weather's go­ing to be like and I don't know things like Dis­ney's staffing sched­ule. How good can we be, and I think the an­swer that we're at is we can be some­where be­tween eight and fif­teen min­utes of ac­cu­racy. So if you tell me you want to pre­dict the wait time for noon at Space Moun­tain on a given day gen­er­ally speak­ing we'll be be­tween plus or mi­nus eight to fif­teen min­utes.

But here's the in­ter­est­ing thing. The big­gest prob­lem that we face isn't that we don't know like, what crowds are like at noon be­cause we know what crowds are like at noon. The big­gest, one of the big­gest prob­lems that we face be­sides weather is that Dis­ney in­ten­tion­ally ma­nip­u­lates those wait times and they don't seem to have a pat­tern in do­ing it. So, you've seen this, you've been to Dis­ney World. Carl - Yup.

Len - Like the Magic King­dom in like, De­cem­ber.

So a cou­ple years ago, this is in­ter­est­ing, this is a tech­nol­ogy thing. In Dis­ney's My Dis­ney Ex­pe­ri­ence app a cou­ple years ago, when they posted the wait time the in­ter­nal data feed in­side of My Dis­ney Ex­pe­ri­ence also had the ac­tual wait time for the ride. Now they didn't show it to you but it was in the data feed, so your phone was get­ting that in­for­ma­tion it just wasn't show­ing it to you. We had ac­cess to that in­for­ma­tion and one of things that we no­ticed is that when it was very crowded at cer­tain rides like Space Moun­tain or Kil­i­man­jaro Sa­faris, or Ex­pe­di­tion Ever­est, Dis­ney would in­ten­tion­ally raise the posted wait time as a sig­nal for you to go some­where else. So, for ex­am­ple over Christ­mas in 2014 Dis­ney raised the posted wait time for Space Moun­tain to 240 min­utes when it knew the ac­tual wait time was 45.

Carl - (laughs) Oh wow!

Len - Right? So, a 200, think about that a three hour and change gap and it's not be­cause they thought that the wait time was go­ing to be 240 min­utes. They were try­ing to send a sig­nal to peo­ple to go some­where else be­cause they knew To­mor­row­land was go­ing to be crowded later on. So they're us­ing, Dis­ney uses wait times as a crowd con­trol mea­sure and the prob­lem is we can't pre­dict when or to what ex­tent they're go­ing to do that. That seems to be largest source of er­ror for us right now.

Carl - Now, I know you said you saw that back in 2014, are you still see­ing that to­day?

Len - Toy Story Ma­nia every night, man!

So if you go to Toy Story Ma­nia, in fact a user just, just emailed me this so last week at one point the posted wait time for Toy Story Ma­nia was like seventy min­utes or some­thing and we had pre­dicted it would be ten, an ac­tual ten minute wait and the user didn't be­lieve us, got in line, and waited seven min­utes and sent me an email on it like no­body in my fam­ily be­lieved me so I took one of my kids be­cause the rest of the fam­ily didn't want to wait an hour in line. It was like 8:30, the park was sup­posed to close at 9:00 some­thing like that, got in line you know, and was done in ten min­utes. The rea­son they do Toy Story Ma­nia is they want to close the park on time so they don't, you have to pay peo­ple.

Carl - Right.

Len - So it's ar­ti­fi­cially in­flated then as well so that's an­other ex­am­ple of it.

Carl - Yeah, I didn't know.

Len - Every theme park­does it right?

Carl - Right.

Len - Dis­ney's not the only, every theme park does it.

Carl - And I did have that same ex­pe­ri­ence be­cause I tested lines, it was shortly af­ter it came out, a year or so af­ter it came out and I was go­ing to say okay how ac­cu­rate is it, lets test it. I looked at their wait, the posted wait, I even tracked it.

Len - Oh that's awe­some, did you re­ally?

Carl - Yeah, I even tracked it, and I said, "Yup, lines is more ac­cu­rate."

Len - So the in­ter­est­ing thing that we do there, so ob­vi­ously we're get­ting real wait times from our users in the park. But, one of the in­ter­est­ing things that we do is we have our crowd counter that pre­dicts stuff, you know 1 to 365 days in ad­vance, but once you're ac­tu­ally in the park we use a com­pletely dif­fer­ent pre­dic­tion tech­nol­ogy to up­date the pre­dic­tions while you're in the park. So to­day once the park opens at, opened at 8 o'clock or 9 o'clock, once we got like ten min­utes worth of wait times we started mak­ing day of pre­dic­tions and those are vastly more ac­cu­rate. We use a straight re­gres­sion method for that, but es­sen­tially what it tells us is what's ac­tu­ally hap­pen­ing in the park that day. So, if you're us­ing a tour­ing plan and you're us­ing the app and you're there in the parks, we will tell you, we'll redo your tour­ing plan based on what's ac­tu­ally hap­pen­ing.

And last week, you men­tioned you were there, I was there too. We ac­tu­ally tested this in the Magic King­dom so two steps into my tour­ing plan the lines rec­og­nized that Peter Pan's wait times were not grow­ing as fast as we ex­pected and it re-routed me over to Peter Pan be­cause, to take ad­van­tage of the lower waits.

And it ended up sav­ing me 35 min­utes that day be­cause Lau­rel was fol­low­ing, was with me in the park fol­low­ing the ex­act same tour­ing plan but a printed ver­sion and she had 35 more min­utes of wait. The only step that lines changed for me us­ing the app was Peter Pan, and it saved me 35 min­utes.

Carl - That's cool, and ac­tu­ally go­ing out and test­ing it against -

Len - Side by side.

Carl - Side by side test­ing that's just, that's price­less right there.

Len - Yeah, I mean, we do it all the time, it's how we know that, you know on av­er­age you'll save four hours in line.

Carl - Right.

Len - Be­cause we've re­cruited fam­i­lies to do this again and again so...

Carl - Yeah, it's the way you should do it. Thank you for do­ing it. Did you all catch the big crowds that hap­pened in Jan­uary and Fe­bru­ary of this year?

Len - We did ac­tu­ally, I think we did fairly well on those yeah.

Carl - Yeah, just curious be­cause ...

Len - Last, so the rea­son why I went back and started look­ing at the ma­chine learn­ing stuff is be­cause we didn't catch last Septem­ber and Oc­to­ber. Those crowds, re­mem­ber those were crazy.

Carl - Right, right.

Len - So we did bet­ter in Jan­uary and Fe­bru­ary and again you know it's this com­bi­na­tion of Dis­ney's try­ing to you know boost their ... They can do sales, they can do pin codes, and we can't al­ways see those things.

Carl - Right.

Len - But, you know we're get­ting bet­ter at it so ...

Carl - Es­pe­cially when they do them out­side the U.S., which is what I think hap­pened early this year.

Len - Yeah, be­cause be­fore Brexit the pound was ac­tu­ally fairly strong against the dollar, it's pretty much col­lapsed since then but I think it's down like what a third or some­thing like that?

Carl - Some­thing like that.

Len - Yeah but that stuff's only go­ing to ef­fect tourism in 2017 so ...

Carl - Okay, so lets move on. Lets talk about the hotel part of Tour­ing­plans that you came up with. I think that's just cool.

Len - Thanks, so we have this thing called the hotel room fin­der and what it is is it's pho­tos of the view you get from every hotel room in Walt Dis­ney World.

Carl - I will say there are some that need to be up­dated.

Len - Yeah, there's a few, we're work­ing on it.

There's 32 or 33 thou­sand pho­tos and we're work­ing our way through it.

Carl - I was, I'm a lover of The Poly­ne­sian so I clicked on The Poly­ne­sian to­day, wait a minute... that con­struc­tion's all done. (Laughs)

Len - Yeah, yeah we're work­ing on that. We had this, our main photographer was a re­tiree who lived in Or­lando and he ac­tu­ally moved here to North Carolina where I'm at now so we've got to go back and redo those. But the idea was this, back in like 2011 when, be­fore the site got re­ally re­ally huge I was an­swer­ing all of my own email, and I still an­swer most of my email but some stan­dard ques­tions get han­dled by customer ser­vice. That year I answered 16 thou­sand emails by my­self, which when you think about it is like al­most like 55-60 a day or some­thing like that.

At least 50 a day, and one of the ques­tions we got most fre­quently was based on some­thing we said in the book. In the book we say, you know, if you want the best hotel rooms in this par­tic­u­lar

re­sort, like the best rooms at Caribbean Beach are these and we ex­plained why. But peo­ple would write in and say you know, I un­der­stand that you rec­om­mend this room but I'm look­ing for some­thing closer to a bus stop, or I'm look­ing for some­thing that's very quiet, or I don't care about quiet, and I don't care about walk­ing, I want the pret­ti­est view, what do you rec­om­mend? We were get­ting thou­sands of these emails a year, I mean, as you can imag­ine.

So, me and Bob are sit­ting around one day, we're at Pop Century kind of walk­ing through and you know we're sit­ting down for a sec­ond and I'm like, "How long do you think it would take to get a photo of the view from every hotel room here?" So, we fig­ured out there's like you know, 30 build­ings, there's 192 rooms I think in each build­ing, what if it takes a minute then you're look­ing at like, a cou­ple days. So we get our photographer out and we're like okay take one build­ing, 192 rooms, take a photo of the view from every win­dow and tell me how long it takes for you to do those pho­tos. It took him less than an hour, right? So we're like okay, well you could con­ceiv­ably do pho­tos for every room in Pop Century in a day? He's like, "Yeah I think I can do it in a day." I was like okay well then we could do every re­sort in a month!

Carl - Yeah, that's im­pres­sive be­cause that's 2,884 rooms-

Len - Yeah that's a lot of rooms-

Carl - In Pop Century-

Len - Yeah, that a lot. So, sorry Pop I think had, it's Art of An­i­ma­tion that has the 192 rooms. But yeah, so we're like okay, so he did, and it turns out that get­ting the pho­tos is the easy part (laughs).

So what we did was this, we de­cided to build maps, and we built a search en­gine on top of it, and it works like this. We have maps show­ing for every build­ing and every floor where those rooms are lo­cated and so we have a lit­tle rec­tan­gle for ex­am­ple that shows you where every room in every build­ing of Caribbean Beach is or every room in every build­ing of Port Or­leans French Quar­ter or River­side. And, if you tell us the kind of room that you booked, like a stan­dard view two queen bed room, we'll show you on the map where all of those rooms are lo­cated floor by floor, and if you click on one of the rooms we'll show you the view you get from that room. In ad­di­tion to that we'll tell you like, you know, the kinds of beds that you have, we'll tell you whether the room is ADA ac­ces­si­ble, and what ameni­ties it has. We'll give you a rat­ing for how quiet we think it is, how far it is from the bus stop, how far it is from the lobby and so on, and all this is search­able so you can find the per­fect room you want. By the way, I've got to say this is an­other ex­am­ple of where Dis­ney won't of­fi­cially ac­knowl­edge that they co­op­er­ate with us, but they to­tally co­op­er­ate with us.

Carl - Right, right.

Len - There's no way we would get that in­for­ma­tion with­out them, and they were fan­tas­tic about it. Yeah so, and things change and what not, but we're try­ing to keep up with it the best we can. The in­ter­est­ing thing was this, it took us the most time to num­ber the pho­tos and build the maps. So, you know when you take a photo on your cam­era it's like Img_1234.jpg or some­thing like that, right Carl?

Carl - Right, ab­so­lutely.

Len - We had to re­name those in a sys­tem so that we could look it up au­to­mat­i­cally. So, you know, we had to re­al­ize that IMG_1234 was re­ally Caribbean Beach, build­ing 34, room 23. That and build­ing the maps took us a year! (Laughs)

Carl - Oh yeah, I can imag­ine.

Len - The or­ga­ni­za­tional sys­tem for all of it was just in­cred­i­ble, but it's done now. The other thing that we added to it was this, we built a fax API, or we hooked into a fax API so that if you tell us you like a par­tic­u­lar room, and you tell us when you're go­ing to check in, and give us your reser­va­tion num­ber, we will au­to­mat­i­cally fax your re­quest for that room to Dis­ney five days be­fore you check in. We've talked to room as­sign­ers across prop­erty about how to do that most ef­fi­ciently to make their lives eas­ier and to make your lives eas­ier. So there's a suc­cess rate for

that of about 70% right now. So if you ask for a spe­cific room or a set of rooms, you'll get one of those rooms 70% of the time.

Carl - That's pretty in­cred­i­ble, that's re­ally re­ally ... Len - It's kind of amaz­ing, the most com­mon rea­son for not get­ting your room is that some­one's al­ready in it. There's ac­tu­ally rooms at The Contemporary where, you know our users, our Tour­ing­plans com­mu­nity, they es­sen­tially oc­cupy the room 365 days out of the year. I won't spec­ify the ac­tual room num­ber be­cause there's al­ready enough competition for it. I'm go­ing through the im­ages one day and I'm like why do we have 53 im­ages from this one room, and I'm like ... So I'm talk­ing to Guy, our Dis­ney­land guy who han­dles the process and he's like yeah lin­ers stay there all the time. So it's one of those rooms where, you can see the Magic King­dom but you don't get charged the Magic King­dom view for it.

And so it's like one of this lit­tle piece of knowl­edge that's passed along from our com­mu­nity, so I'm like you know Ep­cot, there are places within The Contemporary where we ac­tu­ally don't have a photo from a par­tic­u­lar room and like one room next door I've got like 53 pho­tos, be­cause it's the best room.

So that works out re­ally, re­ally well. We talk to room as­sign­ers all the time about it that the hotel fax thing is kind of awe­some. I'm par­tic­u­larly proud of that. The in­ter­est­ing thing too we ac­tu­ally patented that and we did it two ways, ob­vi­ously the abil­ity to look the room is some­thing to pa­tent, but we also patented, we were try­ing to fig­ure out how Dis­ney would use this, so we patented a rev­enue op­ti­miza­tion process on it where you can charge more for your hotel if you al­low peo­ple to choose their spe­cific room and view.

I know Hil­ton is starting to do it now but I don't think they're charg­ing more on it. For Hil­ton Hon­ors mem­bers you can pick your room-

Carl - Right.

Len - View at cer­tain prop­er­ties. I don't think they're yet do­ing any rev­enue op­ti­miza­tion on it but ...

Carl - Now is that, I know you have sub­scrip­tion ser­vices, is that part of the sub­scrip­tion ser­vice or is it out­side the sub­scrip­tion ser­vice?

Len - So, we talked about tour­ing plans those are free, you don't need a sub­scrip­tion to do that.

Al­though you do need an ac­count just so we can track, you know whose tour­ing plans be­long to whom. The crowd counter is sub­scrip­tion based be­cause of the amount of money we spend on

that every year. The room fin­der is free, the fax ser­vice costs money be­cause we pay every month for the fax ser­vice and that's a ton of money.

Carl - Right.

Len - Any­thing that re­ally costs us money ex­ter­nally is the things that we pay for but we try and keep as much stuff free as pos­si­ble.

Carl - You do a good job of it, I mean, logs and ev­ery­thing you do is good stuff. So you have some other prop­er­ties that you deal with with Tour­ing­plans which other prop­er­ties do you work with?

Len - So we cover Uni­ver­sal Or­lando and we do tour­ing plans and crowd coun­ters for that. We haven't yet done hotel pho­tos for that but I'm to­tally in­ter­ested in do­ing that. Then we do Dis­ney­land, and for that we do tour­ing plans and crowd counter, no hotel pho­tos yet, and we're just get­ting ready to launch Wash­ing­ton D.c. early in 2017, and that will have tour­ing plans, so the ... Have you been to Wash­ing­ton D.C.?

Carl - Yes I have, many times.

Len - I re­ally like it for a cou­ple rea­sons, one is all the good museums are free, that's where your tax dol­lars go.

Carl - Right.

Len - But also it's one of the places where you don't nec­es­sar­ily need a car to get around so if you stay any­where in the area that's ser­viced by the Metro, The DC Metro.

You can use that to get around, you know, pretty much any­where, and it's rel­a­tively in­ex­pen­sive and it's fairly ef­fi­cient. So, if you want to go on va­ca­tion and you don't want to worry about driv­ing, D.C.'S a great place, and again most of the museums are free. So we spent most of 2015 and a good chunk of 2016 lit­er­ally go­ing through each of the museums and writ­ing up, you know, re­views of them. We in­ves­ti­gated the sto­ries be­hind every piece of art and if that was at like, the Na­tional Gallery of Art then we tried to ex­plain, like, how a par­tic­u­lar paint­ing fits into the over­all art scheme. So we talk about for ex­am­ple, the French im­pres­sion­ists and we go artist by artist and year by year and we ex­plain how all these pictures fit in to the trend or story.

Over at Air and Space we ac­tu­ally give you the story be­hind all the ma­jor air­craft and all the ma­jor space craft that are there, and the rea­son is some of the times the gal­leries them­selves, some­times they do great jobs at de­scrib­ing why this par­tic­u­lar thing is im­por­tant but they've got thou­sands of things and they can't do the same level of de­tail on each. So I'll give you an ex­am­ple. Over at Air and Space on the sec­ond floor ... Have you been to Air and Space?

Carl - Ab­so­lutely, I have a funny story about that one.

Len - Air and Space gen­er­ates the most funny sto­ries, I don't know what it is. So over at Air and Space, sec­ond floor, there's this tri­an­gle shaped air­craft. It's the Amer­i­can L2M2 Lift­ing Body and the Air and Space mu­seum says that NASA used this as a test ve­hi­cle for de­sign­ing the orig­i­nal space shut­tles. So the space shut­tles, they're es­sen­tially lift­ing bod­ies. They don't gen­er­ate lift through their wings, they get lift from the shape of the body. The shape of the body's the thing that holds the air­craft up in the air. That's what Air and Space says but I was do­ing some re­search on this par­tic­u­lar one that they dis­play and it turns out that the back­story is the most im­por­tant thing. Do you re­mem­ber the TV se­ries "The Six Mil­lion Dollar Man", Carl?

Carl - Yes (laugh­ing) yes.

Len - At the be­gin­ning of The Six Mil­lion Dollar Man, Steve Austin crashes-

Carl - Yes he does...

Len - An air­craft, okay. The air­craft that he crashes is the ex­act ve­hi­cle shown in the Smith­so­nian In­sti­tute.

Len - They re­built it, they flew it once, they do­nated it to the Smith­so­nian. I'm like dude you don't say that it's the lift­ing body for the space shut­tle, you say this is Steve Austin's plane, that's who you say it's for. So I call up the me­dia guy for the Smith­so­nian and I'm like, "Dude, I need you to ver­ify this" right, be­cause they only built like three of them. Ac­tu­ally they built two and when the sec­ond one crashed they re­built it and gave it se­rial num­ber 3, so there are re­ally only two of them. I'm like, "Dude, as far as I can tell this is Steve Austin's plane." So, I call him up, and he's like, he con­firms it right, we think that if this story is true then this is, as far as we can tell this is true, and while he's on the phone he's like, "I've got a Wil­liam Shat­ner story do you want to hear it?"

Which I will not re­peat… but Air and Space gen­er­ates the fun­ni­est, fun­ni­est thing. So any­way, that's one of the things we did so when we launched tour­ing plans for Wash­ing­ton D.C. not only will it give you the most ef­fi­cient tour of Wash­ing­ton D.C., but will tell you the sto­ries be­hind the pieces of art.

Carl - You also do Dis­ney Cruise don't you?

Len - We do the the Dis­ney Cruise Line as well. So that was ex­tremely dif­fi­cult to re­search. We had to go on all the dif­fer­ent

Carl - Tough job ...

Len - Last year we did ... So my fa­vorite ship, you know, prior to the first ship I ever went on was the Fan­tasy. I al­ways said it was my fa­vorite ship but last year, and again this tells you how dif­fi­cult my job is, we did a back to back Alaska and Hawaii cruise on the wa­ter.

And then spent five days at Au­lani and that was one of the best trips I've ever taken yet, I was gone for three weeks. You know, be­fore that my big con­cern was that the Won­der is the old­est ship, it was in need of re­fur­bish­ment, and it doesn't have a Remy, has a Palo, and I was like I don't know if I can spend, I think it was 17 nights to­tal. It was 7 nights Alaska, 10 nights to Hawaii. I'm like, 17 nights on the Won­der is go­ing to be a stretch, but, be­fore that I had done 13 straight, or 11 straight cruises in the Caribbean so I was done with the Caribbean, right?

There's only so many times you can go to Nas­sau. But it was com­pletely dif­fer­ent. When they do the Alaska cruises, it goes out of Van­cou­ver and they change the back­ground mu­sic on the ship to be the back­ground mu­sic of the Wilder­ness Lodge, so Amer­i­can West. You know, the Mag­nif­i­cent Seven, and Sil­ver­ado for you know, Amer­i­can west­ern themes. That re­ally made a huge dif­fer­ence for me on the cruise be­cause I love that mu­sic and they change the menus to be Pa­cific North­west menus, and Alaska is just ... Have you been to Alaska?

Carl - Yeah, ab­so­lutely, did a cruise up there ac­tu­ally. Not Dis­ney, this was be­fore the Dis­ney cruise line. Won­der­ful, won­der­ful trip.

Len - Yeah, you're in these fjords and you know, ob­vi­ously it's deep be­cause you're in the mid­dle of moun­tains but you can see the moun­tains. You could swim there if you needed to, it's not like, you know, when you're in the At­lantic and the Caribbean you could be hun­dreds of miles from the near­est piece of land, but when you're in Alaska you're hun­dreds of feet. (Laugh­ing) From the near­est land, right?

Carl - And even when you're not in the fjords you're in the pas­sage where you've got land on both sides.

Len - Yeah, it's just, it's bizarre, you're in this cruise ship and you can see moun­tains above you, which is just again, com­ing from noth­ing but Caribbean cruises, was just fas­ci­nat­ing. I could've ... I think Alaska is the best cruise I've ... The best desti­na­tion I've ever been on. Then we did Hawaii. We stayed at Au­lani, have you been to Au­lani yet?

Carl - Not yet, no.

Len - It's Dis­ney's best re­sort. Every good idea they've ever had they put in Au­lani. I re­ally like that quite a bit. Then we came home. So yeah, we cover Dis­ney Cruise line, that ad­di­tion to the book just came out this week. So the 2017 edi­tion of the book, that's su­per su­per pop­u­lar. We cover all the des­ti­na­tions. We also did, our co-au­thor Erin Fos­ter did Dis­ney's new river cruises in Europe, and that looks to be su­per in­ter­est­ing be­cause river cruis­ing is a slower, more intimate pace than ocean cruis­ing. You don't go as far ev­ery­day, you just spend more time in each port, and the food is sup­posed to be bet­ter. So, Dis­ney's go­ing re­ally big into river cruis­ing in 2017. I think they're pack­ag­ing some ABD trips with it.

Carl - Adventures by Dis­ney.

Len - Adventures by Dis­ney, right. So we're do­ing that. From the tech­nol­ogy side one of the in­ter­est­ing things we do with the cruise line, and this is free to any­one, is ev­ery­day we check the price of every cabin on every itin­er­ary for every ship that Dis­ney of­fers. So we're get­ting like 50 some thou­sand prices per day, so what we can tell you over time whether the price of your cruise is go­ing up which means you should book now, or it's go­ing down which means you should wait. We'll ac­tu­ally do that his­tor­i­cally, so lets say you want to go on a cruise in sum­mer of 2017, and you've got a spe­cific itin­er­ary in mind, you know like four nights on the Dreamer, seven night East­ern on the Fan­tasy.

You can look at the price trend for that cruise this year, 2016 to see how much they're likely to go in 2017, and whether it's likely to go up or down the closer you get to your cruise. And that saves peo­ple, it can save peo­ple, you know some­times it's a few dol­lars some­times it's thou­sands of dol­lars be­cause de­pend­ing on the cabin you want there's dif­fer­ent book­ing strate­gies that we've iden­ti­fied based on this data. So like if you want the least ex­pen­sive cabin in any state­room cat­e­gory just book now, that's the strat­egy. The other in­ter­est­ing corol­lary to that is if you want the most ex­pen­sive room in a cat­e­gory, like a cat­e­gory four deluxe fam­ily ocean view with ve­randa you should wait til the last minute. You should wait til within like, 60 or 90 days.

Once the can­cel­la­tion win­dow fi­nal­izes for Dis­ney which I think is 60 days, prices drop.

Carl - Ac­tu­ally it's 90 days. I just booked one.

Len - So yeah, once the win­dow closes and every­one's made their fi­nal pay­ments prices drop. So, if you can, we think the sweet spot's some­where around 60 days out. Where you can still get

air­fare with­out be­ing pe­nal­ized for be­ing too close on air­fare but your cruise price is go­ing down. So if you're look­ing for the up­scale ver­sion of each cabin again deluxe fam­ily ocean view state­room with ve­randa and you can wait un­til the last minute go do that. If you're just look­ing for the least ex­pen­sive room in every cat­e­gory, cat­e­gory 11 in­side state­room just book as early as pos­si­ble. So for each desti­na­tion we tell you the best pric­ing strate­gies in the book based on this mil­lions of data points that we've got from that.

Carl - So cool. So, what projects have you got com­ing up?

Len - We've got D.C., we got this ma­chine learn­ing thing that we're do­ing for the crowd counter.

Carl - Cool, any­thing else you want to men­tion?

Len - I think that's it!

Carl - I will, man, it's been great. I re­ally ap­pre­ci­ate the in­ter­view, it’s been great to talk to you.

Im­age Cour­tesy of Len Testa

Carl Trent in­ter­views Len Testa of Tour­ing­plans.com and The Unofficial Guide in this ex­clu­sive in­ter­view for WDW Mag­a­zine.


Newspapers in English

Newspapers from USA

© PressReader. All rights reserved.