Agents vs. algorithms

How much is a house worth? Zestimates can be tool in the trade or thorn in the side

The conversation happens almost every day. And it often starts the same way.

“‘Zillow says,' ‘Trulia says,' ” Re-Max real estate agent Mike Bernadyn hears from clients trying to settle on a listing price. “It's almost nails on a chalkboard for us.”

Because one way or another, there's usually a disappointment — at first.

In the information-overload age, the proliferation of online estimates of millions of homes both on and off market puts the real estate agent in a variety of awkward or advantageous positions, depending on one's perspective. It's a key to an educated public, a tool in the agent's trade, or the thorn in their side that makes them justify their own estimates.

In Bernadyn's experience, online estimates from listing sites often value homes too high.

“Folks are like, ‘Are you kidding me, that's it?' ” when he tells them his suggested price,

he said.

There’s no choice but to make light of the situation, even laugh it off, he said. It’s now a fact of a real estate agent’s life.

A generation ago, the Multiple Listing Service — the holy grail of data on home listings, market trends and forecasts — came in a thick three-ring binder exclusively to the offices of real estate brokers, sending agents in a mad dash about every two weeks between the massive catalog and the copy machine.

They held the key to that information. Now, thanks to websites such as Zillow, Redfin and Trulia, everyone has a copy.

“They do the industry a big favor by helping us distribute that information out there more effectively than we could before,” said Craig Sachse, a Keller Williams real estate agent in Allentown. “But like any great thing, it’s a double-sided coin.”

On the other side is a clientele armed with computer-generated estimates: prebaked expectations of what their homes should sell for put up against their local agent’s calculation, which is almost always different.

The developers of these tools call them a conversation starter. Real estate agents have mixed feelings about the conversation.

Origins of the online estimate

Power to the people — that’s Zillow’s whole shtick, company senior economist Skylar Olsen says.

The Zestimate — an estimate of a home’s market value based on data points encompassing the home’s dimensions and its neighbors, among hundreds of others — predates the Zillow of today, where agents pay top dollar to advertise listings or snag potential clients.

It was born in 2006 on the premise of the democratization of previously protected MLS data, Olsen said. That it ruffles feathers comes as no surprise.

“We were aware it would be provocative,” she said.

In the early days, it was a hurdle to overcome in a new agent-homeowner relationship, said Sean LaSalle, an associate broker with Berkshire Hathaway. People took the website estimates as gold.

“In the beginning it was, ‘This is what my house is worth,’ ” he said.

In reality, home value estimations that appear on these websites are automated valuation models: computer-generated formulas based on data from the MLS. The floodgates opened after a 2014 decision by the National Association of Realtors to require Multiple Listing Services to make its data available to any broker that wants to use it to create an automated valuation model, which provides property values using comparable properties, through a third party.

Redfin, which started as a brokerage, jumped in on the action and created the Redfin Estimate in 2015. The extent of what Redfin spokeswoman Rachel Musiker can say about the Redfin Estimate is that it takes into account hundreds of data points, including data it collects from its local real estate agents.

Plenty of brokers run their own automated valuation models. LaSalle uses his as a launching point before going on a listing appointment. The tough part of the conversation is urging the client away from basing their offers solely on them.

“Under no circumstances do I look at them and say this is the value of a house,” LaSalle said.

The box conundrum

Zillow thinks of homes as boxes, Olsen explains: boxes with three bedrooms, two bathrooms and 1,600 square feet.

That’s the sticking point for local agents: The computer models cannot look in a home. A finished basement, for example, could add $12,000 to $15,000 to the value of a home, LaSalle said. Or the neighbor down the street with a nearly identical exterior and floor plan could have foreclosed.

The box paradigm is less of a problem in cookie-cutter neighborhoods where Zillow can draw data from transactions happening at neighbors’ similar houses. But the fewer neighbors one has, the tougher a job the algorithm has.

In the Lehigh Valley, which has a mixture of densely and not-so-densely populated areas, agents say it is not uncommon for online estimates from a host of websites to be 10-20% off.

Take Bernadyn’s former listing on Newport Avenue in Northampton, for example. Zillow places it at $123,000. Redfin is even higher: Its estimate is $129,000, but its estimated range goes up to $136,000.

The house sold for $116,000. Even smaller disparities in price throw off the marketing of the home, particularly when it’s priced too high, he said.

In Emmaus, LaSalle listed a five-bedroom house on Westminster Drive a month ago for $740,000. In those 30 days, the Zestimate increased by more than $32,000, landing at $761,000 and moving further away from the target.

The larger the list price, typically, the larger the estimated range online platforms provide. The Westminster Drive home could be anywhere between $715,000 and $799,000, according to Zillow.

“If I provided that kind of range, no one would list with me,” LaSalle said.

And then there’s the rare case that defies explanation, like Sachse’s $6 million listing overlooking Hawk Mountain, a 13,800-square-foot mansion dubbed Blue Mountain Estate.

Zillow’s estimate of $424,000 even has its own robot confused. The listing notes: “The list price and Zestimate for this home are very different, so we might be missing something.”

“It certainly doesn’t help to have an estimate up there one-12th of what you’re asking,” Sachse said.

Secluded and “unique” homes like the Blue Mountain Estate are prone to inaccuracies in the Zestimate, Zillow acknowledges in its online FAQ.

For these homes, Olsen said, “a box doesn’t cut it.”

Hitting the mark

Zillow’s first sample of 75 million homes carried a median error rate of 13%, meaning half of Zestimates came within 13% of what the homes actually sold for, and half landed outside the 13% window, Olsen said. Now, with 110 million homes assessed, that window has closed to 4.5%, she said.

The Redfin Estimate launched in 2015 with a median error rate of nearly 2% for homes on the market and just over 6% for homes off the market, according to the company. It says the current median error rate for homes for sale is 1.73%.

Zillow also breaks down its error rate by city and county. Lehigh County’s is 4.7%; Northampton’s is 5.2%.

With the improvements in the algorithm and the gift of time, LaSalle said, public understanding that the estimate is just that — an estimate — has also mildly improved.

Though the exact formula for its estimates are trade secrets, Zillow has made public its efforts to expand it.

The company has crowdsourced ideas for data points over the last couple years, Olsen said, including road noise, rent data and a factor called “view shed”: how much of a view your home has and how many nice things are in it.

And they may soon fight back against the adage that they do not consider the inside feel of a home. Zillow is testing algorithms for predicting quality from indoor photos posted to its website.

“That’s not yet — that’s down the road,” Olsen said.

Zillow and Redfin emphasize in their online materials that their estimates are not appraisals, but starting points, dedicating FAQ entries that defer to the local knowledge of a real estate agent.

“As with most fine print, consumers don’t always look at it that way,” Sachse said. “People put a lot of faith in those models.”

In many cases, the public has the freedom to help change estimates on their homes. On both Zillow and Redfin, a homeowner or their agent can update the facts on their home and request the companies consider these updated facts in a recalculation of the estimate. Last year, Redfin launched an “owner estimate” tool for homeowners to come up with their own estimation to display alongside Redfin’s estimation.

On Redfin, only the owners or agents of homes for sale can request the estimate be taken down, Musiker said. The Zestimate never gets taken down, Olsen said, but her team constantly uses feedback to adjust the algorithm.

That’s tough, Bernadyn says, when small subdivisions can behave like entirely different neighborhoods.

Andy Santana, a Keller Williams agent serving the Lehigh Valley, said he uses the conversation about online estimates as an opportunity to gain rapport with his clients and show the more nuanced aspects of the pricing that only he knows. This, Olsen says, would be the ideal outcome.

“Democratizing this information changes the value proposition of an agent, but I don’t think we’re removing the value of the agent,” she said.

Not even for the senior economist at Zillow, who said when she’s ready to sell, she calls her agent.