What do school ratings tell us?

We are more than a decade into the age of measurable accountability in education. You might have thought we’d have the hang of it by now.

That would mean having metrics we stick with, because they make sense. Instead, Georgia has been more prone to adding new metrics on top of earlier ones that have hardly been perfected.

Take our state’s “Financial Efficiency Star Rating.” The Legislature ordered up this new metrics in 2012 and, five years later, education officials are preparing to release the first school-level results (expected in early 2018). We already have two years of district-level results.

The premise sounds good enough. Perennially, one of the biggest education fights in Georgia concerns whether we’re spending enough on our public schools. A number of fiscal conservatives, myself included, question the wisdom of simply pouring money into the system. And so, voila! — a rating system to tell us how schools and districts are doing.

What we got, however, was a blunt instrument: a rating system of 0.5 to 5 “stars” comparing schools’ and districts’ spending across the state to their performance on the College and Career Readiness Performance Index, or CCRPI.

There’s something to be said for simplicity, as the public is served well by a scale that’s easily understood. One district got four out of five stars, and another only two? The average Georgian can grasp that.

But the average Georgian might also expect all the districts receiving four stars to be getting a similar, or proportionate, bang for their buck. In reality, consider these examples from metro Atlanta:

The city of Decatur’s schools spent about 20 percent more per pupil than the city of Buford’s but scored slightly worse on the CCRPI. Yet both systems got 2.5 stars for “financial efficiency.”

Also getting 2.5 stars were Fulton County schools, which spent almost the same amount as Buford but had a CCRPI that was 16 percent lower.

In the 3.5-star group were both Cobb County schools and Clayton County schools. Cobb spent about 8 percent more but fared 24 percent better on the CCRPI. Yet, somehow, Clayton is deemed to be just as “financially efficient.”

How does the state arrive at these seemingly illogical conclusions? The biggest reason is the matrix used to determine the star ratings. If a district is in the bottom 20 percent of the state in terms of spending, the worst rating it can get is 2.5 stars — no matter how badly its students perform in the classroom. If a district is in the top 20 percent, the best rating it can get is 3 stars — no matter how well its students perform.

The matrix sets up the preposterous possibility of District A spending little money and having none of its students pass the state’s standardized tests, while District B spends a lot of money and has 89 percent of its students pass — and both districts receiving the same rating of 2.5 stars.

Does that really tell us anything about their respective “financial efficiency”?

If a metric doesn’t tell us anything real or credible about how schools and districts compare, it will eventually undercut the very idea of trying to gauge how effectively they spend their money.

If this is what legislators intended when they passed that 2012 bill, they ought to explain what they were thinking. If it’s not, they ought to schedule some time — before those school-level ratings are released — to ask the state officials who built this metric what they were thinking.