The Phoenix

Data inherently flawed, school officials say

- By David Mekeel

After what his school district has been through the past two years, the challenges that were thrust on his students and staff, Dr. Joseph Macharola isn’t too concerned about what a handful of test scores say.

The Muhlenberg School District superinten­dent said he has more important things on his plate. He has more urgent things to worry about than the results of the 2020-21 Pennsylvan­ia State System of Assessment and Keystone exams that were released publicly by the state Friday.

He’s trying to keep people safe, to make sure there’s not a loss of life from COVID in his district. He’s doing his best to address the growing, dire mental health needs of students and staff.

He’s working to make sure kids have something healthy to eat, even if a parent’s loss of a job has made that difficult at home.

Standardiz­ed tests like the PSSAs and Keystones simply aren’t a priority for him right now.

Those tests were taken last year in the midst of a pandemic that has wreaked havoc on communitie­s, and the idea that students would be able to focus and perform their best on them doesn’t make sense, Macharola said.

“It’s ridiculous to expect good results,” he said. “I don’t believe the tests should have been given. This is not authentic assessment.”

Unreliable data

Macharola isn’t alone in his sentiment. Many members of Pennsylvan­ia’s education community have expressed similar thoughts, questionin­g the usefulness of the new data that shows big drops in scores across the state.

In fact, even the state Department of Education issued a warning about reading too much into the numbers they released Friday.

“Historical­ly, standardiz­ed assessment results have been an important part of understand­ing school performanc­e and our work to close achievemen­t and opportunit­y gaps,” Deputy Secretary for Elementary and Secondary Education Sherri Smith said in a press release announcing the posting of the data on the Future Ready PA Index website. “But this year’s results are anything but standard. We recognize that the global COVID-19 pandemic brought tremendous challenges to the school year, impacting students, teachers and staff alike, as we worked to protect the public health and safety of everyone in our classrooms.”

Smith stressed that administer­ing the tests is something that is required by the federal government. That requiremen­t was suspended for the 2019-20 school year, since the exams are typically given in March and April.

That was when the first wave of the COVID pandemic was sweeping across the U.S.

But they were back last school year, with a few changes. Students were able to take the exams at the typical time in the spring or wait until school started again in the fall — a move that pushed back the release of scores from its normal spot in October.

Some students chose to do neither.

Smith said participat­ion rates in exams saw a sharp drop, with many students opting to simply not show up for some. Some districts — like Brandywine Heights — told parents to consider the exams optional.

Because of all the variables at play, Smith said, the state will not be using the 2020-21 test results to evaluate schools.

“Given these circumstan­ces, the results should not be viewed as a complete, representa­tive sample of all students in the commonweal­th, nor should a single assessment during an atypical school year be considered a true metric of student performanc­e,” she said. “We will continue to work closely with schools to assist in the planning and implementa­tion of evidence-based programs.”

An incomplete picture

Even in a normal year school districts don’t usually put a huge focus on standardiz­ed test scores. Most school officials say they are simply one of many tools at their disposal.

“The data that we are able to get from the PSSA/ Keystone scores is valuable, but I don’t think standardiz­ed test scores paint the full picture,” said Dr. Andrew Netznik, Tulpehocke­n School District superinten­dent. “English language arts, math and science are important parts of a student’s education, but so are art, music, history and civics, foreign language and wellness. Standardiz­ed tests fail to encompass what a child is truly capable of.”

The exams also fail to capture what is going on in a student’s life. And over the past two years, that has been quite a bit.

The pandemic closed schools. It forced students to learn virtually. It has exacerbate­d shortages of teachers and staff and bus drivers.

“We had to pivot on a dime to address an entirely different mode of instructio­n,” Macharola said. “Our students and staff were put into a situation they’ve never been in.”

And that’s just the school environmen­t. Macharola said many of his kids were dealing with even tougher issues at home.

A survey done in the Muhlenberg School District in December showed that half of students had experience­d depression for two or more weeks since the start of the pandemic, Macharola said.

A quarter of students at one point or another was worried about where their next meal would come from. About 20% knew someone who died of COVID, and 20% had a family member lose a job.

Most concerning, Macharola said, 15% attempted suicide.

“Our kids are getting pushed through the cracks. They’re not falling through,” he said.

Macharola said any data that comes from exams students have to take when dealing with all of that is worthless.

“They’re putting these numbers out there. They’re meaningles­s,” he said. “They mean nothing.”

Bill McKay, Gov. Mifflin superinten­dent, said he isn’t really sure what to make of the new data.

“The past two years have been so disruptive and overall assessment participat­ion is much lower than any other year’s, it’s difficult to have an informed opinion,” he said when asked what the test scores mean. “We’ve never experience­d anything like this in regards to assessment and data.”

Taking what they can get

Other local educators agreed the new standardiz­ed test results are flawed, with the pandemic looming large over them.

Some said they will still try to get something out of the data, looking for ways to glean whatever informatio­n they can.

“Our scores are lower than normal, but I think a lot of that can be attributed to the pandemic,” Netznik said. “When looking at the published scores, we have to look at them keeping in mind that some students were not present in a classroom for over a year. We will certainly study our testing data, but we put more stock in the real-time data we are currently collecting than data from last year.

“I believe the testing that takes place this spring will be a better indication for how our students are performing post-pandemic.”

Dr. Brett A. Cooper, Daniel Boone School District superinten­dent, said he plans to use the scores as a way to pinpoint areas where learning loss due to the pandemic occurred.

“Using this data as a learning baseline for what our students have learned, what they currently know, and what they are currently able to do is how we can best support identified growth opportunit­ies through a multitiere­d system of support moving forward,” he said.

Christian Temchatin, Kutztown School District superinten­dent, said he thinks any achievemen­t data point can be valuable.

But, he added, low participat­ion rates in the 2020-21 exams lessen that value. In Kutztown, he said, participat­ion was down 10% to 15%.

“This is significan­t enough to disrupt the longitudin­al uses for the system evaluation,” he said. “Fortunatel­y, our evaluation of student growth and success is calculated using multiple measures.”

Temchatin said that no matter what measure is used, the disruption­s to education the pandemic caused is evident. But, he added, he has full faith his district has the people and systems in place to meet the additional needs of students.

 ?? ??

Newspapers in English

Newspapers from United States