Experts: Data on distance learning like ‘the Wild West’
Information on progress, engagement hard to measure
Love it or hate it, the success of distance learning is tough to measure. While some districts say they have “100 percent” engagement, others report numbers much lower — if they are willing to report them at all.
Further complicating the issue, how districts document this information varies by city, town and region. This, experts say, makes it difficult to draw conclusions about performance and progress.
Still, some patterns have emerged: Small districts fare better than large ones in engaging kids. Young children are less likely to participate than older children and, in some cases, participation rates do not stray far from traditional attendance rates.
Hearst Connecticut Media asked multiple districts for numbers of students participating in distance learning. A few reported school-by-school absentee rates; others provided district averages of students logging on every day, while some opted for the number of students logging on “frequently.” For districts without one-to-one devices, completion of paper packets had to be factored in. And some said they did not have “hard numbers.”
Using criteria such as logging on to measure engagement is a slippery slope and educators cannot draw far-reaching conclusions, cautioned Michael Alfano, dean of Sacred Heart University’s Farrington College of Education.
“There are people who are logged in who are not engaged,” he said. “Prior to that, we had people who sat in classrooms all day every day and weren’t engaged.”
It is up to districts to define engagement, which is more esoteric than attendance, said Amy Beldotti, associate superintendent of teaching and learning for Stamford Public Schools. Initially, Stamford Public Schools did not have a common understanding of engagement, and the district had to develop and apply one as the weeks wore on.
“The state has not come out and said, ‘Here’s how you define engagement,’ whereas attendance is simple: You can look at chronic absenteeism across the state because attendance is the same,” she said.
The variability — in resources, participation and progress-monitoring — is going to present problems for students catching up this fall, Alfano said. Districts will need to assess students before school starts to determine the loss of learning because only that data will help teachers tailor instruction to their students’ current levels.
“It’s the Wild West right now,” he said. “We call it distance learning, but it’s crisis education.”
The small-district advantage
In a district of about 3,100 students, Bethel Public Schools Superintendent Christine Carver said only eight to 10 students have not logged on.
Staff are contacting the families of those students who are not participating and working with them.
“We’ve been hounding these people,” Carver said.
Some kids receive more support now than before the pandemic, Carver said. Para-educators work with students who are struggling academically, while counselors help students with behavioral or social-emotional challenges.
Tracking daily participation is challenging, however, because younger students sometimes log on after daycare, while older students work during the day and complete classwork in the evening, Carver said.
In Region 12, a district of 685 students from Bridgewater, Roxbury and Washington, students and teachers already had the technology and devices to support distance learning, Superintendent Megan Bennett said.
The district has contacted every student and family, and with family supports and a staff outreach program, 98 percent of students are regularly attending classes, she said.
Shelton Interim Superintendent Beth Smith said, to date, all 4,662 students have logged on.
“Everyone is accounted for and engaged at some level,” she said.
Meanwhile, New Haven Public Schools, a district with more than 20,000 students, reported about 90 percent of students are engaged.
While the participation is high, the number of students not participating — about 2,000 — is larger than some Connecticut districts.
Size can exacerbate the problems of unequal access, Alfano said. Even if large districts have the resources, coordinating large deliveries of laptops and Chromebooks poses a second challenge.
Alfano said he has spoken to districts that have devices, but getting them delivered has been difficult.
And those logistic issues are made larger by inequities that existed before the crisis, he said.
In Bridgeport, Acting Schools Superintendent Michael Testani said contact has been made with almost all of the district’s 20,000plus students.
“The level of engagement varies day by day,” he said.
Although the district has been accumulating data, Testani said he is hesitant to make it public as he is not sure how accurate the data is and worries how it will be interpreted.
“Engaged and fully engaged are very different,” he said.
Stamford’s associate superintendent agreed.
Stamford schools define “engaged” for middle and high school students as participating in four or more classes. About 85 percent of these students are engaged, while about 94 to
96 percent of elementary school students are engaged, educators there said.
That is likely because younger kids have one teacher, and may not also be working or have caregiving commitments, Beldotti said.
Older students started participating more once the state announced schools would remain closed for the rest of the year, she said.
Younger students vs. older
Data that purports to show engagement as lower among younger children also shows the bar for what counts as engagement is lower.
That is because younger children were learning the basics of engaging with technology in school before the crisis, Alfano said. The pandemic accelerated that process overnight, but they are not as adept as older students.
In Trumbull Public Schools, more than 95 percent of students are logging in every day. That number is slightly higher at the high school level and lowers as the students get younger. By kindergarten, mostly parents are logging on and getting assignments for their children, but the school still counts that as engagement, the assistant superintendent said.
There are similar numbers in New Milford. The intermediate, middle and high schools are seeing engagement percentages from 96 percent to 98 percent. At Northville Elementary School, the number hovered around 95 percent, while Hill and Plain Elementary School vacillated between 90 percent in early April and 76 percent a month later. The dip at Hill and Plain, the superintendent said, was because the district stopped reporting numbers from the preschool, which were initially included.
In Hamden Public Schools, the highest absenteeism rates, 28 and 33 percent, were from two elementary schools.
Principals from a few West Haven Public elementary schools reported they are not collecting data on kindergartners, first- and second-graders.
Weekly participation reports from West Haven schools show different levels of participation across schools and grades, quantified either by rates of logging on or the number of paper packets going home and returning to the district.
High school attendance is specific enough to be broken down by subject matter, with an average of 86.9 percent of students with devices participating. Other principals said they have “no hard numbers” on daily participation.
“From the onset of distance learning, our first goal was to account for every student currently enrolled in the district,” West Haven Superintendent Neil Cavallaro said. “I believe that our administrators, classroom teachers and support staff have done an excellent job given the current circumstances.”
Public schools are learning new and powerful ways to teach through distance learning to students paired with devices, but these developments need to be married with wholesale ways to measure progress, Alfano said. Otherwise, it is difficult to make good comparisons.
“There is no precision, and without precision, accountability doesn’t follow,” he said. “Everyone wants accountability in the system. If we’re going to invest in these tools, we need to know that they’re going to work.”
In Stamford Public Schools, the district is partnering with Connecticut RISE Network to achieve that large-scale data collection Alfano said is necessary. CT Rise had started with Westhill in the fall, before distance learning, but the coronavirus expanded the partnership across the whole district.
The nonprofit that works with urban districts has volunteered to track engagement through the use of Google Classroom, Beldotti said. The number of classrooms using Google Classroom jumped from 150 before the school closures to about 850 by mid-March.
Participation rates and absenteeism
Some districts had absenteeism rates last school year that are on par with online absenteeism rates, while in others, online attendance is reportedly higher.
Over the course of two weeks, about half of Hamden schools upped their average daily engagement from the first to the second week, while the other half saw decreases.
Hamden, a district of 5,412 students in 2018-19, had a chronic absenteeism rate of 14.7 percent of its population. The average absenteeism during the two week-period measured for participation last month was 15.5 percent.
Norwalk Superintendent Steven Adamowski told public officials recently that attendance for digital learning has been similar, if not better, than regular school attendance, with about 95 percent of students logging in each day.
“It’s much harder to be absent if you’re at home . ... However, this is much more than attendance,” Adamowski said. “It depends on how much the student does, how much work they do, how engaged they are in distance-learning lessons.”
Alfano agreed that online participation rates could be higher than attendance rates, given that many parents are also home, he said.
And they could be roughly equal because the kids who are motivated to attend in-person school — or have parents who motivate them — will also show up for online learning, he said.
“But, if someone is going to look at that data and say, ‘What do we need school for?’ that is a huge leap in logic to me,” he said.
Summer-itis
Fairfield Superintendent of Schools Mike Cummings said he could not give an exact figure for participation rates, but parents and teachers tell him a large number of students are logging on daily and keeping up with work.
Participation seems equal across grade levels, but that may not last, he said.
“That will probably change as the weather warms up,” Cummings said. “It happens to some degree when we are in school physically.”
The allure of the sun is one source of changing attendance. But people are also tired of the circumstances.
“People are weary,” he said. “It is great to see the people you care about in a Google Meet but it is not a substitute for the community that a spring concert or awards ceremony or a graduation brings.”
The reliability of education right now is surfacelevel, Alfano said.
“People are trying to survive and get kids to the finish line and close out the academic year and start thinking about the fall,” he said.