NFL-backed youth program touted reduced concussions, but data disagree

As increasing numbers of parents keep their children from playing tackle football for safety reasons, the NFL and other groups have sought to reassure them that the game is becoming less dangerous.

No initiative has received more backing and attention than Heads Up Football, a series of in-person and online courses for coaches to learn better safety procedures and proper tackling drills. The NFL funds and heavily promotes the program. The league and USA Football, youth football’s governing body, which oversees the program, have sold Heads Up Football to thousands of leagues and parents as having been proved effective — telling them that an independent study showed the program reducing injuries by 76 percent and concussions by about 30 percent.

That study, published in July 2015, showed no such thing, a review by

The New York Times has found. The research and interviews with people involved with it indicate, rather, that Heads Up Football showed no demonstrable effect on concussions during the study, and significantly less effect on injuries overall, than USA Football and the league have claimed in settings ranging from online materials to congressional testimony.

As the 2016 youth football season dawns, the revelation will most likely fuel skeptics of football’s claims of reform and discourage parents who want solid information about the sport’s risks for their children.

“Everybody who is involved in trying to improve the safety of youth sports, when parents such as myself are so desperate to have effective solutions, has the responsibility to make sure that any information that they are putting out to the public is accurate, is comprehensive

and is based on legitimate science,” said Elliot F. Kaye, chairman of the U.S. Consumer Product Safety Commission, who has worked with USA Football and the NFL on improving helmet safety. “It does not appear that this met that standard.”

Representatives of USA Football and the NFL said in interviews that they had been unaware that their claims of Heads Up Football’s effectiveness were unsupported by the study, which was conducted by the Datalys Center for Sports Injury Research and Prevention through a $70,000 grant from USA Football.

“USA Football erred in not conducting a more thorough review with Datalys to ensure that our data was up to date,” Scott Hallenbeck, executive director of USA Football, said in an email to The Times. “We regret that error.” He added that the material would be removed from the organization’s print and online materials, and that “our partners and constituents” would be notified of the errors.

Brian McCarthy, an NFL spokesman, said that the league would also include updated information from now on.

Both USA Football and the league said that the questionable data and conclusions were actually preliminary results provided by Datalys five months before the study was published. The lead researchers for Datalys, Thomas Dompier and Zachary Kerr, confirmed in interviews that, despite knowing that the final paper contradicted their preliminary claims, they did not inform USA Football of this until last month, one day after speaking with The Times.

Dompier, the president of Datalys, said in an interview: “We’re the ones that put out the numbers. We’re the ones that kind of blew it.”

In an email, Kerr said that the company had released the early data because, “The results were so compelling, we felt morally obligated to make the youth football community aware of the results.”

In 2014, USA Football asked Datalys, an Indianapolis-based firm that handles all of the NCAA’s injury research, to monitor injury rates during that fall season among six youth leagues that used Heads Up Football and four leagues that did not, covering more than 2,000 players.

In February 2015, Datalys gave USA Football the results: Leagues that used Heads Up Football had 76 percent fewer injuries, 34 percent fewer concussions in games and 29 percent fewer concussions in practices.

In USA Football’s blog post announcing that the safety program “reduces injuries,” Dompier said: “This is compelling data. I am actually surprised by the strength of the association but completely confident in our findings.”

These figures were prominently reported in the media and on websites of youth leagues as a means to show parents that Heads Up Football was scientifically sound. NFL promotional materials have called the program “The New Standard in Football”; a page in its 2015 Information Guide is headlined, “Study Finds USA Football Program Advances Player Safety.”

But last summer, when The Orthopaedic Journal of Sports Medicine published Datalys’ formal paper on the study, the paper did not include the same injury and concussion figures. Its data actually told a far different story about Heads Up Football’s effectiveness.

Rather than looking at Heads Up Football leagues in one category, the paper instead split them into two groups: those that did or did not also belong to Pop Warner Football, a division of youth leagues that has added its own rules to mitigate injuries. Pop Warner leagues have disallowed certain head-on blocking and tackling drills and drastically reduced full-contact practice time, measures that were not a part of USA Football’s program.

As it turned out, only leagues that adhered to Pop Warner’s rules saw a meaningful drop in concussions. Leagues that used Heads Up Football alone actually saw slightly higher concussion rates, although that uptick was not statistically significant. The previously reported drops were clearly driven by a league’s affiliation with Pop Warner, not Heads Up Football.

Similarly, Heads Up Football leagues saw no change in injuries sustained during games unless they also used Pop Warner’s practice restrictions. The drop in practice injuries among Heads Up Football-only leagues was 63 percent, a very meaningful figure. But combined with the in-game injuries, the total reduction became about 45 percent, far less than the 76 percent presented by USA Football and the NFL for the past year and a half.

The authors did not address how the paper’s data contradicted their preliminary conclusions from five months before. Regarding the fact that Datalys did not inform USA Football or the NFL of the discrepancies, Kerr said in an email: “Datalys stands by our decision to release preliminary data in our Feb 2015 release because if we prevented even one youth football player from suffering an injury (sprain, fracture, strain, severe contusion, or concussion), then the release was a success.”

As the 2016 season approaches, the faulty pronouncements about the research continue to be cited by youth programs and football officials as evidence that Heads Up Football makes football safer, especially regarding concussions. During a high school sports conference in Alabama last week, a coach presented a glowing slideshow about the program to fellow coaches and athletic directors, unaware that many of the numbers and statements were not supported by the data.

In May, coaches from Columbia High School in Maplewood, N.J., invited some eighth-graders interested in playing football to a meeting in the cafeteria.

“They basically said they teach Heads Up Football, which reduced head injuries and concussions,” said Jacob Kasdan, one of the students who attended the meeting. “I think they’re struggling to find enough players.”

Jacob went home and asked if he could play this fall. His father declined to sign the forms.