CU Boul­der sci­en­tist says data “fa­tally flawed”

The Denver Post - - DENVER & THE WEST - By Made­line St. Amour

New re­search from a Uni­ver­sity of Colorado sci­en­tist ar­gues that the data used to sup­port new reg­u­la­tions re­quir­ing some fe­male ath­letes to med­i­cally lower their testos­terone is “fa­tally flawed.”

The In­ter­na­tional As­so­ci­a­tion of Ath­let­ics Fed­er­a­tions an­nounced new reg­u­la­tions in April that re­quire cer­tain fe­male ath­letes with higher lev­els of testos­terone to med­i­cally re­duce that level, or they won’t be al­lowed to com­pete.

The as­so­ci­a­tion pub­lished a pa­per in the Bri­tish Jour­nal of Sports Medicine in 2017 claim­ing that elite women run­ners with high testos­terone lev­els per­formed as much as 3 per­cent bet­ter than those with lower lev­els.

A pa­per from Roger Pielke Jr., di­rec­tor of the Sports Gov­er­nance Cen­ter at CU, pub­lished Fri­day found sig­nif­i­cant er­rors in the data that the as­so­ci­a­tion used in its re­search.

Pielke, who au­thored the pa­per along with Erik Boye, a pro­fes­sor emer­i­tus at the Uni­ver­sity of Oslo, and Ross Tucker, a Uni­ver­sity of Cape Town ex­er­cise phys­i­ol­o­gist, said their pa­per doesn’t dis­cuss gen­der and testos­terone lev­els, but rather talks about sci­en­tific in­tegrity.

“Be­fore you can even get to the point of dis­cussing and debating testos­terone in fe­male ath­letes, we be­lieve that the sci­en­tific base should be solid,” Pielke said. “The ev­i­dence isn’t even enough to be­gin to have that dis­cus­sion.”

Pielke will be an ex­pert wit­ness at the in­ter­na­tional Court of Ar­bi­tra­tion for Sport in Switzer­land later this month, where South African Olympic sprinter Caster Se­menya is chal­leng­ing the new reg­u­la­tions. The hear­ing is not open to the pub­lic.

The first is­sue with the study that Pielke and his col­leagues ran into was a lack of trans­parency. Nor­mally, Pielke said that sci­en­tists share data with col­leagues af­ter they pub­lish a pa­per.

The as­so­ci­a­tion had both per­for­mance data and med­i­cal data. Pielke re­quested the per­for­mance data, as he ex­pected the med­i­cal data to be con­fi­den­tial, but said it was dif­fi­cult to even get that.

“Fi­nally, they gave us 25 per­cent of what we asked for,” he said.

They then repli­cated the as­so­ci­a­tion’s statis­tics, and found that their re­sults were very close. But they ran into is­sues when try­ing to recre­ate the data set.

Pielke said they tried us­ing pub­licly avail­able in­for­ma­tion on per­for­mances of fe­male ath­letes. They found that the as­so­ci­a­tion’s data set some­times used the same woman twice, in­cluded dis­qual­i­fied ath­letes who had been dop­ing, copied per­for­mances more than once and also held data that Pielke couldn’t find any­where

else — what he calls “phan­tom times.”

The three re­searchers found that 17 per­cent to 32 per­cent of the data was er­ro­neous, which is “a big num­ber if you’re do­ing re­search that only has 1,000 data points,” Pielke said.

In re­sponse, the as­so­ci­a­tion cor­rected some of the er­rors. Still, though, Pielke said they found er­rors when they repli­cated the statis­tics.

The re­sults from the cor­rected re­search didn’t match up with the as­so­ci­a­tion’s ini­tial find­ings, which “tells you that the data er­rors were ac­tu­ally im­por­tant.”

“All of this is such a sci­en­tific mess ... In most set­tings, if you’re gonna be im­ple­ment­ing poli­cies, you’d say, ‘Let’s clean this up,’ ” Pielke said.

Bad sci­ence leads to bad poli­cies and reg­u­la­tions, he added.

This in­stance is im­por­tant, Pielke said, as sports be­gins to rely more on sci­ence for de­tect­ing those us­ing drugs.

“It forces us to ask, do we re­ally want sport to have solid and ro­bust sci­ence? Be­cause, if we do, then it can’t hide its data, it can’t do it in-house,” he said. “It seems to me that the sport com­mu­nity is not quite ready to ad­here to the 21st cen­tury and the stan­dards we up­hold.”

Pielke said the sports com­mu­nity needs to do two things if it wants to col­lect data to guide reg­u­la­tions. First, it needs to hire a third party to do the re­search. The as­so­ci­a­tion used its own re­searchers for the re­port, which Pielke said wouldn’t be al­lowed in other in­dus­tries.

Se­cond, he said that any of its re­search needs to “fol­low the ba­sic norms of sci­ence.” The data needs to be shared so that oth­ers can try to repli­cate it, and then “let the chips fall where they may.”

The as­so­ci­a­tion has grap­pled with the is­sue of gen­der for a decade, Pielke said, and is­sues sur­round­ing gen­der ver­i­fi­ca­tion date back a half-cen­tury.

In 2011, the as­so­ci­a­tion tried to get sim­i­lar reg­u­la­tions in place, but the Court of Ar­bi­tra­tion for Sport found that there was a lack of ev­i­dence to sup­port them

Newspapers in English

Newspapers from USA

© PressReader. All rights reserved.