Is Facebook tougher on female engineers?

Does Facebook discriminate against female engineers? It’s a question that may take more than one statistical analysis to answer.

The issue has gained prominence, thanks to the research of a former Facebook Inc. engineer, who — according to the Wall Street Journal — found that female engineers get more criticism of their code than their male peers, by a margin of 35 percent. Facebook responded that the analysis was incomplete: If it had taken into account the rank of the engineers, the discrepancy would have disappeared.

The dueling analyses are an example of Simpson’s paradox, one of the trickiest and most intriguing obstacles to human understanding of statistics. It holds that aggregate data can generate a completely different picture than subsets of the same data — but only if something interesting is going on in those subsets.

Consider the Facebook case. In aggregate, the “rate of code criticism” looks pretty misogynistic. But the interesting fact is that women are concentrated among junior engineers. If this subgroup tends to attract more criticism, the average woman will experience a higher rate of criticism than the average man — even if there’s no gender difference within each rank.

Blindness to Simpson’s paradox can be consequential. Back in 1983, the Nation At Risk Report, which spurred the modern education reform movement, presented the troubling finding that SAT scores were going down over time. What it didn’t say is that if you subdivided SAT takers by their high-school class rank, you’d find something completely different: In each subgroup, scores were either holding steady or improving.

The explanation was that more kids from lower class ranks — kids who tended to be from lower-income or minority families — were aiming for college, and therefore taking the SAT in the first place. Because they typically got lower scores, they brought the average down even though their own scores were improving over time. So actually pretty good news, but it wasn’t reported until years after the original report came out.

Here’s another of my favorite examples. From 2000 to 2013, the inflation-adjusted median wage for all U.S. workers rose 1 percent. But if you divided workers by educational attainment, you would find a decline in each subgroup. In this case, the interesting fact is that the share of college graduates in the work force actually increased. Because educated workers tend to make more money, their greater prevalence pushed up the aggregate median wage. This could be perceived as good (more people are getting college degrees!) or bad (wages are going down even for college graduates!).

Simpson’s paradox forces us to grapple with the complications of reality. In the case of Facebook and its female engineers, we might be relieved that women don’t seem to be facing extra criticism for an equivalent job. Then again, it leaves us wondering why there aren’t more women in senior positions. Are they being promoted at similar rates to men? Or are they leaving for some reason? In any case, it’ll take more data — and maybe more than data — to understand and address. O’Neil is a mathematician who has worked as a professor, hedge-fund analyst and data scientist. She founded ORCAA, an algorithmic auditing company, and is the author of “Weapons of Math Destruction.”