Too much real data missing for clear view of harassment stats

As a woman, I’ve been thoroughly outraged by the pervasive sexual assault and harassment that the #MeToo movement has exposed.

As a data scientist, I’ve been intrigued.

It’s a monumental case of missing data. Awful as they are, the reported incidents tell us little about the underlying prevalence of sexual harassment. There’s no reason to think we’ve uncovered even half of it. Judging from what we’ve seen so far, our starting assumption — what statisticians call a Bayesian prior — should be that it’s happening everywhere.

One urgent task is figuring out what cultural conditions allow victims to speak out. In what business, social or cultural environments are we most likely to be seeing what’s really going on, and where can we only infer?

Answering that question requires us to understand why people haven’t spoken out earlier. This clearly involves the repercussions that victims have faced after coming forward. They vary from the nuisance of being defined by something that happened to you rather than something you’ve accomplished, to being doubted and ignored, to losing your job and ability to care for your family, to being shunned and shamed by your community.

Those impediments offer ample reason to doubt that just because we’re not hearing about sexual assault in a given environment, it isn’t happening.

For poor women supporting their families, for example, the risks of speaking out are often too great. Consider hotel workers. Reports of abuse are rare: We heard about it only when Dominique Strauss-Kahn was involved. But the union of hotel workers recently managed to insist on panic buttons for its members. That likely means the problem is ubiquitous.

What about finance? We haven’t heard much about sexual harassment there, although an executive from Visa was recently pushed out. Is it because egregious cases are settled with nondisclosure agreements? Again, no news isn’t good news, and silence eventually becomes a signal in itself.

In other words, there’s a ton of missing data, and it isn’t equally distributed. It’s more likely to be lacking when women are poor, or when the stakes are high, or when the lawyers are very well-paid.

CATHY O’NEIL,