Herron: Statistics and Fairness
Thinking probabilistically about the Trips Directorate.
I am writing this contribution with some trepidation, as wading into a campus debate about an issue like First-Year Trips strikes me as a questionable idea for a faculty member. Nonetheless, one of my advisees asked me last week if I was aware of the controversy surrounding the recent selection of a Trips directorate. Since that time I have read the original Trips editorial as well as several responses. I do not have a dog in this fight, but as someone who teaches statistics at Dartmouth, I hope to see students on campus invoke statistical principles in their discussions and in public debates. Hence this letter.
The one thing we know for certain about the 2018 Trips directorate selection process is the result: Four of 19 selected directorate members are men, and 15 are women. In the op-ed that sparked the controversy, Ryan Spector ’19 wrote that, “[When] nearly 80 percent of Trips’ executives are the same gender, such an imbalance is no longer just a majority. It is ludicrous.” This is an argument that a Trips directorate that is four-nineteenths male (or female) is inconsistent with a gender-blind applicant review process.
I do not know the gender breakdown of the individuals who applied for a directorate position, and Spector himself acknowledges that his claim would presumably be less compelling if the applicant pool were heavily unbalanced with respect to gender. However, let us make a few assumptions. Suppose that the applicant pool were half women and half men, which would be roughly consistent with Dartmouth’s undergraduate student body. Suppose as well that directorate members were chosen at random from this pool. This is a simple model of leader selection, one akin to coin flipping. I assume here that merit in Trips applications was not correlated with gender. In other words, I assume that male applicants are on average equally strong as female applicants and vice versa. Of course, this may not have characterized the recent directorate applicant pool, but I have no information that speaks to this possibility.
Based on this model, the probability that the 19 students chosen for the Trips directorate are all men or all women is approximately 3.8 X 10^-6. This number is not zero, which means that it is possible for the Trips directorate to be completely gender imbalanced even if the process for choosing the directorate were gender-blind. This point is worth noting: A gender imbalance in the directorate is not necessarily evidence of a gender-aware leader selection process. Still, assuming that Trips leaders were chosen randomly from a gender-balanced pool, there is only a minuscule chance that a completely gender imbalanced directorate would result.
We know that the observed Trips directorate is not imbalanced to this extent, but Spector’s argument is that the imbalance in the directorate — four men, 15 women — is a puzzling event nonetheless. Is this argument a strong one? It can be, assuming a gender-balanced pool. A binomial calculation shows that the probability of choosing a Trips directorate composed of four or fewer men, plus the probability of choosing four or fewer women, is approximately 0.019. In practice, empirical researchers often rely on a probability threshold of 0.05 to identify meaningful results. Since 0.019 is lower than this threshold, this suggests that the gender imbalance in the observed Trips pool may not have been due to chance alone. Put another way, the presence of only four men in the recently-selected directorate is odd enough to raise a red flag or two.
A key assumption supporting this conclusion is that the gender distribution of applicants was evenly split between equally qualified men and women. Some may argue against the validity of this claim, but I think it is a fair assumption and, I suspect, probably one that Spector himself held when he wrote his op-ed.
My probability calculation considers gender imbalances that favor men or that favor women. I do this because this position is conservative — not politically, but statistically. The observed gender imbalance in the Trips directorate is indeed one in which men are underrepresented. Nonetheless, if we want to be open-minded about the possibility that the directorate selection process was not gender-blind, we should not condition on the observed imbalance in the directorate.
Spector’s argument about the inherent fairness of the Trips directorate selection process is the sort of claim that probability theory can address. With that in mind, if you have not taken a statistics course at Dartmouth or a course that teaches basic probability yet, it might be worth taking one next quarter. And if you have taken such a course but did not anticipate the argument made here, you might want to consider reviewing relevant material. Without a background in basic probability, it is very difficult to assess when an empirical observation — here, four men and 15 women chosen from a pool of 19 individuals — is typical and not worthy of concern or unlikely and potentially worthy of investigation. This distinction — typical versus unlikely — is fundamental to empirical research in science. It can help inform gut reactions to complicated situations and, as shown in the case of the Trips directorate controversy, strengthen arguments about justice and fairness in the world.
Herron is a government professor and the chair of the program in quantitative social science at Dartmouth.
The Dartmouth welcomes guest columns. We request that guest columns be the original work of the submitter. Submissions may be sent to both firstname.lastname@example.org and email@example.com. Submissions will receive a response within three business days.