NIEHS said, “[w]omen who used chemical hair straightening products were at higher risk for uterine cancer compared to women who did not report using these products, according to a new study from the National Institutes of Health.” That scary line from a recent NIEHS press release got it wrong in multiple ways (more on that, see next section).
This then got picked up in the popular press, such as at CNBC, CNN, CBS News, and many, many others. All with variations on the headline that hair straightening products, like relaxers, are linked/associated with increases in uterine cancer.
The problem is that the data do not support the results. This is yet another example of scientists making health claims based on noise. In fact, the results that NIEHS is reporting is fully consistent with the uterine cancer rate in those people who never used hair straighteners, relaxers, and pressing products! Yes — you read that right — there is actually NO DIFFERENCE in the cancer rate between those who used these products and those who didn’t!
So let’s talk about this. Here are the results of my analysis of their data and here’s the direct link to my report. As you can see I did a lot of examination of the data, and Question 4 (at the end of the report) is where I show that really, this is all a bunch of noise. The reader-friendly version of my analysis follows:
The Noise That Looked Like a Signal
Those of you who read the Toxic Truth Blog on the regular are no doubt familiar with the fact that I talk about how small sample sizes lead to false positive findings (like here, here and here). In toxicology, this means that scientists are saying chemicals are toxic when, in reality, they probably aren’t.
Well, this is that story again, but because of the data we’re talking about I can show you the impacts of small sample sizes quite directly. And I can show you how small sample sizes can mislead researchers into thinking the results they are seeing are “signal” (something important) when in reality it’s just the noise (a manifestation from the uncertainty about the population uterine cancer rate in this case).
The Base Uterine Cancer Rate
In Tables 2 and 3 of NIEHS paper, Chang, et al. (2022) we can see that 30,329 people never used straighteners, relaxers or pressing products.
Now let’s digress for a quick sec. Straighteners, relaxers, and pressing products is a huge category of products. And I’m not exactly sure what NIEHS means by all of this because they never define these terms for us (that I could see). I’m going to assume that when they say straighteners they also include simple straightening irons, but I’m not sure. Maybe they just mean chemical-based products only. And if that’s the case, there is a huge variety of chemicals that are included in chemical straighteners, relaxers, and pressing products. In terms of chemical exposure, I’m fairly certain that simply using a flat iron 4 times in a single year is much different from using a relaxer. But then, even within the class of relaxers, there are varying strengths, there are varying exposures (e.g., your exposure is different between at-home vs salon treatment). So, it would’ve been nice for NIEHS to be more clear. Also, people’s usage patterns of products changes over time as they age. I know when I worked at the hair store that a lot of our older customers were moving more towards wigs and away from relaxers for a multitude of reasons. That’s not captured in this study. Anyway…
We have 30,329 people who never used these products in the year prior to enrollment. 332 of these people developed uterine cancer. That’s a base rate of 1.09%. We have to remember this is a sample from a population. However, with 30,329 people, and the fact that uterine cancer is fairly rare, I’d feel pretty comfortable that 1.09% is going to be a fairly decent representation of the population rate.
However, this study population does not accurately represent the US population — and we need to remember that as it influences the base cancer rate. This study’s population is overwhelmingly white (85.6%), only 7.4% Black, and 4.4% Latina, and only 2.5% all other races and ethnicities. In other words, the study population does not accurately reflect the US population.
What that all means is that the base rate of 1.09% is likely an over-estimate, and that if we had more people in the study, and if those people better represented the US population we would likely have a very different base cancer rate. So that’s the first thing to remember — it’s going to be hard to translate these results over to the US population just because of those demographic differences.
What Are Plausible Numbers of People Developing Uterine Cancer Assuming the Base Rate is 1.09%?
If we assume that the base rate really is 1.09%, then are the rates of uterine cancer that NIEHS reported actually different from the base rate (that is are they a true signal), or are these numbers likely noise and entirely plausible?
Let me say this another way — NIEHS reports that 1,464 people reported using straighteners, relaxers, or pressing products 4 or fewer times in the year prior to enrollment in the study. Of these 1,464 people, 12 went on to develop uterine cancer during the study.
NIEHS also reports that 1,572 people reported using these products more than 4 times in the year prior to enrollment in the study. Of these 1,572 people, 26 went on to develop uterine cancer during the study.
Okay, so that’s a cancer rate of 0.82% for the people using the products 4 or fewer times.
And that’s a cancer rate of 1.65% for the people using the products more than 4 times.
Well, those sure do seem different from the base rate of 1.09%, right? Case closed, right? Right?
No — the case is just beginning.
Dig Deeper Into Those Numbers
So, when doing statistical analyses like these, we need to be on the lookout for numbers that just don’t make sense. People who used these products is a very small number compared to the total number who didn’t use these products. We have 30,329 people who never used these products. But people who ever used the products is 3,036. And now consider that uterine cancer is a rare type of cancer with a base rate of 1.09%.
Put that together — rare cancer, effects maybe 1% of the population. And if we’re only sampling from 3,000 people, that means at best we’re ever only going to catch 30 cases on average. What that means is that sometimes we’ll catch more than 30, and sometimes we’ll catch less than 30.
That means we need to get a handle on what this variability looks like. Could it be that even with sampling 1600 people we might have a lot of variability? And if so, could it be that the 26 cases that NIEHS is seeing in the more than 4 uses per year group is actually just due to random sampling/chance? And if it were due to random sampling/chance, that would make this a false positive. And if it’s a false positive, and NIEHS just pushed a major press release, and their scientists are out there telling the public about this false positive, couldn’t they be alarming or scaring the public for no reason at all?
So, What Are Plausible Numbers of People Developing Uterine Cancer Assuming the Base Rate is 1.09%?
So, I have the entire analysis online and you can go read it now. Or you can read the condensed version here:
I’m going to use the binomial distribution to simulate the number of people who may develop cancer using the base rate of 1.09%.
Let’s start off with the less than 4 uses of these products scenario. Look at the histogram above. This is a binomial distribution using 1.09% as the base rate. Here I am asking the question — what are the plausible number of people who have uterine cancer with a 1.09% rate, when my population is 1,464 people — the same as the number of people NIEHS reported in the less than 4 uses group.
As you can see, plausible values range from 7 out to 27. In the NIEHS study they saw 12 people. Based on this simulation study 12 is a very plausible number — in other words, at a uterine cancer rate of 1.09%, in a group of 1,464 people, we would expect to see anywhere from 7 to 27 people with uterine cancer, and NIEHS saw 12 — so it all shakes out.
Cool — what about if we have 1,572 people? Again, using that uterine cancer rate of 1.09%, how many people would we expect, of 1,572, to have uterine cancer? Well, look at the histogram above. You can see that the plausible values span from 7 to 28. NIEHS saw 26 people. Yep — 26 is definitely plausible. It’s a bit on the high side of things, but it’s definitely in the realm of possibility.
Okay, that’s really cool, too. Wait. Hold up!
So, NIEHS says that this group that uses straighteners, relaxers or pressing products more than 4 times in the year before study enrollment has a straight up linkage with increased uterine cancer.
But, I just found that 26 people is actually a plausible number when using the same cancer rate as the comparison group that didn’t use straighteners, relaxers, or pressing products.
So, that means that the uterine cancer rate could be the same between the group that used those products more than 4 times in the year prior to enrollment in the study.
And that would mean there’s no treatment effect.
So that would mean NIEHS’ report and bottom line is actually a false positive.
Which means NIEHS didn’t actually see an effect. Instead, their results are plausibly explained by the 1.09% cancer rate in the group that didn’t use any straighteners, relaxers, or pressing products.
Let’s Recap This
Just so we’re all on the same page.
NIEHS says that people who used straighteners, relaxers, or pressing products more than 4 times in the year prior to enrollment in their study had an increased risk of uterine cancer.
However, that’s not right. NIEHS’ results are actually consistent with the cancer rate in the control group — the people who never used straighteners, relaxers, or pressing products.
So what on Earth happened here?
And the answer is — I have some thoughts and facts.
Perhaps NIEHS’ scientists didn’t perform the simple statistical test I did. They didn’t look at this in the simplest way possible to see if the number of people with uterine cancers they are seeing is plausible given their base cancer rate in the control group and given the size of the populations they are using. To be frank — it’s easy to forget this step. But it’s an important step to do — and it should be the first thing we do just to save us time (and doing pointless analyses).
Sampling bias happens. The number of people in the groups that ever used relaxers is tiny, especially by human studies standards. But what really matters is the number of cases — that is, the number of people with uterine cancer in those groups. That is what is truly small, and that is what is driving the statistical power equation, not the total number of people in the group. NIEHS has very little ability to detect reliably cancer patients in these small groups — even though they are in the low thousands — it’s still less than 100 people with uterine cancer that they’ll be able to detect. So what is happening is that NIEHS is getting a false positive result due to sampling bias. When you sample, sometimes you’re on the high end of the distribution, sometimes you’re in the middle, and sometimes you’re in the low end. You don’t know ahead of time. Which is why sampling bias is like gambling.
I’m going to guess here, but I think there’s a possibility that this data violates the assumptions required of the Cox Proportional Hazards test. I don’t know for sure, because I can’t get the raw de-identified data, but the NIEHS scientists never said that they tested that they met all of the assumptions — so that’s another potential issue here.
But really, this is a tale of sampling bias. Again. Seems like that happens a lot in public health and toxicology.
Bottom Line
NIEHS’ study does not provide any evidence that straighteners, relaxers, or pressing products cause or is associated with uterine cancer.
Instead, the data in NIEHS’ study clearly states that use of straighteners, relaxers, or pressing products actually results in uterine cancer rates that are in line with those who don’t use these products.
Burgoon, L.D. Re-Analysis of NIEHS Hair Straightener and Uterine Cancer Data.
