The class action lawsuit, Thames vs Mars, Inc, where Jenile Thames is suing the maker of skittles candies, is based on the European Commission’s ban of titanium dioxide, a colorant in skittles. But here’s the rub — the European Commission based their ban in part on a study published in a highly questionable journal.
A Highly Questionable Journal
The paper in question is Kurzawa-Zegota, et al. (2017; Journal of Nanoscience and Nanotechnology, 17, 9274-9285). It was published in the Journal of Nanoscience and Nanotechnology, published by American Scientific Publishers.
In my opinion, and I’m joined by Elsevier and Clarivate, the Journal of Nanoscience and Nanotechnology is highly questionable. It used to be a journal that published monthly — I say used to be because it stopped publishing in December 2021. It was delisted by Elsevier’s Scopus service (Elsevier prefers the term “discontinued”), for “publication concerns” according to Cortegiani et al. (F1000Res, 2020; 9: 415). It was also delisted from the Journal Citation Report (JCR). This is significant as these databases do not just delist/discontinue journals for no good reason — they realize there are serious ramifications when this happens. However, when JCR and Scopus both delist a journal, that is a very strong indication that something is not right and in my opinion the journal is highly questionable.
What Were the European Food Safety Authorities Conclusions About the Paper?
The European Food Safety Authority (EFSA) panel concluded in their report that the paper was “of high relevance” to their assessment of the genotoxicity of titanium dioxide (TiO2). This disagrees slightly with their official reliability score, which was reliable with restrictions.
EFSA specifically states that Kurzawa-Zegota, et al.’s concludes that titanium dioxide nanoparticles:
- Increased a DNA damage marker (micronucleus and comet assay) in white blood cells (specifically lymphocytes) from healthy subjects and colon cancer patients
- Showed signs of DNA strand breaks in the same type of blood cells
The EFSA panel also stated in their summary of the study that the “[r]elevance…of the diseased patients is not certain…”, meaning their opinion should be based more on the healthy volunteers.
But what’s important to note is what EFSA’s panel did not consider — the study design, the statistical analyses, the fact that this was a study where participants weren’t matched based on various “confounding” variables, the fact that the number of participants was really small. This is made clear in their evaluation approach found in Appendix D.
So, What’s Wrong With This Paper?
In my opinion this study is unreliable for a number of reasons, and the results reported are likely false positives (I’ll expand these ideas in a future post).
- The data do not appear to be normal; for instance, 19% of the untreated controls from “healthy individuals” in the micronucleus assay (Figure 6 of the paper) would be negative numbers if the data were actually normal
- The statistical analysis methods do not match the study design
- The statistical approaches being used are inappropriate for the way the data are being handled
- The different groups (healthy individuals, polyposis coli, and colon cancer patients) have wildly different demographic characteristics, which makes it difficult to effectively compare the groups, and impossible to make causal arguments
- There is insufficient methodological controls to prevent false positives
- Samples were collected from individuals, but not run in all of the assays
- Based on the dissertation that the paper is based on, there is insufficient recording of potential confounders
- The analysis did not account for potential confounding appropriately
All of these issues can lead to an increased false positive rate.
Shouldn’t Government Agencies Avoid Questionable Journals?
Yes, government agencies do need to avoid questionable journals when looking for the best science. But it’s hard to identify questionable journals. US Environmental Protection Agency scientists have published in the Journal of Nanoscience and Nanotechnology in the past (see here, here, here, here, and here) — so clearly even the Agency’s scientists believe the journal to be reputable.
Government Agencies Need To Vet the Actual Studies
So if it’s so hard to vet these journals, what is a government regulatory agency to do?
Well, a good first start is to see if the journal is listed in Scopus and JCR. If the journal is currently “discontinued” or “delisted”, then in my opinion (and the opinion of either Scopus or JCR) the journal is highly questionable — meaning: stay away!
However, the easiest way to avoid this type of mess is for scientists, especially at regulatory agencies, to always check the science. Check the study design. Check to see if there are statistical issues with the data. Check to see if the data are stated, or assumed, to be normal, but return invalid data when you bring it back to the population level (e.g., the population with the sample mean and standard deviation returns negative values when those are impossible). Check the vehicle controls to see if their means are smaller or larger than what you anticipate (Bayesian statistical approaches help with this). Check the sample sizes — is there a potential for sampling bias to lead to false positives (studies with 20 people in them is simply not enough — would you take a drug tested on only 20 people)? Re-run the statistical analyses — especially for every t-test and ANOVA. This can be done even with just summary data.
Not Evaluating The Entire Study — This is Just The Tip of the Iceberg
What troubles me is that it’s clear that there is a pattern emerging. Why didn’t EFSA’s scientific panel find the problems with the studies they cited in their review of the literature on genotoxicity and titanium dioxide (check back as I’ll be posting my critiques of these articles soon)? Why is it that EPA didn’t find the problems with the study they used to define their PFOS and PFOA health advisories?
In the case of EFSA it’s clear: they weren’t looking for the same problems I found. They make clear what they are looking for in Appendix D of the EFSA report — there is no mention of looking at the statistical analysis. There is no mention of re-running the statistical analysis. There is no mention of considering potential biases from the study design or statistical analysis. In fact, the words “bias” and “statistical analysis” do not even appear in Appendix D.
And the problem at EPA appears to be the same as EFSA’s. They aren’t looking for ALL the right things. In order to realize there are problems with the study design and the statistical analysis you need someone looking for these things. You need someone re-running the statistical analyses. You need someone thinking about the study design and asking if the correct statistical analysis is being used. You need someone thinking deeply and critically about these studies. That’s simply not happening.
There’s an ongoing bias across the field of toxicology, in governments (including Congress and the Courts) and academia, that peer review is some kind of indelible stamp of approval. That peer review makes papers and their results infallible. For those who believe that, I present Exhibit A: Dr. Wakefield’s retracted article that ramped up the anti-vax movement. I also present Exhibit B: Kurzawa-Zegota, et al. I have lots of other examples.
We need to bring back critical thinking. We need to stop regulating safety based on fear. We need to be thoughtful and critical. We need to be skeptical. We need to become a nation of critical thinkers again.
Want to Support Our Investigations?
The Toxic Truth Blog is focused on getting at the truth in science. I started the Special Investigations Unit at Raptor Pharm & Tox, Ltd to focus on scientific integrity. That’s part of what we do, beyond our consulting and technology product development. Unfortunately, these types of investigations take time, we have bills to pay, and we’d like to expand this service. If you’d like to help fund our Special Investigations Unit operations, drop me a line at burgoon.lyle@raptorpharmtox.com.
We’ll continue to post more investigations on titanium dioxide and other chemicals on the Toxic Truth Blog as they develop.
Thanks for reading and thank you for your support!
