UPDATE: Here is the exact quote of Administrator on which this post is based:
“The FDA does this on a regular basis, which is actually why I was a little surprised at the comments tagged in the proposal…this isn’t necessarily new … manners of accessing science in the regulatory process. FDA routinely uses the double-blind scientific studies for their work where they have two teams of researchers replicating the same data, both teams don’t know each other exist and where they are. We don’t do that at the agency, we don’t do that across the entire federal government. I’ve believed for a long time that federal research would be more accepted by the public if you used the double-blind standard for everything. I don’t know if we have the funding to do that for everything but they certainly do that at the FDA. But there are things that we can do … to help the regulatory process by putting the data out there for the public to see that I think would really benefit to make our regulations stronger.”
Yesterday at the EPA’s Science Advisory Board meeting, EPA Administrator Andrew Wheeler made comments on the agency’s proposed Restricting Science Rule that raised eyebrows for anyone who understands the basics of health studies. In his defense of the rule (which the scientific community agrees will severely hamstring the agency’s ability to rely on the best available science in its decision-making), Wheeler asserted that the EPA should be more like the FDA in its data transparency. The FDA uses double-blind studies and the EPA should be taking that approach, he suggested. Um, what?
To understand exactly how misguided this statement is, let’s take it from the top.
Dear Administrator Wheeler, this is what a double-blind study is
Double-blind studies involve study designs where both researchers and study subjects don’t know which subjects are given the placebo versus the treatment. These studies are a great way that researchers can reduce bias in scientific studies since those involved in reporting and collecting data won’t be influenced by knowledge of a potential effect. These study designs are especially useful in clinical trial research for new drugs, where scientists are interested in the efficacy of a drug, the presence of side effects, and other health outcomes.
The EPA and FDA study fundamentally different things
What would a double-blind study even mean in the EPA context? Your guess is as good as mine.
Here’s the problem: There is a fundamental reason that the EPA is different than the FDA. The EPA is studying environmental contaminants, i.e. pollutants that are out in the world. As a result, we have to rely on observational data of people living their lives in the world, with all the variability in pollution exposure that comes with that. We cannot control who is and isn’t exposed to pollution if we want to study populations. There are of course natural experiments that scientists can learn from but as I explain in a recent piece in Science, it is not possible or ethical for scientists to expose groups of people to harmful levels of pollution—something that would be required if we wanted to attempt a double-blind study. This is very different than the FDA context where researchers in clinical trials have a level of control over which study subjects are provided drugs.
Air pollution studies with Scuba gear?
But let’s entertain Administrator Wheeler’s idea, shall we? If we wanted to design a double-blind study on an EPA issue like air pollution, here’s what that would look like. You would need groups of people who are and aren’t exposed to harmful levels of air pollution. But to make it double-blind, both those people and the researchers conducting the study would need to not know whether they were breathing clean or dirty air (until after the data was collected). To do this, the participants can’t breath the ambient air because we wouldn’t be able to control the pollution level of the environment, i.e. it would mess up the study.
Thus, a double-blind EPA air pollution study would need to involve something like study subjects living their lives in scuba gear, where neither they nor the scientists studying them would know whether their scuba tank contained clean or contaminated air.
It is very easy to see how this is completely unworkable. Does Administrator Wheeler need a crash course in study design?
The FDA is not a poster child for the transparency Wheeler claims to want
Administrator Wheeler had complimentary remarks for FDA disclosure of scientific data in its decision-making. This is a head-scratcher for anyone who follows FDA decision-making.
The FDA is not always forthcoming in disclosing detailed information about its decisions on drug approvals, food additives, and other agency regulatory actions.While the FDA has made many improvements in process, leadership, and transparency, when it comes to drug review and decisions, information disclosure is still lacking. For other products like medical devices, the standard is less rigorous, and primary data is often not publicly available. Also of note, even FDA advisory committee members might only see summary reports from companies and FDA reviews, not the raw data itself. Finally, label changes and safety alerts may or may not be based on publicly available data.
Importantly, the FDA is certainly not following the extreme requirements in the EPA Restricting Science Rule. Like EPA, the FDA handles much confidential business information as well as personally identifying information. It is true that some studies involving this kind of information are released publicly in ways that protect the sensitive information. For example, by statute, large pivotal clinical trials are required for companies seeking approval for new drugs. When drugs are approved, the studies the FDA relied on are released publicly and more information is now becoming available through Clinicaltrials.gov. The FDA is not, however, releasing raw data in the way that the EPA Restricting Science Rule suggests. This isn’t the model of disclosure that Administrator Wheeler claims to be aspiring toward.
Administrator Wheeler’s comments prove EPA desperately needs science advice
In conclusion, Administrator Wheeler’s comments suggesting that the EPA should mimic FDA’s use of double-blind studies makes zero sense. It’s almost as if he is in need of science advice from experts. The EPA Science Advisory Board could provide such expertise, if only the EPA will let them.
Thankfully, the SAB voted yesterday afternoon to fully consider the EPA Restricting Science Rule in addition to providing individual comments in a near-term consultation. If the administration makes good on its promise to release the final rule by the end of the calendar year, then the SAB advice would come later, but regardless of the timing these are issues where the EPA desperately needs science advice and the SAB must step up.