The Unit 3 reactor at the Dresden Nuclear Power Station near Morris, Illinois is a boiling water reactor with a Mark I containment design that began operating in 1971. On June 27, 2016, operators manually started the high pressure coolant injection (HPCI) system for a test run required every quarter by the reactor’s operating license. Soon after starting HPCI, alarms sounded in the main control room. The operators shut down the HPCI system and dispatched equipment operators to the HPCI room in the reactor building to investigate the problem.
The equipment operators opened the HPCI room door and saw flames around the HPCI system’s auxiliary oil pump motor and the room filling with smoke. They reported the fire to the control room operators and used a portable extinguisher to put out the fire within three minutes.
Fig. 1 (Source: NRC)
What Broke?
The HPCI system is part of the emergency core cooling systems (ECCS) on boiling water reactors like Dresden Unit 3. The HPCI system is normally in standby mode when the reactor is operating. The HPCI system’s primary purpose is to provide makeup water to the reactor vessel in event that a small-diameter pump connected to the vessel breaks. The rupture of a small-diameter pipe allows cooling water to escape, but maintains the pressure within the reactor vessel too high for the many low pressure ECCS pumps to deliver makeup flow. The HPCI system takes steam produced by the reactor core’s decay heat to spin a turbine connected to a pump. The steam-driven pump transfers water from a large storage tank outside the reactor building into the reactor vessel. The HPCI system can also be used during transients without broken pipes. The HPCI system’s operation can be used by operators to help control the pressure inside the reactor vessel by drawing off the steam being produced by decay heat.
The HPCI auxiliary oil pump is powered by an electric motor. The auxiliary oil pump runs to provide lubricating oil to the HPCI system as the system starts and begins operating. Once the HPCI system is up and running, the auxiliary oil pump is no longer needed. At other boiling water reactors, the auxiliary oil pump is automatically turned off once the HPCI system is up and running—at Dresden, the auxiliary oil pump continues running.
Why the Failure was Reported
On August 25, 2016, Exelon Generation Company (hereafter Exelon) reported the HPCI system problem to the Nuclear Regulatory Commission (NRC). Exelon reported the problem “under 10 CFR 50.73(a)(20(v)(D), ‘Any event or condition that could have prevented the fulfillment of the safety function of structures or systems that are needed to mitigate the consequences of an accident.’”
Why It Broke
Exelon additionally informed the NRC that the HPCI system auxiliary oil pump motor caught fire due to “inadequate control of critical parameters when installing a DC shunt wound motor.” The HPCI system auxiliary oil pump motor had failed in March 2015 during planned maintenance. The failure in 2015 was attributed by Exelon to “inadequate cleaning and inspection of the motor” which allowed carbon dust to accumulate inside the motor.
How the NRC Assessed the Failure
The NRC issued an inspection report on December 5, 2016, with a preliminary white finding for the HPCI system problem. The NRC determined that the repair of HPCI system auxiliary oil pump motor following its failure in March 2015 resulted in the motor receiving higher electrical current than needed for the motor to run. Consequently, when the HPCI system was tested in June 2016, the high electrical current flowing to the auxiliary oil pump motor caused its windings to overheat and catch fire. The NRC determined that the inadequate repair in March 2015 caused the failure in June 2016. The NRC proposed a white finding in its green, white, yellow, and red string of increasing significant findings and gave Exelon ten days to contest that classification.
During a telephone call between the NRC staff and Exelon representatives on December 15, 2016, Exelon “did not contest the characterization of the risk significance of this finding” and declined the option “to discuss this issue in a Regulatory Conference or to provide a written response.” With the proposed white finding seemingly uncontested, the NRC issued the final white finding on February 27, 2017.
Why the NRC Reassessed the Failure
It took the NRC over two months to finalize an uncontested preliminary finding because Exelon essentially contested the preliminary finding, but not in the way used by the rest of the industry and consistent with the NRC’s longstanding procedures over the 17 years that the agency’s Reactor Oversight Process has been in place.
Instead, Exelon mailed a letter dated January 12, 2017, to the NRC requesting that the agency improve the computer models it uses to determine the significance of events. Exelon whined that NRC’s computer model over-estimated the real risk because it considered only the failure of a standby component to start and the failure causing a running component to stop. Exelon pointed out that the auxiliary oil pump did permit the HPCI system to successfully start during the June 2016 test run and it later catching on fire did not disable the HPCI system. Exelon whined that the NRC’s modeling was “analogous to the situation where the starter motor of a car breaks down after the car is running and then concluding that ‘the car won’t run’ even though it is already running.”
The NRC carefully considered each of Exelon’s whines in its January 12 letter and still concluded that the failure warranted a white finding. So, the agency issued a white finding. With respect to Exelon’s whine that the auxiliary oil pump burned up after the HPCI system was up and running, the NRC reminded the company that the operators shut down the HPCI system in response to the alarms—had it been necessary to restart the HPCI system, the toasted auxiliary oil pump would have prevented it. It is not uncommon for the HPCI system to be automatically shut down (e.g., due to high water level in the reactor vessel) or to be manually shut down (e.g., due to operators restoring the vessel water level to within the prescribed band or responding to a fire alarm in the HPCI room) only to be restarted later during the transient. The NRC’s review determined that their computer model’s treatment of a “failure to restart” would yield results very similar to its treatment of a “failure to start.”
The auxiliary oil pump’s impairment reduced the HPCI system to one and done use. In an actual emergency, one and done might not have cut it—thus, NRC issued the white finding for Exelon’s poor performance that let the auxiliary oil pump literally go up in smoke.
The NRC conducted a public meeting on May 2, 2017, in response to Exelon’s letter. I called into the meeting to see if Exelon’s whines are as shallow and ill-conceived as they appear in print. I admit to being surprised—their whining came across even shallower live than in writing. And I would have bet it impossible after reading, and re-reading, their whiny letter.
What’s With the Whining?
Does Exelon hire whiners, or does the company train people to become whiners?
It’s a moot point because Exelon should stop whining and start performing.
Exelon whined that the NRC failed to recognize or appreciate that the auxiliary oil pump is only needed during startup of the HPCI system. During the June 2016 test run, the HPCI system successfully started and achieved steady-state running before the auxiliary oil pump caught fire. Workers put out the fire before it disabled the HPCI pump. But the NRC’s justification for the final white characterization of the “uncontested” finding explained why those considerations did not change their conclusion. While the auxiliary oil pump did not catch fire until after the HPCI system was successfully started during the June 2016 test run, its becoming toast would have prevented a second start.
Exelon expended considerable effort contesting and re-contesting the “uncontested” white finding. Had Exelon expended a fraction of that effort properly cleaning and inspection the auxiliary oil pump motor, the motor would not have failed in March 2015. Had Exelon expended a fraction of that effort properly setting control parameters when the failed motor was replaced in March 2015, it would not have caught on fire in June 2016. If the motor had not caught on fire in June 2016, the NRC would not even have reached for its box of crayons in December 2016. If the NRC had not reached for its box of crayons, Exelon would not have been whining in January and May 2017 that the green crayon instead of the white one should have been picked.
So, Exelon would be better off if it stopped whining and started performing. And the people living around Exelon’s nuclear plants would be better off, too.