The owner of the two boiling water reactors (BWRs) at the Susquehanna Steam Electric Station in northeastern Pennsylvania notified the Nuclear Regulatory Commission (NRC) on April 2, 2018, that workers’ mistakes rendered an emergency core cooling system on Unit 1 vulnerable to being disabled by an earthquake at the same time that another emergency core cooling system was out of service for work on its power supply system. This is good news—not in having two safety systems impaired while the reactor operated, but in how quickly the problem was detected and corrected.
Fig. 1 (Source: Nuclear Regulatory Commission)
The Emergency Core Cooling Systems
Susquehanna Unit 1 is a model BWR/4 reactor with a Mark II containment design that was placed into commercial operation in June 1983. In case of an accident that drains cooling water from the reactor vessel, Unit 1 is equipped with an array of emergency core cooling system (ECCS) pumps that will automatically start and provide makeup water. The ECCS include one steam-driven high pressure coolant injection (HPCI) pump, four motor-driven low pressure coolant injection (LPCI) pumps, and more motor-driven core spray (CS) pumps. The LPCI and CS pumps are split into two divisions of two LPCI pumps and two CS pumps each. Each division is powered from separate electrical buses, backed by separate emergency diesel generators, to increase the chances that enough pumps survive whatever challenge is experienced to provide adequate makeup cooling water flow for the reactor core.
The Situation
During the early afternoon of December 1, 2017, workers moved pipe sections into the room housing the Division II core spray pumps and staged this material on the floor as close as six inches from one of the two air conditioning units for the room.
At 7:48 am on December 2, the power supply to the Division II low pressure coolant injection pumps was removed from service to enable its voltage regulator to be replaced.
The Problem
At 10:30 am on December 3, an operator noticed that the materials staged in the core spray pump room were not seismically restrained and were close to one of the room’s air conditioning unit. The Operations department conservatively assumed that an earthquake could case the pipe sections to move into and damage the air conditioning unit. If that occurred, the heat from the running core spray pump motors could warm the room above the temperature that electrical equipment was qualified to endure. The Operations department declared the Division II core spray pumps inoperable due to their potential loss in event of an earthquake.
The Unit 1 operation license allowed the Division II low pressure coolant injection pumps to be out of service for up to 7 days while the reactor continued operating. This allowed outage time relied on other ECCS pumps being available in case an accident happened. The discovery that the Division II core spray pumps were also inoperable undermined that reliance. The operating license for Unit 1 required the reactor to be shut down within 7 hours with both the Division II low pressure coolant injection and core spray pumps inoperable.
The Solution
At 1:35 pm on December 3, the Division II low pressure coolant injection pumps were restored to operable following replacement of the voltage regulator on their power supply. Their restoration ended the need for the reactor to be shut down and returned the unit to the need to restore the Division II core spray pumps to service within 5 days (the 7-day clock started on December 1).
Around 4:00 pm on December 3, workers completed the removal of the pipe sections from the Division II core spray pump room. Doing so ended the need to shut down the reactor as all ECCS pumps were restored to service.
The Armchair Viewpoint
The Engineering department analyzed the temperature in the Division II core spray pump room with both motor-driven core spray pumps running and only one of two air conditioning units in the room operating. The second air conditioning unit was assumed not to be running due to damage from the pipe sections hitting it during an earthquake. The engineering analysis concluded that the room temperature would have remained below the temperatures used to qualify safety components in the room and that the core spray pumps would have performed their safety function successfully.
UCS Perspective
The staging of the replacement pipe sections without seismic restraints in the Division II core spray pump rooms near its air conditioning unit could have resulted in an air conditioning unit becoming damaged during an earthquake. That potential vulnerability was not recognized the next day when the Division II low pressure coolant injection pumps were taken out of service for maintenance to their power supply. The defense-in-depth approach to nuclear safety gets undermined when multiple layers are missing and/or impaired concurrently.
It would have been better had the pipe sections not been staged improperly or had that mistake been identified before it was compounded by the intentional disabling of additional ECCS pumps the next day. But dozens of activities are ongoing each and every day at a nuclear power plant. And materials temporary stored in the core spray pump room—a confined area infrequently accessed by workers on a daily basis—made detection of their improper configuration less than readily evident.
The mistake was identified by the Operations department less than two days after it was made and a day after it was compounded by taking other ECCS pumps out of service. It would have been easy not to have discovered the subtle mistake, but it was found. Once found, it would have been easy to presume that the core spray pumps would have functioned despite the potential loss of one of two air conditioning units in the room. But the Operations department lacked an analysis to support that presumption and declared the pumps inoperable. That conservative call accelerated the solution to the problem. Within about 185 minutes, the low pressure coolant injection pumps were restored to service. And within 330 minutes, the pipe sections were removed to eliminate the potential hazard to the air conditioning unit in the core spray pump room. The Operations department handled this matter very well. The Operations department handled this matter very well.
Defense-in-depth is frequently discussed in terms of equipment—two redundant pumps provided when only one needs to run for the necessary safety function to be fulfilled. This case illustrates how defense-in-depth also has an important role to play in human performance reliability. The Maintenance department placed the pipe sections in the core spray pump room. They should have stored the material properly, but failed to do so. The Operations department caught the mistake and caused it to be promptly remedied. And the Engineering department reviewed the mistake to determine its safety significance.
This event also reveals an unintended consequence from defense-in-depth applied to human performance reliability—when the first defense-in-depth layer succeeds, backup layers are not tested. Here, the first layer failed but the second and third layers came through. The next best thing to perfection is having a highly reliable first layer backed by a highly reliable second layer backed by a highly reliable third layer and so on.