The Nuclear Regulatory Commission (NRC) sent a special inspection team to the River Bend nuclear plant near St. Francisville, Louisiana on February 8, 2016, to investigate an event in which cooling of the reactor core was interrupted for over an hour on January 10 when the reactor was shut down. The NRC’s special inspection team identified four violations of regulatory requirements characterized as Green findings, the least serious among the agency’s green, white, yellow, and red classifications.
How the Event Unfolded
At 2:37 am on January 9, 2016, one of the 230,000 volt transmission lines connecting the plant to the offsite electrical power grid developed a fault (essentially a short circuit). That fault caused a transient as electricity being produced by the plant stopped flowing through the faulted line and was rerouted to the remaining transmission lines. During that electrical transient, the voltage of the electrical power supply to both of the reactor protection system’s (RPS) divisions dropped low enough to trigger the rapid, automatic shut down of the reactor from 100 percent power. Following the unplanned shutdown, management decided to cool the reactor down and enter an outage for maintenance and inspection tasks.
At 2:47 am on January 10, the reactor was shut down with the temperature of the water flowing through the reactor vessel cooled to 128°F. The reactor water was being cooled by the one pump within the Residual Heat Removal (RHR) system (Fig. 1). Both of the recirculation pumps were running to promote flow through the reactor core. RHR Pump A was running. It was taking water from the suction piping between the reactor vessel and Recirculation Pump A. RHR Pump A sent this water through two heat exchangers in series where it was being cooled by the plant’s cooling water system. The cooled water was returned to Feedwater Line A which carried it back into the reactor vessel.
Fig. 1 (Source: Union of Concerned Scientists)
The RHR system is a low pressure system. Its piping and components are not designed for the high pressure that exists within the reactor vessel when the plant is operating. Interlocks are installed that automatically close valves to isolate the low pressure system from high pressure conditions. One such interlock closes valves F008A and F053A (see Fig. 1) when the pressure inside the reactor vessel exceeds 135 pounds per square inch (psi).
Plant procedures directed workers to bypass this specific interlock when the RHR system is cooling the reactor water and its temperature is 200°F and below. Bypassing this interlock is intended to improve the reliability of the RHR shutdown cooling operation. In this configuration, the single RHR pump may be the only cooling mode available. Bypassing this interlock does not unduly jeopardize the low pressure RHR system because other interlocks remain available to automatically close the necessary valves should reactor vessel pressure rise above 200 psi.
To bypass the 135 psi interlock, workers install a wire, called a jumper, in parallel with the interlock’s relay. If reactor vessel pressure exceeds 135 psi, the relay opens to break the logic circuit. Normally, the break would trigger the automatic closure of the valves. With the jumper installed, the relay’s opening does not break the logic circuit.
When workers tried to install the jumper, its metal tip contacted a nearby part of the panel causing an electrical short. The short caused the very outcome that the jumper’s installation sought to prevent—the automatic closure of valves F008A and F053A and the loss of shutdown cooling. As these valves closed, RHR Pump A automatically shut down to protect it from overheating by running but not moving water.
Workers looked at restoring shutdown cooling by either restarting RHR Pump A or by starting RHR Pump B instead. They decided it would be quicker to restore cooling using RHR Pump A. Time was a critical factor because the reactor water’s temperature was increasing about 1°F per minute.
At 2:50 am, just three minutes after shutdown cooling was inadvertently lost, the control room operators dispatched a worker to go out into the plant to manually re-open valve F008A. But the valve was located in an area where radiation levels were above background levels. The worker had to interface with the Radiation Protection staff to get the paperwork completed to allow that radiation area to be entered. It took awhile to fill out the paperwork. The worker was finally able to open valve F008A at 3:51 am, more than an hour after being sent out to open it.
The control room operators restarted RHR Pump A at 4:01 am. The reactor water’s temperature had risen to 196.7°F. About ten minutes later, the reactor water’s temperature had decreased to 175.8°F. The workers restored cooling with about three minutes to spare. Had the reactor water’s temperature risen above 200°F, the plant would have transitioned from Operational Mode 4, Cold Shutdown, into Operational Mode 3, Hot Shutdown (Fig. 2). Entering a higher Operational Mode is only legal when all the tests necessary to verify availability of safety equipment needed in that mode have been successfully completed. Because those tests had not been performed, the plant may have entered a mode without the safety equipment needed to manage risks in that configuration.
Fig. 2 (Source: Nuclear Regulatory Commission)
The NRC’s special inspection team determined that neither the unplanned reactor shut down nor the subsequent loss of shutdown cooling in January 2016 were first-time occurrences at River Bend. On November 27, 2015, one of the 230,000 volt transmission lines connecting the plant to the offsite electrical power grid developed a fault (essentially a short circuit). That fault caused a transient as electricity being produced by the plant stopped flowing through the faulted line and was rerouted to the remaining transmission lines. During that electrical transient, the voltage of the electrical power supply to both of the RPS’s divisions dropped low enough to trigger the rapid, automatic shut down of the reactor from 100 percent power. Normally, the RPS’s two divisions are powered from independent supplies. On November 27, 2015, and on January 9, 2016, workers had reconnected both RPS divisions to the same power supply. When that power supply experienced problems, it affected both divisions of the reactor protection. Under the normal configuration, a single power supply problem could not affect both divisions to cause an automatic shut down of the reactor.
The NRC’s special inspection team concluded that while some corrective actions had been taken following the November 27 event, those actions did not resolve the underlying problem and therefore failed to prevent the January 9, event from occurring.
The NRC’s special inspection team further identified that the procedure governing the lineup for configuring electricity supplies to plant equipment had been revised in 2012 to state a strong preference for powering the two RPS divisions from separate sources due to “the superior protection” it provides against “unintended actuations caused by voltage transients.” Even though powering both RPS divisions from the same power source caused the November 27 event, this configuration remained in place to cause a repeat event on January 9.
On June 23, 1994, workers attempting to bypass the 135 psi interlock inadvertently dropped one end of the jumper into the electrical panel. The metal tip caused an electrical short that automatically closed valves F008A and F053A, which in turn caused RHR Pump A to stop running. The plant was in Operational Mode 4 at the time. Workers restored shutdown cooling in 18 minutes. To prevent electrical shorts in the future when bypassing the 135 psi interlock, workers revised the procedure to require that the logic circuit be de-energized while the jumper was being installed. Once the jumper was properly installed, the procedure had worker re-energize the logic circuit. Five years later, the procedure was revised to allow the logic circuit to remain energized during installation of the jumper. The procedure directed workers to use a special type of jumper that had a rubber protective cover for its metal tips as protection against electrical shorts.
The NRC’s special inspection team found that while workers used the revised procedure on January 10, they failed to use the special jumper specified by the procedure. Instead, they used one of the older jumpers with unshielded metal tips to essentially recreate the June 23, 1994, event.
NRC Sanctions
The NRC’s special inspection team identified four violations of regulatory requirements, all classified as green—the least severe in the agency’s green, white, yellow and red hierarchy:
- Failing to follow procedures when bypassing the 135 psi interlock by using an unapproved jumper.
- Failing to take effective corrective action to preclude repetition of a significant condition adverse to quality after the November 27, 2015, unplanned reactor shutdown.
- Failing, since the practice was initiated in 2012, to access the risk of simultaneously powering both RPS divisions from the same source.
- Failing to develop adequate procedures needed by plant workers to respond in a timely and effective manner to the unplanned loss of shutdown cooling.
The NRC binned the first three violations in the Initiating Events cornerstone of the agency’s Reactor Oversight Process and the fourth violation in the Mitigating Systems cornerstone.
UCS Perspective
“Rinse, Lather, Repeat” are common instructions provided on bottles of shampoo, presumably for protection against consumers drinking the product hoping to get clean, manageable hair.
“Event, Survive, Repeat” are all-too-common explanations for why River Bend and other reactors operated by Entergy receive so many visits from the NRC’s special inspection teams. It happens so often that special inspection team members rack up lots of frequent flyer miles.
Nuclear plants are large industrial facilities employing hundreds of workers who perform thousands of tasks every week. It would be impossible for any nuclear plant not to experience equipment malfunctions and worker miscues.
Top performing companies learn as much as they can from each malfunction and miscue so as to adjust what they do and how they do it seeking to reduce the number of such problems in the future. Reducing the number of problems has the double benefit of improving safety and reducing costs.
Other companies, like Entergy in this case, either don’t extract such lessons or don’t make the adjustments based on the lessons needed to lessen recurrence of the problems.
The NRC cannot force companies to be great at learning lessons and applying remedies. But the NRC can, and should, prevent companies from being lousy at it. When a company demonstrates again and again its inability to find and fix safety problems in a timely and effective manner as required by NRC’s regulations, the NRC must stop enabling poor performance.
River Bend’s procedures would have avoided the January 9 unplanned reactor shut down and the January 10 undesired loss of reactor core cooling, but only if they had been followed.
NRC’s regulations will prevent inadequate procedures from someday factoring in an accident, but only if they are enforced.
Failing to follow procedures is wrong.
Failing to enforce safety regulations is wrong.
Two wrongs still don’t make it right.