The Arizona Public Service Company’s Palo Verde Generating Station about 60 miles west of Phoenix has three Combustion Engineering pressurized water reactors that began operating in the mid 1980s. In the early morning hours of Thursday, December 15, 2016, workers started one of two emergency diesel generators (EDGs) on the Unit 3 reactor for a routine test. The EDGs are the third tier of electrical power to emergency equipment for Unit 3.
When the unit is operating, the source of power is the electricity produced by the main generator (labeled A in Figure 1.) The electricity flows through the Main Transformer to the switchyard and offsite power grid and also flows through the Unit Auxiliary Transformer to in-plant equipment. If the unit is not operating, electrical power flows from the offsite power grid through the Startup Transformer (B) to in-plant equipment. When the main generator is offline and power from the offsite power grid is unavailable, the EDGs (C) step in to provide electrical power to a subset of in-plant equipment—the emergency equipment needed to protect the reactor core and minimize release of radioactivity to the environment. An additional backup power source exists at Palo Verde in the form of gas turbine generators (D) that can supply power to any of the three units.
Fig. 1 (Source: Arizona Public Service Company)
I toured the Palo Verde site on May 11, 2016. The tour included one of EDG rooms on Unit 2 as shown in Figure 2. Each unit at Palo Verde has two EDGs. The EDG being tested on December 15, 2016, was manufactured in 1981 and was a Cooper Bessemer 20-cylinder V-type turbocharged engine. The engine operated at 600 revolutions per minute with a rated output of 5,500,000 watts.
Fig. 2 (Source: Arizona Public Service Company)
Assuming one of the two EDGs for a unit fails and there are no additional equipment failures, the remaining EDG and the equipment powered by it are sufficient to mitigate any design basis accident (including a loss of coolant accident caused by a broken pipe connected to the reactor vessel) and protect workers and the public from excessive exposure to radiation. Figure 3 shows the major components powered by the Unit 3 EDGs—a High Pressure Safety Injection (HPSI) train, a Low Pressure Safety Injection (LPSI) train, a Containment Spray train, an Essential Cooling Water Pump, an Auxiliary Feedwater Pump, and so on.
Fig. 3 (Source: Arizona Public Service Company Individual Plant Examination)
Because the EDGs are normally in standby mode, the operating license for each unit requires that they be periodically tested to verify they remain ready to save the day should that need arise. At 3:02 am on December 15, 2016, workers started EDG 3B. Workers increased the loading on EDG 3B to about 2,700,000 watts, roughly half load, at 3:46 am per the test procedure.
Ten minutes later, alarms sounded and flashed in the Unit 3 Control Room alerting operators that EDG B had automatically stopped running to due low lube oil pressure. A worker in the area notified the control room operators about a large amount of smoke as well as oil on the floor of the EDG room. The operators contacted the onsite fire department which arrived in the EDG room at 4:06 am. There was no fire ongoing when they arrived, but they remained on scene for about 90 minutes to assist in the response to the event.
Operators declared an Alert, the third most serious in the NRC’s four emergency classifications, at 4:10 am due to a fire or explosion resulting in control room indication of degraded safety system performance. The emergency declaration was terminated at 6:36 am.
Seven weeks later after the fire had long been out, the oil on the floor long since wiped up, and all sharp-edged metal fragments long gone, and any toxic smoke long dissipated, the Nuclear Regulatory Commission (NRC) dispatched a special inspection team to investigate the event and its cause. The NRC dispatched its special inspection team more than a month after it authorized Unit 3 to continue operating for up to 62 days while its blown-up backup power source was repaired. The Unit 3 operating license originally allowed the reactor to operate for only 10 days with one of two EDGs out of service.
Workers at Palo Verde determined that EDG 3B failed because the connecting rod on cylinder 9R failed. It was the fifth time that an EDG of that type at a US nuclear power plant experienced a connecting rod failure and it was the second time that Cylinder 9R on EDG 3B at Palo Verde. It had also failed during a test in 1986.
Examinations in 2017 following the most recent failure traced its root cause back to the first failure. The forces resulting from that failure caused misalignment of the main engine crankshaft. (In this engine, the crankshaft rotates. The crankshaft causes the connecting rods to rise and fall with each rotation, in turn driving the pistons in and out of the cylinders.) The misalignment was very minor—the tolerances are on the order of thousands of an inch. But this minor misalignment over hundreds of hours of EDG operation over the ensuing three decades resulted in high cyclic fatigue failure of the connecting rod.
Workers installed a new crankshaft aligned within the tight tolerances established by the vendor. Workers also installed new connecting rods and repaired the crankcase. After testing the repairs, EDG B was returned to service.
NRC Sanctions
The NRC’s special inspection team did not identify any violations contributing to the cause of the EDG failure, in the response to the failure, or in the corrective actions undertaken to remedy the failure.
UCS Perspective
The NRC’s timeline for this event isn’t comforting.
The operating licenses issued by the NRC for the three reactors at Palo Verde allow each unit to continue running for up to 10 days when one of two EDGs is out of service. The Unit 3 EDG that was blown apart on December 15 could not be repaired within 10 days. So, the owner applied to the NRC for permission to operate Unit 3 for up to 21 days with only one EDG. But the EDG could not be repaired within 21 days. So, the owner applied to the NRC for permission to operate Unit 3 for up to 62 days with only one EDG.
The NRC approved both requests, the second on January 4, 2017. More than a month later, on February 6, 2017, the NRC special inspection team arrived onsite to examine what happened and why it happened.
Wouldn’t a prudent safety regulator have asked and answered those questions before allowing a reactor to continue operating for six times as permitted by its operating license?
Wouldn’t a prudent safety regulator have ensured the cause of EDG 3B blowing itself apart might not also cause EDG 3A to blow itself apart before allowing a reactor to continue operating for two months with a potential explosion in waiting?
Whether the answers are yes or no, could that prudent regulator please call the NRC and share some of that prudency? The NRC may be many things, but it’ll seldom be accused and never be convicted of excessive prudency.
Where’s a prudent regulator when America needs one?