Disaster by Design/ Safety by Intent #52
Safety by Intent
The discovery of significant corrosion to the reactor vessel head at the Davis-Besse nuclear plant in Ohio gave the Nuclear Regulatory Commission (NRC) a figurative black eye. On the same day in April 2002 that the NRC announced it rated Davis-Besse one of the top performing nuclear plants in the country, the agency reported that the corrosion spanning several years at the plant had compromised safety margins more than any event since the Three Mile Island accident in March 1979.
The well-deserved black eye overshadowed what had been stellar performance by a regulator with eyes wide open seeing a safety problem and swiftly acting to effectively resolve it in a timely manner. Prior commentaries have chronicled the NRC’s shortcomings. This commentary covers the history before the NRC snatched defeat from the jaws of victory.
Oconee Eye-Opener
In March 2001, workers at the Oconee nuclear plant in South Carolina saw something neither they nor workers at the nation’s other pressurized water reactors expected to ever see—signs that water had leaked from the reactor vessel from cracks in the control rod drive mechanism (CRDM) nozzles that penetrated the dome-shaped head.
The control rods that regulate the power produced by the reactor core are connected by long metal poles to their motors mounted on a service platform above the reactor vessel head. These poles pass through the CRDM nozzles. The CRDM nozzles function as sleeves allowing the motors to move the poles as necessary to withdraw or insert control rods (Fig. 1).
Fig. 1 CRDM nozzle (Source: Nuclear Regulatory Commission)
The upper ends of the CRDM nozzles had flanges where the control rod drive mechanisms were bolted on. The flanged connections provided tight seals to prevent water inside the CRDM nozzles from leaking out. To prevent water from leaking out of the vessel around the edges of the CRDM nozzles, the lower ends of the nozzles are connected to the inner surface of the head by what are called J-groove welds. The CRDM nozzles are vertical whereas the head’s inner surface is curved. While the angles between the CRDM nozzle and the inner surface varied depending on whether the CRDM was near the center or nearer the edge of the head, the J-groove weld located at these intersections was believed to be where forces caused by expansion and contraction of metals being heated up and cooled down would be the largest.
Stress corrosion cracking is a longstanding problem in pressurized water reactors. Impurities in the water can find microscopic crevices in the metal walls and head of the reactor vessel. These impurities can start a corrosion process that gets accelerated by forces on the metal from thermal stresses. Because they were clearly the places where stresses were the largest, the standard industry practice reviewed and accepted by the NRC was to only inspect the J-groove welds and not examine the rest of the CRDM nozzles. The theory was that if the J-groove welds did not exhibit evidence of significant stress corrosion cracking, the remaining low-stress portions of the CRDM nozzle would be in even better condition.
But cracks formed in the CRDM nozzles at Oconee above the J-groove weld areas that were being inspected. The cracks grew slowly over time until they passed entirely through the walls of some CRDM nozzles, allowing water to leak out.
Nuclear plants have multiple means to detect leakage—radiation detectors that sense radioactive gases or particles within the leaked water, humidity detectors that sense the increasing amount of moisture inside containment, and level detectors that sense more water entering collection sumps in the containment’s basement. The cracks at Oconee were large enough to allow water to leak out, but small enough to limit the leak rate to below that amount that can be detected.
The leakage had entirely stopped by the time workers conducted their inspections at Oconee. The reactor had been shut down, depressurized, and the head removed to allow spent fuel assemblies in the reactor core to be replaced with fresh fuel. There was no longer pressure over 2,000 pounds per square inch to force water through the tiny cracks in the CRDM nozzles.
Fig. 2 (Source: Nuclear Regulatory Commission)
So workers did not see water spurting through the nozzles or observe puddles on the floor. Instead, they saw “collars” of white powder on the outer surface of the head surrounding some of the CRDM nozzles (Fig. 2). The water in pressurized water reactors is borated to help control the reactor power level. Boric acid is dissolved into the water. Workers can adjust the boron concentration of the water during reactor operation to compensate for fuel depletion and other factors. As water leaking from the CRDM nozzles evaporated, boric acid in the form of white powder was left behind as tell-tale evidence of a cracked and leaking nozzle.
NRC’s Ad Hoc Triage
Because cracks in CRDM nozzles outside of the J-groove weld region were unanticipated, neither the nuclear industry nor the NRC had a pre-planned response ready to go. Consequently, the NRC was forced to figure what was causing the unexpected cracking, which reactors might be particularly susceptible to cracking, and what to do about it to prevent it from undermining safety. Answering these questions would require considerable homework, and safety dictated that the NRC not take too long to find the answers.
Fortunately, while CRDM nozzle cracking outside of the J-groove weld regions was unanticipated, the factors contributing to cracks forming and growing in reactor vessel materials had received considerable attention by the nuclear industry and the NRC. The nuclear industry instituted the Materials Reliability Program (MRP) in the late 1990s. The NRC had closely monitored the development and implementation of this program. The MRP’s efforts included defining guidance on what components to inspect and how best to inspect them as well as methods to mitigate the initiation and growth of cracks.
The NRC issued Bulletin 2001-01, “Circumferential Cracking of Reactor Vessel Head Penetration Nozzles,” on August 3, 2001, to the owners of U.S. pressurized water reactors. The NRC required that owners look to see whether CRDM nozzles at their reactors also had the unexpected cracking.
The bulletin reflected a timely response by the NRC. The bulletin was issued roughly five months after the cracking problem first surfaced, but in time for owners to incorporate any additional inspections into reactor refueling outages already scheduled for the fall of 2001.
The bulletin also demonstrated thoughtful triage by the NRC. The bulletin did not require that all owners take the same measures on the same timeframe. Instead, it required that all owners take the appropriate measures to address the problem. The bulletin established a process to be used to determine how susceptible individual reactors were to CRDM nozzle cracking. More susceptible reactors had to take more steps more expeditiously. Less susceptible reactors needed to take steps, but at a justifiably slower pace.
NRC’s 20/20 Vision
The NRC required the owners of the most susceptible reactors to inspect their CRDM nozzles by December 31, 2001, even if an outage that fall had not been planned. The NRC permitted owners of less susceptible reactors to inspect their CRDM nozzles at later times, but required them to review past inspection records to confirm that no signs of degradation had been identified.
Fig. 3 (Source: NRC 2002 Regulatory Information Conference)
The CRDM nozzle inspections at the more susceptible reactors proved the NRC to have 20/20 vision. Eight of the top twelve most susceptible reactors had cracked CRDM nozzles that were leaking cooling water (Fig. 3). A ninth reactor had cracked CRDM nozzles, but the cracks had not yet penetrated all the way through the nozzles’ walls and leaked.
NRC’s Sustained Oversight
A little over a month after the NRC issued Bulletin 2001-01, terrorists piloted hijacked aircraft into the World Trade Center and the Pentagon. The NRC responded to the tragic events by identifying measures to be taken by owners to lessen nuclear plant vulnerabilities to sabotage attacks. My review of the CRDM nozzle cracking documents and the post-9/11 security documents did not find any evidence or even hints that either effort was hampered by insufficient resources or management oversight. Instead, it was clear that the NRC had the capacity and wherewithal to sustain the CRDM nozzle response plan while embarking on an equally important security response plan.
NRC’s Overlooked Success
Had it not been for Davis-Besse, the NRC’s response to the CRDM nozzle cracking reported at Oconee would have been an unqualified regulatory success. The agency responded quickly to an emerging hazard by accurately defining susceptibility of other reactors to this shared threat and requiring owners to take actions within timeframes determined by the identified threat level. As plant owners took the steps mandated by the NRC, the results confirmed that the NRC’s response plan was spot on.
Disaster by Design
The NRC properly identified Davis-Besse among the twelve reactors most susceptible to CRDM nozzle cracking. Davis-Besse appears as the green circle amid the red triangles on the left-hand side of the “Inspection Confirm Rankings” graphic (Fig. 3). It was the seventh most susceptible pressurized water reactor to this safety problem. The NRC had required that Davis-Besse’s CRDM nozzles be inspected by December 31, 2001. But the NRC granted the owner’s request to postpone the safety inspections until spring of 2002. When the deferred safety inspections were finally conducted, Davis-Besse went from being green circle on the graphic to a red triangle the size of the Great Pyramid.
After determining that highly susceptible reactors needed to do X, Y, and Z by a specified deadline for safety reasons, the NRC should not accept less than X, Y, and Z by that deadline. Setting a safety bar and then allowing reactors to limbo beneath it invites disaster—and Three Mile Island, Chernobyl, and Fukushima show that disaster sometimes accepts the invitation.
—–
UCS’s Disaster by Design/ Safety by Intent series of blog posts is intended to help readers understand how a seemingly unrelated assortment of minor problems can coalesce to cause disaster and how effective defense-in-depth can lessen both the number of pre-existing problems and the chances they team up.