Substation Physical Security Initiatives
GridSME Reliability & Compliance Team • Mar 30, 2020

Have you ever been curious about what events lead to the increase of regulations towards increasing physical security protection of your electric substations? In this article we will take a dive into the events that lead to the standards and how those standards effect your substations.


In the electric energy industry, a predominant compliance driver has focused on cyber security of the bulk electric system
controls. In 2013, the focus on cyber security compliance issues for electric utilities was augmented with some new physical
security requirements. In this article, I’ll introduce you to the events that led to the development of these new regulations and how you can development and implement your new physical security protocol that meets these new guidelines.


Metcalf Substation Attack

The Executive Order seeks to mitigate well known, and long-standing, cyber security supply chain risks. There is no doubt that
on the night of April 15, 2013, several very well-informed attackers caused physical damage to Pacific Gas & Electric’s large 500
kv/230kv Metcalf Transmission Substation located south of San Jose, California. Beginning at about 1:00 AM, the attackers cut
two fiber communication lines. After they were finished, they resealed the telecom vaults and spread garbage in the area to help draw attention away from their actions.


At 1:31 AM the attackers began shooting at the substation transformers and circuit breakers. Ten of the 11 transformers were
struck. It appeared that the attackers only shot at the “hot” transformers (one was down for maintenance). By 1:45 AM the
transformers had begun to shut down, presumably, due to low cooling oil levels and low oil pressures.


The subsequent investigation after the shooting identified 116 impact points on 22 pieces of equipment and 52,000 gallons of
transformer oil was spilled onto the base of the substation foundation. It was also determined during the investigation that the slow response time of the employees and inaccessibility of the substation allowed the perpetrator(s) enough time to get escape before the police arrived on scene.

TIMELINE OF THE EVENTS
1:00 AM Attacker(s) cut two fiber communication lines
1:31 AM Attacker(s) began to shoot transformers and circuit breakers
1:41 AM 911 called
1:48 – 1:50 AM Investigators estimated the attacker(s) stopped shooting
1:51 AM Police arrived on scene but couldn’t enter substation due to the gates being locked.
3:25 AM Utility electrician arrived on scene


Metcalf Attack Regulatory Response
After the attack, there was a buzz within the electric energy and security industries, which was filled with conjecture as to the true motivation(s) behind the attack. The local transmission system operators were able to effectively bypass Metcalf, which resulted in no reported electricity outages in the surrounding area.


The “buzzing” reached a crescendo on March 7, 2014 when the Federal Energy Regulatory Commission (FERC) – the government entity regulating the interstate transmission of electricity, natural gas, and oil in the US – directed the North American Electric Reliability Corporation (NERC) to submit a proposed physical security reliability standard for electric transmission substations within 90 days.


The focus of this proposed standard was to “Identify and protect facilities that if rendered inoperable or damaged could result in widespread transmission grid instability, uncontrolled electric network separation, or cascading failure within a transmission interconnection.”


Such a demand from FERC is not abnormal; however, their mandate for a rapid turnaround and production by NERC standards committees was unheard of. Everyone realized FERC was taking the Metcalf event seriously.
The response from NERC was the development of NERC Critical Infrastructure Protection (CIP) Standard 014, “Physical Security.” The final ballot closed on May 5, 2014 – passing at 85%. The NERC Board of Trustees adopted CIP-014 on May 13, 2014. FERC approved this new standard on July 17, 2014 resulting in an effective implementation date of October 1, 2015.


This may seem slow; however, in the regulatory environments of FERC and NERC this was “SUPERFAST!”


CIP-014, Substation Physical Security Standard

The CIP-014 requirements are summarized in the table shown below but the you can view the full
CIP-014 standards by visiting NERC.com or by (clicking here).


REQUIREMENT GOAL
R1 Attacker(s) cut two fiber communication lines
R2 Independent Review of Initial Risk Assessment (R1)
R3 Coordination Between Grid Operator (e.g., ISO) and Owner/Utility
R4 Threat and Vulnerability Assessment
R5 Development and Implementation of Physical Security Plan for Critical Substations
R6 Qualified Third-Party Assessment of Plans Developed in R4 and R5


Since the initial introduction of CIP-014, all North American electric utilities have implemented plans to address the requirements specified within the legislation and most have been audited to ensure the effectiveness of their plans.


California Doesn’t Want to Be Left Behind
While FERC and NERC were busy developing new physical security standards for large electric transmission substations, California lawmakers also showed their concern regarding the Metcalf attack by passing new legislation in June 2015 called Senate Bill 699 (aka, SB-699).


The new legislation addressed two major areas when it effective on January 1, 2016:

  • Directed the California Public Utilities Commission (CPUC) to explore policies and practices related to physical security of
    electric distribution assets (not transmission – that is FERC/NERC’s jurisdiction!)
  • Directed the CPUC to consider adoption of new standards and rules to address any physical security risk to the distribution system of California’s electric corporations so as to ensure “high-quality, safe, and reliable service.”


SB-699 Development

The actual legislation included in SB-699 is not very detailed. The document is only a few paragraphs long. So, to develop the detailed response to SB-699, the CPUC held three workshops to gather information from the affected California utilities and get a sense of the necessary parameters to include in the actual rules.


The first workshop was held on May 2, 2017 and was focused on information sharing, sensitive data protection, and confidentiality of critical energy infrastructure information. The workshop also established proceeding rules of engagement for input and testimony on sensitive subjects.


The second workshop was held on May 31, 2017 where the discussions were about state, federal, and industry standards and
responses including CIP-014.


On June 21, 2017, the final workshop was held to address how SB-699 informs CPUC response and responsibilities. The meeting
also included discussions on threat assessment, critical substation protections, and incident response resiliency.


Of note, on July 12, 2017, as part of the SB-699 response development process, an administrative law judge issued a ruling requesting straw proposals from the stakeholders on what the SB-699 rules should include.


January 2018 – CPUC Staff White Paper
After digesting the workshop notes and straw proposals the CPUC issued a very informative white paper entitled Security
and Resilience for California Electric Distribution Infrastructure: Regulatory and Industry Response to SB-699. The primary
topics in the paper included:

  • Electric utility physical security in the Post-Metcalf era
  • Distribution asset security and resiliency in California
  • Incident reporting and tracking best practices
  • Exchange of and access to highly confidential and sensitive information
  • Utility general rate cases informing physical security efforts, and
  • Recommendations


This white paper gives you an excellent sense of the State of California electric distribution grid and the challenges with keeping the critical infrastructure information secured from general knowledge – especially from the terrorists and threat actors.


January 22, 2019 – Physical Security Decision
Almost a year after the CPUC white paper was issued – the CPUC approved the “Physical Security Decision (D.19-01-018).”
This made California the first U.S. state to adopt rules to guard the electric distribution grid against terrorist attack.


The decision established general criteria for the identification phase and criteria for the assessment phase, which clearly
defined what parameters are used when deciding what distribution substations should be tagged as critical, and what are the rules that should be followed for the physical security assessments of the identified subs.


Identification Criteria
The general rules in the Physical Security Decision for identifying and declaring which distribution substations are critical
include those that:

  • Are needed for crank path, black start, restoration of the regional grid
  • Are an electric power source for a military installation
  • Serve a regional water and wastewater facilities
  • Serve a regional public safety (e.g., 911 center)
  • Serve a major transportation facility (e.g., Los Angeles International Airport)
  • Serve a Level 1 Trauma Center
  • Serve > 60,000 electric meters


Assessment Phase Criteria
Once the more critical distribution substations are identified – as noted above – the assessment phase will look at such issues as:

  • Existing system resiliency and/or redundancy solutions
  • Spare assets to restore a particular load
  • Existing physical security protections
  • Potential for emergency responders to identify and respond to an attack in a timely manner
  • Location/proximity to gas pipelines, geographical challenges, impacts of weather, etc…
  • History of criminal activity affecting the substation


Cost Recovery
The decision does allow for the investor-owned utilities to file separate applications for cost recovery associated with their respective distribution security programs. Although the Distribution Security Program documents are considered security-sensitive and cannot be publicly released, the investor-owned utilities may file a public version of the unaffiliated third-party review and CPUC approval in their cost recovery requests.


Security Decision Timeline
According to the Security Decision, each utility’s Security Plan Report is due to the CPUC within 30 months of the approval of the Decision (estimated to be July 2020). The third-party reviews should be completed by April 2021.


Conclusion
The events of the Metcalf substation shooting were profound and a bit rattling to the electric energy utilities and regulators. CIP-014 came out of the event thus affecting most major North American electric transmission utilities. Now California Senate Bill 699 and the derivative Security Decision of January 2019 has been issued thus affecting most electric utilities in California. These two events show a trend towards increased physical security protection of electric substations.

There are solar panels and power lines in the background.
By Torri Pugh 02 May, 2024
Insights from GridSME: California's Renewable Energy Roadmap GridSME's Subject Matter Experts, Gary Brown and Lowell Watros, contributed valuable insights to this article for Reuters. Their expertise helped provide information and analysis on California's latest grid plan, particularly concerning grid improvements, renewable energy projects, and an expected surge in solar and storage projects. With their input, the article offers a comprehensive understanding of the developments in California's energy landscape and the implications for the renewable energy sector. Below are some of the key insights and takeaways from the article.
By Torri Pugh 30 Apr, 2024
🚨 NERC Alert: Assessing 6 GHz Communication Impact on Electric Industry NERC has issued a Level 2 Alert concerning 6 GHz Communication Penetration in the Electric Industry, targeting Distribution Providers (DP), Distribution Provider UFLS-Only (DP-UFLS), Generator Owners (GO), and Transmission Owners (TO). The alert aims to evaluate reliance on 6 GHz technology and potential interference severity. Respondents must address a series of questions by July 8, 2024, to assess the situation comprehensively. Although only registered entities can respond directly, the alert is publicly available. Action items include acknowledging the alert and completing the questionnaire. Recommendations include identifying critical circuits and conducting baseline performance tests. It's crucial to note that these recommendations are distinct from requirements, as clarified by NERC. This alert underscores the necessity of assessing 6 GHz communication's impact on industry reliability and security. Stakeholders should stay informed and promptly address the alert's requirements. For further details, visit NERC's website: Link . See our LinkedIn post about the NERC Alert: here .
Share by: