Unsolicited Response

Patrick Miller joins Dale Peterson to discuss the months top three stories and then give their prediction, win and fail for the month. The stories

1) What to make of the US Government's efforts in ICS security in the first 100 days of the Biden administration.

2) The Pulse Secure VPN vulnerabilities and active exploitation, the impact on ICS asset owners, and what should be done.

3) ICS Security Training ... with Joel Langill back in the game Patrick and Dale look at the offerings.

TweetMe: @digitalbond

Get my Friday News & Notes: https://friday.dale-peterson.com/signup

Dale's article published on 4 May 2021. It shows how the Industrial Edge parallels Level 1 devices in capabilities using AWS as an example. And leaves a hanging question at the end. 

TweetMe: @digitalbond

Sign Up For My Friday Newsletter at https://friday.dale-peterson.com/signup

The ICS CYBERSEC 2021 event in Israel challenged speakers by limiting sessions to ten minutes. So Dale Peterson went with the theme and focused on how doing less in two important areas can actually help improve OT and ICS security.

1) Requiring less of engineers and operators (security is not everyone's responsibility)

2) Less cyber hygiene, more efficient risk reduction driven action

TweetMe: @digitalbond

Sign Up To My Friday Newsletter: https://friday.dale-peterson.com/signup

Dale's article originally published on 27 April 2020

Subscribe to my Friday ICS Security News & Notes email

TweetMe: @digitalbond

Here is a debate between Eric Byres and me from S4x14, January 2014 that centers on the most important issue of Level 1 / PLC insecurity.

The question in the debate: Is Eric Byres a SCADA Apologist or a SCADA Realist?

You will see from the clips at the start there was disagreement on stage between the two friends.

It came down to then, and in 2021, on whether you believe the fact that PLC's are insecure by design / lack authentication / will do whatever they are told by legitimate user or attacker is a problem that needs to be prioritized and can be addressed in the short run.

I add about six minutes of 2021 comments at the start and then 6:20 the debate starts. At 30:50 you get questions from the audience, and it is fun to see all the ICS security pioneers in the smaller S4 room mixing it up.

Tweet Me! @digitalbond

My Weekly Newsletter: friday.dale-peterson.com/signup

Last week a Bloomberg article covered the Biden Administration's plan for a 100-day sprint to secure the power grid. I'll comment on the three focus areas the article lays out and more broadly on 100-day efforts.

This is my article originally published on 20 April 2021.

Tweet Me! @digitalbond

My Weekly Newsletter: friday.dale-peterson.com/signup

Rapid 7 recently made a "Strategic Investment" in SCADAfence's $12M B round and announced integration with the SCADAfence product. Dale Peterson interviews Justin Prince of Rapid 7 on their OT vision, what the integration will and won't do, where the Rapid 7 solution will sit, and future product directions. You can use your judgment on the status and completeness of the Rapid7 OT offerings.

Then at 37:05 Mark Hyman, an OT Security recruiter with Verge Management Group, joins Dale to answer some OT Security job hunting questions that came in after Dale's recent article on solving the OT Cybersecurity Staffing challenge.

Subscribe to my Friday ICS Security News & Notes email.

This is a book that an ICS security professional should give to friends and family to read so they know why they do what they do. Nicole guides the lay person through her compelling journey to understand the 0day market and its impact on the security of the systems we all rely on. The ICSsec pro will find it to be interesting except for the parts on ICS / critical infrastructure where it is a historical fiction ... historical incidents extrapolated to their most dire possible results rather than presented in their true context.

This was a very difficult book to review. I'm conflicted because the story is engaging and will keep the lay person turning the pages. The 0day market through line is well told, and the theme and major points Nicole is making are clear and compelling. And yet the parts I know in detail, ICS security and critical infrastructure, are portrayed in a light that is misleading, and even deliberately misleading. Misleading because a lay reader, including government policymakers, would almost certainly conclude the US critical infrastructure at this moment is compromised and a click or two away from the Russians, or other adversaries, causing a major catastrophe.

The Story ... The Positive

Imagine you are trying to tell your Mom or Dad, your Husband/Wife/Partner, your close friend about cybersecurity risk and how it could affect their lives, their communities and their country or region. This is hard. If you get into the technical details you will lose them. If you try to add too much nuance (HT: RLee) you will lose them. There needs to be a captivating story that makes the non-technical audience keep reading even if they don't care about the tech.

Nicole has succeeded in this area by inserting herself into the story. She is not the heroine of the story. Instead she is the observer, the Nick Carraway from The Great Gatsby, who is observing the players in the 0day world who are neither pure heroes nor pure villains. She begins her journey naively at S4x13 in Miami Beach and investigates for over seven years. Never actually reaching a point of knowing the market, and yet she describes what she knows and what cannot be known.

The best parts of the book are when Nicole is an active character, walking through the world and talking to the players. You feel her frustration, fear, intimidation, dread and disgust. You want her to come out the other side with some answers or even the answer. Although to her credit she does not force a solution. The ending is actually more muddled than the beginning. It makes for a less satisfying journey, and it is more accurate.

Nicole's journey is the highlight of the book. The reason why you can recommend it to your Mom or Dad. It would have been even better for the lay reader had she not tried to add in the history, the details. It does not go deep into the technical details like Kim Zetter's Countdown to Zero Day, which can be viewed as a positive or negative. My view is if you are not going to push for technical accuracy, then less is better. Still the book is an interesting read as my family members can attest.

The Theme

I'm sympathetic with the theme that the US Government's focus on offense, in this book primarily the accumulation of 0days, has made the world more dangerous. We see this offense focus clearly and unapologeticly stated by NSA and Cyber Command across multiple leaders. The dominance of offensive theory and capabilities makes for a less stable world.

My hope for any policy makers reading this book is they reject the current philosophy of "we can't defend so we need to be able to attack first and potentially cause even greater damage". Nicole repeatedly shows where US actions to buy 0days resulted in an unexpected and negative result.

What is less certain is whether the 0day market was inevitable whether the US participated, or even led, in the early years. Nicole bemoans that "the cyberarms market was an incoherent mess". There were buyers and sellers reaching agreement, so it was not an incoherent mess. It was unregulated and led to undesirable outcomes in the past and likely in the future. However unless there are agreed upon cyber norms, similar to biological and chemical weapons, this was and is to be expected.

The Technical ... The Negative

I'm only qualified to comment on the ICS / Critical Infrastructure part of the book. My guess though is if you are part of the Vulnerability Equities Process (VEP), 0day market, Ecko Party, ... the parts of the book that discuss your area will be frustrating. I say this because anyone reading the ICS / Critical Infrastructure part of the book would come out with an incorrect understanding of the current capability of Russia and other adversaries to cause a catastrophic event using existing deployed exploits of the US critical infrastructure.

There is not a lot of factual detail in the book, again good for the lay person reader, and therefore creating an errata list wouldn't be a compelling case. In the ICS area, there was one major mistake on page 297:

It was an act of unprecedented digital cruelty, but the Russians stopped just short of taking lives. Six hours later, they flipped the power back on in Ukraine, just long enough to send their neighbor, and Kyiv's backers in Washington a clear message: "We can torch you".

This clearly implies that the Russians stopped their attack and turned the power back on in Ukraine. What actually happened was the Ukrainians went out to the substations and manually brought them back on line and operated them manually for many months. The SCADA system was down for about a year. Nicole was right that a "clear message" was sent.

This error on its own in a 400-page book would not be an issue. The issue is that every incident is presented in its worst possible light. Often not wrong by a strict parsing of the text, but misleading. A great example is Wolf Creek Nuclear plant on page 397:

the Russians were inside our nuclear plants ... The code made clear that Russia's hackers had breached the most alarming target of all: Wolf Creek, the 1200 mega-watt nuclear power plant near Burlington, Kansas. This was no espionage attack. The Russians were mapping out the plant's networks for a future attack; they had already compromised the industrial engineers who maintain direct access to the reactor controls ... And the goal wasn't to stop the boom. It was to trigger one.

Although she doesn't state it, this quote and the surrounding text would almost certainly be read as the Russians were in the nuclear control and safety systems. The reality is that an adversary had breached the office network at the Wolf Creek Nuclear Power Plant, but they had not yet been able to breach the ICS that controlled the nuclear plant nor the safety systems that would need to fail to cause "the boom".

Nicole wrote on page 392, "The technical community will argue I have overgeneralized and oversimplified, and indeed, some of the issues and solutions are highly technical and better left to them." When I had my interview with Nicole and wrote this review, this sentence kept running through my mind. After much introspection and consideration of this point, I do believe that this Wolf Creek example and many others in the book would lead the lay person to an incorrect understanding of the current state.

How different would a reader's understanding be if the Wolf Creek incident would have said the Russians were just outside the control and safety systems. Yes, they were knocking on the doors where accounting, HR, and other office functions take place, but they had not yet gotten in to plant operations or safety systems.

Another specific example is related to the Bowman Avenue Sluice Gate. To her credit Nicole notes in an early section that this is not Arthur R. Bowman dam in Oregon. However in the concluding chapter she writes,

"We've caught Iranian hackers rifling through our dams."

An Internet connected, ~5 meter wide, ~1 meter high sluice gate that keeps a neighborhood from flooding a couple of times a year is not a national security event and not worth noting as a reason for perilous concern in the concluding chapter.

Beyond the ICS security specifics, and probably more important, are the unsubstantiated contentions that the Russians and adversaries are in our systems and a click away from causing a catastrophic event. There are many in the book's text and in the interviews.

• Page 297 "By now, Russian hackers were so deeply embedded in the American grid and critical infrastructure, they were only one step from taking everything down".
• Pivot Podcast "Russia's in our government networks, they are in the grid, they've gotten into the power plants, we've seen them break into nuclear plants" "the worst case scenario is just one more minute away is because no one has actually used these accesses to turn off the power yet; it's two clicks away."  
• Page 380: "Russia invisibly worked their way into an untold number of nuclear and power plants around the country."

There are many more examples where the book's clear message is that the adversaries, Russians, Chinese, North Koreans, Iranians are able to cause a critical infrastructure catastrophe. The facts don't indicate this. As noted in the summary, Nicole has taken historical incidents and either extrapolated them to their most hysterical or left out the a sentence or two that would give the reader the correct impression. This approach is consistent throughout the text.

If the goal is to grab the lay reader by the shoulders and shake them saying this is important, it is a successful deception. Still it is nearly as scary without the hyperbole.

Recommendations

The final chapter includes a set of recommendations that are underwhelming. Vendors need to have a security development lifecycle (SDL) and put out better systems. The end users, the people need to be more security aware. In this area I don't fault Nicole because there are not easy answers. It might have been better to leave this chapter off.

One interesting suggestion was on Page 398:

We could start by passing laws with real teeth that mandate, for instance, that critical infrastructure operators refrain from using old, unsupported software; that they conduct regular penetration tests, that they don't reuse manufacturers' passwords; that they turn on multifactor authentication; and that they airgap the most critical systems.

This is NERC CIP, sans the air gap, that has been around for a decade plus.

If you've made it to the end of this book review, I hope you understand where the book succeeds and fails. Who it is written for, and who it is not written for. You and I are not the intended audience. The journey is compelling; the themes are on target; and maybe we should not get too upset that the specifics go beyond reality and are taken to their most extreme possibility.

Subscribe to my Friday ICS Security News & Notes email.

Dale Peterson interviews NY Times Reporter and author of the book This Is How They Tell Me The World Ends.

Some of the highlights include:

4:00 - the story of how Nicole began her 0day journey at a dinner at S4x13

19:42 - was the 0day market inevitable?

26:05 - how incentives have caused the problem and good help solve the problem

30:30 - the ICS / critical infrastructure attack detail in the book For regular followers, the last part is the most applicable.

This book was not written though for the ICS security pro, so some of the earlier discussion may help you understand who the intended reader is and the message Nicole is trying to get out there.

Subscribe to my Friday ICS Security News & Notes email.

Three answers.

1. Women

Women represent 51% of the population and 57% of the college graduates in the US. They comprise less than 10% of the OT Security workforce. 

Solving the problem could be as simple as adding women to the OT Security workforce until they reach close to their population percentage. Encouraging and recruiting them into the field, and treating them fairly once they are in, is key. (Of course the OT Security community should be welcoming to all and any addition will help with the shortage in the workforce. Women are singled out in this answer due to the numerical impact.)

2. Stop Searching For Unicorns

The OT Security unicorn has domain specific engineering, automation, IT and IT security skills and experience. OT Security unicorns do exist. I’ve seen a few. They are rare, and not the answer to the OT security workforce problem. Instead find people with one of those skills, an ability to learn, good communication skills so they can supplement the knowledge areas they lack, and a desire to be in OT Security.

3. Hire OT Security Professionals

Asset owners' OT Security programs fail when they try to force engineers and others in Operations to work on OT Security 10 or 20 percent of their time. Partially because they already have more than 100% allocated to existing job functions.

Asset owners' OT Security programs fail when they force an engineer and others in Operations to work primarily on OT Security when they don't want to. Engineers are fully capable of being OT Security pro's, if they want to. However, most would prefer to do the job they trained for and enjoy.

OT Security is a profession.

One last thought, this is not an OT v IT or OT is different than IT issue. There are many specializations under the big umbrella that is called IT. As Patrick Miller says, "it's all T". You need a workforce trained on the appropriate technology.

Subscribe to my Friday ICS Security News & Notes email.