CISA has issued a large number of documents during the Biden administration. Perhaps a flood the zone strategy to prove they are on it and how much they care. I’ve admittedly become a bit numb to reading them as they preach good practices for others with little CISA accountability.
And as I started into the well written platitudes in CISA’s 2024 – 2026 Strategic Plan I thought more of the same, until page 9. By the time I got to page 15 I was hopeful.
Before we get to the hope, let’s look back at the CISA 2023 – 2025 Strategic Plan. The Objectives in the 2023 – 2025 plan, page 6, are very different than the Objectives in the 2024 – 2026 plan, page 7. The 2024 – 2026 document is silent on why the changes were made or how CISA did at meeting the 2023 – 2025 objectives.
On page 5 of the 2024-2026 plan there is a section: Achieve Impact or Fail Fast. Here’s the pull quote:
Where we determine that a given program, service, or capability is not resulting in expected impacts, we will be disciplined in “failing fast” and making best use of our resources to pivot with agility.
Fantastic. Right approach. Where did CISA fail and what were the pivots?
The Hope
Frequent readers, viewers and S4 attendees know I’ve been on a metrics rampage for a few years now. Few in OT security have metrics and CISA, Department of Energy, and others in government are the worst because they point out industry’s and other government ageny’s shortcomings without a look inward.
The 2024 – 2026 Strategic Plan could change this as there are real Measures of Effectiveness for each Objective. They are much better than the previous Strategic Plan measures and some are quite good. One of the best examples is Objective 2.2: Drive implementation of measurably effective cybersecurity investments. Here are the first two Measures of Effectiveness.
1 | Increase in the average number of Cybersecurity Performance Goals effectively adopted by organizations across each critical infrastructure sector.
2 | Where possible, reduction in confirmed impactful incidents in organizations that have adopted a higher number of Cybersecurity Performance Goals.
Measure 1 is a metric on how effective CISA has been in getting critical infrastructure to implement the CPG’s. Important, but not sufficient because it does not show that the CPG’s are effective.
Measure 2 solves this by comparing the impactful incidents, in some unspecified way, between entities that implemented the CPG’s and those that did not. Do implementing the CPG’s reduce cyber risk to critical infrastructure?
The Measures of Effectiveness vary a great deal in quality, and many are potentially quite good.
The Reality?
If CISA is serious about these Measures of Effectiveness there should be a data source and calculation for each measure and an initial score. And the data source, calculation and FY 2023 score should be published now / soon / no later than the start of FY 2024 in October.
If this is not all available on October 2nd, there’s no need to wait to put out a grand document. Create a page on the CISA site as they become available and update with results at some cadence. Move from the grand sounding language to reality, with the full expectation that CISA and the country will not succeed at improving all of these Measures of Effectiveness.