So this morning, I came across this article. It describes a call to action, given by Sen. Sheldon Whitehouse (D-R.I) at the FCW (Federal Computing Week) “Big Issues Conference”, about the NIST Cybersecurity Framework (CSF). In general, it makes me feel better about the world that the sentiments in it are being expressed – and that this lawmaker considers it a “big issue”. I care about security. It’s refreshing that lawmakers do too.
However, “caring about it,” while absolutely a fantastic starting point, is not – as a practical measure – good enough. The second step is doing something about it. And it’s here where I start to have criticisms. Before I get into this in detail, I want to STRESS HEAVILY that I’m not calling any of this out to slam any particular political point of view… I also honestly happen to think that this lawmaker is most likely coming from a good place. So let me start with reiterating “props to him” for caring about the topic generally — and further “attaboys” for paying attention more specifically to the CSF and, by extension, the regulatory and security considerations for civilian agencies.
All that said, I do think there is “room to grow” in the specifics – and I level that criticism at all parties in this discussion. I think, if you unpack this discussion, it illustrates why it is incumbent on us, as an industry, to better inform lawmakers about security issues.
Here’s what I mean. From the article:
“The NIST Framework has never been adequately validated,” Whitehouse said at FCW’s Big Issues Conference on Nov. 1.
Whitehouse said that he wonders whether agencies have accepted the NIST Framework because it’s effective or because “compliance demands so little effort.” Whitehouse said that the framework needs to be tested.
One way to test the framework would be to assign a white hat hacking team to attempt to breach a system that’s compliant with the framework.
There are two things going on here that I have criticisms about. The first is the use of “compliance” to describe the CSF. Yes, it is true that federal agencies have a regulatory obligation to use the CSF to manage cybersecurity risk (that’s per the May executive order.) For reference, it says: “Effective immediately, each agency head shall use The Framework for Improving Critical Infrastructure Cybersecurity (the Framework) developed by the National Institute of Standards and Technology, or any successor document, to manage the agency’s cybersecurity risk.”
So the answer to why agencies accepted it? It’s because they are required to.
However, there’s more going on here. These same agencies have also been required to use the NIST Risk Management Framework (RMF) for nigh on the past decade (since 2010 to be precise about it.) The CSF and the RMF are not the same thing. If there’s a problem in the federal government – specifically civilian agencies – with risk management (spoiler alert: there is), the issue isn’t the CSF. It’s instead with compliance efforts more generally. Maybe it’s that the RMF describes a risk management model that is at a level of sophistication beyond what most organizations are ready to meet (“in spirit” anyway) without significant work and investment — work they can’t don’t have time/staff to put in and investment they’re not getting. Or maybe it’s that risk management is difficult to get right and pretty much nobody is doing it well as it is. Or maybe it’s something else. But the core issue – and there absolutely is one – isn’t the fault of the CSF.
Then there’s the question of “compliance” with it in the first place. If you read the CSF, you’ll notice it’s not about compliance. Like not even a little bit. From the framework:
Building from those standards, guidelines, and practices, the Framework provides a common taxonomy and mechanism for organizations to: 1) Describe their current cybersecurity posture; 2) Describe their target state for cybersecurity; 3) Identify and prioritize opportunities for improvement within the context of a continuous and repeatable process; 4) Assess progress toward the target state; 5) Communicate among internal and external stakeholders about cybersecurity risk. The Framework complements, and does not replace, an organization’s risk management process and cybersecurity program
What exactly would “compliance” with that look like? That they use the common taxonomy? That they communicate or prioritize? The short version is that it’s hard to imagine because it’s not written in such a way that it’s about compliance in the first place: it’s not intended to be (per feedback from the community at large) and, as an artifact, it isn’t written that way. SP800-53 is arguably about compliance. The RMF is (arguably in concept but for sure in terms of how it’s used) about compliance. The CSF? It’s about something else. The fact that we have made it be used the way it is belies a misunderstanding (I think) of what the document is for. A discussion about how best to use it that way is like me arguing with my neighbor about the best way to use a bottle of shampoo as a hedge trimmer. The whole conversation is flawed from the start. Should we choose to pursue the discussion to it’s logical conclusion, we might wind up with some compromise that lets you (with significant expenditure of effort) trim hedges, but wouldn’t asking a landscaper the right way to do it be a whole lot more productive?
The second thing is the utility of drawing a line between a pentest and efficacy of the framework. If you want to do pentests of federal agencies, I’m all over that. However, what you’re testing, should you do so, isn’t the utility (or lack of it) of the CSF. There are two reasons for that. One is that you could find numerous examples of places that use it that have terrible security – and examples of places that don’t that have great security. The same is true of PCI – or HIPAA – or the ten commandments. When I was a QSA (PCI DSS assessor), we used to joke that “the difference between a compliant or non-compliant environment is how hard you look”. The same is true here. The second is that the measuring instrument proposed (pentesting) doesn’t measure what you want to test (risk management practices.) If you want to test patch management efficacy, application hardening practices, incident response capability, or any number of other things, a pentest could be a great way to do that. Risk management? Not so much. I will assume you know already why that is (since it would take me another thousand words to argue why this is true), so will allow it to stand on it’s own. But trust me, it’s not a great instrument for that.
I could go on and pick at other things about the call to action, but I feel like I’ve made my point. The underlying current is that the executive branch and the legislative branch are having a detailed discussion about something that neither of them fully understands. Mandating the use of the CSF and RMF (simultaneously) for risk management isn’t addressing the root cause people are concerned about. Mandating the use of the CSF at all misunderstands what it is for. Setting up pentests to validate their utility also doesn’t solve for what’s wrong.
How do you address these things? The first step is caring about the topic. We’re there. The second step is having an informed, no-bullshit discussion about it. We’re close to being “bullshit free” I think – or at least we’re farther along than we were. That’s good. Where we need work is on “informed”. We’re nowhere near “informed” yet. How do we get informed? We, as an industry, need subject matter representation at the table for these discussions. And I DO NOT mean more people who also don’t know anything about the topic — but yet say they are “experts” because of the Dunning-Kruger effect. I mean people who really get it. I could list dozens of examples (Ron Ross, Ed Felten, Susan Landau, Gene Spafford) and I’m sure you could too. We as an industry need to put them forward — and politicians need to listen to them.