SC Magazine recently published an article by Ciaran Martin (previously CEO of the UK’s National Cyber Security Centre, and a member of Garrison’s advisory board): Fix a broken market, and we can resolve so many cybersecurity issues. It’s a good read and I recommend it, but Ciaran’s key statement is this: “the industry clearly needs a new model for defining and measuring the effectiveness of cybersecurity products and services to improve decision-making regarding cybersecurity risks and how to manage them”.
That’s something we at Garrison certainly know a thing or two about.
Buying cybersecurity technology is a challenging business, because buyers need to weigh up three key factors: security, cost, and usability (where I’m considering usability here both from the end user’s perspective and from the IT department’s perspective).
Cost at least is relatively easy to measure. And while assessing usability is a serious challenge, it is at least one that organisations have plenty of experience of. This is what the Proof of Concept is for – allowing both the IT department and the end users to get some real hands-on experience with the technology to see what it’s like to use in practice. Sure, we all know of plenty of horror stories where the wider rollout doesn’t perform nearly as well as the Proof of Concept – but buyers know at least in principle how to avoid these. Define clear success criteria for the PoC, and make sure that the PoC deployment is as representative as possible. That’s not always easy – and as a result some PoCs go better than others – but it’s a well-understood and universal process.
By contrast, we’re in full agreement with Ciaran that the assessment of security is poorly understood. As the Debate Security study that Ciaran references found, commercial buyers don’t think the current mainstream approach to security assessment is working – and this is creating perverse economic incentives for vendors.
The exception to this is with one small proportion of buyers, who do seem generally to believe that their processes for assessing security are working effectively – at least when seen through a narrow lens. These are buyers for whom security has long been a serious business – and they’re found almost exclusively in the “national security” elements of government. But while the security assessment processes used here may be effective from a narrow technical point of view, there are a different set of economic issues here, driven above all by a lack of standardisation because every national government has its own particular assessment process. For Garrison, some of the key processes we run up against are:
– The NSA’s NCDSMO “Raise the Bar” programme in the USA
– The NCSC’s “Cross Domain Industry Pilot” programme in the UK
– A wide variety of individual national spins on Common Criteria. There’s at least some level of “commonality” in Common Criteria, but at least for our buyers it’s never quite as common as one might hope!
It’s easy for vendors like us to moan about this patchwork landscape of assessment standards, and there’s no doubt that there’s real work required of us for each of them. But despite what people may say, I’m quite convinced that none of them represents bureaucracy for bureaucracy’s sake. Each nation’s approach is a reasonably well thought-through approach to trying to determine the answer to a difficult question: does this product deliver enough security for my needs? In fact, I’ll make a geeky admission: I rather like going through these assessment processes. When you put a lot of effort into building a security product, it’s rather nice to engage with people who really want to know whether you’ve done it right, and have the time to spend on getting into the detail.
But clearly it’s not scalable. It’s fair enough that major nations set their own individual approaches for their national security needs: after all, they’re typically trying to protect against each other! But if the wider set of buyers is to move on from today’s ad-hoc (or nonexistent) assessments, it’s not going to work if each buyer does their own thing.
So here’s Garrison’s own take: yes, it would be great if buyers could get better at assessing the security of cybersecurity products – but let’s not let that develop as a fragmented patchwork of different approaches. Ciaran suggests three ways that things could develop: industry-led, sector-led or government-led. I’m hearing a lot of views that government intervention is going to be required, but that’s a cause for real concern because a government-led approach is almost bound to end up with a fragmented landscape of national standards. It seems to me that buyers have a real interest in avoiding that, so let’s hope for a collective buyer-led approach that can get on the front foot and get ahead of any regulatory action.