March 5, 2019 | Categories: Blog CISO |

Why You Should Be Using Metrics to Measure and Demonstrate Success

When it comes to cybersecurity, how do you know if your efforts are performing the way that you want them to? Usually, people might think their cybersecurity is performing well because nothing’s gone wrong yet, they haven’t experienced a data breach, or they haven’t been informed by a third party or regulator that it’s insufficient.

However, this is a flawed way to look at your security program. If you really want to identify your strengths and weaknesses, you have to be able to understand the metrics that define cybersecurity success — not just on a day-to-day basis but in the long-term.

The reason why so many companies fail to understand the strengths and weaknesses in their security program is that they don’t know how to identify the right metrics or what metrics they should be looking at to measure it.

Why are metrics so important to demonstrate how your security program is performing? To start, you can only improve what you measure. If the only way you can identify success with your cybersecurity is by not experiencing an incident, you’re not seeing the bigger picture. Without any measurement, you won’t be able to confidently say that your efforts were the reason you were safe, your security could be considered purely an act of artistry.

Metrics can be shared with your peers, your executives, or both. They’re a great way of communicating the successes of your efforts and allowing you to identify weaknesses, which can help you be more eligible to receive the help you need, whether it’s additional tooling, a larger budget, or more bodies on your team.

How Can You Use Metrics to Grow Your Security Program?

Security wants — and needs — to have a seat at the big table. With strong metrics, you can easily create reports that can be shown to your board so everyone is on the same page with how your efforts are making a difference for the company.

With any security program, it’s important that you know what’s working and what needs attention. And if you want to add additional security products, you need to have a baseline for what you actually need so you don’t end up wasting time and money on products that aren’t going to give you any benefit.

Plus, if you want to add more staff, how do you know who to hire? How do you know how many people you need? Metrics can provide insight into where your cybersecurity team may be lacking, and allow you to know what type of experience you need to build that up. Detailed metrics on your security program can also help to drive team performance, explain security to your board, justify past and future expenses, and increase transparency.

So, what makes a good metric? It’s good to know that not all metrics are created equal. For your security program, you’ll want to identify metrics that enable decision-making and are easily measured or understood. Good metrics are meaningful, consistent, and quantitative, and they align with your organization’s risk profile.

Bad metrics — yes, they do exist — are difficult to gather and understand. The data might be inconsistent or require a lot of explanation. They might even be meaningless, but most importantly, they won’t be able to be used to drive decisions. Here are a few examples of bad metrics that you should avoid:

  • The number of critical vulnerabilities — in this example, “critical” needs to be defined because it could mean a number of different things
  • Unpatched systems — this should align with your policy; if it doesn’t, it’s not going to be worth anything to your team
  • Level of risk — if this is completely opinion-based, it’s not going to help you or your team understand what needs to be improved
  • Results from Pentests performed by different firms, using different scales — this data needs to be normalized in order to be beneficial
  • Findings from audits against different frameworks — this data will need to be mapped
  • Anything that takes more than one hour each month to gather — if you’re spending this much time on something, it’s too much time and you’re not going to glean anything helpful from it or be able to explain it to others

As for good metrics, here are a few examples that you should consider using in your reports. Note that each of these includes a concrete measurement and time period:

  • Number of closed vulnerabilities each month
  • Number of potential attacks each month
  • Number of breaches in a year
  • Number of phishing emails that were reported each month
  • Number of employee outreach events each quarter
  • Percent of systems found in compliance with patch policy each month
  • Number of defects found in software QA, per release

For good metrics, there are plenty of places you can get information from, including logs for your firewall, web filter, and antivirus system. Internal testing results such as phishing, audits, awareness training, employee outreach sessions, penetration tests and vulnerability scans, and help desk tickets can also provide insights and data. External testing like audits, vendor assessments, software QA and security assessments, and more are also great sources. And data from platforms like SAP, a CRM, or Salesforce will also be useful.

When you go to choose your metrics, you’ll want to ensure that you’re using the best starting point. You’ll also want to check that you’re pulling from data that is meaningful to your goals and the points you’re trying to make — not just because it’s there.

The data you want might be in metrics that you’re not currently collecting data for, so it’s good to be open about the datasets that you want to pull. However, it can be a challenge to find new data without the quest to find what you need becoming a major distraction. So, while it’s good to identify new opportunities, don’t let it stop you from creating something useful from the information that you already have.

How Should You Display Your Metrics?

Depending on who you’re preparing your metrics for, there are a few different ways that you can build a report to share with them. Any report is going to rely on the same data that you pull. But by considering who is going to be receiving your metrics, you can create a report that helps underscore and detail the different elements for its audience in order to tell the story you want.

A monthly scorecard is a great way to communicate with your security team. A scorecard can also help you manage your security operations and tasks by understanding where your day-to-day activities stand. A scorecard typically is designed as a one-page roll-up that can be presented to your CIO or just used internally. With this type of report, you can show your current operations, any risks that you’re managing, and a legend that shows the thresholds for each item detailed in the scorecard.

Another report you can use is a quarterly dashboard. This is a place where you can store and update all your metrics on a quarterly basis to show peers and direct management where your current operations stand.

For a high-level view, an annual presentation can show the overall value that your efforts are creating. This type of report is a great way to demonstrate the power of your efforts and the direction of your security program to executives in your organization or the board.

Now that you’ve identified a stronger approach to information security metrics, create a strategy that fits within your organization’s environment. There is no right way to identify and report on metrics, instead, you’ll want to ensure you’re preparing the right data for your risks and the audience you’re talking to. Align your reporting with existing goals and efforts to ensure that your information is better received from the start.

It’s also important to know that you might not be able to get all these right the first time. If you need help, working with an experienced security partner can help you determine what the best metrics are for your security program and what the best method is for displaying them. At MRK, our CISOs have decades of cybersecurity experience in organizations of all sizes. We get to know the risks your organization faces and can assist as an integral part of your security team.

Interested in learning more about how we can help you create better metrics reports? Get in touch with our team today to learn more.