The Four(ish) Appsec Metrics You Can’t Ignore
Download as PPTX, PDF0 likes3,754 views
The document discusses the importance of application security metrics to effectively communicate progress and manage risk in software development. It highlights four key metrics: policy compliance, flaw prevalence, fix rate, and tailored metrics, which can help teams mitigate vulnerabilities and improve application safety. The document emphasizes that the choice of metrics should align with business goals and the specific needs of development teams.
The Four(ish) Appsec Metrics You Can’t Ignore
- 1. The “Fantastic 4” Metrics You Can’t Ignore When Reducing Application Layer Risk
- 2. 2 Why application security metrics?
- 3. 3 Why application security metrics? Sometimes you need: 1. To communicate to your sponsors what you’re doing with the money they provided for the program. 2. A way to communicate with your development teams that is anchored in something more than just encouragement. 3. A tool to show yourself how much progress you’re making.
- 4. 4 It’s a hostile environment out there
- 5. 5 It’s a hostile environment out there Applications have been a top vector for data breaches over the last five years because they’re not coded with security in mind. The software industry’s shift to composing applications via pre-built—some would say “pre-0wned”— components has made it more challenging for security teams by introducing risk via the software supply chain. So application security is important, but how do you show progress?
- 6. 6 Four key metrics to save the day
- 8. You have to have some way of measuring the quality of applications; it should be aligned with the needs of the business. A lot of your program measurements are going to be anchored in how well your portfolio does against a policy. But what sort of pass rate should you expect?
- 9. 9
- 11. 11 When vulnerabilities are all around you might feel like your world is on fire. Let’s try to get our arms around how common some of these fatal flaws really are.
- 12. 12 Top vulnerabilities by industry
- 14. You know that guy, the one who always insists that the hole you’re in isn’t as deep as you think it is…
- 15. It turns out that’s true of AppSec. There are a lot of people out there making their applications safer, never accepting “no” for an answer. And it turns out that tracking the flaws fixed can be powerfully motivational.
- 16. 16
- 17. 17 Source: Veracode State of Software Security vol. 6: https://info.veracode.com/state-of- software-security-report-volume6.html How? Empower developers • Customers in the financial services and manufacturing verticals are successfully fixing between 65% and 81% of the flaws found in their applications. Applications undergoing remediation coaching (readouts) reduce application risk 2.5x more than those that don’t, as measured by average flaw density per MB Source: Veracode State of Software Security vol. 6: https://info.veracode.com/state-of-software-security-report-volume6.html
- 19. 19 Which One?
- 20. 20 Which One? It depends. Just as there’s no “one” lineup of a super hero team, you may find you need a different set of metrics depending on the goals of your program—developer training completion, for instance, or percent of applications undergoing automated testing. Ultimately it’s up to you, and the needs of your business.
- 21. 21 Answers Key Questions for CISOs • Which industries are doing the best job of reducing application-layer risk ? • Do I have more serious vulnerabilities than my peers? • What percentage of vulnerabilities do my peers remediate? • How many of our applications should pass the OWASP Top 10 when initially assessed? • What are the Top 10 most common vulnerabilities in our vertical? • How can I reduce more risk in my organization’s applications?
Editor's Notes
- #3: Sometimes it seems like application security programs are a never ending chasm. Why do we need to measure? For several important reasons— Sometimes you need to communicate to your sponsors what you’re doing with the money they provided for the program. Sometimes you need a way to communicate with your development teams that is anchored in something more than just encouragement. Sometimes you need a tool to show yourself how much progress you’re making.
- #4: Sometimes it seems like application security programs are a never ending chasm. Why do we need to measure? For several important reasons— Sometimes you need to communicate to your sponsors what you’re doing with the money they provided for the program. Sometimes you need a way to communicate with your development teams that is anchored in something more than just encouragement. Sometimes you need a tool to show yourself how much progress you’re making.
- #5: Don’t forget – it’s a hostile environment out there. Applications have been a top vector for data breaches over the last five years (see the Verizon Data Breach Reports) because they’re not coded with security in mind. The software industry’s shift to composing applications via pre-built—some would say “pre-0wned”—components has made it more challenging for security teams by introducing risk via the software supply chain. So application security is important, but how do you show progress?
- #6: Don’t forget – it’s a hostile environment out there. Applications have been a top vector for data breaches over the last five years (see the Verizon Data Breach Reports) because they’re not coded with security in mind. The software industry’s shift to composing applications via pre-built—some would say “pre-0wned”—components has made it more challenging for security teams by introducing risk via the software supply chain. So application security is important, but how do you show progress?
- #7: We’re going to walk through four ways to look at your portfolio of applications and benchmark it against other organizations. Each metric has its strength but they’re definitely better together. Let’s get started…
- #9: I think of policy compliance as the bedrock measurement (pardon the pun) of AppSec. You have to have some way of measuring the quality of applications; it should be aligned with the needs of the business. A lot of your program measurements are going to be anchored in how well your portfolio does against a policy. But what sort of pass rate should you expect? There’s the problem…
- #10: Let’s look at the industry perspective for a second—using a kind-of-generic policy, the OWASP Top 10, to look at how your peers are doing. Not well. (Highlight relatively high pass rate in FinSvc which is still bad news – more than 50% fail)
- #11: OK, so now that we understand how bad the problem is, how do we fix it? One way is to understand the types of issues we’re facing.
- #12: When vulnerabilities are all around you might feel like your world is on fire. Let’s try to get our arms around how common some of these fatal flaws really are.
- #13: It’s important to note that not every application is subject to the same risks. For instance, SQL Injection, a leading cause of data loss, is only present in about 30% of applications (40% if you’re in the government). Note though that crypto is found in 45 to 80% of applications depending on industry—a problem if your customers or regulators require you to protect sensitive data.
- #14: Okay, so we know how good (or bad) our applications are, and we know what the nature of the vulnerabilities are. Now what?
- #15: You know that guy, the one who always insists that the hole you’re in isn’t as deep as you think it is… It turns out that’s true of AppSec. There are a lot of people out there making their applications safer, never accepting “no” for an answer. And it turns out that tracking the flaws fixed can be powerfully motivational.
- #16: You know that guy, the one who always insists that the hole you’re in isn’t as deep as you think it is… It turns out that’s true of AppSec. There are a lot of people out there making their applications safer, never accepting “no” for an answer. And it turns out that tracking the flaws fixed can be powerfully motivational.
- #17: Some industries are making huge progress in fixing the vulnerabilities they’ve found. In fact across all Veracode’s customers, they fixed 3 out of every four flaws found by automated scans last year.
- #18: So a strategy for reducing risk is “real-time” developer education, enabling developers to go faster.
- #19: Policy, top flaw prevalence, fix rate. What’s our fourth metric?