What Works in Cybersecurity: Perimeter Security Appliances

Which yes/no technical question of the form “Do you implement X” splits the world most successfully according to the likelihood of suffering a cyber incident? The perfect question would see none of the firms answering yes suffer an incident, whereas all firms answering no suffer an incident.

We don't have the perfect question, but this post highlights evidence of a question where firms saying yes are more than 10 times as likely to suffer an incident. Yes, I’m going hard on the intriguing cliff hangers because my first post “What Works in Cybersecurity: MFA” didn't get many views.

The aim of WWiC isn't to dunk on one vendor. Instead I want to generalize and talk about the efficacy of perimeter security appliances:

A perimeter security appliance is a network device, such as a firewall or VPN, designed to protect an organization’s internal network by monitoring and controlling incoming and outgoing traffic at the network’s boundary.

The studies we cover will mention specific devices, so let’s get into it.

tl;dr

In 2023, organizations adopting Cisco ASA devices were estimated to be 4.7 and 11.6 times more likely to report a cyber insurance claim than those who didn’t, with evidence provided by two cyber insurers (Coalition and AtBay). Perimeter security appliances built by vendors like Citrix, Fortinet and SonicWall are also correlated with elevated claims rates.

The forensic data shows firewalls and VPNs are some of the most common initial access vectors used by ransomware gangs (see Coalition and Arctic Wolf). Further, Mandiant found the most commonly exploited CVEs affected perimeter security appliances made by firms including Palo Alto Networks, Ivanti, and Fortinet.

The colossal effect sizes and high proportion of IAVs lead me to rate efficacy at just 1.5. The only saving grace is that these devices can contribute to security if properly configured and maintained. Unfortunately, that’s not done by the typical user lacking in resources. Further even when configured properly, these devices typically run on proprietary operating systems that have been plagued by 0-days recently. 

Insurance Data

The correlational evidence is so strong that it has to go first. Two insurers have correlated adoption of specific perimeter security appliances and cyber insurance claims.

Coalition’s 2024 Cyber Claims Report shows organizations with Cisco ASA devices were almost 5 times as likely to suffer a cyber incident. This is all based on claims observed in 2023, during which Cisco's ASA product had a nasty 0-day vuln (e.g. CVE-2023-20269 affecting VPN functionality) that was exploited by the ransomware group Akira.

Coalition also found evidence that of elevated claims rates associated with Fortinet devices. Notably, these correlational statistical effects are durable over time.

AtBay’s effect sizes are even stronger. Cisco ASA is associated with an 11.6 times higher cyber insurance rate among their policyholders (as is Citrix's SSL VPN). Fortinet, Sonicwall and other VPN's are also associated with elevated claims rates, which suggests this effect is broader than one vendor alone. AtBay suggests the problem lies with self-managed VPNs.

As we discussed in the first post, the univariate analysis could be vulnerable to confounding variables. This is why we should triangulate across multiple data sources.

Digital Forensics Investigations

Forensic evidence contradicts the theory that perimeter security appliances are just coincidentally adopted by the kinds of companies who suffered breaches at a higher frequency. 

Coalition's 2025 Cyber Threat Index shows the majority of ransomware claims started with a compromised VPN or firewall, with attackers using a mixture of software exploits and credentials.

Arguably, stolen credentials are not the device failing, but instead the user who lost the credentials. But even here there is nuance. Credentials can be stolen by compromising devices, such as when CVE-2022-40684 was used to steal VPN credentials for over 15,000 Fortigate devices. Second, the vendors could design products to nudge customers towards enabling MFA, which would reduce the incidence of stolen credentials.

Many other firms provide data that supports this view that perimeter security appliances are commonly compromised during ransomware incidents. For example, Arctic Wolf also found VPNs (along with RDP and RMM solutions) were the Initial Access Vector in the majority of ransomware incidents:

Mandiant’s 2025 M-Trends Report found that the most frequently exploited CVEs affect perimeter security appliances:

This type of finding has been replicated sufficiently often to conclude that insecure perimeter security appliances are among the most common root causes in cybersecurity incidents. However, this kind of statistical evidence cannot explain why.

Technical Reasoning/First Principles

The weirdest aspect of the correlational evidence is that adopting perimeter security appliances is associated with being more likely to suffer an incident.

This suggests firms who haven’t adopted these products face less risk. The dumb conclusion is that turning off your VPN/firewall and allowing connections from any network address will reduce risk. I won’t waste time explaining why this is wrong.

The core problem is that firms with perimeter security appliances make different InfoSec decisions because they believe the devices effectively secure the network perimeter. Such firms might allow RDP connections from users connected from the internal network including VPN users, whereas a firm without a perimeter security appliance might decide to disable the protocol altogether. 

This scenario is an example of risk homeostasis, by which defenders target a fixed level of risk. If the theory holds, the perceived reduction in risk associated with adopting a perimeter security appliance is offset by other decisions (such as enabling RDP) that increase risk back to the target-level. In such a world, all defenders face the same risk level because controls are offset by riskier behaviour.

Yet we do not observe that firms have the same likelihood of breach regardless of whether they adopt these controls, as would be predicted by risk homeostasis (assuming they have the same risk targets). Instead, firms adopting certain perimeter security appliances are more than 10x as likely to suffer an incident. 

To explain this, we need to layer “a false sense of security” onto “risk homeostasis”. If firms over-estimate the risk reduced by adopting the appliance, then they will make even riskier decisions and push themselves above the target risk level. The perimeter security appliance did not directly lead to higher likelihood of incident, rather the appliance lead the customer to make bad decisions under the allusion their boundary was secure.

This raises the question of why perimeter security appliances create a false sense of security. Imo, the problem lies in the gap between the vendors’ marketing and the reality of how these devices are compromised:

1. Ransomware gangs can reliably find unpatched vulnerabilities in perimeter security appliances.
2. Failing that, ransomware gangs can reliably steal the login credentials of authorized VPN users.

Evidence for both factors can clearly be seen in the DFIR reports.

Discussion

The solution for most defenders is to stop assuming a perimeter security appliance can reliably enforce a network perimeter and to build an InfoSec program accordingly. Now there are exceptions here—if you rapidly apply firmware updates and enforce MFA for all remote access accounts, you probably can build an InfoSec posture around these devices. But the reality is that most defenders cannot do that at present.

From a manufacturer perspective, this is likely an infuriating post to read. How can vendors be blamed for their customers not applying patches or enforcing MFA? Answers can be found in the extensive writing on secure by design and the less extensive writing on secure by default.

Defenders shouldn’t even need to patch vulnerabilities that were easily avoidable. For example,  Fortinet’s recent stack overflow vulnerability could have been eliminated by engineering methods that have been around since the 1990s. In 2007, Steve Christey declared buffer overflows were an unforgivable vulnerability. In this framing, vendors are at fault for not eliminating these vulnerabilities (not the defenders who didn't patch in time).

For stolen credentials, secure by default reframes the lack of MFA as the vendors failing to design choice architecture that nudges customers towards MFA. Bob Lord likes the example of modern cars beeping when the seat belt isn’t buckled in. This transforms seat belts from a problem of educating passengers about the relative benefits of safety vs convenience (hard) into a problem of passengers stopping an annoying noise (easy). To follow secure by default, vendors should work out the equivalent way of transforming MFA from a user decision about security vs convenience (hard) into a much simpler decision.

Final Judgement

An optimistic judgement would try to quantify efficacy for customers with perfect configuration and maintenance, which is how the device manufacturers would like to be judged. With this framing, perimeter security appliances might get a 3 or 4 star rating. Although, the constant drum of 0-day vulnerabilities impacting these products creates risk even for perfect customers who patch quickly.

Imo, the optimistic approach to scoring is irresponsible given we know the vast majority of defenders are under-resourced. The typical customer is slow to patch and inconsistently enforces MFA for remote connections. For that reason, the efficacy of perimeter security appliances should be rated for the typical user. Based on that approach, I am giving perimeter security appliances 1.5 stars for effectiveness.

The evidence base is, imo, worth 4 stars. We have huge effect sizes, among the biggest I have seen in empirical cybersecurity. The causal interpretation is further evidenced by forensic evidence (consistent across vendors) showing these devices are the root cause of a good chunk of incidents. There is also a robust mechanistic explanation, rooted in behavioural rather than technical reasoning.

To get up to 5 stars for evidence, we’d need a research design that can statistically capture causality, and also that can differentiate efficacy based on configuration.