How to Test for the Best Endpoint Protection

All software tests are not created equal. A fact that is particularly true when it comes to cybersecurity software.

A test designed to reward vendors for every alert generated, no matter the likelihood of it being a real intrusion, creates winners based on volume, not effectiveness. It does not test real world enterprise security. But it may explain why today some security operations center (SOC) personnel and analysts are overwhelmed with up to thousands of threat alerts a day. Products that are built to win these types of tests don’t deliver what really matters: preventing, effectively and efficiently, real threats to the enterprise environment.

When Everything is a Threat

In theory it may be a good idea that leading Endpoint Detection and Response (EDR) solutions tend to capture everything – legitimate threat or not. But in practice, it is a problem that has become a major reason for so many successful cyberattacks and data breaches.

When analysts can’t respond to every alert, studies show they tend to turn down the noise by picking and choosing, or worse, tuning out and hoping for the best. In fact, on page 4 of a recent industry report, International Data Corporation (IDC) concluded that nearly a third of enterprise SOCs ignore more than 30 percent of all alerts

The issue is further exacerbated by the sheer number of data sources and feeds in the typical enterprise IT environment. Indeed, an IBM study concluded that despite the continuing investment in more security tools – more than 50 in the typical enterprise -- to monitor these data sources and detect cyberattacks, the net result was that the enterprises were more vulnerable to attack than ever.

With so many data sources and too many tools to monitor those sources, it is no wonder security analysts feel the deck is stacked against them. Alert fatigue is becoming endemic in the industry, as analysts find that their prime task is no longer remediating real threats but often trying to determine if the alert threat is real or just another false positive (FP). 

Doing it Right: Real World Testing

Real world testing is the right way to measure an EDR security solution. These tests put products into an environment that represents how they perform against an attack by real threat actors, whose actions are mixed in with legitimate traffic. And they don’t count alerts, they determine if the SOC can discover the threat in those series of alerts. One test we think gets it right is done by SE Labs.

SE Labs, is a private, independently owned firm based in the UK, that specializes in advanced cybersecurity testing. A new comparative EDR test performed by SE Labs replicated the real-world techniques used by a variety of active cyberattack groups. This SE Labs test, the Enterprise Advanced Security Test,  rewarded participants based on accuracy of threat detection, while penalizing for false positives. Virtually all other major EDR tests do not penalize for false positives, rewarding product behavior that looks good on paper, but in the real world has been shown to hinder SOCs to do their job effectively.

The SE Labs test simulated a typical enterprise environment in which there is a cacophony of good and normal activity occurring at the same time as the malicious activity. The tests also more accurately replicated the current threat landscape by simulating several different attack groups simultaneously rather than just one or two in isolation, as is typical in most tests. Moreover, while using the techniques employed by various known attack groups, the test attackers also went “off script” and freelanced variations on those techniques. Again, this is in dramatic contrast to most industry tests.

The combined result of these test scenarios creates a far more accurate picture of how EDR solutions behave in a real enterprise environment.

Finally, it’s worth noting that the SE Labs test highlights another critical aspect of enterprise security by weighing its findings based on the severity or importance of the threat to the enterprise itself. As security professionals know, all threats are not created equal. The SE Labs comparative EDR test put that consideration front-and-center in its evaluation criteria.

Our EDR capability in Symantec Endpoint Security Complete (SES Complete) received a perfect score on SE Labs first comparative EDR test. We led the field with a 100% score for Total Accuracy, meaning we were perfect with 100% detection while reducing the alert noise. Meanwhile most of the other vendors struggled with this tradeoff between detection and lots of noise.

Of course this was not a surprise. Symantec solutions have long been recognized as the gold standard in endpoint and EDR security. They have consistently scored well in every industry test. 

An EDR solution must balance between detection and noise. Testing that doesn’t reflect the real world rewards noise. Real world testing can show you what products can reach a balance, great detection and extensive data collection, without overwhelming the SOC with alerts. So, consider the EDR solution rated the best by the test that is the industry’s most comprehensive and accurate: Symantec Endpoint Security Complete.