Symantec Participates in Latest MITRE ATT&CK® Evaluation

In order to enable customers to get an independent evaluation of our product’s effectiveness, we’re pleased to announce our participation in MITRE’s new evaluation program that assesses cyber security products, using APT29 as an example adversary. The program aims to provide organizations with a third-party, analytical look at a product’s capabilities and visibility into hacker techniques. While we’ve worked over the years to develop the most advanced products for endpoint security, we believe that well thought out and performed third-party evaluations are key to helping validate our technology and prove our effectiveness in the face of modern threats. 

These measurements are difficult for any company to do on their own.  And yet understanding a product’s strength in detection and prevention has never been more critical.  Starting in 2018 (According to Symantec’s Internet Security Threat Report) half of new targeted attack groups Symantec exposed were uncovered through their use of living off the land tools. There was also a 25% increase in the number of groups using destructive malware. And today Targeted Ransomware has become one of the biggest cyber crime challenges facing enterprises, resulting in loss of business and millions of dollars in damage. 

To help cyber defenders better understand what they’re up against, the MITRE ATT&CK® framework was developed to collect and categorize the tactics and techniques cyber attackers use to infiltrate organizations. The framework has become something of a standard used by cyber security professionals for insight into adversary behavior, providing a common language for the security industry to use. All the information is publicly available, and provides a valuable tool to help both organizations and vendors identify gaps in capabilities. At Symantec, we’ve relied on ATT&CK® as a guiding framework for our Endpoint Security solution. Our approach focuses on both detection and prevention across the entire ATT&CK® chain, with our capabilities mapped to every stage of the chain so that we can deliver the best protection throughout, and as early as possible in, an attack.

Tried and tested

MITRE’s ATT&CK® Evaluations program is the latest third-party testing we’ve collaborated with, but Symantec has a long history of participating in such testing. For nearly 10 years, we’ve ranked consistently high on AV-TEST’s evaluation of antivirus and security software. Similarly, Symantec has tested with AV-Comparatives since 2004, from the very beginning of its operations. And just last year, SE Labs conducted an in-depth evaluation of Symantec Endpoint Security Complete for breach security, looking across the entire attack chain. The results showed that we were effective against a variety of exploits, fileless attacks and malware, “comprising the widest range of threats in any currently available public test.” We understand the importance of such evaluations in understanding the strengths of our products, but because they can show us areas for improvement and blind spots in our detection and prevention. 

Why an ATT&CK® Evaluation

The strength of the ATT&CK® Evaluation lies in its emulation of actual adversary behavior for testing.  In Round 2 of testing, MITRE tested products based on techniques used by APT29, also known as Cozy Bear or The Dukes, a hacker group believed to be linked to the Russian government. The group has been tied to cyber attacks on the Pentagon, other Western governments, organizations and agencies around the world, but it has become most infamous for its hack on the Democratic National Committee in the run-up to the 2016 US Presidential elections. According to MITRE, APT 29 was chosen as the attacker group to emulate because it uses more sophisticated implementations of tactics and Living-off-the-Land methods, such as PowerShell commands. The group’s attacks can be difficult to detect and defend against, making it a good example for assessing how organizations measure up against an advanced adversary. Additionally, APT29’s international reach makes this example particularly relevant for governments and organizations. 

ATT&CK® Evaluations used a simulation of APT29 to identify the strengths and weaknesses of each vendor’s defense approach and visibility into adversary techniques. We particularly appreciate that ATT&CK® Evaluations involved close collaboration with each vendor to identify their capabilities, using the common lexicon of the ATT&CK® framework. According to MITRE, evaluations are measurable and repeatable allowing participants to make continual improvements and better focus their future defense investments.

In Round 2 of testing, MITRE tested dozens of Enterprise Windows ATT&CK® techniques across 10 ATT&CK® stages, excluding the “Initial Access” and “Impact” stages which were beyond the scope of the test. These techniques have been found to be used by APT29 in previous breaches. 

Compared to other third-party evaluations which tend to focus on prevention capabilities, MITRE measures the number and types of detections. The logic behind this makes sense as, ultimately, you can’t prevent or respond to what you can’t see. Therefore, visibility is the name of the game in the ATT&CK® Evaluations. High visibility contributes to an increased chance of detecting stealthy attackers before they have fully breached an organization. Low visibility, by contrast, increases the time it takes to detect attackers and leads to a greater chance of breach.

Detection and prevention – across the attack chain

In reality, both detection and prevention are crucial for confronting advanced threats. There is no Protection without Detection.  Detection can stop threats from getting on the machine in the first place.  And lessons from Detection can be converted into prevention rules to prevent future breaches.  Without Detection threats can’t be found, the attack surface can not be reduced.   Symantec Endpoint Security focuses on providing protection not only across the entire MITRE ATT&CK® chain, but beyond it, at the pre-attack and post-breach stages – with detection, prevention and remediation as early as possible. 

Usually a vendor waits till they see the results of a test to decide if they want to talk about it.  But we feel this test is important.  We want to be publicly benchmarked.   And we look forward to sharing our thoughts on the final testing.  We’ll be back soon to share the results.