MITRE Engenuity Publishes Managed Security Services Provider Evaluations

MITRE Engenuity has published an evaluation of managed security service providers. The evaluations “highlighted results across 16 providers and assessed provider capabilities in their ability to analyze and describe adversary behavior.” The participants in this first round of evaluations included Atos, Bitdefender, BlackBerry, BlueVoyant, Critical Start, CrowdStrike, Microsoft, NVISO, OpenText, Palo Alto Networks, Rapid7, Red Canary, SentinelOne, Sophos, Trend Micro, and WithSecure.

Note

• I’m a big fan of the ATT&CK-based product evaluations but a few caveats: (1) It takes a skilled/experienced security person to understand the results and compare across the tested products. This is not a simple ranking. (2) This latest test is not really of broad MSSPs, more a test of vendors with Managed Endpoint Detection and Response offerings based on their own products. (3) Each year, Engenuity tests something new: there is no year-to-year continuity, so the next version of these products and services will change as far as which are more effective.
• The MITRE ATT&CK framework is the Industry accepted way to describe cyberattack details. The jury is still out on its applicability in assessing security capabilities of individual products that make up the cybersecurity ecosystem. MSSPs offer a much broader range of cybersecurity tools and capabilities that support managed service providers.
• As much as 50% of organizations are using MSSPs these days. The evaluations don’t rank, qualify or disqualify any of the assessed vendors, but rather give you the results of a ten-step response to the TTPs of OilRig (an Iranian Government aligned ATP.) Consider not only how these MSSPs performed, but also if your team, insourced or outsourced, is considering all aspects of these ten steps. Drill down on making adjustments where gaps are found.

Read more in

• MITRE Engenuity Publishes First-Ever ATT&CK Evaluations of Security Service Providers
• MITRE Engenuity Launches Evaluations for Security Service Providers