Summiting the Pyramid: Bring the Pain with Robust and Accurate Detection

By Michaela Adams, Roman Daszczyszak and Steve Luke • December 16, 2024

David Bianco’s The Pyramid of Pain introduced the world to the idea that if defenders focused on identifying and detecting adversary tactics, techniques, and procedures (TTPs), it would be harder for adversaries to evade detection. The higher up the Pyramid a defender can detect, the greater the cost imposed on the adversary.

In 2023, the Center for Threat-Informed Defense created and released Summiting the Pyramid. Summiting the Pyramid (STP) provided a methodology to score analytics against the pyramid of pain, helping defenders create more robust detections against adversary behavior. The methodology scores analytics against the pyramid of pain and changes the way we assess about detection engineering by scrutinizing the components within the analytic. Since its release, Summiting the Pyramid has been used by organizations to improve their detections of adversary behavior, and the Sigma analytics repository now has an STP flag to score the robustness of open-source analytics.

The initial research broke ground on the new dimension of robustness, and immediately the community asked for more. Two requests in particular inspired our next research area:

• Fewer false positives. How can we write robust detection analytics that are resistant to adversary evasion without introducing too many false positives?
• Determining analytic robustness scoring for network-based analytics. How could we expand the scoring framework to include analytics from both a network-based sensing and host-based sensing perspective? The initial release of Summiting provided a framework for scoring host-based analytics, primarily from the Windows Operating System (OS).

In partnership with AttackIQ, Fortinet, IBM Security, Lloyds Banking Group, and The Microsoft Corporation, Summiting the Pyramid now includes resources for defenders to build accurate, robust analytics for host and network data that are difficult for an adversary to evade.

Build a Robust Detection

Ideally, our detections will be accurate and resistant to adversary evasion over time.

An accurate analytic has low false positives and low false negatives (i.e., good precision and recall). A robust analytic will remain accurate, even as adversaries implement new and sneakier versions of ATT&CK techniques.

The methodology now provides three actionable steps to build robust detections:

1. Identify sets of observables which will trigger no matter how a technique is implemented, known as a spanning set.
2. Select spanning sets which are most specific to malicious behavior, focusing on resistance to adversary evasion first, and then on accuracy.
3. Add false positive exclusions, making the detection more accurate. These exclusions use specific values of fields that are difficult for the adversary to modify. This is necessary to preclude an adversary “hiding” within the exclusion itself.

Summiting the Pyramid has introduced Detection Decomposition Diagrams (D3) to facilitate this process. These D3 visuals give defenders a view across multiple implementations of a technique to identify analytic and event observables for robust detections. D3 visuals include benign and malicious implementations of the technique. Observables which span across multiple implementations provide higher robustness; that is, resistance to adversary evasion over time. Other observables may be used for better accuracy rates.

There will be tradeoffs between robustness and accuracy when building more robust detections. For some technique a detection’s, resistance to adversary evasion might be more important than accuracy, or vice versa. The robust detection guidance and D3 visuals provide defenders the tools and steps to evaluate the tradeoffs, determine how each component impacts the detection, and build the most impactful detection for their environment.

Expanding to Network Detections

Summiting the Pyramid now has two frameworks for scoring detections: host-based model and network traffic model. These are separate because OS-based robustness looks different from network traffic robustness.

The host-based model consists of three event-robustness columns. It measures the increasing cost or difficulty for the adversary to avoid host-based sensors. This includes host-based network events. To incorporate these events, the rows have been updated to account for detections or events which may occur on the outside boundary (Level 2) or within the defender’s environment (Level 3).

The network traffic model scores detections based on the increasing cost or difficulty for the adversary to avoid sensor visibility into the relevant network protocol. This model has two columns: protocol header and protocol payload. Network traffic robustness increases as you move into the header, since a defender will not need to rely on obfuscated network traffic payloads to detect adversarial activity.

With these two models, defenders can analyze robustness from both a host-based and network perspective.

Increasing Difficulty on Adversaries in v2

Summiting the Pyramid provides a framework to understand how adversaries can evade detections, helping defenders build more accurate, robust detections. In this continuation of Summiting the Pyramid, defenders can:

• Utilize step-by-step guidance and D3 visuals to build robust detections which are accurate and resistant to adversary evasion over time.
• Scoring to network detections and improve them through pre-scored observables.
• Use the Summiting the Pyramid analytics repository.

Future work will include how to build more robust detections against non-malicious (living-off-the-land) techniques and will expand the Summiting the Pyramid framework to more OS’s and environments.

Get Involved

We would love to hear about how you’re using our work! If you have any feedback, technical questions, concerns, or contributions you’d like to make to the project, use the Contact Us form or submit an issue on Github.

© 2024 MITRE. Approved for Public Release. Document number CT0128.