Organizations are increasingly adopting the threat led approach to penetration testing. This approach essentially advances the boundaries of conventional penetration testing by seeking to adopt the tactics, techniques and procedures of an advanced threat actor aggressively targeting a critical system.

The Bank of England-backed CBEST scheme and CREST-accredited STAR scheme are the two man programs to have emerged that have adopted threat led penetration testing. The first wave of CBEST tests has already occurred; with the Bank of England having urged the UK’s systematically important financial institutions to take the test.

A STAR test is similar to a CBEST test in many ways, but different in that it has no input from the regulator. However, it’s no longer just the United Kingdom; Hong Kong and the Netherlands are developing their own schemes that aggressively test critical systems in a live environment. Even if there is no requirement to conduct threat lead penetration testing, the approach can benefit organizations across the globe who are interested in conducting more effective assessments of their security posture.

Although such schemes are in their infancy, I’ve noticed a rapid development, particularly within the STAR scheme – the change from a linear to iterative process.

The STAR Test

A basic breakdown of the project steps for both a CBEST and STAR test is shown below:

Provider Project Phase
Threat Intelligence (TI) Define number of Critical Functions (CFs) in scope
Construct targeting report based on open source research around the CFs
Develop Threat Actor profiles specific to the CFs and based on the results of the targeting report
Develop scenarios, with approximately one scenario per CF
Deliver Threat Intelligence report
Penetration Tester (PEN) Conduct penetration test based on results of the Threat Intelligence Phase of the project

The table above demonstrates two main factors:

  1. There are two distinct providers involved in a CBEST/STAR test; the Threat Intelligence and the Penetration Tester.
  2. The process flows lineally from start to finish.

But this is starting to change. The linear flow is beginning to be replaced with a more iterative cycle that involves far greater dialog between the threat intelligence and penetration test provider. This not only results in a more cohesive project for the client, but it also demonstrates a profound change to threat led penetration testing. This is highlighted through the presentation of the attack scenarios.

Linear versus Iterative Process

Within the conventional, linear threat led penetration test, the scenarios are presented to the client as threat scenarios that could possibly work against a Critical Function (see below for stages 1-6). This is based on the previous reconnaissance and estimation of the capabilities of the threat actor. However, it is common for a scenario to quickly become unworkable early in the penetration test stage due to some unseen control within the client environment. Often, only 1 out of 5 scenarios become viable.

  1. Define number of Critical Functions (CFs) in scope
  2. Construct targeting report based on open source research around the CFs
  3. Develop Threat Actor profiles specific to the CFs and based on the results of the targeting report
  4. Develop scenarios, with approximately one scenario per CF
  5. Deliver Threat Intelligence report
  6. Conduct penetration test based on results of the Threat Intelligence Phase of the project

Within more iterative model of a threat led penetration test, the threat intelligence provider initially delivers a far more slimmed down scenario that really just designates the ultimate effect on the Critical Function as opposed to a blow-by-blow road map for the penetration tester to work through. The scenarios are subsequently updated as the penetration tester discovers nuances within the client network.

This is the fundamental difference between the linear and iterative approach to threat led penetration testing. In a sequential test the scenarios are either achievable or not. In an iterative style test, on the other hand, all of the scenarios should be achievable by the end of the test.

The Benefits of an Iterative Process

With so many scenarios now viable, organizations will, understandably, seek security assurances. To allay these concerns, I suggest the development of an additional framework to examine factors around the actor featured within the scenario. Factors might include:

  • Motivation
  • Level of resourcing
  • Technical skill
  • Access to advanced tools

These factors could all be used to effectively ‘benchmark’ a scenario. The penetration testers would begin at the lowest level, but increase each factor over time.

The Outcome

The completion of this iterative approach will result in a test that will reassure the organization that their system is secure against a constellation of actors with a defined set of skills and resources. Organizations can then better prioritize security spending and allocate resources as appropriate.