Testing set-up

The what-when-where must be formalized:

• What: a well-defined lists of targets
• When: the timeframe for the testing
• Where: Which environment
• Tester location: On-prem, special facility, remote

And also the how-who:

• How:
• tests to run
• methodology
• Who
• Internal test team
• External: independent and/or accredited vendors

Testing environments

The environment selected for testing plays a major role in the preparation.

Testing in production and pre-production environments

Pre-production may share almost all the management functions like logging, monitoring, and incident handling. Therefor, they are treated together.

Collateral damage during the test: logging and monitoring

Tests on production impact any logging up to and including the targets, and possibly systems the targets connect to. Monitoring on the targets or connections or infrastructure used to connect for the testing is impacted.

Even if the testing is planned in stealth mode, or with restrictions on authorized disruptions like denial of service, it cannot be excluded that the testing causes incidents on the production. Production incidents start a chain of events and actions with a potential big cost in effort. The organization must be prepared to stop this test-triggered incident handling before it is out of control.

Security weakening: IAM

The authentication and access control components will be those of production. That can get in the way of tool-based testing, if you get to run any tools at all. Introducing backdoors for production testing has its drawbacks as well: backdoors tend to be abused and weaken the security.

If scenarios involve 4-eyes control, it is very tricky to allow the testers to have multiple accounts to test those scenarios on production. There is a good reason to demand 4-eyes control.

"Rogue" connections

To test systems in production requires connectivity to the target systems from sources that would normally not be authorized. The extra connectivity can be created physically, using dedicated cabling to specific physical locations. Establishing connectivity can be logical via enabling limited routes to the targets from locations deemed safe enough, like a room with strictly limited badge-based access.

Testing in test environments

Testing environments are by nature much easier to work in. The key issue is to ensure a stable set-up for the security tests. They are going to test all parts of the targets so the complete test set-up for the targets must be exclusively reserved for these tests. Another reason for this exclusive access is the log inspection to see what was logged to evaluate incident response information, and for quickly seeing impacts of tests on the targets without "clutter" from other tests.

Access to the test environments (accounts, credentials) are rarely a problem.

As security testing requires a set-up that is as close to finished as possible, it will be scheduled near the end of testing just before moving to (pre)production. That timeframe is always hectic and often instable. Combined with the exclusiveness desire, conflicting interests with other tests are likely.

Good communication with the test leads and project management is recommended.

Testing in development environments

On the point of access or tooling, the development environment is the easiest. Stability on the other hand is naturally at risk. Developers will likely not be blocked from access during the test, just informed not to change stuff impacting the test, not a strong safeguard.

Good communication with project management is recommended.

Statement of Work

The statement of work (SoW) is the formal contract between the tester and the client. It contains the usual contractual items like acceptance criteria, payment information, legal terms and conditions. It must contain the clear definition of the contracted work related to the:

• Why: the objectives of the testing
• What: scope of the work, listing in-scope and out-of-scope targets
• How: the testing methodology, and whether it is white, gray or black box testing
• Restrictions: typically excluding Denial of Service, or shared services that have dedicated testing
• Reporting: not just final, but also end-of-day, and potentially high and critical findings when found
• Remediation assistance

Test timeslot

Test timeslot planning depends on the test triggers. All of these triggers come with their own constraints and surprises. That is what makes test planning sometimes a hard job.

The seemingly simplest test is one that is scheduled with a defined frequency. One would expect that you assign slots, do the tests at the defined slot, and you are done. These tests are usually tests in pre-production or production.

There are elements that can get in the way of the original planning. Production test environment may be blocked during certain business periods were any interruption would be very unwelcome. If there is an attack going on the extra alerts and events caused by testing are unwanted distractions.

Pentest related to change projects do have a plan when they expect the system to be in a state that allows correct testing. Experience shows that a plan rarely is stable for the whole lifetime of the project. However, as pentesting is most often executed by an external, independent company, even a shift of a week may be difficulty for the test team, especially near the planned date. Ignoring the changed planning means testing an unfinished product and probably dealing with an unstable environment. These elements may render the test results unusable.

For project pentesting there is another frequent problem. The test may fail to approve the result as acceptable. Fixes will be required, as well as a retest. Project planning may not have foreseen the delay. The pentest company may not have resources for the retest at the ideal point in time.

Continue reading: 

test execution