Test triggers

Security testing triggers

Functional security testing like authorization enforcement, logging of security events etc. the testing should be part of standard Quality Assurance. Secure configuration standards compliance should also be business as usual.

In other cases the motivation for a security test is grounded in the security policy, that itself may be driven by regulations. Some typical triggers can be for instance:

- Changes

The introduction of new systems or new versions of systems may introduce new issues or make others more severe. New issues may arise in conjunction with existing systems, the configuration of security may pose problems due to unfamiliarity with the impact of settings.

The modifications may also solve previously reported vulnerabilities, which typically can only be closed after a new test.

- Regulatory requirements

Many industries are (heavily) regulated, including more security controls over time. Their license to operate may depend on acceptable test results. Providers of systems to any regulated organization likely requires the provider to have attestations from testing the system.

- Policy of the organization

The organizations' policy typically contains controls related to security testing, to prove to their people, their clients etc. that they take care of their security.

- Incidents

Incidents uncovering vulnerabilities not found or not covered by previous testing should lead to a test on other systems that may have the same issues.

- Security intelligence reports

Intelligence-based prioritization of new or extra tests to deal with active or upcoming attacks may be required, and this focus on real and present danger just makes sense.

When and how to define testing objectives?

The principle:

For every definition of a control, it must be defined simultaneously how compliance will be established, in sufficient, practical ways.

The way to look at it is putting the change or creation of a component between security brackets:

• The opening bracket is where you define security controls, and how compliance shall be validated
• The closing bracket is where you check compliance with the controls according to the rules set forth initially, with the security tests defined in the opening bracket

If you open a bracket, it must be closed.

No surprise attacks

Avoid the typical "surprise!" effect: security testing finds flaws or oversights, yet, the project was unaware of the issues till they were found by testing and reported. This is an example of a testing closing bracket without an opening bracket.

It can be underestimated what it means to implement a control. Controls may be defined without realizing the implications or how to do it at all. A single control may touch upon many targets and different type of targets. If that is not made explicit, it is also hard to know how to check compliance with the control.

Checking compliance through security tests implies defining which tests on which targets are providing reliable compliance assurance. If there is no way to establish compliance, the control has no power. Spend your time elsewhere.

Specific testing objectives

THE security test does not exist

There may be some misunderstanding around security testing. It happens often that a test request sounds like this "Please do a security test of XYZ, before the end of next month.". This request misses the point that many choices exist for what to test, where to test, and who does the testing, and is maybe optimistic on the when to test, and may forego the possibility the test could be failed.

The clarification "pentest" does not help. There are still too many variations and assurances that could result. It has no precise meaning so a mismatch between test and expectations is likely.

Identify the objectives associated to testable targets

The main parameters to define before testing are the security objectives, the scope specifying the targets, the controls to be tested, the test coverage, the expected time box, and the restrictions on testing. It is up to the organization to define these parameters for the planned pentest, and then match this plan with specific tests and assign these to qualified testers.

The objectives are mostly defined by two parties: the business owners of the targets and the security function. Business owners are driven by compliance and commercial reasons. They know the regulatory environment of the organization and the security requirements in these regulations. They need evidence that their systems meet those requirements as a minimum. Security requirements are defined in approved policies, standards, guidelines and principles.

Check if planned tests would meet the objective

Each security requirement relevant for the targets should be tested to have sufficient assurance for the desired security of the targets. Testing requirements can be factored into specific tests meeting typical testing competences.

See addendum: common tests

Test coverage

Once the targets, the requirements and the testing types are defined, test coverage is next. The following is a selection of bullet points showing the diversity of options, without details that would deviate from the main objective of this document.

Testing general security functionality:

The list would include:

• Security Logging and safekeeping
• Audit logging
• Monitoring capabilities
• Management access control
• Security configuration
• Key management
• Authentication
• Access control

A generic security test suite might include security test cases to validate both positive and negative requirements for security controls such as:

• Identity, authentication & access control
• Input validation & encoding
• Encryption
• User and session management
• Error and exception handling

Targets

Different types of targets require different testing. Here we list typical tests for common target types.

Network:

• Network devices: configuration guidelines and policies
• Communication protocols: security configurations
• Filtering: rules
• Segmentation/zoning/zero trust principles and practice
• Event generation
• DLP

Systems:

• Services running
• Hardening guidelines and compliance
• Software control of versions, licenses
• Configuration guidelines and verification
• Malware protection
• Access control, esp. Privileged access control
• Security Event generation
• Patching policy and practice

Applications:

• Installation rights
• Secure installation guidelines and configuration
• Patching, version management
• IAM issues
• Connection security
• Code issues

Common security tests

Continue reading: 

choose the right testers