Managing Technical Security Testing - part 3: Choose the right testers
Independent testing
The first question to answer is how independent the testing needs to be from the business and IT owners of the targets. For regulatory testing, the testers' organization must be independent from the organization owning the targets. It requires that the creation, maintenance or operation of the targets is not even partially done by the testers' organization.
At the other end, in agile mode, the tester responsible for security testing may be part of the team. The process and the results are inspected like for any other role in the team.
Security testing may be a part of the security organization within the organization itself. The level of independence depends on the reporting lines of security. If security testing is part of the IT department, there may be conflicts of interest.
Testers' specializations
While testers may have broad competences, they usually focus on specific areas: DB security, system hardening, switches and firewalls, webservers and reverse proxies and web application firewalls, email systems, cloud connectivity, mobile clients, SAP systems, data lakes, mainframe, … There is a long list of testing disciplines and expertise.
Please also check specific requirements for security tests that are mandatory for your industry branch. These may require testing providers to be accredited for the specific security testing.
Check your testers for the specific skills, they may have narrower or broader skill sets than advertised.
Infrastructure testing
The test based on running scans on the targeted networks is a core one. The findings can be a annotated, huge list, ordered by severity. The severity ratings are often context-agnostic: they would be the same regardless of the system and its location in the infrastructure.
For severity ratings over the years some rating methodologies have become standard. This is especially useful if you have multiple sources of ratings.
There are many offerings for these scans, with have a wide variety of impact to deploy, depth of view provided and contextualizing options.
Outdated versions and critical vulnerabilities must be investigated and be put in context. These adjustments are the basis of a prioritized to do list. While scans easily list thousands of issues, the priority list tends to be much smaller. Testers that provide good digests are a great asset. The skill to handle scan-based testing is being able to explain issues and severities and decide, in cooperation with the infrastructure team, the right priorities and plans to resolve those.
Application testing
Web applications
Today the applications that are most likely to be tested are web applications. They are externally facing so security issues are very visible, they are providing important business functions for the organizations, they are used for supporting remote access by providers, customers and staff.
Contrary to infrastructure testing, the non-profit organization OWASP was founded early in the rise of web applications and has since been a key resource for all application testing. The most famous series of documents are the OWASP top 10 (web) application security issues. Please note that limiting your testing to the OWASP top 10 issues is the very basic, essential level of testing only.
They also have much more elaborate guidance on application security and application security testing: the "OWASP SAMM", the software assurance guide, and the "OWASP Application Security Verification Standard", and the "OWASP web security testing guide".
Office automation/end-user computing/shadow software
Office tools like MS excel and MS access can be used to build applications. They have their specific risks and they require specific skills to do security testing. These applications are initially build with minimal code, and using basic skills, yet they can grow to serious levels of complexity and large impact if broken.
Specific classes: mobile apps
Mobile apps are yet another special category within applications. OWASP provides testing support.
"The OWASP Mobile Application Security (MAS) flagship project provides a security standard for mobile apps (OWASP MASVS) and a comprehensive testing guide (OWASP MASTG) that covers the processes, techniques, and tools used during a mobile app security test, as well as an exhaustive set of test cases that enables testers to deliver consistent and complete results."
Specific enterprise-level software
Many organizations deploy enterprise level solutions. There are certainly databases and possibly data lakes or data mining solutions in use. There are widespread systems like SAP systems, there may be mainframes in place, …
These pieces of software come with dedicated solutions and configurations for security. While a generalist may be able to do some security testing, it is best to look for more in depth understanding.
Large in-house systems build by or for the organization have the additional problem that knowledge may be found only within the company, putting the independence of the reviewer/tester in question.
Building management systems and SCADA systems
While many of the systems in this domain are linked to physical security measures, they are actually networked applications with centralized control systems.
They may be sharing infrastructure with the other IT systems, especially network components and network usage. They may be black boxes with little information on the internals. They may not respond well to standard network scans. If they are linked to safety testing must be done by competent people understanding the potential impact of testing scenario's, to avoid any safety risks.
A particular problem is quit common: organizations may share buildings or floors or meeting rooms with other tenants. That typically implies shared HVAC, and also technical infrastructure located in shared locations. Testing any shared infrastructure runs into restrictions and liabilities.