The colorful world of security testing

11-11-2024

The offensive security world is full of colors like red, blue, purple, black, grey, and white. Whether the last three are colors is a grey area. But what do these colors mean and which pieces of the security composition require which color?

The color of the tester's hat

When you look at older westerns the color of a person's hat is often a hint on the role: is he good or bad? Woke alert: we talk about cowBoys and white signifies "good" in this particular context.  

In offensive security the convention is that the white hat is worn by the defender, the black hat is worn by a malicious hacker (some people define hacker as always malicious; that was certainly not the case initially when it meant skilled technician), the grey hat is a hacker with a conscience. 


The white hat hacker works fully transparently and is authorized to test systems with due care to avoid damages and collateral damages. The methods being used are cleared by the organization. The findings are kept strictly within the organization. They work for the organization or are hired by the organization from a vetted company offering pentesting services.

The black hat hacker has no restrictions, neither on the methods used, nor the damage incurred, nor the distribution and publication of the outcome. The results may be used to support criminal activities.

The grey hat hacker operates at the border of the law, and may not have any formal link with he organization. They may expect a finders fee or to boost their reputation. They would try to follow the responsible disclosure guidelines depending on the organization's response to the report.


Which hat do you want your tester to wear? You want your systems to be professionally tested. White hat hackers are the preferred way forward. Regulations often require that you use independent testing of your security posture. So you hire white hat hackers from a specialized organization, or you convince the auditor that your internal testers are independent (enough). 

In case you are contacted by a hacker claiming vulnerabilities are present in your systems, make sure you find out if the hat is grey or black. Cooperate with grey hats to solve confirmed issues, and give them credit. If that is all you need to do you got a great deal.

The impact of the color of "the box"

The hacker's goal is to test the security of your box, the target for testing.  

The black, white and gray boxes

You can provide various levels of information to help the hacker to check if the box can be opened. The information can be extremely limited: the box is there, make my day. There is no information on the material of the box, the type of lock, what is in the box, nothing. This is the black box problem. For testing across the internet, you get the domain name. Period.

The other extreme is that you get full information on everything: the plan of the box, the type of keys, the materials it is build with, the internal structure, boobytraps, alarms, … This is the white box variant.

The grey box is in between the white box and the black box. You get some information on the target, but not everything. This option may be a good choice for flexibility in testing: the level of information can move from black to white over time. The sliding view allows to test various aspects in one go.

What type of box for which test objectives?

  • I want a realistic situation mimicking malicious hackers, so I choose a black box approach.

If your question is to see if a blind hack can be done within the given timeframe with no prior information, black box will give you an indication of the complexity of breaking in.

The problem is the time frame you allow. Most of us will not have the money to let the attempted hacking to go on forever. On the other hand, real hackers may be patient if the expected gain is high enough.

Your detection mechanisms may work almost immediately as going low and slow is too time consuming for the testers. Real hackers have no such large time frame.

  • I want to find all problems regardless of how hard they would be to find, so I pick the white box approach.

This type of test can find all problems regardless of the difficulty of finding the flaws. In practice, this goal will not be achieved as again, testing will be time boxed.

The findings of this test will often be met with skepticism on the probability or the likelihood that an external hacker could find the vulnerability without the inside information.

  • I want to test the time it takes for an outsider without information to find problems, and to test my detection mechanisms to see if and when they pick up the attack, and rest assured that we don't rely on security by obscurity. I will start with black box testing, then move to gray box and finally white box.

The color of the team

Time to talk about the true colors of the testing team: red, blue, and purple. Again, disclaimer; these colors have connotations in some countries. I do not think it was a democratic process to assign these colors.


Meet team blue. Organization have internal testing teams (upstaffed with external if needed) to produce secure systems. Typically they use standards, guidelines, best practices, configuration templates, … that are checked and adapted as required. They look at their systems from a defensive perspective. They are specializing in preventive counter measures and detection systems.

Red is used everywhere to signal danger and risk. Red teams are attacking teams. They do not care about standards, guidelines etc., they just look for any way to break through the barriers, preferably undetected. Staying hidden does not matter in hit-and-run style hacks so being detected does not always invalidate the test result.

Back to kindergarten: what do you get if you mix red and blue? Purple! (if you mix them well and in the right proportions). Purple team testing is simultaneously using a red and blue team and facilitates the communication during testing between the two viewpoints and findings to improve the overall benefit of the testing.


Which teams will you have? You must have a blue team function, no discussion: it is mandatory to build security into the acquisition and development processes, as well as in the deployment and operational steps.

The red team serves as a major quality control on all efforts, across the board. If they can physically enter the building and computer room, get access to a phishing attack, that works as well as via a sophisticated network and computer hacking.

To increase maturity of both blue and red teams, and heighten the level of assurance a purple team is a good instrument.

Summary

The guide for colors in testing:

  • The color of the hat is a grayscale indicator about legal and ethical behavior: the better, the whiter

  • The color of the team indicates the teams stance: defensive (blue) or offensive (red) or both are cooperating during testing (purple).

  • The color of the box is a grayscale indicator about knowledge of the target: the more knowledge, the whiter

What are the main objectives for testing and which version fits those objectives?

  • Depth of analysis:

    • White box: most complete view on vulnerability
    • Black box: outsider view on vulnerability
    • Grey box: black box with tunable extra information to allow to speed-up the testing
  • Trust: look at the hat
  • If the hat is not white, the tester may bite
  • Test comprehensiveness:

    • Purple team:
    • Broadest effort with focus on both defense and offense
    • Get better offensive results through insider insight
    • Get first-hand information on the vulnerability and exploitation
    • Feedback on proposed fixes
    • Red team: the "raw" truth
    • Go outside of the comfort zone: anything goes
    • The scope is not defined by the security organization (but restrictions must be considered and defined)
    • Blue team: security quality control within the existing frameworks