There are many different types of security assessments an organisation can perform and they are not always easy to keep separately in our minds. The key is to understand what differentiates them from their commonly confused cousins.
A vulnerability assessment is designed to find as many flaws as possible in order to make a prioritised list of remediation items. It is a technical assessment designed to yield as many vulnerabilities as possible in an environment, along with severity and remediation priority information. Best used when security maturity is low to medium, when you need a prioritised list of everything that’s wrong, where the goal is to fix as many things as possible as efficiently as possible.
A penetration test is designed to see if a mature defense can stop an attacker from achieving one or more specific goals. It is a technical assessment designed to achieve a specific goal, e.g., to steal customer data, to gain domain administrator, or to modify sensitive salary information. Due to the fact that a penetration test is designed to achieve one or more specific goals, they should not be commissioned by low or medium security organisations in most cases. Performing a penetration test against a low or medium security shot will simply yield recommendation all-time-greats such as, “implement patching across the organisation”, “disable inactive users” and one of my favourites “understand where your sensitive data is.” I would not recommend wasting money on a penetration test unless you have already undergone at least 5 vulnerability assessments and then remediated everything that was found. Penetration tests are for testing security that is assumed to be strong.
A red team engagement is designed to continuously test and improve the effectiveness of a company’s blue team by mimicking real-world attackers. This is more a service than a one-time engagement or assessment. The central purpose of a red team is to improve the quality of the corporate information security defences, which, if one exists, would be the company’s blue team. Red Team services should always have the following five elements: Organisational Independence, Defensive Coordination, Continuous Operation, Adversary Emulation and Efficacy Measurement. They are best used when an organisation has covered the basics of strong vulnerability management and has at least some capability to detect and respond to malicious or suspicious behaviour in the environment. If an organisation is still struggling with basic asset management, patching, egress traffic control and other fundamentals, it’s usually best that they get those solved before hiring or building a “Red Team”. A good rule of thumb is: if you don’t have a blue team, you probably don’t need a red team.
An audit is technical and/or documentation-based, and focuses on how an existing configuration compares to a desired standard. This is an important point. It doe not prove or validate security; it validates conformation with a given perspective on what security means. These two things should not be confused. Organisations use audits to demonstrate compliance. Importantly, compliance should not be used to demonstrate security.
White/Grey/Black-box Assessments are used to indicate how much internal information a tester will get to know or use during a given technical assessment. The levels map light to internal transparency, so a white-box assessment is where the tester has full access to all internal information available, such as network diagrams, source code, etc. A grey-box assessment is the next level of opacity down from white, meaning that the tester has some information but not all. The amount varies. A black-box assessment is an assessment where the tester has zero internal knowledge about the environment, i.e. it is performed entirely from an attacker perspective.
White-box assessments are best used with vulnerability assessments because you want to find as many issues as possible, regardless of how the tester came to discover them. Grey-box assessment are often used when people are confused about the difference between a penetration test and a vulnerability assessment.
A Risk Assessment, like a threat model, is extremely broad. Both in how they are understood and how they are carried out. At the highest level, a risk assessment should involve determining what the current level of acceptable risk is, measuring the current risk level, and then determining what can be done to bring these two in line where there are mismatches. Risk Assessments commonly involve the rating of risks in two dimensions: probability and impact, and both quantitative and qualitative models are used. In many ways, risk assessments and threat modelling are similar exercises, as the goal of each is to determine a course of action that will bring risk to an acceptable level.
Risk Assessments should be considered the umbrella term for determining what you have that is of value, how it can be attacked, what you would lose if those attacks were successful, and what should be done to address the issues. It is important that when someone says they’re doing a risk assessment that you delve deeper into exactly what is meant by that, i.e., what approach or methodology will be used, what the artifacts will be, etc.
A threat assessment is a type of security review that’s somewhat different that the others mentioned. In general it pertains more to physical attacks than technology, but the lines are blurring. The primary focus of a threat assessment is to determine whether a threat (think terrorist threat) that was made, or that was detected some other way, is credible. The driver for this assessment is to determine how many resources – If any – should be spent on addressing the issue in question. These assessments are best used in situations where someone has made a claim around performing an attack in the future, or such a potential is uncovered somehow. The goal is to learn whether the situation is worth spending resources on addressing.
Threat modelling is not a well understood type of security assessment to most organsiations, and part of the problem is that it means many different things to many different people. At the most basic level, threat modelling is the process of capturing, documenting, and (often) visualising how threat-agents, vulnerabilities, attacks, countermeasures, and impacts to the business are related for a given environment. As the name suggests, the focus often starts with the threat agent and a given attack scenario, but the subsequent workflow them captures what vulnerabilities may be taken advantage of, what exploits may be used, what countermeasures may exist to stop/diminish such an attack, and what business impact may result. Organisations should be using threat modelling early and often, and they should definitely form part of the development process. They are a way of ensuring that known potential attack scenarios can actually be handled by a given security posture. They can also be extraordinarily illuminating from a pure documentation and visibility standpoint. Seeing your potential threat-actors, how they’re likely to attack your app or system, using what vulnerabilities and exploits and what it will likely do to your organisation is often a sobering experience. They are especially useful for showing non-security related people how compliance and security products do not a security program make.
A bug bounty is designed to use the benefits of a crowd to discover as many vulnerabilities as possible. The central concept is simple: security testers, regardless of quality, have their own set of strengths, weaknesses, experiences, biases, and preferences, and these combine to yield different findings for the same system when tested by different people. In other words, you can give 100 experienced security testers the exact same testing methodology and they’re likely to find widely different vulnerabilities. The bug bounty concept is to embrace this difference instead of fighting it by harnessing multiple testers on a single assessment.
Bug bounties are best used when you have already performed one or more standard vulnerability assessments (which should have included both automated and manual testing) and then you have remediated everything that was found. Consider them an optional step between classical vulnerability assessments and a penetration test, which, as noted above, does not seek to find all issues rather to confirm that the security posture is where it needs to be by pursuing specific goals.
