Introduction
Most organisations now accept the need for a Security Awareness Training programme in some form or other. But what’s the use of this programme if you’re not measuring its effectiveness? More importantly, what if you’re not measuring the right thing?
One metric many practitioners focus their energy on is measuring click rates in phishing simulations, believing sometimes that a lower or higher click rate reflects how effective their awareness programme is. This approach overlooks other more critical (and useful) metrics such as the reporting rate on the simulations.
In this blog, we will explore why measuring positive behaviour change should be at the forefront of any cyber-security awareness strategy. For example, shifting the balance to focus more on reporting rates, rather than measuring the absence of a "bad" behaviour, like click rates.
The Traditional Emphasis on Click Rates
Phishing simulations have become a staple of cyber-security training programs. These simulations involve sending fake phishing emails to employees to see if they are lured by the bait, by clicking on malicious links or disclosing sensitive information. Measuring the click rate, i.e., the percentage of employees who take the bait, has been a common way to gauge an organization's susceptibility to phishing attacks. While this metric provides some insights, it has its limitations.
The Limitations of Click Rates
- Click rates are subjective: they are highly influenced by the complexity of the simulation, its theme and the area of business the recipient is in; it can even be influenced by the weather or what the employee ate for breakfast! How your awareness teams write these simulations will greatly influence the click rate.
- False sense of security: relying solely on click rates can create a false sense of security. A low click rate might suggest that employees are vigilant, but it does not necessarily mean they are fully prepared to handle real-world threats. Some employees may be lucky or cautious in simulations but fall victim to a well-crafted phishing attack.
- Misleading behavioural insights: click rates do not provide insights into employee behaviour beyond their initial response. Does a high click rate mean your employees wouldn’t recognise real phishing emails, or that they’re simply (understandably) getting tricked into clicking a simulation imitating their company’s emails- something they have implicit trust in and would more likely engage with?
As we have mentioned in a previous blog, it can be hard for staff to contextualise training delivered after the fact due to ego and cognitive biases. We are very good at convincing ourselves that we didn’t actually click (it must be some mistake) or that there was a very good reason we acted the way we did, we will take on information that justifies that and ignore information that conflicts with our current world view. This can make it hard for organizations to gain valuable data that could help them understand why certain employees clicked on links and what they can do to prevent it.
- Building mistrust between the business and your security teams: placing too much emphasis on click rates may discourage employees from reporting when they’ve genuinely interacted with malicious emails, fearing repercussions for their click actions.
- Business impact: driving mistrust of all emails through simulations, especially ones which imitate your business processes, could drive busy employees to just ignore all emails, including business internal communications, or emails from legitimate suppliers, potentially having a detrimental effect on your business operations.
The Crucial Role of Reporting Rates
Instead of fixating on click rates, organizations should prioritize measuring factors like the reporting rate—the percentage of employees who report suspicious emails or activities. Here's why reporting rates are a more critical metric:
- Early detection and response: a high reporting rate indicates that employees are actively engaged in the organization's security efforts. When employees report suspicious activity promptly, security teams can respond quickly, mitigating potential threats before they escalate.
- Continuous improvement: reporting rates provide a wealth of data that can be used to refine cyber-security training programs. By analysing reported incidents, organizations can identify recurring trends and tailor their training to address specific weaknesses, focus on less engaged parts of the business and reward those who are.
- Fostering a culture of security: emphasizing reporting rates promotes a culture of security where employees are encouraged to be proactive in safeguarding the organization. When employees feel supported to report, this fosters “self-efficacy”, and they become valuable assets in the fight against your business’s cyber threats.
- Real-world readiness: Unlike click rates, reporting rates mirror real-world behaviour. In a real phishing attack, the goal is not to prevent every click but to ensure that employees report any suspicious activity promptly. A high reporting rate indicates that employees are prepared for the unpredictable nature of cyber threats. Many phishing simulation platforms provide data on the time taken to report, an additional metric to understand your company’s engagement in reporting.
Building positive secure habits
A shift to measuring a “positive” behaviour or habit employees can develop around their interactions with emails, such as reporting, can be taken even further by applying behavioural nudges such as:
- Priming employees from time to time as they visit their email, to keep the threat of phishing alive and reminding them to report.
- Prompting them as they are about to click a link in an email from an unknown sender, embedding a simple behaviour such as double checking the sender and discouraging clicks in such emails.
- Nudging them to stop and think if they start to enter credentials having clicked a link.
Whereas the punitive post-click training approach of phishing tests can promote fear and inaction, these supportive nudges which improve confidence and self-efficacy, can build key behaviours into habits that employees will carry with them into all risky situations. We’ve recently seen clicks in emails from unknown senders reduce by 76% in just a few months at one client and have been able to measure a doubling of reporting rates at another. (If you want to see what this looks like in action, book a 15-minute demo here.)
Conclusion
While measuring click rates in phishing simulations has its place in cyber-security training, it should not be the principal focus. Instead, the reporting rate, reflecting employees' ability and willingness to report suspicious activity, is a more vital metric for assessing an organization's cyber security posture. Embedding coping strategies, such as checking the sender or checking for urgency/manipulation, as habits and measuring reduced overall clicks on links from unknown senders with tools such as Redflags® can take this even further.
By placing greater emphasis on positive behaviour change such as that measured through reporting rates, organizations can better prepare their employees to be the first line of defence against cyber threats. In this ever-advancing digital landscape, fostering a culture of security and early threat detection is paramount, and measuring positive behaviour change is the key to achieving these goals.