It’ll never happen to me…

We conducted a post phishing-test survey with a client and found that people held some intriguing perceptions…

Phishing test-and-train solutions have their limitations, and they should definitely not be viewed as the only option available for providing guidance on phishing. However, we recognise that they are a useful part of the security awareness armoury – and have found phishing tests to be useful to provide a point-in-time view of the susceptibility of the organisations we work with to malicious emails.

Following a recent phishing test conducted with a client, we ran a survey to gauge organisational attitudes to the problem. This uncovered a range of insights which we’ve been able to feed into our technology. But there was one result which stood out in particular – people’s assessment of how informed they are.

The table below summarises the responses from our question asking people how well informed they felt they were about phishing. The first row are the results for people who did not click the link in our test. The second row are for people who clicked the link and entered their password into a test web page.

We found it striking that the results from these two groups are virtually identical: those who had, in effect, “failed” the test felt they knew as much about phishing as those who “passed” the test. How might we interpret that outcome? Here are a few ideas…

• Overconfidence. The 60% of people who felt “fully” or “well” informed about phishing, yet clicked the link and entered their password, fell foul of overconfidence effects. It is important to know that people often overestimate their performance of a task – so perhaps we need to remind them that “it can happen to you”. This is just one of many cognitive biases we consider in designing our solutions.
• Changing perceptions. The test itself changed perceptions of how well-informed people felt about phishing. Remembering that the survey followed the test, perhaps the test was a “lesson learned” and those who “failed” then felt better informed. Although in this case no training or guidance was offered immediately following test “failure”.
• Other factors. There may, indeed, be a common distribution across the organisation of the extent to which people are informed about phishing, and other factors were more important in determining whether people “failed” the test. For example, they were especially busy, in a hurry, not paying attention, phishing simply wasn’t front of mind at the time etc. We strongly believe that an ongoing / drip feed approach to security awareness is critical to addressing these factors.
• Too difficult. Detecting phishing is simply too difficult a problem for people to do effectively. We hear this all the time (funnily enough normally from techies/tech vendors). Although our test was hard and tailored to the organisation in question, it wasn’t impossibly difficult and included several factors which could have raised suspicions. In addition, many more people “passed” the test than “failed” it. Incidentally, the aim of security awareness isn’t to eradicate security issues entirely (much like technical solutions can’t be 100% effective), but rather to reduce the likelihood of incidents.
• Differing perceptions. Perceptions differed of what being well informed, or otherwise, really meant – people’s level of expectation in what knowledge or expertise is required to be considered to be “informed” varied. This is undoubtedly true – and another reason not to rely on people’s own perception of their ability/knowledge/understanding.

Whilst this survey cannot be considered conclusive evidence – our methodology and sample size undoubtedly fell some way short of Office of National Statistics standards for robustness – it is interesting to speculate as to the cause of these results. The truth probably lies in some combination of the above factors.

As a final point, it is concerning, given the criticality of the problem from a security perspective, that 40% of people felt only “somewhat” or “not” informed about phishing. But that is why the organisation in question are working with us to use our Redflags™ product to provide effective security awareness!

Survey image copyright andreypopov / 123RF Stock Photo