First, let's talk about what "failure" is and is not in the context of Security Awareness Training (SAT). Failure is not when a company gets breached due to social engineering. Wait, what?? All the outstanding training in the world does not guarantee that an individual will follow it when the moment of testing occurs. Soldiers are trained to highest levels to deal with the stress and trauma of battle, but still come back scarred, having likely made mistakes in spite of their training.
Security Awareness Training "failure" is when a user is left without the knowledge to act appropriately in a given situation. In other words, when they haven't been properly prepared to handle phishing emails and voice based scams or choose a strong password, SAT has failed them. The user clicking on the phishing link is just the end result of this failure.
I submit that SAT failure occurs for (at least) three reasons:
1) Training does not reflect real life security encounters.
I am sick to death of training courses telling users that they should watch out for misspellings and grammar mistakes in the email body, as if phishers don't have access to Word's spell/grammar check. While its certainly true that phishing emails can and do originate from these places, the user is actually provided a false sense of security if that is their sole basis for detection. What if your company employs or works with individuals whose first language isn't English? End users will receive legitimate emails with spelling/grammar mistakes and their training will be for nothing.
Trainer tip: If you want to talk about misspellings, forget the body of the email and focus on the sender's domain! That's more of an effective phishing mitigation technique than looking for misspellings in a paragraph will ever be.
Additionally, how often have we heard to choose a "secure" password like "7Jkw8$hQ"? Look at that thing. Covers all the complexity rules, meets the minimum 8 character corporate standard, a true thing of beauty...
Trainers say this because they've never cracked a password hash in their whole career and don't know any better (See #2). With our cracking rig, that would take about 2 seconds to crack (NTLMv1 that is. About an hour for NTLMv2).
Trainer Tip: Emphasize length over complexity, every time. "this here password is fantastic" is much stronger than an 8 character complex password and is way easier to type!
Good SAT has to be thoroughly relevant and a bit scary. Users must understand the consequences of clicking that link or opening that attachment. It's not good enough to simply say "don't do this", they must come to understand why. If they cannot link their improper action with a loss of company data in vivid, graphic detail, you aren't doing your job.
2) Instructor does not have the right experience to be effective.
The best instruction I've ever received, security-wise or other, has been from those who have been there - actually doing the work. They aren't professional trainers. They are simply experts in their field because they've spent years actually doing the things they are training about. When you go on YouTube to learn how to replace a toilet, do you find 17 year old Billy's video and follow that, or do you find some crusty graybeard who's been plumbing his whole life? That's not to say good trainers can't be young, it just means they must be *experienced*.
Let me emphasize this: the WORST thing you can do for your org is to simply relegate your SAT to computer based training. Forcing a user to watch some lame video of a "hacker" so they can click Next as fast as possible before guessing the painfully obvious quiz answers is an egregious waste of company resources, not to mention people's lives.
Do you company's data a favor, pay for high quality live training that will make an impression and get your users talking about security! Fostering a culture of security in your enterprise is one of the best things you can do for your organization's security posture.
3) Training is boring.
This is a personal pet peeve of mine. How often have we watched someone with awesome content who simply could not deliver it well. I'm not talking about people who are nervous during their presentation. That's certainly understandable. I'm talking about people who deliver in a bland, monotone, and otherwise horrible way - devoid of anything close to resembling emotion.
Information Security is one of the most interesting fields in existence today, and one need not look far for stories that will not only make you laugh (or cry), but will also drive home the points you need to make as a trainer.
Trainer Tip: Learn how to tell a good story. Practice in the mirror if you have to, but do not settle for mediocrity. Your training should be engaging and memorable, and when it comes to Information Security, nothing beats a good story! Don't know how to do that? Start here. Then here.
Pro Trainer Tip: Tell stories about your failures. When people observe you as a human, not an "expert", you will connect with them immediately and have their complete attention.
If you want to deliver solid infosec training, you must focus on the right things, have the experience to back it up, and be able to connect to your audience. Miss any one of these three and the quality of your training will suffer. Put up screenshots of real phishing emails, do a quick demo of what a compromise looks like, tell your audience how you failed then succeeded, and people will connect with you not only as an expert, but also as one who is in the trenches with them. There is no better place to be as a trainer.
But I'm just organizing the training for my company!
Hire a pentester! They will (hopefully) know their stuff and have the experience to back it up. Have a quick call with them to make sure they don't bore you to tears, and off you go. Don't know who to hire? Call me biased, but I think these guys are pretty good. :-)