Measuring Security Behaviour Change: From Completion Rate to Actual Risk Reduction

At the end of every compliance period, someone in most Australian organisations generates a report that shows what percentage of staff completed the mandatory security awareness training. If the number is high enough, the programme is considered successful. If it is low, a reminder email goes out and the number climbs. The process is administratively clean, easily auditable, and almost entirely disconnected from whether the organisation's security posture has improved as a result of the training.

Completion rate measures whether people clicked through a module. It does not measure whether they absorbed anything from it, changed any behaviour because of it, or are better equipped to make a security-relevant decision than they were before. A staff member who clicked through a phishing awareness module on their phone while distracted has a completion record identical to one who engaged carefully, took notes, and discussed the content with their team. The metric cannot distinguish between them. If completion rate is the primary measure of your awareness programme, you do not know what your programme is achieving.

The Metrics That Actually Reflect Behaviour

The metrics that provide genuine signal about a security awareness programme's effectiveness are the ones that measure behaviour rather than activity. Three categories stand out as both meaningful and measurable for most organisations.

Phishing susceptibility rate -- the percentage of staff who click on simulated phishing attempts -- is the most widely used behavioural metric and the most useful one, provided it is measured over time rather than as a single data point. A single simulation tells you where the organisation is today. A series of simulations over 12 months, with increasing sophistication of the scenarios, tells you whether the programme is producing improvement, stability, or regression. The trend line matters more than any individual number. Pair this with the report rate from those same simulations -- the percentage of staff who reported the simulation as suspicious rather than clicking -- and you have a two-dimensional picture of where your staff awareness actually sits.

Reporting Rate as a Leading Indicator

The phishing report rate is arguably the most useful single metric in a security awareness programme because it is a leading indicator rather than a lagging one. A staff member who reports a suspicious email before clicking it has contributed to the organisation's defence rather than simply failing to harm it. An organisation with a high report rate has an early warning system embedded in its workforce. An organisation with a low report rate -- even one with a low click rate -- has a workforce that is individually cautious but not collectively contributing to detection.

Tracking report rate over time reveals things that click rate does not. An organisation where click rate is dropping but report rate is also low may be producing staff who are more hesitant but no more likely to alert the security team. An organisation where both metrics are improving is building a genuinely more security-aware workforce. The combination tells a story that either metric alone cannot.

Incident Patterns as Programme Feedback

Security incident data is an underused source of feedback for awareness programmes. If your incident records over a 12-month period show a recurring pattern -- for example, a disproportionate number of credential compromise incidents originating from a particular team, or a pattern of data handling incidents in a specific location -- that is information about where your awareness programme is not reaching effectively. It is also information about where targeted intervention is likely to have the most impact.

Useful indicators to track from incident data include:

• The proportion of incidents involving a human action as an enabling factor (clicking a link, approving a fraudulent request, misconfiguring access)
• Whether those incidents are concentrated in particular teams, roles, or locations
• Whether near-misses -- incidents that were caught before causing damage -- are increasing, which may indicate that reporting culture is improving
• The time between an incident occurring and it being reported internally, which reflects whether staff know what to report and feel safe doing so

Combining Quantitative and Qualitative Measures

Quantitative metrics -- click rates, report rates, incident counts -- tell you what is happening. Qualitative measures tell you why, and they are worth adding to your measurement framework. Short surveys after training campaigns, conversations with team managers about what staff are raising as questions, and structured feedback sessions after tabletop exercises all provide insight that numbers cannot. An organisation where staff say they understand why phishing is a risk, know what to do when they suspect an attack, and feel comfortable reporting without fear of blame is in a different position from one where numbers look similar but staff feel unclear or unsupported.

Reporting to leadership and boards on awareness programme effectiveness should reflect this richer picture. A report that shows click rate and report rate trends, connects them to incident patterns, and provides qualitative insight from staff feedback is a genuinely useful governance document. A report that shows completion rates is an administrative record. The standard you hold your measurement framework to signals how seriously the organisation treats the programme overall.

We help organisations build measurement frameworks for their awareness programmes that go beyond completion rates. If you are looking to understand what your programme is actually achieving and how to use that to improve it, get in touch at info@cyberlinx.com.au.