How to Protect Against AI-Enabled Phishing and Social Engineering

Phishing has always been the most reliable initial access technique for attackers because it targets people rather than systems, and people can be deceived in ways that technical controls cannot always prevent. For most of the past decade, the defences against phishing focused on training people to spot the signals of a poorly crafted message: grammatical errors, suspicious sender addresses, mismatched URLs, and generic salutations. Those signals were useful because commodity phishing was often poorly constructed. That advantage is largely gone.

AI language tools allow attackers to generate grammatically correct, contextually appropriate, and individually personalised messages at a scale that was previously only available to nation-state actors with large operational teams. A phishing message that references a recent company announcement, uses the recipient's correct name and role, and reads as though it came from someone who knows the organisation is no longer the product of significant attacker effort. It can be produced in seconds. The defensive posture needs to reflect this shift.

What Has Changed in the Threat

Three changes are most relevant to how organisations should adjust their defences. First, the quality bar for phishing messages has risen significantly. Controls and training that relied on people identifying low-quality messages as suspicious will miss a higher proportion of attacks than they did before. The tells that were common, poor grammar, awkward phrasing, obvious translation, have largely been eliminated as a reliable signal.

Second, voice and video have become usable attack vectors at scale. Synthetic voice and video generation, previously expensive to produce convincingly, can now create realistic impersonations from limited source material. Social engineering attacks that use audio calls or video calls as part of the scenario, a fake executive calling to request an urgent wire transfer or a fake IT support person requesting credential reset assistance, are more credible and harder for targets to detect without verification procedures.

Third, the research phase of a targeted attack is faster and cheaper. AI tools can process publicly available information about an organisation and its people to identify plausible attack scenarios, relevant relationships, and appropriate language for a specific target. Spear phishing that previously required hours of manual research can now be prepared in minutes.

Controls That Address AI-Enhanced Attacks

The most effective controls against AI-enhanced phishing address the attack where it is most constrained: the exploitation phase rather than the delivery phase. Controls include:

• Phishing-resistant multi-factor authentication that does not rely on one-time codes that can be stolen or relayed in real time, but instead uses hardware keys or passkeys that are bound to the specific site and cannot be phished
• Conditional access policies that require additional verification for high-risk actions such as accessing sensitive data from a new device or location, even when a valid credential is presented
• Strict email authentication including DMARC at reject policy, which prevents attackers from spoofing your domain in the targeting of your own staff and partners
• Verification procedures for out-of-band requests, particularly financial transactions, credential changes, and access grants, that require a second channel of confirmation independent of the original message
• Endpoint detection that can identify post-exploitation activity even when the initial access method bypasses email filtering

What Awareness Training Still Needs to Cover

Awareness training remains valuable but needs to be updated to reflect the changed threat. Training that teaches people to look for spelling errors and generic salutations is training them to look for the wrong things. Updated training should focus on the nature of the request rather than the quality of the message. Requests that create urgency, requests that bypass normal processes, and requests that involve credential submission or financial action outside of established workflows should trigger verification regardless of how convincing the message appears.

Training should also cover the specific scenarios that AI-enhanced social engineering uses: callback phishing where the initial message is benign and the attack happens in the follow-up call, voice impersonation of executives or IT support, and messages that reference real internal events or individuals in ways that appear to indicate insider knowledge. Practising recognition of these scenarios, particularly in phishing simulations, builds more useful muscle memory than training that focuses on identifying poorly crafted messages.

Detection and Response Assumptions

Assume some phishing will succeed. The defensive architecture should account for this: credential theft that results in account compromise should be detectable through behavioural monitoring before significant damage is done. Controls that limit what a compromised account can access, that detect unusual access patterns, and that require verification for high-impact actions all reduce the consequences of a successful phishing attack. The goal is not zero successful phishing. It is limiting the blast radius when phishing succeeds, which it will.

To discuss your organisation's defences against modern phishing and social engineering, contact Cyberlinx at info@cyberlinx.com.au.