Loading
Loading
We use strictly necessary cookies to run this site, and analytics cookies to understand how it's used. See our Privacy Policy for details.
AI-written phishing, deepfake executive impersonation, and automated reconnaissance — what actually changed, and the defenses that still work regardless.
Most coverage of "AI hacking" focuses on attacks against AI systems — prompt injection, model theft, data poisoning. That's a real and growing category, but it's not the one most organizations will actually run into first. The more immediate shift is AI being used as a tool to make already-familiar attacks faster, more convincing, and cheaper to run at scale. Here's what that actually looks like in practice, and what changes about defending against it.
For years, one of the most reliable signals in security awareness training was "look for bad grammar and awkward phrasing" — a real, useful heuristic when a lot of phishing originated from non-native-English-speaking operations working from templates. Generative AI removed that tell almost entirely. A phishing email can now be fluent, contextually appropriate, and personalized to the target's actual job function and recent public activity, generated in seconds rather than hand-written. The defense has to shift accordingly: the reliable signal isn't how an email reads anymore, it's what it's asking for — an unusual payment, a credential re-entry, an urgent deviation from normal process — regardless of how polished the writing is.
The clearest real-world case remains Arup's HK$200 million (~US$25.6 million) loss in February 2024: a Hong Kong finance employee was convinced to make fifteen wire transfers after a video call featuring deepfaked versions of the company's CFO and several other colleagues, built entirely from publicly available meeting and conference footage. No system was breached — Arup's own CIO described it as technology-enhanced social engineering, not a cyberattack in the traditional sense. The lesson that generalizes: any organization whose executives appear in public video (earnings calls, conference talks, webinars, LinkedIn) has enough source material circulating for a plausible synthetic version to exist. Verifying an unusual instruction through a channel the request itself didn't suggest — calling a known number back, not replying on the same thread — is the actual mitigation, because it doesn't depend on being able to spot a deepfake in the moment, which is a genuinely hard problem even for specialists.
Before an attacker sends anything, they typically research a target — org structure, vendor relationships, who reports to whom, what tools a company uses. AI didn't invent this step; it collapsed the time it takes to do it from hours of manual searching to a task that can run largely unattended. That matters less for the sophistication of any individual attack and more for volume — the same economics that made mass, untargeted spam profitable now apply to attacks that look individually targeted, because the targeting research itself got cheap.
None of this requires a fundamentally new security program — it requires the existing fundamentals applied more consistently, because the old assumption that a well-crafted, personalized, or verbally convincing request is inherently more trustworthy no longer holds:
This is exactly the kind of scenario our Security Consulting engagements are built to pressure-test — not whether your systems can be technically breached, but whether your actual approval processes hold up against a convincing, well-timed request that doesn't touch a single vulnerability.
Tell us about your environment and goals — we'll help you scope the right engagement.