British engineering giant Arup revealed as $25M deepfake scam victim
Scope of the Problem: Deepfakes vs Governance
- Many argue this isn’t primarily about AI, but about weak internal controls: no organization should let one person move $25M to a new account based on a single instruction channel.
- Others emphasize that modern attacks are more targeted and sophisticated than “Nigerian prince” scams, combining fake lawyers, deepfaked executives, urgency, and secrecy.
- Some suspect an “inside job” or at least compromised internal systems, noting unanswered questions about how the meeting was scheduled and participants identified.
Authentication and Cryptography
- Repeated calls to treat video/voice like email: inherently untrusted, requiring cryptographic signing, PKI, or trusted apps to authenticate instructions.
- Pushback notes PKI and key management are hard in practice; users struggle with verifying identities, revoking keys, and avoiding Sybil attacks.
- Suggestions include:
- Private-key signing for high-value instructions (not necessarily blockchain).
- Company video systems with strong account auth and clear “guest vs internal” labeling.
- Out-of-band verification: callbacks to known numbers, written confirmation, or multi-person approval.
Human and Cultural Factors
- Commenters stress culture: people are trained to authenticate themselves but not to authenticate others, especially superiors.
- “Secret” or “urgent” large transfers should be a red flag, yet social pressure, fear of missing a big deal, or abusive management can suppress questioning.
- Some families and teams adopt “secret passwords” or use tools like Signal safety codes to verify identity, though many users don’t understand these features.
In-Person vs Remote and Future Risks
- One view: as deepfakes advance, only in-person communication is truly trustworthy; expect more travel and less trust in telecom and VR.
- Others counter that fraud exists in person too; the real fix is process design and cryptographic channels, not abandoning technology.
- Concerns are raised about platforms (e.g., videoconferencing providers) training AI on user data, potentially enhancing their ability—or an attacker’s—to convincingly impersonate executives.