The rise of generative artificial intelligence has led to powerful, easily accessible and scalable tools that can be exploited by fraudsters and lead to a wide range of cybersecurity issues, from data breaches to malware and resulting in various forms of theft.

At the beginning of this year, CNN.com published an article about a deepfake scam which illustrates an increasingly alarming fraud problem that should concern every executive. A finance worker was tricked into paying $25 million to a fraudster who was able to look at the company’s CFO during an AI-enhanced video conference call.

This is a disturbing foreshadowing of what the threat landscape may become.

GenAI’s ability to create deepfakes that look and sound convincing is just one example of its potential utility for bad actors. The technology can also be exploited to fuel email phishing scams, allowing the criminal to draft emails in the style and syntax of a trusted person or source. Another malicious use case involves manipulating data or forging documents to support fraudulent transactions. It is relatively easy to use GenAI to combine one or more of these methods to make it more difficult to prevent or detect fraud.

In May, the San Francisco Division of the FBI issued a warning about the “growing threat” posed by cybercriminals using artificial intelligence to “conduct sophisticated phishing/social engineering attacks and voice/video cloning scams.”

Certainly, cybersecurity is a challenge that precedes the development of GenAI. But the rapid evolution of technology has only escalated the potential threats.

Both the persuasiveness and speed of development of AI-enhanced threats may well mean that defending against these types of threats is increasingly beyond the capabilities of many traditional risk management protocols. The Deloitte Center for Financial Services predicts that GenAI could leads fraud losses to $40 billion in the US by 2027, up from $12.3 billion in 2023.

Defensive measures

Many organizations already take cyber threats seriously, but the age of GenAI fraud is a game changer across the entire business landscape. Some defensive measures to consider include:

• Learning technology. Familiarize yourself with the basics of GenAI, including algorithms, data sources, trends and techniques, to better understand their strengths. Staying informed about advances in AI technology, especially use cases that are relevant to your industry, can help you better understand potential risks and vulnerabilities and determine what capabilities you may need to improve.
• Knowing your GenAI vulnerabilities. It is important to identify which of your protocols security is most likely to be compromised or misled by content produced by GenAI – whether voice, video, audio, documents or otherwise. A risk identification process can help uncover these vulnerabilities and can include activities such as risk hackathons and brainstorming sessions. Examples of areas where organizations can focus include strengthening access and approval processes by including multiple levels of approval and multi-factor authentication to verify staff identities. Another example is strengthening third-party document verification processes.
• Carrying out regular training of the workforce. Fraud schemes will likely continue to evolve, and your organizational understanding should keep pace. Both employees and key stakeholders should know how to identify potential GenAI threats, as well as how to respond appropriately to a breach. Consider updating your security processes to include a wider variety of data when evaluating and validating documents, requests or transactions.
• Combining organizational expertise. Cyber ​​threats are now complex enough that an effective defense can often require a multidisciplinary approach. Consider collaborating with other departments in your organization, such as IT and HR, so you can comprehensively assess AI-based fraud risks and develop your own internal knowledge and skills.
• Share what you learn. Once bad actors discover a weakness anywhere, other organizations are likely to be targeted soon. Sharing your findings can help protect multiple companies.

While no threat can be completely eliminated, the likelihood of being affected by a GenAI-enabled fraud scheme can be reduced. It is as important as ever for organizations to think proactively about defense and risk management, and to regularly reassess and update their protocols in response to the rapidly evolving threats enabled by GenAI.