Broken English no longer a sign of scams as crooks tap AI bots like ChatGPT: Experts

SINGAPORE – Bad grammar has long been a telltale sign that a message or a job offer is likely to be a scam.

But cyber-security experts said those days may be over as generative artificial intelligence (AI) chatbots like ChatGPT have helped scammers craft messages in near-perfect language.

Cyber-security experts said they have observed improvements in the language used in phishing scams in recent months – coinciding with the rise of ChatGPT – and warned that end users will need to be even more vigilant for other signs of a scam.

Risks associated with ChatGPT were categorised as an emerging threat in a special-issue report in March by security software firm Norton, which said scammers will tap large language models like ChatGPT. While not new, the tools are more accessible than before, and can be used to create deepfake content, launch phishing campaigns and code malware, wrote Norton.

British cyber-security firm Darktrace reported in March that since ChatGPT’s release, e-mail attacks have contained messages with greater linguistic complexity, suggesting cyber criminals may be directing their focus to crafting more sophisticated social engineering scams.

ChatGPT is able to correct imperfect English and rewrite blocks of texts for specific audiences – in a corporate tone, or for children. It now powers the revamped Microsoft Bing, which crossed 100 million active users in March and is set to challenge Google in a fight for the search-engine pie.

Mr Matt Oostveen, regional vice-president and chief technology officer of data management firm Pure Storage, has noticed the text used in phishing scams becoming better written in the past six months, as cases of cyber attacks handled by his firm rose. He is unable to quantify the number of cases believed to be aided by AI as it is still early days.

He said: “ChatGPT has a polite, bedside manner to the way it writes… It was immediately apparent that there was a change in the language used in phishing scams.”

He added: “It’s still rather recent, but in the last six months, we’ve seen more sophisticated attempts start to surface, and it is probable that fraudsters are using these tools.”

The polite and calm tone of chatbots like ChatGPT comes across as similar to how corporations might craft their messages, said Mr Oostveen. This could trick people who previously caught on to scams that featured poor and often aggressive language, he added.

Users are 10 per cent more likely to click on phishing links generated by AI, said Mr Teo Xiang Zheng, vice-president of advisory and consulting at Ensign InfoSecurity. The numbers are based on the cyber-security firm’s phishing exercises that made use of content created by AI.

He added that within Dark Web forums, there has been chatter about methods to exploit ChatGPT to enhance phishing scams or create malware.

In December, Check Point Software Technologies found evidence of cyber criminals on the Dark Web using the chatbot to create a python script that could be used in a malware attack.

Software company BlackBerry reported in February that more than half of the tech leaders it surveyed believe there will be a successful cyber attack attributed to ChatGPT within a year. Most respondents are planning to double down on AI-driven cyber security within two years, said BlackBerry.