AI is not the answer to AI-enabled fraud
A top banker has got in trouble by referring to employees whose jobs will be replaced by AI as “lower-value human capital.” But he’s just saying the quiet part out loud: compliance officers have been expected to work like inefficient computers for years already anyway. Standard Chartered CEO Bill Withers used the unfortunate phrase when describing how the bank plans to keep hitting its profitability targets by cutting 15% of back office staff, and has tried to backtrack a little after a predictable storm of criticism, including from the former president of Singapore.
It would be nice to think of bank employees deployed to fight financial crime as super-effective old-school gumshoes, with a bottle of bourbon in the bottom drawer and an inexhaustible stock of one-liners, but in reality their jobs are more like something from ‘The Office’ than ‘The Big Sleep’.
Thousands of people sit in cubicles in Warsaw or Bengaluru, checking through transactions flagged as possibly abnormal by banks’ automated systems, and confirming that 99% of them are, in fact, normal. Anything that might conceivably be abnormal gets sent up the chain, where someone more senior will almost certainly decide it wasn’t.
It is a ruinously expensive process and, as far as we can tell, completely ineffective. The best estimates we have for the size of the criminal economy suggest it has grown, untroubled, along with everything else for decades despite all the laws, fines, and prosecutions that we’ve thrown at the problem.
But banks’ financial crime compliance isn’t about stopping financial crime at all: it’s about stopping banks from being fined, as Standard Chartered has previously been, enormous sums in both the U.S. and the UK. As long as banks can be sure that AI checks boxes in ways that satisfy regulators, then they’ll be happy.
This is a little bit worrying because AI is already getting very good at fraud. I am very alert to attempts to trick me but was sufficiently fooled by an AI email yesterday to forward it on to someone. Fortunately, a very similar one arrived (purportedly from someone else) a few minutes later, alerting me to my mistake. Money laundering is a laborious activity and criminal gangs will be as keen as banks to cut their back office expenditure, and AI could help automate the processing of the many small transactions that add up to a large amount of money. Phishing and smurfing are just a couple of its use cases, however.
“Scammers can leverage AI to scrape data from social media and dating platforms to identify vulnerable targets (e.g. lonely individuals, recent retirees, or people interested in finance) for pig butchering scams,” notes TRM Labs. “Bots can sustain long-term, emotionally persuasive conversations without tiring or making mistakes, making the scam more scalable.”
And that’s before you get onto AI’s ability to exploit cryptocurrencies and smart contracts to really start rampaging through the crypto world. “TRM observed a roughly 500% increase in AI-enabled scam activity over the past year. The convergence of generative AI, programmable financial infrastructure, and global crypto liquidity has altered the economics, velocity, and scalability of fraud,” the blockchain analytics firm said in a follow-up report. This is very bad indeed.
So, although I can see why people are annoyed that Withers referred to his bank’s employees in such a disparaging way, I am more troubled that he’s planning to replace them with AI, rather than to retain them and train them in how to counter it.
It was remarkable, however, to see Warren Davidson, chair of the illicit finance subcommittee at the House of Representatives Financial Services Committee, draw precisely the wrong conclusion from all this. Although he was correct in condemning the defensive nature of compliance, and its focus on generating paperwork over results, he then got lost in praising the White House’s decisions to attack corporate transparency legislation.
“As we focus on risk, we must also ensure that tools like artificial intelligence are fully deployed to counter the AI-enabled crimes of today,” he said, without realising that, without reliable information to train the AI models on, this is as useless an approach as the one he says has failed. If you don’t know who owns what, neither will a computer, no matter how cleverly it can pretend to be human.
I sincerely hope he listened to the testimony of Carole House of the Atlantic Council who forcefully pointed out the harm to national security and to ordinary Americans caused by the United States’ failure to create even an approximation of a decent corporate registry, as well as the historic idiocy of its current crypto policy. “Without a secure identity foundation, AI agents will simply scale up fraud at a speed and volume that human investigators can’t possibly track, destroying trust in the whole system,” she said in testimony that I highly recommend you take a look at.
I very much doubt Davidson was listening, however, because that is not the direction the Republican Party is going in right now. I would write more about that, but frankly it’s all too depressing, and I’d rather move on.
So let me point you towards this excellent paper on how online scam marketplaces work, with criminals using the messaging app Telegram and the stablecoin Tether to launder hundreds of billions of dollars. It argues that our current approach of sanctioning exchanges is futile since their owners just shut them down and switch to a new platform that works in the same way but hasn’t yet been sanctioned.
“As long as the underlying digital infrastructure remains permissive, criminal syndicates will simply migrate to new channels. To move from reactive disruption to systemic prevention, the international community must shift its focus toward the structural enablers of these marketplaces,” Elliptic’s Tom Robinson argues. Elliptic is unusual among blockchain analytics companies in being willing to name Tether as a major vector for money launderers. Its rivals tend to just say “stablecoins,” I have no idea why.
Meanwhile, there is something grimly depressing about the fact that — against the backdrop of, well, everything — the UK has postponed June’s illicit finance summit due to “scheduling issues in the international calendar.” Everyone is so busy dealing with the consequences of illicit finance, that no one has time to talk about illicit finance.
A version of this story was published in this week’s Oligarchy newsletter. Sign up here.