Machine learning–aided scams
Credit to Author: Anastasiya Gridasova| Date: Fri, 04 Oct 2019 14:20:36 +0000
New technologies are clearly changing the world, but not the human psyche. As a result, evil geniuses are devising new technological innovations to target vulnerabilities in the human brain. One vivid example is the story of how scammers mimicked the voice of an international CEO to trick the head of a subsidiary into transferring money to shady accounts.
What happened?
The details of the attack are unknown, but the Wall Street Journal, citing insurance firm Euler Hermes Group SA, describes the incident as follows:
- Answering a phone call, the CEO of a U.K.-based energy firm thought he was speaking with his boss, the chief executive of the firm’s German parent company, who asked him to send €220,000 to a (fictitious, as it later turned out) Hungarian supplier within an hour.
- The British executive transferred the requested amount.
- The attackers called again to say the parent company had transferred money to reimburse the U.K. firm.
- They then made a third call later that day, again impersonating the CEO, and asked for a second payment.
- Because the transfer reimbursing the funds hadn’t yet arrived and the third call was from an Austrian phone number, not a German one, the executive became suspicious. He didn’t make the second payment.
How was it done?
Insurers are considering two possibilities. Either the attackers sifted through a vast number of recordings of the CEO and manually pieced together the voice messages, or (more likely) they unleashed a machine-learning algorithm on the recordings. The first method is very time-consuming and unreliable — it is extremely difficult to assemble a cohesive sentence from separate words without jarring the ear. And according to the British victim, the speech was absolutely normal, with a clearly recognizable timbre and a slight German accent. So, the main suspect is AI. But the attack’s success had less to do with the use of new technologies than with cognitive distortion, in this case submission to authority.
Psychological postmortem
Social psychologists have conducted many experiments showing that even intelligent, experienced people are prone to obeying authority unquestioningly, even if doing so runs counter to personal convictions, common sense, or security considerations.
In his book The Lucifer Effect: Understanding How Good People Turn Evil, Philip Zimbardo describes this type of experiment, in which nurses got a phone call from a doctor asking them to inject a patient with a dose of medicine twice the maximum allowable amount. Out of 22 nurses, 21 filled the syringe as instructed. In fact, almost half of nurses surveyed had followed a doctor’s instructions that, in their opinions, could harm a patient. The obedient nurses believed they had less responsibility for the orders than a doctor with the legal authority to prescribe treatment to a patient.
Psychologist Stanley Milgram likewise explained the unquestioning obedience to authority using the theory of subjectivity, the essence of which is that if people perceive themselves as tools for fulfilling the wills of others, they do not feel responsible for their actions.
What to do?
You simply cannot know with 100% certainty who you are talking to on the phone — especially if it’s a public figure and recordings of their voice (interviews, speeches) are publicly available. Today it’s rare, but as technology advances, such incidents will become more frequent.
By unquestioningly following instructions, you might be doing the bidding of cybercriminals. It’s normal to obey the boss, of course, but it’s also critical to question strange or illogical managerial decisions.
We can only advise discouraging employees from following instructions blindly. Try not to give orders without explaining the reason. That way, an employee is more likely to query an unusual order if there’s no apparent justification.
From a technical point of view, we recommend:
- Prescribing a clear procedure for transferring funds so that even high-ranking employees cannot move money outside of the company unsupervised. Transfers of large sums must be authorized by several managers.
- Train employees in the basics of cybersecurity, and teach them to view incoming orders with a healthy dollop of skepticism. Our threat awareness programs will help with this.