Although less than 0.001% of transactions were fraudulent, with over £1bn worth of transactions per year, fraud was amongst the largest expenditures of the organisation and posed a high reputational risk. The fraud team wanted to move from a rule-based system to Machine Learning to improve their fraud detection process and capacity.
The rule-based system involved constant manual calibration by a team who struggled to keep up with the growing volume of transactions, the huge amount of data they generate, and the evolving tactics of fraudsters. The rules being applied were becoming increasingly complex, staff costs were growing, and access control was becoming difficult to manage. Given all the moving parts and the sensitive nature of financial transaction data the company collected, a secure platform like SherlockML with full auditing capabilities and a centralised place to manage permissions was critical.
Processing 8 million transactions a month and generating 2TB of data, the dataset was too large to be handled by most computers. Training the models was computationally intense and, given the evolving tactics of the fraudsters, had to be retrained regularly. SherlockML allowed the team to scale computational resources depending on the needs of the project, and to iterate quickly.
SherlockML also allowed the company to easily deploy the trained model on new data and provided the environment for effective version control.
SherlockML provides unlimited customisation. Cutting edge techniques (t-Distributed Stochastic Neighbour Embedding) were used to overcome the technical challenge of training a model on a dataset in which only a tiny percentage of data points are fraudulent. A balanced random forest significantly outperformed the previous system, and unlike black-box systems, provided transparency crucial for financial compliance.
The greater accuracy of the AI-based system meant that 93% of fraudulent transactions that would have slipped through the previous system are now detected.
Fewer legitimate transactions are blocked increasing customer satisfaction and revenues.
The fraud detection team saw significant improvements in productivity, saving millions in staffing costs whilst allowing them to scale.