SOVRN banner

Tuesday, December 17, 2019

Fraud detection in real time

A couple of weeks ago I found this article about Codere, the Spanish gambling house. The article explains how Codere discovered serious inconsistencies in their yearly accounts, adding up to €20m (£17.9m/$22.0m) in losses. A very serious loss that reminded me of another customer I worked for a few years ago.
By chance, that customer I'm referring was also in the gambling sector and also had operations in South America, indeed they operated worldwide, but I was consulting for them because they wanted to address several kinds of fraud they had discovered in their slot machines in certain countries in America. So this article sparked my memories, and, as I think it was an interesting project and there some lessons to learn, here you have a small article describing it.

General View


As I said, the customer I was prospecting at that time -I will not disclose its name-, operated casinos all around the world with a lot of success. However, from time to time they discovered certain kinds of frauds or even bugs in their slots machines or casino systems they wanted to address. The main problem with this kind of issues it is not to find them out, but when do you find them out. If you have a system that alerts you as soon as a fraud -or a bug- is happening, you can stop the loss immediately and remediate it quickly. If, on the contrary, you allow it to run for days, weeks or even months, the loss increases as a snowball running downhill.

Problems to Address


One example of a bug was in relation with one of their slots machines. It was not their more popular machine, it offered a new game that was only used by a marginal group of users. However, at the end of a given month, when the usage data of the machines arrived the headquarters, the team of business analysts found that this specific type of machine had increased in popularity in an amazing 4000%!!
For some weird reason suddenly everybody wanted to use that machine. So they decided to understand why.
The next discovery was that, in spite of the spike in the number of users, the machine was losing huge amounts of money, but the ratio of winning/losing games did not favor the gamblers, so something very funny was going on.
After retiring the machines from the casinos and sending a team of engineers to find it out, they discovered the real reason of the sudden popularity of the slot machine and why it was losing money: the software had a bug, a very subtle one. 
The machine allowed gamblers to pay with cash, including bank notes, returning change of the amounts not used. The bug consisted in the wrong recognition of a very particular kind of bank note when running one single bet in the machine: when a $20 note was inserted in the machine, and just one bet was used -valued in $1-, the machine returned $99 in change instead of $19, as if the note introduced was a $100 one.
It was a very weird bug, as it didn't happen if the gambler used any other number of bets than one, and only with $20 notes. But word spread quickly among gamblers and they quickly exploited this bug on their benefit. Unfortunately for the company, they needed a couple of months to find it out.
[Note: I used $ as the currency to explain it in a simple way, but the real currency was not in dollars]

To try to stop this things to happen again the company opened a tender to invite vendors to provide a solution. At that time I was working for IBM and we had a wonderful product for real-time/streaming data analysis: Streams. Indeed, it is still being sold by IBM, and allegedly is the best product in this category of software -I'll write a comparison about this products in the future, hopefully-.

The Solution


My proposal was relatively simple: Streams would gather all the information from the different systems installed on the casinos and would analyze it in real-time(1). Then, using stored historical information, Streams would decide it the information was normal or if there were any anomalies that an operator should check in detail. Then, the information would also be stored in the historical databae to be part of the knowledge base for future reference in the search of anomalies, with an additional piece of information: once an operator investigated the detected anomalies, it would annotate them to inform the system if they were worthy to investigate or if they were false possitives ths should be ignored, improving the accuracy of the system.
If you think about this, it resembles an Artificial Intelligence system... 

Obviously, this is more easily said than done. The project involved not only the use of the streaming technology, but the processes to gather the information from the slot machines and other casino systems, the design of the data-warehouse to store the historical information, the algorithms to detect the possible frauds in real-time, etc. 
To add value to the project, my proposal also included monitoring the video cameras in real time in search of suspicious behaviours, not only from the players, but from the maintenance personnel: we were told of cases where the maintenance people acted in connivence with malicious gamblers installing the most exotic devices inside the machines to win illegally.

But not only fraud could be addressed. Another interesting feature that could be addressed with this project was preventive maintenance and detection of failure in the systems. As it happens with cars or other kinds of machinery, there are scheduled maintenance intervals, but sometimes failures happen before the maintenance periods are met, so by monitoring the systems in real time you could learn when a system is about to fail, by using historical information and anomaly detection. Then, by approaching a 100% availability of the machine, it means that the system is almost 100% of the time available to produce a benefit.
In sum, we realized that by just decreasing the fraud in the systems by a 2% -a very easy and non-ambitious target-, the project would not just pay for itself but produce additional benefits in a 10 month period. The other added benefits -reducing maintenance periods and other kinds of fraud- were not included in the proposal as we wanted to stay on the safe side, but we estimated that they could add another 12-15% increase in the final results of the systems being monitored. Regarding fraud detection, the 2% estimation was also a very conservative one. Our previous experience showed we could easily achieve a 18-23% of fraud reduction, which is a significant amount of money as they estimated the undetected fraud in more than $100M

Aftermath


What happened after all? The customer decided not to buy our solution. We may have done if for free and claim only a percentage of the fraud detected, and it would have been the project of the year but our internal rules did not allow that, and anyway the customer didn't want us to do it anyway.
As I said, they asked several vendors and consultancy firms to propose solutions, and they trusted the most their own internal auditor, one of the big 4, so there was no room left for a pure technical provider.
Their auditor-consultant recommended to just audit the results as they were doing, and not spend money on anybody else: just in them... And the customer believed them. It happens a lot, once you earn the trust of your client, it is more likely that they will follow you even if it is not the best possible solution.
In any case, that company was later involved in internal fights and the CIO left the company, so the project quite likely would not have been deployed.

Conclusion

 

Well, it is you who have to reach your own conclussions, right?
Just kidding. Regarding only the streaming project, real-time (streaming) analysis of information is a very powerful tool that can provide immediate results and mitigate, if not totally avoid, future problems. And if used in combination with AI/ML techniques and the proper use of historical information it is just fantastic.
There has been a lot of advance in several open-source platforms to address those kind of issues and with relatively very small investment a lot can be achieved with these solutions, but its use is not as widespread as I'd like it to be. 
We'll have to work on this.

---------------------------------------------------
(1): When I mention real-time in this context I really mean "streaming", or that the information is analyzed just as it arrives and an answer is provided as soon as possible. For computer scientists, real-time means that the information needs to be received and processed in a fix interval of time, as otherwise it would be lost or would be meaningless. For instance, real-time when operating a rover in Mars is about 9 minutes: approximately 4.5 minutes are needed for a radio signal send by the rover to reach the Earth and another 4.5 minutes for the answer to get back So if the rover finds an obstacle, its real-time to request instructions to avoid it is 9 minutes. If we are controlling an autonomous vehicle, real-time is under a second, as it is the time the vehicle may have to decide the best course of action to avoid an accident (after this second, the accident could be unavoidable).