How can artificial intelligence be used in business analysis to prevent fraud? That’s the question more than 300 Scotiabank employees set out to answer in the Bank’s first-ever AI-kathon competition.
Participants from 83 teams were presented with datasets representing credit card transactions from e-commerce sites as well as anonymized customer online behaviours from those same sites. The multi-stage virtual competition challenged participants to uncover insights through business analysis and AI modelling to effectively prevent fraud, while balancing growth.
According to the Government of Canada’s Anti-Fraud Centre, Canada lost $531 million to fraud in 2022. With e-commerce showing no signs of slowing down in the future, reliable fraud detection models become even more critical, helping to avoid massive losses going forward.
During the June hackathon, teams were presented with two datasets — public and simulated — which provided anonymized or simulated customer transactional data, such as spending amount and credit limit, along with what’s known as clickstream data, providing insights on all sorts of customer behaviour on the website, such as the items browsed or whether a new credit card is being used. Each of these data points can indicate the possibility of fraud and collectively be used in an AI model that can reliably flag and block fraud attempts.
While all teams produced high model accuracy, judges ultimately awarded first place to Team FraudSeer, made up of James Cao, Manager, Strategy and Data Scientist, Retail Credit Risk; Juno Jiang, Senior Data Scientist, Retail Credit Risk; and Sophie Zhang, Senior Manager, Collections Strategies, Retail Credit Risk.
Photo: Members of Team FraudSeer
“Having a fantastic model on paper is not enough. We also need a compatible AI platform for deployment and a strategy that optimizes risk and reward,” Zhang said.
Team FraudSeer’s model did accurately detect fraud, but what set their team apart was their clear presentation of complicated AI concepts, business implications and hypothetical next steps for integrating their model into existing systems, judges said.
The team members were familiar with data analysis given their work with credit at the Bank, but the competition format allowed them to explore a new issue and experiment with new innovative approaches, Zhang noted.
Team FraudSeer’s winning approach incorporated machine learning (a subfield of AI), data engineering and hyperparameter tuning, which essentially involves optimizing the parameters used to identify patterns in data, Cao said.
In simple terms, Team FraudSeer used machine learning algorithms to find patterns in the data they were given, and then applied those patterns on new transactions to assign them a probability of fraud. Certain factors like whether a gift card is being used, which operating system is being used, or where the transaction is taking place can all provide useful background information that is used to calculate the likelihood of fraud.
AI can enhance decision-making, customer experience, and operational efficiency. AI will continue to shape the future of the financial industry.”
“There’s no perfect model. Every model has its own drawbacks or shortcomings,” Cao said. Balance is key, especially in the case of fraud detection, Cao added. On one hand, if your model is not sensitive enough, it will fail to detect fraud. On the other hand, an overly sensitive model will block legitimate transactions, which creates a negative experience for customers.
Another element is the importance of data ethics and safeguarding user privacy. Scotiabank has taken a leadership position in ethical data and AI and was one of the first Canadian financial institutions to operationalize and implement a tool to assess data ethics in AI and ML models.
The AI-kathon was part of the Bank’s ongoing investigation of innovative use cases for AI as the technology continues to develop.
When Yannick Lallement, Vice President, Corporate Functions Analytics and Chief Artificial Intelligence Officer, had the idea for the event last year, he worried there might not be enough participants. The overwhelming interest the event ended up garnering from Scotiabank employees is a sign of the enthusiasm for AI innovation, says Lallement.
“We believe that AI is the driving force of innovation and transformation in banking. Our AI-kathon showcased how our internal talent can harness the incredible potential that lies within AI. AI can enhance decision-making, customer experience, and operational efficiency. AI will continue to shape the future of the financial industry,” he said after the event.
Cao says he looks forward to continuing conversations to identify opportunities within the Bank to innovate using AI: “Competition is one thing, but how can we keep the momentum going?”
