Scotiabank recently launched its next-generation Global AI Platform with the goal of improving the customer experience, an exciting move made possible by game-changing technology innovations behind the scenes. Here’s a glimpse into the technology journey that made this possible.
The tug of war between legacy data and modernization
You may have heard data experts speak of data gravity. It’s a concept that when data accumulates in common areas across an organization, that data exerts a gravitational pull for additional compute services and applications to be created. So, what does this mean for businesses? When companies around the globe devised their data strategies over the past decade, many moved toward building large Hadoop clusters — collections of computers that together can perform computations on big data sets — which in turn led to large amounts of data being created, needing compute services. While this strategy worked well in the short term, many companies were, in parallel, having to address modernizing their infrastructure toward the cloud, given the widely recognized benefits of cloud. So, how does a company continue investing in legacy pockets of data with compute services while also modernizing its infrastructure by moving to the cloud?
At Scotiabank, to start, we decided to move away from massive Hadoop clusters and separate our data and compute services while still allowing the data to be accessed as before. We accomplished this by creating a hybrid data and analytics environment by adopting a high-performance object storage system and containerized deployments with Kubernetes, which means we can manage data as units on-premises for existing applications. This setup also enables us to take a staged approach to modernizing our infrastructure for cloud, making it possible to modernize while maintaining legacy data. And, very shortly, we’ll be able to leverage the scale of cloud for compute-intense analytics.
This hybrid data and analytics architecture has proven key to Scotiabank’s modernization and expansion into the world of readily available and highly scalable infrastructures. As we complete this current phase of delivery on our Global AI Platform, our analytics teams will no longer need to worry about the availability of compute capacity, nor about designing solutions based on where the data resides today. Rather, they can focus on business logic that drives value for the bank and ultimately customers, knowing that the platform can deploy the solutions we need where it’s most appropriate.
The role of reusability in data analytics
As the technology platforms have evolved, another key focus for us has been ensuring our analytics teams are equipped to drive insights and great customer experience by efficiently getting analytics assets, such as machine learning models, into production. The requirements for getting machine learning models into production include auditability, reproducibility, performance monitoring, and model registration. To address that, we’re creating machine learning development and operations (MLOps) pipelines that will create reusable components in the overall production process and make implementation of new machine learning models faster, more efficient, more robust and easier to maintain. Overall, we’re making significant investments in what will be a smooth end-to-end process that integrates at every step the efforts of our engineering and data science teams into the Global AI Platform.
Where are we going next?
While we’ve made a significant leap forward in technology capabilities that support our data and analytics journey, we’re only still getting started. With a strong focus on further automation in our data domain, and expanded cloud-native integrations, we’ll certainly continue to enhance the customer experience.
