Situation & Goal:
La Mobilière is in the middle of their cloud transformation journey. The entire IT infrastructure is to be moved to the cloud. A huge undertaking, of which event-streaming is also a central part of. Event-streaming is especially important because it provides data connectivity and is the connection between the legacy and the cloud world. From a business perspective, la Mobilière wants to become quicker and more flexible by moving to the cloud. With Kafka as a Service they are decoupling their data streams in space and time, enabling them to integrate and deploy applications faster and more flexible, as well as analyze and evaluate data efficiently.
“With the help of SPOUD we were able to quickly evaluate the different options for event-streaming on the cloud. Thanks to their experience, we could assess, test and build a PoC in only three months. The PoC is now the foundation for the bigger scoped project.”
Tirone Gigandet, Team Lead, La Mobilière
Their current on-prem Kafka Connect architecture introduced by SPOUD is vividly used. For this reason, la Mobilière was faced with the question, “How can we do event-streaming and messaging on the cloud?” There are numerous solutions for this but finding the right one for the requirements and needs is hard and requires a lot of experience with event-streaming to properly evaluate.
To solve the challenge with event streaming in the cloud, SPOUD came into play. The goal was to answer the question “How can la Mobilière do event-streaming in the cloud and especially if we can cover all the needs using Confluent Cloud (SaaS)”. The question was answered by evaluating the current architecture, defining the technical requirements, and to build a PoC around the assumptions.
About La Mobilière:
La Mobilière aims to offer its customers optimal and holistic advice on insurance solutions. For this reason, they are pioneers in digitalization. It was founded in Berne in 1826 and has been headquartered there ever since.
La Mobilière has an on-prem Kafka. This instance needs to be replaced by Kafka as a Service to properly run in the cloud. Before everything can be moved to the cloud, there’s a transition period, where some data-streams are on-prem, while others are in the cloud. During the transition, the infrastructure on the cloud must still be able to exchange data with applications on-prem (“backward-compatibility”) as well as with new applications in the cloud (“forward-compatibility”). This hybrid cloud approach ensures that during the transition phase, the two worlds are still able to exchange data where needed.
La Mobilière’s sophisticated cloud strategy also presents a few key technical challenges. Especially since the event-streaming infrastructure will operate on the cloud, governance policies and zero-trust security models must be considered when evaluating a possible implementation. Security demands on the cloud and on-premises are quite different, adding an extra layer of complexity. So, the main technical challenges to be considered, are:
- Enterprise governance rules that require certain safety, legal and
- organizational constraints to be fulfilled
- Integration of roles and security patterns
- Integrating zero-trust environments on-premises and on the cloud
- Automation (infrastructure as code) with e. g. HashiCorp’s Terraform
- Integrating monitoring with tools on the Azure Cloud
- Self-service provisioning of service accounts, topics and acls
- Replication of data and schemas from on-prem
- Replication of data from the cloud to be accessed on-prem
But the question remained: how can this even be done? The engineers at la Mobilière and at SPOUD evaluated the situation and came to the conclusion that Confluent Cloud could solve most of their needs. To test their hypothesis, they set up a PoC.
Solution & Benefit:
SPOUD consulted the team at la Mobilière with their knowledge about Kafka, Kafka as a Service and Confluent Cloud. By setting up the PoC, SPOUD illustrated where’s room for improvement regarding security and interoperability. SPOUD built the basis to estimate the effort to build Kafka as a Service and have thus laid the foundation for the project.
Our goal was to kickstart the Kafka cloud migration at la Mobilière. In 3 months, we evaluated the best possible solution to do event-streaming on the cloud and set up a PoC to test our evaluation.
In detail, we did:
- Architect for zero-trust hybrid Confluent Kafka as a Service Cloud setup
- Built a PoC cluster, to prove cloud integration possibilities – using the latest and greatest APIs
- Integrate Kafka with new Confluent Governance
- Automated setup of Kafka with an “Infrastructure as Code” approach
- Integrated Confluent Cloud monitoring API with the existing observability system on the Azure Cloud
Right now, la Mobilière and SPOUD are working on building the complete Kafka as a Service infrastructure – stay tuned for an update on that front.
Interested event-streaming on the cloud?
If you want to get an even deeper understanding of the project, you can reach out to us, we’re more than happy to elaborate.