Migrating an Analytics and Big Data Platform to Microsoft Azure and Kubernetes
Our customer, an analytics platform, intended to consolidate all development projects within the Microsoft Azure Cloud. niologic supported the customer in the setup of a Kubernetes infrastructure and Microsoft HDInsights as Cloud Enablement. Both the frontend and the Big Data components were migrated.
Challenge
Our client wanted to consolidate their development of the analytics platform on Microsoft Azure. Previously, there had been several individually administered Hadoop and Docker nodes, which were to be replaced by managed solutions. At the same time, the customer wanted to migrate out of Amazon AWS.
Procedure
Along with the customer’s development team, niologic planned the platfrom’s new infrastructure and security checks of the (penetration tests).
Frontend, middleware and backend systems were provided based on a clearly separated network topology. A mix of Software-defined networking (SDN) was used in the Cloud and in the Kubernetes-Cluster. The resulting tiering of services guarantees that only the permitted services can communicate with each other.
In the subsequent implementation phase, niologic put the infrastructure for Microsoft HDInsights (Hadoop/Spark) and Microsoft Azure Kubernetes Solution (Kubernetes as a Service) into effect. A Microsoft system partner with Gold certification was responsible for integrating the services into the customer’s Active Directory as well as for digital rights management between the cloud and the Microsoft world.
After finishing the development and production environment, container registration and monitoring solution based on Microsoft products were implemented additionally.
A final security test, carried out by an external partner, ensured that the new platform was tested and prequalified before it was put into production. After finalizing all work, the migration was completed by changing DNS entries.
Prior to the migration, niologic had already prepared the developers for using Docker for analytics applications. After the migration, niologic took over the training of developers extending this knowledge by Kubernetes after the migration.
Result and customer value
The project took up a total of 2 weeks of planning and 2 weeks of implementation. By combining the latest technologies (Microsoft HDInsights and Microsoft Azure Kubernetes Service), the customer was able to migrate their analytics services without any failure. Furthermore, training the developers before and after the migration guaranteed that they immediately were able to use new features after the migration.
By using the cloud provider’s Software-as-a-Service solutions, administration costs could be significantly reduced for Hadoop and Docker or now Kubernetes. The development team was now able to prioritize and carry out most of the tasks itself.
With comparable costs for cloud providers, the ROI was approx. 12 weeks due to savings on external services.
