Description
Automate managing complex applications in a cloud native way using Operators written in Python
Many of us are using Kubernetes in production. A Kubernetes Operator is a way to automate packaging, deploying, and managing of a Kubernetes Application. It is a software alternative to a human operator who has deep knowledge of how to set up, deploy, and manage a particular piece of infrastructure and what to do if it isn’t behaving correctly. Let’s see how we can automate all of this while staying in the Python ecosystem.
It will be helpful to know some basic concepts of Kubernetes(Deployments, Services, Pods, Configmap etc.) and Celery(docs.celeryproject.org) to get the most out of this talk.
Talk is divided into four phases.
Phase I - Problems and Opportunities We're going to see some simple examples/problems where a lot of manual effort is involved so as to connect audience to the problem. We're going to discuss problems with configuration management, database cluster setup and introduce the focus problem of the talk which is around automating the setup of a Celery cluster.
Phase II - Incrementally Approaching the Solution We're going to incrementally approach the automation each of the manual steps involved in running a Celery cluster in Production. We're going to discuss the extension capabilities in Kubernetes using CRDs and Custom Controllers which are going to help us manage our Celery cluster automagically.
Phase III - Celery Operator in action We're going to see the code of custom controller and the whole operator in action. We create the newly defined celery resource and see how the operator works on bringing up the worker and flower deployments and handles autoscaling based on queue length.
Phase IV - Conclusion and Q&A We're going to talk about different use-cases and what is world doing with Operators. We'll discuss the next steps for the Celery operator and some resources to help build operators. We'll end the talk with a Q&A.
Slides for the talk are available on - https://bit.ly/europython20-ppt Celery Operator POC I built for this talk is open source - https://github.com/brainbreaker/Celery-Kubernetes-Operator