Description
We started as a development team of two people working on 2 containerised services. While we were concerned about scale, we weren't concerned about scaling the team itself, so we built our services using simple makefiles and each service had its own compose file. After a few months we found ourselves working with ~20 developers from 3 different teams, across several countries, collaborating on the same project, with 25 containerised python microservices.
A major growing pain we experienced was the “onboarding” process for our new team members. In order to automate and expedite the process, we wrote deployment scripts intended to easily create new dev environments. It didn’t work out as planned and the consequence was the vast majority of developers debugging/testing their own code remotely, rather than on their local machines.
After taking some time to reflect on the topic, we set out to solve the challenge of maintaining a stable and scalable dev environment within a growing team. These efforts can be summarized as the following:
- Simplify container orchestration for local development environments
- Build time optimization for local and CI processes
- Maintaining and developing on multiple python environments (Python2/Python3/PyPy) with minimal setup overhead
- Taking care of different caching aspects related to our CI and local builds
- Enabling debug of the code in the container using its python interpreter
- Creation of a monitoring solution for inter-service communication
This was and continues to be a journey towards improving our processes and mindset of growing a development team while reducing unnecessary pain. After learning the lessons above, we are excited to share our experiences with the community in an effort to help us all grow and improve together.