CI/CD Thoughts

CI/CD is an area I wasn't all that excited about not that long ago. But, I'm coming to see it as one of the most important aspects. Though, it isn't an area which is without it's share of challenges.

I think that one of the biggest reasons for developers to get exciting about CI/CD is that it can help automate a lot of things which we don't think of as "development tasks" and likely aren't that all excited about. Like running automation tests or performing deployments. We may be interested in writing tests and interested in the results. But, in my experience, very few devs are passionate about manually running tests. 

And the same goes for deploying builds to various environments. We generally run locally in a debug capacity. And that process generally looks a lot different from the other environments that we may be interested in. Like pushing a build out to a QA or Production server or some other environment. I think a lot of developers would love to have this step automated as much as possible. And, in many cases it can be automated completely depending on your CI/CD tooling of choice and how much effort you're willing to put into it.

All of that upfront effort also comes with another benefit; reduced learning curve. Without CI/CD automation, there is another suite of things you need to train new team members on. And that can be costly. It is also wastes a lot of time.

Learning curve reminds me of another reason; personal projects. I have gone through a number of personal projects over the years. And, one of the biggest problems has always been that, often,  by the time I circle back to a project it has been years and I'm running on new hardware and need to refresh myself on a lot of those things. This has lead to me to abandon getting back into projects in the past. Having a proper CI/CD pipeline would mean that I don't need to worry about that. There would be a build environment on my development server which is configured as necessary, and the pipeline is already configured to do what needs to be done.

I have to take a point here and acknowledge something; when I talk about coming back to projects years later on different hardware, it does occur to me that there is little reason (generally speaking) to assume that the same thing can't happen to the dev server. So, I do have to say that a major motivator for me on this topic has been a migration toward Docker and toward cross platform development tools. Specifically, .Net Core and web based projects. There may still be people who have issue. Though, I imagine virtualized environments would be a solution here. Having your build environment inside a VM image would allow you to backup and migrate it rather easily. Though, for personal projects that would take up too much disk and too much hassle. So, I feel the need to leave YMMV warning there.

Also, it is not all up sides. Perhaps the easiest complaint is that CI/CD pipelines tend to act as a lock-in mechanism. I can't take my YAML from GitLab and drop it into GitHub or Azure DevOps or Jenkins. Not only is it not modular, but thing which may work out of the box in one ecosystem might need complex custom integrations or vice versa. 

How these systems evolve is also a bit of a pain. Which is to say that they rarely do. Or at least not in the traditional sense. Which makes sense. New versions often bring changes. Changes can break automations quite easily. So, there are often clean cuts from one iteration to the next. And this usually frees the CI/CD devs to make rather drastic changes which they often do. I have a few different projects in Azure DevOps. Some are using a pipeline YAML file, and another one is using is using an older, completely different graphical approach. While I'm sure that we could translate one to the other, I feel reasonably confident that we couldn't do so easily.

Lastly, that reduction in the learning curve means removing things developers need to learn. And means a risk of losing certain skills which may become necessary. For instance, framework upgrades may be something teams do infrequently and which may require changes in the build and deployment environments. It is then possible that the people who created the CI/CD process either quit or perhaps lost the skills over time and then need to relearn them when these things happen.

I won't argue that the negatives exists. But, I do think that they are outweighed by the pros. Most teams and individuals either aren't worried about upgrading or migrating their CI/CD tooling, or they are excited enough about the prospect to put the time into it. After all, the point is to have a repeatable process which "just works". So, you shouldn't be too concerned about it changing too often if at all.

As for the learning curve problem. I think most situations would realize more time gained from the automation than the time lost retraining when those cases arose. And, it may be a non-issue depending on the cadence (IE - it happens often enough that there are always trained individuals, or it literally happens never).

At the end of the day, the realization is that CI/CD is much more exciting than I think a lot of developers give it credit for. I also think that this aversion from developers has lead to a bunch of fairly robust and simple tools. I still need about 20 attempts to get my first pipeline working 100% when I'm new to the stack. But, this normally means less than a day. And after that, each new project takes less than an hour to get online. After which, it doesn't take too many builds for me to be back in the black on time spent.