How can you set up a CI/CD pipeline to speed up software development? Here’s an IT leader’s overview of key principles and concerns, with an eye to teams starting from scratch
One of the better ways to get started with CI/CD: Dig into why you should get started with CI/CD at all.
“When getting started with CI/CD systems, they should be viewed as a force multiplier for your product teams,” says Josh Komoroske, senior DevOps engineer at StackRox. “Investing in setting up CI/CD system properly will pay dividends in engineering and testing time as it has the potential to vastly lower the time to market for new features.”
CI/CD: A primer for leaders
CI stands for continuous integration, and it involves building, testing, and merging developers’ various code changes into a shared repository such as GitHub or a container registry, usually multiple times per day.
CD, on the other hand, refers to either continuous delivery or continuous deployment, which are sometimes used interchangeably. Either way, this typically refers to later stages of the software pipeline, and especially to how new code moves into production.
(For more on CI/CD basics, check out What is CI/CD? We’ll revisit that primer a bit later on in this article. Read Getting started with CI/CD: 4 success factors, for more advice.)
Komoroske notes that there are multiple measurable gains to be had from a well-conceived CI/CD pipeline. Take testing as a major example.
“CI/CD can lower the amount of time it takes to test, and conversely, allows the testing of more code. By running a test suite in CI, a developer can get feedback on their feature faster,” Komoroske explains. “In the meantime, it frees them up to do more development work instead of running tests. You can also run tests on every single commit instead of once at the end of a release cycle. This greatly improves code quality and reduces bugs.”
CI/CD can shrink feedback loops and speed up features, enhancements, and fixes.
CI/CD can also shrink feedback loops and speed up features, enhancements, and fixes: “CI/CD should lower the iteration time between development and feedback. Proper CI can give feedback to developers moments after they push code,” Komoroske says. “It also lowers the amount of time it takes to deploy new features. Proper CD can enable the deployment of new features moments after they are reviewed and approved.”
These all sound like wins from a modern IT and business standpoint. But Komoroske did mention a prerequisite to these benefits: Investing in setting up CI/CD system properly.
4 keys to setting up a CI/CD pipeline
We’re here to explore, with help from Komoroske and other experts, what goes into that “proper setup.” This isn’t a step-by-step technical guide to setting up the perfect CI/CD pipeline (in fact, some experts would say that such a software development utopia doesn’t exist). Rather, it’s a leadership-oriented overview of some of the key principles and concerns that should guide the setup of a CI/CD pipeline based on the specific needs of your organization, with a particular eye toward teams that are starting from scratch. Let’s dig into four main areas:
1. Embrace key principles: Simplicity, repeatability, flexibility
“Fundamentally, succeeding with CI/CD requires transitioning from a process that is very manual and slow in many organizations to one that is much more automated and rapid,” Red Hat technology evangelist Gordon Haff noted. “Some companies can end up going from one or two releases per year to a release of one or more per week or even per day.”
But if you want to realize CI/CD’s promises – starting with faster and more frequent deployments, but also including improved testing and code quality and reduced operational burdens – then choose design principles for your pipeline and processes carefully.
Let’s start with simplicity: It’s a must. Too much overhead – in terms of implementation, operations, optimization, and other contexts – can immediately eat up any of the benefits and then some.
“Remember the golden rule: YAGNI – you ain’t gonna need it.”
“Remember the golden rule: YAGNI – you ain’t gonna need it,” says Nate Berent-Spillson, senior delivery director at Nexient. “The key to building your first CI/CD pipeline is to stick to the essentials based on only your immediate need.”
We’ll talk tools in a moment, but suffice it to say that you want to avoid what Berent-Spillson once described to us as “tool salad” (not to be confused with one of the healthy recipes you pledged to consume more of in 2020). If you’re experimenting or otherwise just starting out, Berent-Spillson recommends getting going with open source and free tools; you can add commercial products as needed later.
Simplicity begets all manner of other principles and benefits, such as widespread buy-in and cultural acclimation. In fact, it’s a defining characteristic of successful CI/CD.
“Your goal is to create an automated process that takes code from your source repository through compilation, testing, and deployment, and notifies the right people when it’s done,” Berent-Spillison. “If you do only that, you’ll have done quite a lot.”
Simplicity can also guide your long-term strategy: Build your pipeline incrementally – start with just CI and work up to CD, for example – and order key steps in the process with simplicity as a guiding logic.
“When getting started, run the high-impact, low-cost things first, like unit tests,” Komoroske advises. “From there, move on to low-impact, low-cost things like code linting and formatting. Run the high-cost things last, like integration tests or test deployments.”
Repeatability is a related guiding principle. Since automation is intrinsic to CI/CD, it’s important that steps in the pipeline are easily re-run, especially given the importance of testing (and re-testing) in a CI/CD pipeline.
“CI/CD allows humans to focus on what they do best and lets the automation take care of the repeatable minutiae,” Komoroske says. “Repeatability is key, so developers should structure builds so that they are as deterministic as possible and have all the information required to be re-run.”
While your pipeline should be well-defined and understood, avoid rigidity where possible, whether it’s a product of dogmatic thinking, a lack of foresight, or other reasons. Flexibility is important, especially when you take the long view. Komoroske recommends, for example, enabling simultaneous deployments.
“If there are multiple developers building features at the same time, it’s not unreasonable to have them all deploying test deployments,” Komoroske says. “Structure things to allow for this possibility.”
2. Understand the building blocks of a CI/CD pipeline
The ideal CI/CD pipeline differs from team to team: Keep this in mind when you start considering tools.
Dissuade yourself – early in your CI/CD exploration – of the notion of an ideal pipeline that works for every team or organization. Your path to a well-oiled, productive CI/CD pipeline will be paved by some fundamentals, but the actual route differs from team to team. This becomes especially important to keep in mind when you start considering tools.
The “What is CI/CD?” primer mentioned at the outset has a good simple visual of the fundamental building blocks of a pipeline. The CI phase covers build, test, and merge. The CD phase depends on whether your “D” stands for “delivery” (repository) or “deployment” (production); In some organizations, the terms “delivery” and “deployment” might be used interchangeably, and it’s not worth losing sleep over these semantics.
“Remember that CI/CD is really a process, often visualized as a pipeline, that involves adding a high degree of ongoing automation and continuous monitoring to app development,” the article notes. “Case-by-case, what the terms refer to depends on how much automation has been built into the CI/CD pipeline. Many enterprises start by adding CI, and then work their way towards automating delivery and deployment down the road, for instance as part of cloud-native apps.”
At some point, you’ll map tools to these key building blocks. There’s a vast menu of choices out there.
“There are many tools that can automate the tasks in CI/CD pipelines,” says Tsvi Korren, field CTO at Aqua Security. “From open source to cloud providers, these tools include several key features:
connections to repositories where code and other artifacts are stored; a procedural language or UI that outlines the steps to integrate the artifacts into an application; connections to testing tools and staging environments; and connection to DevOps automation tools that can provision and configure the infrastructure for application deployments.”
Again, remember Berent-Spillson’s “YAGNI” and the virtue of simplicity here. Too many tools cobbled together might turn into an IT equivalent of Frankenstein’s monster.
Korren shares a relatively simple illustrative scenario of translating the basic components of a CI/CD pipeline to recognizable tools or platforms; keep in mind, this is just a single illustration of what this might look like.
“For example, a pipeline based on open source can include Nexus as a repository, Jenkins as a CI platform, and Kubernetes for CD of containers,” Korren says.
Then the process could look something like this: “Developers will commit their code, which triggers a build of an image by the CI. The resulting image is pushed to the registry and is used in automated testing and staging scenarios. When testing criteria are satisfied, a deployment manifest will be sent to Kubernetes, which will pull the images and run them as containers, completing the continuous development cycle of an application,” Korren explains.
Next, you need to choose the right applications for CI/CD and double down on testing:
3. Choose the right applications for CI/CD – and the right teams
Just as you’ll want to spend adequate time identifying the right tools for your pipeline, you’ll want to ensure you’re moving the right applications through that pipeline. You’re not necessarily going to want to force all of your software products into CI/CD. Korren offered a hint above (i.e., containers) as to the types of applications well-suited for CI/CD.
“A major consideration for modern CI/CD is which applications to include in the process,” Korren notes. “In general, applications that need more post-deployment configuration and have complex internal dependencies on co-location of components will be less suitable for assembly and release through a pipeline. When applications are built with a cloud-native mindset, they will be broken up into small components that operate independently, and can be deployed asynchronously through automation.”
Simplicity can once again guide your early choices; application fit can also be a matter of picking the right service with which to start.
“Beginning with a small project that is relatively isolated is ideal – or perhaps a portion of a project, such as the front end of a line of business app.”
“Beginning with a small project that is relatively isolated is ideal – or perhaps a portion of a project, such as the front end of a line-of-business (LOB) app is considered a safe starting point,” says Tom Iverson, managing principal, cloud and DevOps at RBA. “An internal LOB app allows you to experiment and be more aggressive in your approach for maximum return.”
This approach to application selection can work well in tandem with the aforementioned strategy of starting with CI and then growing into CD later: “Once your CI is well-defined and your code coverage is acceptable, then you can begin experimenting with CD tools and techniques,” Iverson says.
A complex monolithic application might not be a seamless fit for CI/CD; similarly, a monolithic culture or team setup might pose issues as well. Teams with a budding (or maturing) DevOps culture are a good bet.
“When starting out on a CI/CD journey, try to pick a team that is likely to succeed,” Iverson advises. “A high-performing team that is not afraid of change is a good choice.”
4. Double down on testing
If you wanted to distill CI/CD down to key terms, automation would most certainly need to be on your shortlist. Testing should be there, too. It’s a crucial facet of a healthy CI/CD pipeline and is fundamental to shortening feedback loops, improving quality, and achieving the “continuous” in CI/CD without sacrificing quality.
"Allow for extra time to write good tests."
“Organizations must also allow for extra time to write good tests,” Iverson says. “This extra effort will pay off in the long run as your software grows.”
So make sure this is a priority early in your CI/CD efforts. Don’t treat “YAGNI” as an excuse to take shortcuts.
“Do not skimp on testing and telemetry,” Berent-Spillson says. “Automated tests and code-coverage metrics need to be part of your pipeline from the get-go.”
Prioritizing testing will have compounding benefits in terms of continuous – there’s that word again – improvement and optimization. Good tests also help foster buy-in, especially in organizations that might be culturally disinclined to bake too much automation into their software pipeline.
“[Testing and telemetry] create confidence in your new automated process,” Berent-Spillson says. “Anyone who wants to can verify that the pipeline is properly running the checks that used to be done by hand. That’s especially important when the pipeline is new and people are getting used to taking their hands off deployment.”
MORE ON CI/CD
SOURCE:Paper.li
Comments