Why does air-gapped environment require rolling your own CI/CD solution? Plenty of examples of air-gapped Jenkins and/or Argo Workflows. Was this just an educational exercise?
Argo Workflows does not live up to what they advertise, it is so much more complex to setup and then build workflows for. Helm + Argo is pain (both use the same template delimiters...)
Jenkins, like many tools with extreme flexibility, sucks as much as you make it suck. You can pretty easily turn Jenkins into a monstrosity that makes everyone afraid to ever try to update it. On the other hand, you can also have a pretty solid setup that is easy to work on. The trouble is that the tool itself doesn't guide you much to the good path, so unless you've seen a pleasant Jenkins instance before you're likely to have a worse time than necessary.
What particular issues do you have with it? My company uses it at scale (dozens of different instances, hundreds of workers, thousands of pipelines) to support thousands of applications and we are reasonably happy with it. DSL is incredibly helpful at scale. IAC is incredibly helpful at scale. It requires a good amount of upkeep, but all things underpinning large amounts of infrastructure require a good amount of upkeep.
We've minimized our usage of the DSL, there is no way for devs to debug it without pushing commits, and it means you have to implement much of your CI logic twice (once for local dev, once for ci system).
IMO, ci should be running the same commands humans would run (or could if it is production settings). Thus our Jenkins pipelines became a bunch of DSL boilerplate wrapped around make commands. The other nice thing about this is that it prepares you for easier migrations to a new ci system
When was the last time you used Jenkins? I don't get the hate. Not only from you, but lots of people on the internet. What makes Jenkins stand out IMO is the community and the core maintainers, they are perhaps moving slow, but they are moving in the right directions. The interface looks really nice now, they've done a lot of ux improvements lately.
both the old & new interfaces to Jenkins are riddled with bugs, work seems to be maintenance mode, across the plugin ecosystem too
If you are talking about Jenkins-X, that is a different story, it's basically a rewrite to Kubernetes. I haven't talked to anyone actually using it, if you go k8s, you are far more likely to go argo
It seems like a simple CI/CD in an airgapped environment might be simpler to implement than to (1) learn and (2) onboard an off-the-shelf solution when your airgapped requirement limits your ability to leverage the off-the-shelf ecosystem.
Since you're exercising, you can take it to the next level where you don't specify the next step but the inputs to each task, allowing you to infer the DAG and implement caching...
> Build a dependency graph (which jobs need which other jobs)
> Execute jobs in topological order (respecting dependencies)
For what it’s worth, Python has graphlib.TopologicalSorter in the standard library that can do this, including grouping tasks that can be run in parallel:
Why does air-gapped environment require rolling your own CI/CD solution? Plenty of examples of air-gapped Jenkins and/or Argo Workflows. Was this just an educational exercise?
Jenkins sucks but is insanely reliable
Argo Workflows does not live up to what they advertise, it is so much more complex to setup and then build workflows for. Helm + Argo is pain (both use the same template delimiters...)
Jenkins, like many tools with extreme flexibility, sucks as much as you make it suck. You can pretty easily turn Jenkins into a monstrosity that makes everyone afraid to ever try to update it. On the other hand, you can also have a pretty solid setup that is easy to work on. The trouble is that the tool itself doesn't guide you much to the good path, so unless you've seen a pleasant Jenkins instance before you're likely to have a worse time than necessary.
Are you sure, because last time I used Jenkins it actively sucked. The interface was a total mess and it doesn't surface results in any useful way.
What particular issues do you have with it? My company uses it at scale (dozens of different instances, hundreds of workers, thousands of pipelines) to support thousands of applications and we are reasonably happy with it. DSL is incredibly helpful at scale. IAC is incredibly helpful at scale. It requires a good amount of upkeep, but all things underpinning large amounts of infrastructure require a good amount of upkeep.
We've minimized our usage of the DSL, there is no way for devs to debug it without pushing commits, and it means you have to implement much of your CI logic twice (once for local dev, once for ci system).
IMO, ci should be running the same commands humans would run (or could if it is production settings). Thus our Jenkins pipelines became a bunch of DSL boilerplate wrapped around make commands. The other nice thing about this is that it prepares you for easier migrations to a new ci system
When was the last time you used Jenkins? I don't get the hate. Not only from you, but lots of people on the internet. What makes Jenkins stand out IMO is the community and the core maintainers, they are perhaps moving slow, but they are moving in the right directions. The interface looks really nice now, they've done a lot of ux improvements lately.
both the old & new interfaces to Jenkins are riddled with bugs, work seems to be maintenance mode, across the plugin ecosystem too
If you are talking about Jenkins-X, that is a different story, it's basically a rewrite to Kubernetes. I haven't talked to anyone actually using it, if you go k8s, you are far more likely to go argo
It seems like a simple CI/CD in an airgapped environment might be simpler to implement than to (1) learn and (2) onboard an off-the-shelf solution when your airgapped requirement limits your ability to leverage the off-the-shelf ecosystem.
This was more like an educational exercise
Since you're exercising, you can take it to the next level where you don't specify the next step but the inputs to each task, allowing you to infer the DAG and implement caching...
You can do this with cue/flow, but have not turned it into a full CI system. The building blocks are there
Never heard of cue/flow will definitely check it out
cue/flow is a Go package and what powers the scripting / custom tools part of CUE. It's built on top of the language and a graph resolver.
I have another custom flow implementation that I find more ergonomic: https://hofstadter.io/getting-started/task-engine/
Why use Docker as a build job execution engine? It seems terribly unsuited for this.
It's widely used among DevOps Engineers so hence I picked Docker as it makes it easier to understand
> We need to:
> Build a dependency graph (which jobs need which other jobs)
> Execute jobs in topological order (respecting dependencies)
For what it’s worth, Python has graphlib.TopologicalSorter in the standard library that can do this, including grouping tasks that can be run in parallel:
https://docs.python.org/3/library/graphlib.html