Any suggestions for a self-hosted CI that can also be run locally?
from onlinepersona@programming.dev to programming@programming.dev on 15 Apr 13:26
https://programming.dev/post/28655287
from onlinepersona@programming.dev to programming@programming.dev on 15 Apr 13:26
https://programming.dev/post/28655287
I left Github a while ago and have been relying on simple pre-push scripts in my workflow, but would like to be able to test PRs from others without putting my machine at risk. Besides codeberg and radicle (neither of which have reliable CI), I also have a build machine, where I could run CI jobs, however it is important that the CI jobs can also run locally so that external people do not require access to the build machine.
Is there a CI that can do those things (run locally and remotely)?
#programming
threaded - newest
Gitlab runners can run locally
Not anymore
Oh, thanks for letting me know
I don’t think that’s accurate, the post is from seven years ago. Additionally there are a lot of materials online that indicate your still can - virtualizare.net/…/how-to-run-gitlab-runner-local…
I think there is a misunderstanding, what running locally means.
You can run a gitlab runner on your local machine, but it needs to pulls it’s jobs from git. It also requires gitlab to register your runner, so it can’t really work for new contributors to use themselves.
Ahh, I see what you mean.
At that point I feel like you may as well just use makefiles. Did that at an old company, it had params for local deployment testing vs CICD. This also let’s you define how you break the local deployment tests, as usually you can’t really fully test a CICD locally.
Earthly!
earthly.dev/earthfile
@szicari@programming.dev it should be noted that they're shutting down the open source project. However, a fork is apparently forming. But it's good to know.
Boo! I didn’t know that, but thanks for letting me know.
Do you have a link to that? There’s a blog entry “Earthly Switches to Open-source” From July 2023. Are they undoing that?
Anti Commercial-AI license
@onlinepersona@programming.dev https://earthly.dev/blog/shutting-down-earthfiles-cloud/
@onlinepersona don't do it. Create makefiles or whatever that runs the build as a series of Podman/Docker commands or whatever, then just put as little CI config as possible around it. You'll thank me when you need to switch CI system.
I can’t upvote this comment enough. I grow so angry at Gitlab ci and GitHub actions. Even Jenkins got in on the junk.
Just use normal build tools and you can use whatever cruft you want around it with just a few lines instead of monster ci file that goes out of date next year.
I set up Forgejo with Woodpecker CI some days ago and it’s been great so far
Are you able to run woodpecker locally from the repository? As in can
woodpecker runin the checked out repository run the CI jobs?It also has a CLI tool that I know can re-run your pipeline locally for debugging, so just running it normally should also be possible. Haven’t used either so far though.
I can’t find documentation about that unfortunately 🧐 There’s
woodpecker-cli execbut after testing that on the example pipeline, it does nothing even with verbose logging.Do you have a functional example somewhere?
Anti Commercial-AI license
Put as much of your testing in shell scripts, or even better, Ansible playbooks, so that you can run them locally. That way your CI system just does
ansible-playbookThere’s a very good Ansible collection for podman, so you can orchestrate the unit tests to run inside a container for full isolation
good luck
Woodpecker with Ansible. Woodpecker will give container environment and using Ansible will reduce dependency on the CI tool.
Woodpecker has a alpine linux based container for Ansible. It will take some time to setup, but will make the life much easier.
I’m attempting this setup as well. It’s been a struggle but i am also new to a lot of this.
Why ansible? I’m not sure how that fits in. Does that make running it locally easier? An example of working setup that I can checkout and run would be useful.
Anti Commercial-AI license
As I mentioned it is to reduce dependency on CI tool. You may have to shift the tool in the future and if you use a lot of commands specific to the CI tool, that is going to be a nightmare.
Ansible is agent less and only needs SSH access. You can SSH into your local system, from the same local system. Need to add few entries in your SSH
configandknown_hosts. Essentially everything in Ansible are shell commands. So you are not really that much locked into Ansible.On the question,
If you mean making it easier compared to remote, on the surface level, the answer is ‘no’. But it makes CI pipeline easier to run independent of your environment. Ansible is here to reduce dependency on a specific tool.
Bonus point is you can also create a working but basic CD system with Ansible.
buildbot.net
Great timing. I’m interested in this as well. I am currently attempting an ansible setup that runs podman containers in a couple lxc incus containers (developnent setup to mimic production) with forgejo and woodpecker on the other lxc container but it has been a battle.
Currently unable to figure out why the ‘general.community’ modules won’t get recognized by ansible.
I use forjero with forgero runners.
Basicly 100% compatible with GitHub actions and all locally run via podman.
Strong recommend. It’s all designed to work together and everything just works.
forgejo.org/docs/latest/…/runner-installation/
Isn’t Forgejo runner still in alpha though? How stable is it?
I can’t speak for general use. But use it to:
It works perfectly for me and I have not run into issues. But it might be bad for other people. I just know it works well for me.
Surprised to not see Gitea here, thats what I’ve been using for awhile now for my little projects
gitea has had some organizational problems so a lot of people have been using forgejo instead, which is just a community fork of gitea plus some more features
Oh yeah I keep forgetting about that. One of these days I’ll jump to Forgejo
Is that easy to run for contributors? Can you just
gitea-run-ciand it’ll run the CI locally in your checked out repository?I know what you mean, but do you not read the diff? Are you working on codebases that are so obfuscated that you can’t spot a malicious command?
What if they pull in a new dependency with a CVE or that executes malicious code? How am I supposed to check that? Or what if I miss a bug in the justfile or shell script?
Anti Commercial-AI license
Run your CI in a sandbox.
For example gitlab allows you to run in a docker image.
Unless the attacker knows a docker CVE or is willing to waste a specter style 0-day on you, the most they can do is waste your cpu cycles.
Yep. Hell, be very paranoid and run it in a container on a runner VM on your box if you like.
And you can use podman or sysbox there.
docs.dagger.io
I remember seeing dagger trying to solve exactly this problem around 3 years ago, but it was still in alpha at that time. Not sure how good it is now.
After perusing the docs, this looks more like it. Thank you. I’ll just have to explore how it can be combined with projects that use nix and those that don’t. My biggest issue with CIs has always been caching, but as the saying goes “there are 2 hard problems in computer science…”
Anti Commercial-AI license
You may try out github.com/melezhik/sparky which is a local / remote task runner with nice front end and scripts could be written on many languages