Any suggestions for a self-hosted CI that can also be run locally?
from onlinepersona@programming.dev to programming@programming.dev on 15 Apr 13:26
https://programming.dev/post/28655287

I left Github a while ago and have been relying on simple pre-push scripts in my workflow, but would like to be able to test PRs from others without putting my machine at risk. Besides codeberg and radicle (neither of which have reliable CI), I also have a build machine, where I could run CI jobs, however it is important that the CI jobs can also run locally so that external people do not require access to the build machine.

Is there a CI that can do those things (run locally and remotely)?

Anti Commercial-AI license

#programming

threaded - newest

iii@mander.xyz on 15 Apr 13:55 next collapse

Gitlab runners can run locally

onlinepersona@programming.dev on 15 Apr 17:39 collapse

Not anymore

iii@mander.xyz on 15 Apr 18:16 next collapse

Oh, thanks for letting me know

jacksilver@lemmy.world on 16 Apr 03:42 collapse

I don’t think that’s accurate, the post is from seven years ago. Additionally there are a lot of materials online that indicate your still can - virtualizare.net/…/how-to-run-gitlab-runner-local…

anton@lemmy.blahaj.zone on 16 Apr 11:42 collapse

I think there is a misunderstanding, what running locally means.
You can run a gitlab runner on your local machine, but it needs to pulls it’s jobs from git. It also requires gitlab to register your runner, so it can’t really work for new contributors to use themselves.

jacksilver@lemmy.world on 16 Apr 15:14 collapse

Ahh, I see what you mean.

At that point I feel like you may as well just use makefiles. Did that at an old company, it had params for local deployment testing vs CICD. This also let’s you define how you break the local deployment tests, as usually you can’t really fully test a CICD locally.

szicari@programming.dev on 15 Apr 14:05 next collapse

Earthly!

earthly.dev/earthfile

projectmoon@forum.agnos.is on 15 Apr 18:15 collapse

@szicari@programming.dev it should be noted that they're shutting down the open source project. However, a fork is apparently forming. But it's good to know.

szicari@programming.dev on 15 Apr 20:01 next collapse

Boo! I didn’t know that, but thanks for letting me know.

onlinepersona@programming.dev on 16 Apr 08:21 collapse

Do you have a link to that? There’s a blog entry “Earthly Switches to Open-source” From July 2023. Are they undoing that?

Anti Commercial-AI license

projectmoon@forum.agnos.is on 26 Apr 17:38 collapse
bkhl@social.sdfeu.org on 15 Apr 14:15 next collapse

@onlinepersona don't do it. Create makefiles or whatever that runs the build as a series of Podman/Docker commands or whatever, then just put as little CI config as possible around it. You'll thank me when you need to switch CI system.

TrumpetX@programming.dev on 15 Apr 23:26 collapse

I can’t upvote this comment enough. I grow so angry at Gitlab ci and GitHub actions. Even Jenkins got in on the junk.

Just use normal build tools and you can use whatever cruft you want around it with just a few lines instead of monster ci file that goes out of date next year.

dave@programming.dev on 15 Apr 14:15 next collapse

I set up Forgejo with Woodpecker CI some days ago and it’s been great so far

onlinepersona@programming.dev on 15 Apr 15:06 collapse

Are you able to run woodpecker locally from the repository? As in can woodpecker run in the checked out repository run the CI jobs?

dave@programming.dev on 15 Apr 20:14 collapse

It also has a CLI tool that I know can re-run your pipeline locally for debugging, so just running it normally should also be possible. Haven’t used either so far though.

onlinepersona@programming.dev on 16 Apr 09:47 collapse

I can’t find documentation about that unfortunately 🧐 There’s woodpecker-cli exec but after testing that on the example pipeline, it does nothing even with verbose logging.

Do you have a functional example somewhere?

Anti Commercial-AI license

footfaults@lemmygrad.ml on 15 Apr 14:43 next collapse

Put as much of your testing in shell scripts, or even better, Ansible playbooks, so that you can run them locally. That way your CI system just does ansible-playbook

There’s a very good Ansible collection for podman, so you can orchestrate the unit tests to run inside a container for full isolation

drspod@lemmy.ml on 15 Apr 22:39 collapse

inside a container for full isolation

good luck

sorter_plainview@lemmy.today on 15 Apr 15:03 next collapse

Woodpecker with Ansible. Woodpecker will give container environment and using Ansible will reduce dependency on the CI tool.

Woodpecker has a alpine linux based container for Ansible. It will take some time to setup, but will make the life much easier.

PokerChips@programming.dev on 15 Apr 16:27 next collapse

I’m attempting this setup as well. It’s been a struggle but i am also new to a lot of this.

onlinepersona@programming.dev on 15 Apr 18:10 collapse

Why ansible? I’m not sure how that fits in. Does that make running it locally easier? An example of working setup that I can checkout and run would be useful.

Anti Commercial-AI license

sorter_plainview@lemmy.today on 16 Apr 02:23 collapse

As I mentioned it is to reduce dependency on CI tool. You may have to shift the tool in the future and if you use a lot of commands specific to the CI tool, that is going to be a nightmare.

Ansible is agent less and only needs SSH access. You can SSH into your local system, from the same local system. Need to add few entries in your SSH config and known_hosts. Essentially everything in Ansible are shell commands. So you are not really that much locked into Ansible.

On the question,

Does that make running it locally easier?

If you mean making it easier compared to remote, on the surface level, the answer is ‘no’. But it makes CI pipeline easier to run independent of your environment. Ansible is here to reduce dependency on a specific tool.

Bonus point is you can also create a working but basic CD system with Ansible.

refalo@programming.dev on 15 Apr 16:23 next collapse

buildbot.net

PokerChips@programming.dev on 15 Apr 16:25 next collapse

Great timing. I’m interested in this as well. I am currently attempting an ansible setup that runs podman containers in a couple lxc incus containers (developnent setup to mimic production) with forgejo and woodpecker on the other lxc container but it has been a battle.

Currently unable to figure out why the ‘general.community’ modules won’t get recognized by ansible.

mholiv@lemmy.world on 15 Apr 18:42 next collapse

I use forjero with forgero runners.

Basicly 100% compatible with GitHub actions and all locally run via podman.

Strong recommend. It’s all designed to work together and everything just works.

forgejo.org/docs/latest/…/runner-installation/

dave@programming.dev on 15 Apr 20:16 collapse

Isn’t Forgejo runner still in alpha though? How stable is it?

mholiv@lemmy.world on 15 Apr 20:42 collapse

I can’t speak for general use. But use it to:

  1. Build Rust artifacts
  2. Rebuild static sites, upload them to a bucket, then clear the CDN cache.

It works perfectly for me and I have not run into issues. But it might be bad for other people. I just know it works well for me.

vfsh@lemmy.blahaj.zone on 15 Apr 19:50 next collapse

Surprised to not see Gitea here, thats what I’ve been using for awhile now for my little projects

brian@programming.dev on 15 Apr 20:17 next collapse

gitea has had some organizational problems so a lot of people have been using forgejo instead, which is just a community fork of gitea plus some more features

vfsh@lemmy.blahaj.zone on 15 Apr 20:29 collapse

Oh yeah I keep forgetting about that. One of these days I’ll jump to Forgejo

onlinepersona@programming.dev on 16 Apr 07:24 collapse

Is that easy to run for contributors? Can you just gitea-run-ci and it’ll run the CI locally in your checked out repository?

drspod@lemmy.ml on 15 Apr 22:37 next collapse

would like to be able to test PRs from others without putting my machine at risk

I know what you mean, but do you not read the diff? Are you working on codebases that are so obfuscated that you can’t spot a malicious command?

onlinepersona@programming.dev on 16 Apr 09:36 collapse

What if they pull in a new dependency with a CVE or that executes malicious code? How am I supposed to check that? Or what if I miss a bug in the justfile or shell script?

Anti Commercial-AI license

anton@lemmy.blahaj.zone on 16 Apr 10:25 collapse

Run your CI in a sandbox.
For example gitlab allows you to run in a docker image.
Unless the attacker knows a docker CVE or is willing to waste a specter style 0-day on you, the most they can do is waste your cpu cycles.

timbuck2themoon@sh.itjust.works on 16 Apr 10:55 collapse

Yep. Hell, be very paranoid and run it in a container on a runner VM on your box if you like.

And you can use podman or sysbox there.

PsychoWiz@lemmy.world on 16 Apr 05:55 next collapse

docs.dagger.io

I remember seeing dagger trying to solve exactly this problem around 3 years ago, but it was still in alpha at that time. Not sure how good it is now.

onlinepersona@programming.dev on 16 Apr 08:17 collapse

After perusing the docs, this looks more like it. Thank you. I’ll just have to explore how it can be combined with projects that use nix and those that don’t. My biggest issue with CIs has always been caching, but as the saying goes “there are 2 hard problems in computer science…”

Anti Commercial-AI license

melezhik@programming.dev on 20 Apr 05:02 collapse

You may try out github.com/melezhik/sparky which is a local / remote task runner with nice front end and scripts could be written on many languages