Python has too many package managers (dublog.net)
from chaospatterns@lemmy.world to python@programming.dev on 08 Jul 2024 19:31
https://lemmy.world/post/17360173

#python

threaded - newest

henfredemars@infosec.pub on 08 Jul 2024 19:36 next collapse

I’ve tried several and all of them left be disappointed eventually.

ericjmorey@programming.dev on 09 Jul 2024 02:13 next collapse

What did you end up using for your workflow?

best_username_ever@sh.itjust.works on 09 Jul 2024 04:17 collapse

Not OP. I ended up using hatch to make packages because it’s simple and does everything without feeling like a jumbled mess of tools like Poetry. It feels unix-like and didn’t prevent from doing my job.

flying_sheep@lemmy.ml on 10 Jul 2024 14:07 collapse

Hatch is great. It’s easy to get started but I wouldn’t call it simple. Flit is simple, because it’s limited.

Hatch is complex enough to allow you to override everything, which makes it not simple, but also not complicated.

demizerone@lemmy.world on 11 Aug 2024 01:55 collapse

I landed on hatch. The way it handles venvs and python versioning is awesome, and can also use UV. I haven’t used UV yet tho.

eager_eagle@lemmy.world on 08 Jul 2024 19:51 next collapse

and it used to be worse (at the time that XKCD was published)

ericjmorey@programming.dev on 09 Jul 2024 02:12 collapse

That means it’s improving! I hope the pace of improvement can accelerate.

joyjoy@lemm.ee on 08 Jul 2024 20:10 next collapse

Unfortunately, PEP 582 was rejected, so PDM has deprecated the functionality. But it does support PEP 621, which can’t be said for Poetry. There’s a PR.

eager_eagle@lemmy.world on 08 Jul 2024 20:22 collapse

Poetry’s compatibility with PEP 621 is underway, but it’ll likely be delayed to version 2.

Poetry provides a lot of the same functionality for quite some time now, which complicates the quick adoption, but it’ll happen.

ZephyrXero@lemmy.world on 08 Jul 2024 20:23 next collapse

Having more than one obvious way is unpythonic

sugar_in_your_tea@sh.itjust.works on 09 Jul 2024 01:30 collapse

Unfortunately, pip, the obvious way, kind of sucks. So I use poetry, which seems to work nicely.

richieadler@lemmy.myserv.one on 09 Jul 2024 01:44 next collapse

… except for the part where the dependency definition doesn’t follow the latest approved PEP, and the default constraint with ^ add an upper limit that causes problems.

I moved to PDM.

sugar_in_your_tea@sh.itjust.works on 09 Jul 2024 02:11 collapse

Eh, by the time I get everyone on board and convert all of our repos, poetry will probably have support for the latest PEP.

richieadler@lemmy.myserv.one on 09 Jul 2024 19:27 collapse

Given the glacial pace I’ve been seeing, I would’t be so sure… But understandable if you have many repos and need to reach consensus.

You can reduce some impacts adding explicit >= constraints instead of ^.

sugar_in_your_tea@sh.itjust.works on 09 Jul 2024 19:59 collapse

The thing is, things are working reasonably well right now, so updating to be in sync with the latest PEP isn’t super impactful, whereas switching from requirements.txt -> poetry and pyproject.toml was a big change. So we’ll probably switch eventually, but since we have over a dozen repos and several teams across timezones, it isn’t a priority.

I’ll certainly take a look though.

proton_lynx@lemmy.world on 09 Jul 2024 04:03 next collapse

I’m using Poetry as well. But I really hope Rye works out so we can finally end this madness.

chaospatterns@lemmy.world on 09 Jul 2024 04:18 collapse

Interesting. I just learned about Rye today. Has anybody tried it? Does it live up to the promise?

anzo@programming.dev on 09 Jul 2024 04:51 next collapse

Check “uv”. Builds on top, is coming good.

FizzyOrange@programming.dev on 09 Jul 2024 06:44 next collapse

Haven’t tried Rye but I have used uv (which Rye uses to replace pip). Pip install time went down from 58s to 7s. Yes really. Python is fucking slow!

NostraDavid@programming.dev on 10 Jul 2024 12:05 collapse

I tried it a little - being able to run rye sync and not even having to worry about Python versioning is sooooo nice.

PenisWenisGenius@lemmynsfw.com on 09 Jul 2024 04:56 next collapse

I don’t know what the solution is but like 50% of everything on github that’s in python is broken from dependencies and of those, half of them are fixable if you put a substantial amount of effort into it. It sucks. I’ve spent 2 entire days trying to get any AI tts to work and have gotten nowhere. I’ve tried 'em all, just about. None of them work on Debian.

SandbagTiara2816@lemmy.dbzer0.com on 09 Jul 2024 18:17 collapse

Wait, what’s wrong with pip?

(Disclaimer: I grade my Python proficiency slightly above beginner. I used it for some research in college and grad school, and I’ve made a few helpful scripts at work, but I am not anything approaching an expert)

sugar_in_your_tea@sh.itjust.works on 09 Jul 2024 18:30 collapse

It does its job well, but it doesn’t do much more than that.

The main workflow w/ Pip is:

  1. install whatever you need to get your app working
  2. pip freeze > requirements.txt so the next person can just pip install -r requirements.txt instead of figuring out the requirements
  3. either edit requirements.txt manually to do updates, or pip freeze again later

There are some issues with this:

  • dependencies that are no longer needed tend to stick around since they’re in the requirements.txt
  • updating dependencies is a manual process and pretty annoying, especially when different packages have different dependencies
  • requirements.txt doesn’t have any structure, so to do something like separating dev vs prod dependencies, you need two (or more) requirements.txt files

It’s totally fine for one-off scripts and whatnot, but it gets pretty annoying when working across multiple repositories on a larger project (i.e. what I do at work with microservices).

Poetry improves this in a few ways:

  • poetry.lock - similar to requirements.txt, in that it locks all dependencies to specific versions
  • pyproject.toml - lists only your direct dependencies and certain exceptions (i.e. if you want to force a specific dependency version)
  • package groups - we tend to have local (linters and whatnot), test (deps for unit tests), and the default group (prod deps), so you can install only what you need (e.g. our CI uses test, prod uses no groups, and local uses everything)

There’s a simple command to update all dependencies, and another command to try to add a dependency with minimal impact. It makes doing package updates a lot nicer, and I can easily compare direct dependencies between repositories since there’s minimal noise in the pyproject.toml (great when doing bulk updates of some critical dependency).

TL;DR - pip is fine for small projects, alternatives are nice when dealing with large, complex projects because it gives you nicer control over dependencies.

SandbagTiara2816@lemmy.dbzer0.com on 09 Jul 2024 19:39 collapse

Good to know, thank you for educating me!

callcc@lemmy.world on 08 Jul 2024 20:56 next collapse

Pip-tools!

richieadler@lemmy.myserv.one on 09 Jul 2024 01:46 collapse

You can use uv now and have Rust speeds for the same functionality.

ericjmorey@programming.dev on 09 Jul 2024 02:11 next collapse

I’m hoping uv is able to make itself to Python what Cargo is to rust.

Seems like Pixi is close but confined to the Conda ecosystem.

callcc@lemmy.world on 09 Jul 2024 06:44 collapse

Interesting. I might have a look. Actually I don’t have a problem with speed though. I spend most time not dealing with pip and pip-tools but reading docs, programming and fixing the weirdest bugs

ericjmorey@programming.dev on 09 Jul 2024 02:26 next collapse

This article is an excellent update on the state of Python package management tools.

flying_sheep@lemmy.ml on 10 Jul 2024 14:05 collapse

I agree, although it undersells hatch and has a title that made me roll my eyes. Very well researched, none of the usual hot takes and clichés.

Gsus4@programming.dev on 09 Jul 2024 04:58 next collapse

and Conda takes 20 minutes to do anything these days… 😂 send help

GabberPiet@lemmy.world on 09 Jul 2024 06:05 next collapse

Mamba is much faster!

NostraDavid@programming.dev on 10 Jul 2024 12:01 collapse

Might want to look at Pixi - written in Rust, BTW, but effectively seems to be a drop-in replacement that’s at least faster than Conda.

edit: link: prefix.dev

rutrum@lm.paradisus.day on 09 Jul 2024 13:06 next collapse

I would love similar movement with regards to doc comment standards. My company uses numpy and its too verbose, and the style guide makes zero mention of type hints so we keep winging it. And with not many tools for enforcing the standards (like what type to actual write for a parameter) its an ongoing battle among the team.

corsicanguppy@lemmy.ca on 09 Jul 2024 17:44 next collapse

… and every one of them destroys single source of truth for installed state. Most of them have no validation potential, and most of them are imperfect upgrades/downgrades.

If you don’t know why all that’s important, just remember why fired all our mentor types 20 years ago after the Y2K bust and so you had noone to teach you why. Mistakes like lennart’s Cancer didn’t grow in the light of rich knowledge sharing.

onlinepersona@programming.dev on 09 Jul 2024 18:47 next collapse

I’d rather have too many options than the bullshit situation C/C++ has and has had for decades. Open any C++ project and try to get all the dependencies easily using the standard tooling. It’ll vary on every distro and OS.

Anti Commercial-AI license

thesporkeffect@lemmy.world on 09 Jul 2024 19:51 next collapse

Rye

By the creator of Flask and many other things

NostraDavid@programming.dev on 10 Jul 2024 12:00 collapse

It comes with built-in ruff and uv, and can handle Python versioning for you!

Not sure if they want to eventually dissolve Rye into uv or what, but for now it’s one of the best (better than Poetry, IMO)

edit: it also uses the pypoetry.toml standard, something Poetry doesn’t (because Poetry predates pyproject.toml becoming a standard).

One possible downside: I’ve heard Rye doesn’t honor XDG, which means it has its own location for its config. I don’t mind, but perhaps you do.

A second possible downside is that Rye doesn’t let you centralize your venvs, so each .venv goes into each project folder, so no using virtualenvwrapper with workon to jump between projects. zoxide can alleviate that problem, or presuming you have a ~/dev folder or similar, you could write a bash function that ls’ that folder and lets you select a folder via fzf? Go ask ChatGPT about it or something.

edit2: link: rye.astral.sh/guide/installation/

muntedcrocodile@lemm.ee on 10 Jul 2024 06:17 collapse

I just pip in a venv