I'm sorry but you are showcasing everything that's wrong with the python packaging /environment ecosystem. You list 10 tools which all work to varying degrees of overlap between each other. Which one should one use? And you don't even need to answer that because the answer changes every year. Last year it was poetry, this year it's uv. Next year it will be some other silly attempt.
This is a bad joke. It's a mature language where the answer to: "I want to manage my programming language version and installed libraries" is that you have to try a dozen different tools, each of which will cover some but not all of your requirements to do this one simple thing.
> I've used all of these tools successfully (at different times and/or for different use cases).
And you see nothing wrong with that? Pretty much every modern language has one way (or at most just a small handful of ways) to build a library and manage your dependencies/environment. The reason is that packaging and environment management is a side show. A necessary evil we have to do so that the actual crux of our work can be done. When I set out on a project, I don't say: "I want to spend a week figuring out which is the current environment management tool supported by the python mindshare". I don't want to deal with a dozen different ways of installing the dependencies of a package. This is insane. When I pick up a language, I want to know what is the way of managing dependencies and packaging up my stuff. And I don't want this to change on a yearly basis, doubly so for a mature language.
> If you think a language that has just one tool is better, then it just means your use-cases are very narrow.
Yes, I prefer that there's a choice of tools for say time series analysis or running a web service. Competition in those areas is good. That's how innovation is driven forward.
When it comes to package and environment management, I don't want innovation, I want stability. I want one agreed way to do it so that I don't need to fuck around with things which are completely orthogonal to the work I actually want to do and that will put bread on the table. I don't want to spend brain cycles keeping up to date with yet another hare brained way of defining what is a python package.
In my view, the reason why we are in this quagmire is because the roots of python are in a fairly simple scripting language, and the packaging has not well escaped these roots. You can have loose python files, you can have directories containing python files which are automagically modules, then we had to hack in a way to package collections of modules and manage their different versions. It's all hacks upon hacks upon hacks, and it has to be backwards compatible all the way to being able to run a loose python file.
I'm not saying this is easy to solve. It's the posterchild of the "Situation: there are 15 competing standards" XKCD. The time to solve it would've been 15 years ago while Guido was still the BDFL. There are too many stakeholders now to get any sort of consensus.
Well, that heavily depends on what you want to do. Python has a number of concerns when it comes to code and package management, some of which are not present in most other languages. Here's an incomplete list, off the top of my head:
1. installation and management of installed packages
2. management of Python versions present in an environment
3. management of virtual environments (Python version + packages installed and ready to use, in an isolated context)
4. building of distribution packages (nowadays pretty much only wheels, with or without native extensions, which depend on the target platform)
5. publishing of distribution packages (to PyPI or a compatible repo)
6. defining repeatable deployment environments (a subset of #1)
Most developers face some combination of above problems, and various tools offer solutions for certain combinations -- with a few covering all of them, to different levels of quality. It is crucial to understand your needs and select the tool(s) that offer the right solutions in the way that fits your usual workflow the best.
This article [0] is a good starting point to understanding the current Python packaging landscape, with a clear overview of which problem is covered by which tool.
And in this very thread, people seem to accept that this is fine. There's nothing wrong with the fact that there are 6 separate tools just for building a package, some support publishing, some don't, some also manage environments or python versions? Which of these are currently supported, which are deprecated, which are going to become deprecated in 2025?
But also there's a tool which only does publishing (twine)? The diagram is not even correct, because conda itself requires package building, except it's about building conda packages, which are cross-platform/language and separate from building a python package.
Why is there a separate set of tools for package management and package publishing?
The blog post is indeed helpful to allow someone new to python to at least see what options there are and roll the dice, but it will also raise extremely serious alarm bells that there's something fundamentally rotten at the core of the python ecosystem.
The fact that I need to read some unofficial post from 2023 to gain an overview of the python packaging and environment management ecosystem is itself completely nuts. And I can guarantee you that by now this blog post is getting outdated, because some mad genius is cooking the new best tool and ready to unleash it on the unsuspecting world.
The only case where this question is rhetorical is when you do not really use Python enough to require deciding which management tool to use. Which tells me everything I need to know about you.
Yes you are right. I do not want to care about which management tool to use. Programming is a difficult to discipline as it is, the tools should make it easier, not more complicated. Some people might do programming purely as an exercise of self-fulfilment and they don't mind. For me it's a tool, a means to achieve some ends. If you think the python landscape and tools are in an optimal state, more power to you. Meanwhile, I have a dozen quants complaining at me that they are wasting time on irrelevant crap learning yet another set of pacakging tools and that the technologists need to figure out one consistent tool (or at least a stable set of tools) for managing packaging/environment concerns instead of inventing a new one each year.
But yes, let's go throwing around thinly veiled insults instead. This tells me everything I need to know about you :)
This is a bad joke. It's a mature language where the answer to: "I want to manage my programming language version and installed libraries" is that you have to try a dozen different tools, each of which will cover some but not all of your requirements to do this one simple thing.
> I've used all of these tools successfully (at different times and/or for different use cases).
And you see nothing wrong with that? Pretty much every modern language has one way (or at most just a small handful of ways) to build a library and manage your dependencies/environment. The reason is that packaging and environment management is a side show. A necessary evil we have to do so that the actual crux of our work can be done. When I set out on a project, I don't say: "I want to spend a week figuring out which is the current environment management tool supported by the python mindshare". I don't want to deal with a dozen different ways of installing the dependencies of a package. This is insane. When I pick up a language, I want to know what is the way of managing dependencies and packaging up my stuff. And I don't want this to change on a yearly basis, doubly so for a mature language.
> If you think a language that has just one tool is better, then it just means your use-cases are very narrow.
Yes, I prefer that there's a choice of tools for say time series analysis or running a web service. Competition in those areas is good. That's how innovation is driven forward.
When it comes to package and environment management, I don't want innovation, I want stability. I want one agreed way to do it so that I don't need to fuck around with things which are completely orthogonal to the work I actually want to do and that will put bread on the table. I don't want to spend brain cycles keeping up to date with yet another hare brained way of defining what is a python package.
In my view, the reason why we are in this quagmire is because the roots of python are in a fairly simple scripting language, and the packaging has not well escaped these roots. You can have loose python files, you can have directories containing python files which are automagically modules, then we had to hack in a way to package collections of modules and manage their different versions. It's all hacks upon hacks upon hacks, and it has to be backwards compatible all the way to being able to run a loose python file.
I'm not saying this is easy to solve. It's the posterchild of the "Situation: there are 15 competing standards" XKCD. The time to solve it would've been 15 years ago while Guido was still the BDFL. There are too many stakeholders now to get any sort of consensus.