My philsophy is simple. If the program is intended to be distributable, just use Go. If it does not require port stuff, use docker. If you have an IT team or someone to hand you a computer with OS and Python version installed that everyone else in the org uses, use venv.
If you have to work with ports, you have to distribute programs, or your libraries depend on C or OS stuff, then start consulting where you do not have to manage the codebase or have no committment to it after getting paid.
It's more complicated to write machine learning software in go than it is to write portable apps in python. Same goes for a lot of uses cases for python outside of backend servers or similar web related use cases. You can't really just "use go" for a lot of the things people use python for, at least not realistically
I have seen my fair share of ML Python codebases. Distribution is a mess, onboarding new people is a mess. The thing I would says just works is OS level configuration things like Kubernetes or NixOS are proven technology that works and there are enough resources for issues that can be self-debugged instead of opening tickets/ gh issues or reaching out to support. But as these are much complicated technology, you need domain experts and should not pressure ML engineers or data scientists to figure this out. I have seen Python packaging to be such a mess it is easier to teah to Python engineers ML or DS, then ML engineers proper package handling and distribution. The very existence dozens of packaging solutions show that engineer would rather create something from scrath rather work with existing tools.
I mean, I completely agree with that. I'm a MLE and I absolutely, utterly hate how much of a mess it can be and how much time is spent just helping interns getting their env set up reliably (we now have a pretty reliable setup/docs but that was after a few painful onboardings). I just think that using another language for some of what python does would be even more painful, just not on the packaging side of it
Choosing a language based on its distribution capabilities is the wrong criteria. Instead, decide based on what it enables you to do, and deal with the distribution later. The distribution won't matter if your project is not successful anyway.
Hilarious that this is being downvoted. Can you imagine professionals in any other industry being so pathetic? “Oh man, making bridges is sooo hard, I won’t stand by my work on anything that is above the ground. Making highways is sooo hard, I won’t stand by my work for anything that has to hold a lot of weight. Making food safely is soooo hard, I won’t stand by my work for anything that requires chilled storage and/or cooking to an appropriate temperature.”
Grow up. Have some respect for yourself, your work, and the industry.
Indeed. Docker solved distributing and running python programs like 10 years ago. You can even run CUDA and pytorch in docker nowadays. And the usual answers you see on HN every time someone brings up "just use docker" on those threads, is "but I don't wanna """learn""" docker". Takes 10 min to get a python container running with 0 experience in Docker.
