Wow, this model addresses a lot of the issues I have with different Linux distributions. Mainly, finding where the heck it installed that program in the maze of file/folder locations. Sometimes it's etc, opt, var/lib, bin, usr/bin, apps... Gwahhhhh!
Then, you go to modify the configuration, and it turns out that the configuration file is being copied and generated from a different location. It's endless misdirection which changes in each distribution based on the package manager.
I think I may try out Gobo the next chance I get. This is pretty cool.
- /var/ is read-write state data and will persist between boots. /var/log/ is for log files.
- /run/ is read-write scratch on a tmpfs, and will be cleaned up between boots. So, UNIX sockets and other temporary files go here.
- /etc/ is per-system configuration overrides. There is a goal to move to "empty /etc/", meaning you should be able to remove all files in /etc/ and still have a functioning system.
- /opt/ is "vendor space". The OS should never touch /opt/.
The names are weird, but that's the intention behind the new scheme. If you want to change configuration, the default config might be shipped read-only in /usr/, but it should be overridable in /etc/. This is why systemd ships read-only unit files in /usr/, which are symlinked into /etc/ to be "enabled".
> /etc/ is per-system configuration overrides. There is a goal to move to "empty /etc/", meaning you should be able to remove all files in /etc/ and still have a functioning system.
I find this goal to be rather frustrating.
It means that software will hardcode it's default configuration when it cannot find a file in `/etc`, and it makes it very hard to deduce what exactly can be configured.
I strongly dislike software that will simply apply a default state when it cannot encounter a certain configuration in it's configuration file for this reason.
I favor that software simply refuse to start with a clear warning as to why, when it cannot find a configuration option in it's configuration file.
Applying a default on a missing key is another case of weak typing; I would hope that lessons were learned by now that guessing what the user wants in case of error is a bad idea.
What is even worse is software that simply ignores malformed keys, and applies a default value anyway, giving rise to not only innocent mistakes by typoes, but some malicious exploits that were found by putting zero-width spaces into keys to trick administrators viā social engineering.
I partially agree, but I think a good middleground is for default configuration to be in a place like /usr/share/, and allow for admin configuration to happen in /etc/. This way, if you upgrade, you don't need a complicated merge to merge new defaults + old configuration. I've seen a lot of configuration migrations gone wrong in my life.
That would mitigate the problem of easily finding out what options there are, but it would still be a form of weak typing where a best guess is substituted for a mistake.
If the system administrator truly intend for a value to be default, he should explicitly declare it so in the configuration file.
Interpreting the system administrator's act of forgetting something as his intention for it to be a default value, seems like a bad idea to me.
As for merge, I find it as annoying if software not support configuration directories. `foo.conf` should exist alongside `foo.conf.d`, and if the latter exist, it's contents should be parsed as well.
On an update, the system can simply install a file in `foo.conf.d` which contains the default of any new keys added from the last version.
> I strongly dislike software that will simply apply a default state when it cannot encounter a certain configuration in it's configuration file for this reason.
Isn't sensible default what make certain software a lot easier to use then others? Apple seems to be good at this. Ten years ago Ubunutu brought this to the Linux world to an extent. Personally I like the configurability of Linux, but I am past the days of spending hours tweaking it and I am quite happy if someone supplies me with sensible defaults.
Having said that, as a developer I agree with what you said, it's often easier if a library call gives an error than tries to do something smart / just carries on (JavaScript comes to mind), so there is a balance depending on what the user wants to do with the software.
Default configuration exists for multiple reasons: 1) to provide minimally working software so that the user isn't burdened by unnecessary configuration, 2) to facilitate easier testing of minimal functionality, 3) to keep configuration DRY, and 4) to stop-gap missing configuration.
Of those, only 4) relates to typing. But even then it often bears no relation to real typing, because often there's no schema or typing at all, and usually application has no idea what version a given configuration relates to.
In addition to achieving the above 4 points, we also need to be able to differentiate between default and modified configuration, so the application can override defaults successfully. You can keep default and modified configs separate a number of ways. You can hard-code it in the application, which is fine for the application. But it's a pain in the ass for the user. And as a packager, you cannot expect files in /etc/ to always be "defaults", as the user/administrator expects these to be configured by themselves at any time. And if you want to upgrade software, there needs to be a way to differentiate the versions of default configuration to compare the modified configuration to.
The best way to do this is to package default configuration in /usr/share/. The user then has the job of upgrading their modified configuration in /etc/, and they can use /usr/share/ as a reference, in combination with some UPGRADING guide.
Distros have long had a convention to package a default distro-specific configuration in /etc/, so that when the app first starts, it can work out of the box. But then when you'd upgrade a package, the distro would have to figure out what to do with the once-default-but-now-modified files in /etc/. This just adds unnecessary burden on the distro. Instead, the user needs to look at their own manual configuration and update it if necessary. But the default configuration for the system should remain virgin and independent, in /usr/share/.
And of course, none of this applies to configuration that can't be default. The application should always die if it doesn't find required configuration.
However, it's not hard to figure out where files were installed, and what files are from. The package manager keeps track of all of that, and you can ask it. Example for debian or ubuntu:
# list all files from package
dpkq-query -L <package>
dpkg-query -L <package> | grep bin/ # see just commands
# find what package a file is from
dpkg-query -S /path/to/some/thing
I use equivalent commands in Archlinux (pacman ...) and they certainly exist for redhat/fedora but I'm not familiar with those.
For stuff not installed as packages, hopefully it's either in a single directory in /opt, or among a small collection of stuff spread in /usr/local, or in a self-contained directory in your home folder.
That of course assumes that the post-install scripts don't move files around. Mac OS has a similar utility called "pkgutil" which supposedly tracks all files belong to programs that come as "pkg" files -- typically system utilities that don't make sense to ship as clickable bundles. But the pkgutil database is often useless. For example, the MacGPG installer used to "install" its files under "/tmp/private", but that was simply a temporary install directory which was immediately moved by a postinstall script to an untracked location.
macOS has App Folders, since the beginning when it was call OS X, and they're kinda like GoboLinux, but even better! Just copy this folder anywhere and it runs stand-alone and self contained.
But almost nothing is just a plain App Folder on macOS anymore. Starting over a decade ago, anything from Microsoft or Adobe or Google even Apple themselves, requires you to double-click run an installer that puts suspicious bits everywhere, just like on Windows. They want to auto-launch and auto-update and integrate and engage etc. And if you want to run Photoshop or Office or Chrome, what are you going to do, not let it install all its junk all over? Every regular person certainly will just let it, and all non-trivial companies will take maximum advantage. Skype and Office were automatically installing crappy Firefox extensions and stuff, so Firefox had to do some annoying complex extension-install-signing thing to try to prevent that ...
This is why Apple and Microsoft went towards sandboxed App Stores. The user won't choose less-abusively-packaged software, but even Big Software Corp can feel pressure to get its app into the store and sandboxed.
Linux distros never had this problem or needed this solution, because they re-package the software they distribute. If xscreensaver or calibra or virtualbox are programs people want to run, but their authors do annoying or dumb things, that can be patched by the distro packager, and the package manager file database will remain in quite good shape.
It kinda seems like your example says more about the specific behavior of the MacGPG installer than about the general reliability of macOS's package database.
You said it, stop switching distros, master it and enjoy. By mastering the distro I mean knowing enough to be productiva with it. I almost only use debian or alpine(docker and servers) and gentoo on mi laptop
You can certainly list files in packages in other distros, but the beauty of the GoboLinux approach is that it dispenses with loads of complexity. You don't need much of a package database at all.
Reducing complexity is good in and of itself and is always a win as long as you don't lose important functionality.
The notion of packages or installers vomiting files all over the place on systems really needs to die. Mobile OSes mostly got this right, and MacOS has been moving in this direction. Windows and older Linux distributions are the only things left that treat the system like a stir fry of random shit. Of the two this is probably easier to fix on Linux, since Windows is hamstrung by a ton of legacy requirements that are much harder to work around.
I'll be THAT guy as well! On Gentoo, "equery f <package>" displays a list of files installed by that package, "equery b <path>" says which package owns that file or directory, if any. (f and b are the short versions of "files" and "belongs", respectively)
I mostly use RHEL and SLES because that's what's required by my job. Their package databases are helpful sometimes. But usually it's just a symlink nightmare hidden across 5+ different directories. That is if you can even install anything past the license checks that fail to phone home because you are in a private VPC.
I do like Arch based Linux distributions. In my opinion they are the best all around for servers and even containers. Pacman is an excellent package manager.
FHS and distros that follow it aren't perfect, but it does make a big difference. I prefer Fedora-based and Arch-based flavors for that reason. It's usually pretty easy to find stuff.
Backtrack used to do something like this and while it was easier compared to a lot of other debian/ubuntu based things, I still ran into some pain points.
A great command to keep handy is
ls -la `which <program>`
That will usually find you the binary right away, and if it's a symlink or something you'll see it in the ls output. It's especially useful to me because (generally speaking) I put stuff I build myself in ~/bin rather than `/usr/local/bin` so I can quickly tell if I'm using my own build or the package manager's.
trivia: `which` is not the same which on all distros - on Ubuntu for example it's a super simple shell script named `which` which supports one single param (-a), not the compiled binary that you have on say Arch - and the features are different between them - I can't for example run `which --skip-tilde which` on Ubuntu, invalid usage.
Many scripters who need wide compatibility of "which" fall back to the builtins `type` and `command` as they resolve any bash aliases set and give you a more truthful (sic) output than relying on `which` and bash is bash on all platforms:
# command -v which
/usr/bin/which
# type which
which is hashed (/usr/bin/which)
$0.02 for your next adventure down the `which` rabbit hole. :)
Very true, I missed that thanks - to illustrate how that could be important (to the scripter):
kaneda@tetsuo:~ which type
kaneda@tetsuo:~ type type
type is a shell builtin
kaneda@tetsuo:~ command -v type
type
This was on Ubuntu, the which script doesn't resolve type or command silently, while on say Arch it's verbose ("can't find type in (paths ... ...)" so even a failure-to-find emits differently for the `which` crowd.
Interestingly Arch address multiple bin directories mess by symlinking /bin, /sbin, and /usr/sbin to /usr/bin. Stali, an experimental distro (originally a suckless.org project), goes that a bit further by symlinking /sbin to /bin and /usr to /.
Big picture, this is known as "the usr merge" which has been happening for probably a decade, each distro is approaching it at their own speed as they're not all exactly identical in their FHS layout (some distros add a bit of this and that on top).
I'm not sure I agree that most uses of Linux are single-user. I know this (likely) isn't what you meant, but at a minimum most every system, even containers, will have at least two users: root and the non-privileged user account. It may not be two humans but it is two users. On my current desktop (where I'm the only human user) I have 53 entries in /etc/passwd. Each have different permissions and access to resources.
But even if you consider the above to be a single-user setup (which is reasonable) I still think there's value in consistency. I personally get a ton of value from having CentOS and RHEL systems being laid out the same as my Fedora Laptop and Desktop system, and having all that knowledge transfer both ways. Being able to add a user to a "single user" setup too is very useful. It would suck if adding a second user required completely changing everything about the filesystem.
> I prefer Fedora-based and Arch-based flavors for that reason. It's usually pretty easy to find stuff.
> Backtrack used to do something like this and while it was easier compared to a lot of other debian/ubuntu based things, I still ran into some pain points.
FTA you linked:
> Most Linux distributions follow the Filesystem Hierarchy Standard and declare it their own policy to maintain FHS compliance. GoboLinux and NixOS provide examples of intentionally non-compliant filesystem implementations.
Debian and Ubuntu adhere to FHS; they're not more difficult in this regard than Arch or Fedora (or distributions based on these). GoboLinux and NixOS don't because they serve a higher goal. GoboLinux for user-friendliness/explorability and has backwards compatibility (which is hidden via optional kernel extension), NixOS aims to solve dependency hell (among various other goals). Learning curves are an obvious drawback.
> ls -la `which <program>`
Backticks should be avoided; use ls -la $(which <program>) instead (avoid the $ in Fish).
My pleasure! It can feel brutal at first to learn Linux, especially with so much choice out there (aka fragmentation?), but honestly it's the best thing I ever did for myself both personal and career-wise. Can't recommend at least basic Linux proficiency enough if you are in a technology sector especially.
Sadly Gobo has few users, so it doesn't enjoy as much support as other mainstream distributions.
You may want to consider NixOS, which is equivalent to Gobo except for the fact that linking across packages is static instead of dynamic. This has very deep consequences.
Nix has tons of developers, it's one of the most active projects on Github. Past the initial difficulties, most common usecases are really easy.
In almost any system, finding out where what files belong to a package is a trivial operation for the package manager to list.
The idea that a package is “installed somewhere” rather than “installed” and has files that “belong to it” seems to be a Windows-ism in mentality, where it is common to start applications by navigating to the folder they are installed in, and starting an execution there from an absolute path.
Many packages' files consist of nothing more than adding more fonts to the system, it seems a bad idea to me, to have all that in a per-package folder, rather than simply in `/usr/share/fonts`, and have the package manager track what file therein belongs to what package.
This runs somewhat counter to my practical experience. On Windows, most programs come with an installer and end up launchable with a shortcut from the start menu. On Linux, on the other hand, unless you're pulling from the package manager, then you've probably got a .tar.gz archive to manually extract somewhere. I see things have been changing only recently with the advent of Snap/Flatpak/AppImage.
I find that most Unix software installs via a `make install` directive, which keeps track of what files are installed where, and are then uninstalled with a `make uninstall`, if one not rely on a package manager that installs does the same internally.
/bin and /usr/bin make less sense these days since the root partition is usually big enough to boot and to hold all the programs the system needs. But it’s important to have somewhere outside the package-manager’s control for manually installed binaries. Most installers default to directories like /opt/foo/bin and I tend to put user-built programs into /usr/local/bin.
I ran across it ages ago. I think you could use something like GNU Stow to achieve the same thing unless I am missing something. You install programs to ~/.local/pkg, then you link it to ~/.local/{bin,etc,include,lib,man,share,var}. Unless of course it is an Electron app.
I use zpkg to handle this, it has been made specifically for this use case.
Then, you go to modify the configuration, and it turns out that the configuration file is being copied and generated from a different location. It's endless misdirection which changes in each distribution based on the package manager.
I think I may try out Gobo the next chance I get. This is pretty cool.