When you download an installer, unless you reverse engineer it you also have no idea what it's doing.
As others said, you must trust the domain you're downloading from in both cases.
I really don't understand why people still argue about it. When you study security threat models, you should immediately understand that there's no difference from a security point of view. Perhaps the only alternative that's more secure is when you install the package from a package manager or app store... as in that case you have some sort of guarantee that the binaries are "vetoed" by someone who knows what they're doing it, hopefully.. i.e. you transfer your trust to the package manager's owners/maintainers... but you still need to trust the package publisher is not bent on trying to get you, because if they are, they will still find ways around package managers.
