This might be a frequentist vs bayesian thing, and I am bayesian. So maybe other people would have a different view.
I don't think you need to have any information to have a probability distribution; your distribution already represents your degree of ignorance about an outcome. So without even sampling it once, you already should have a uniform probability distribution for a random number generator or a coin flip. If you do personally have additional information to help you predict the outcome -- you're skilled at coin-flipping, or you wrote the RNG and know an exploit -- then you can compress that distribution to a lower-entropy one.
But you don't need to sample the distribution to do this. You can have that information before the first coin toss. Sampling can be one way to get information but it won't necessarily even help. If samples are independent, then each sample really teaches you barely anything about the next. RNGs eventually do repeat so if you sample it enough you might be able to find the pattern and reduce the entropy to zero, but in that case you're not learning the statistical distribution, you're deducing the exact internal state of the RNG and predicting the exact next outcome, because the samples are not actually independent. If you do enough coin flips you might eventually find that there's a slight bias to the coin, but that really takes an extreme number of tosses and only reduces the entropy a tiny tiny bit; not at all if the coin-tossing procedure had no bias to begin with.
However the objective truth is just that the next toss will land heads. That's the only truth that experiment can objectively determine. Any other doubt that it might-have-counterfactually-landed-tails is subjective, due to a subjective lack of sufficient information to predict the outcome. We can formalize a correct procedure to convert prior information into a corresponding probability distribution, we can get a unanimous consensus by giving everybody the same information, but the probability distribution is still subjective because it is a function of that prior information.
The best introduction that I can recommend is this type-written PDF from E.T. Jaynes, called "probability theory with applications in science and engineering": https://bayes.wustl.edu/etj/science.pdf.html
It requires a lot of attention to read and follow the math, but it's worthwhile. Jaynes is a pretty passionate writer, and in his writing he's clearly battling against some enemies (who might be ghosts), but on the other hand this also makes for more entertaining reading and I find that's usually a benefit when it comes to a textbook.
