It is a bit of a problem with archeology which is also recognized by archaeologists.

That and how the whole story of hundreds of years of civilization is sometimes just an embellished projection based on a handful of symbols scratched in stone somewhere.