You don't have evidence for this. You are simply viewing others as a reflection of yourself.
If there are people at the tail end of the spectrum there must be people in between people who are centered and people who are extreme. Where and how these groups aggregate is hard to measure experimentally but it is clear that humans can lie and if they are not in the center of the spectrum and more evil than normal than likely they won't reveal it.
I like to think I have sufficient theory of mind that my self isn't the only evidence for my views. I'm able to admit that my thoughts are colored by myself and my experiences, just like yours.
Some people are not psychopathic but get labeled as if they are because of a lack of understanding. Others truly do have psychopathic tendencies. Even then, they are people first, and I'd wager they are a product of environment in more cases than not.
If there are people at the tail end of the spectrum there must be people in between people who are centered and people who are extreme. Where and how these groups aggregate is hard to measure experimentally but it is clear that humans can lie and if they are not in the center of the spectrum and more evil than normal than likely they won't reveal it.