US doesn't seem to have access to UK criminal records as I've travelled to NY without declaring convictions and been OK. No idea if just lucky or there's no data sharing.
There is data sharing, what you've done is given the US the ability to refuse access and additionally to refuse a visa - as they required you to self-declare and they consider it a fraudulent attempt to gain access to the US if you have not done so.
That they do not consistently enforce at time of entry is luck on your part in that entry instance, but either way they now have the ability to freely deny entry forever due to you not following their rules.
Sucks to be you now, the US system does not believe in rehabilitation and if you've been arrested you need a visa. You are not entitled to use the visa waiver program that exists between the UK and US and allows visa-less movement with only an ESTA.
If you want to visit the US in the future, or your career requires such visits... I suggest applying for a B1/B2 visa, if they think you're no risk they may even grant you a multi-year multi-visit one. If they think you're too risky they'll only grant a single timed visit visa, or you'll be fully declined.
