I actually encountered a similar situation years ago, where I tried to bisect for a regression. Problem was, old code wasn't testable for my regression, and I had to apply a patch to it to be able to automatically test it. I kept getting nonsensical results out of the bisect however. I later realized that my patch to make old code testable introduced the very regression I was looking for!