> I do it a few more times. It’s not always the 20th iteration, but it usually happens sometime between the 10th and 40th iteration. Sometimes it never happend. Okay, the bug is nondeterministic.
That’s an incorrect assumption. Just because your test case isn’t triggering the bug reliably, it does not mean the bug is nondeterministic.
That is like saying the “OpenOffice can’t print on Tuesdays” is non deterministic because you can’t reproduce it everyday. It is deterministic, you just need to find the right set of circumstances.
From the writing it appears the author found one way to reproduce the bug sometimes and then relied on it for every test. Another approach would have been to tweak their test case until they found a situation which reproduced the bug more or less often, trying to find the threshold that causes it and continuing to deduce from there.
"Deterministic" is .. something of a moveable feast. We'd generally agree that "software is deterministic in that if you provide the same inputs to the same executable machine code it will return the same value", which is nearly always true unless someone is irradiating your processor or trying to voltage-glitch it.
But there's a lot hidden in "same inputs", because that includes everything that's an input to your program from the operating system. Which includes things like "time" (bane of reproduction), memory layout, execution scheduling order of multithreaded code, value of uninitialized memory, and so on.
> Another approach would have been to tweak their test case until they found a situation which reproduced the bug more or less often, trying to find the threshold that causes it and continuing to deduce from there.
Yes - when dealing with unknowns in a huge problem space it can be very effective to play hotter-colder and climb up the hill.
If I understood correctly - the Math.Abs() value would be positive roughly half the time, regardless of the steps taken to get there. That seems definitively nondeterministic.
You don’t call Math.abs() on its own, you need to give it a number. Regardless if it is positive or negative, it should always return a positive (that’s what an absolute value is). The issue here is that it was returning a negative number when given a negative value, which is wrong:
> We rerun the repro. We look at the logged value. Math.abs() is returning negative values for negative inputs. We reload and run it again. Math.abs() is returning negative values for negative inputs. We reload and run it again. Math.abs() is returning negative values for negative inputs.
Regardless, that is beside the point. I was not arguing either way if this was a deterministic bug or not, I was pointing out that the author’s conclusion does not follow from the premise. Even if the bug had turned out to be nondeterministic, they had not done the necessary steps to confidently make that assertion. There is a chasm of difference between “this bug is nondeterministic” and “I haven’t yet determined the conditions that reproduce this bug”.
My interpretation was it was replaced with the identity function (e.g. just returning the original value). But it's only replaced if the code is determined to be a hot spot. So it would work correctly until the code was in a tight loop, then it would start failing once passed a negative number.
That’s an incorrect assumption. Just because your test case isn’t triggering the bug reliably, it does not mean the bug is nondeterministic.
That is like saying the “OpenOffice can’t print on Tuesdays” is non deterministic because you can’t reproduce it everyday. It is deterministic, you just need to find the right set of circumstances.
https://beza1e1.tuxen.de/lore/print_on_tuesday.html
From the writing it appears the author found one way to reproduce the bug sometimes and then relied on it for every test. Another approach would have been to tweak their test case until they found a situation which reproduced the bug more or less often, trying to find the threshold that causes it and continuing to deduce from there.