http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1298487014.1298488395.13658.gz Rev3 WINNT 5.1 mozilla-central opt test reftest on 2011/02/23 10:50:14 s: talos-r3-xp-013 REFTEST INFO | Loading a blank page REFTEST TEST-START | file:///C:/talos-slave/test/build/reftest/tests/layout/reftests/svg/dynamic-text-04.svg REFTEST TEST-START | file:///C:/talos-slave/test/build/reftest/tests/layout/reftests/svg/dynamic-text-04-ref.svg REFTEST TEST-UNEXPECTED-PASS | file:///C:/talos-slave/test/build/reftest/tests/layout/reftests/svg/dynamic-text-04.svg | image comparison (==) REFTEST IMAGE: data:image/png;base64,iVBOR... REFTEST INFO | Loading a blank page I would expect this extremely intermittent pass of the test that bug 623405 marked as failing on WinXP to just be, as I said mistakenly of the other test there, [[[ Honestly, I don't have the slightest idea what made me think that "drawn in a random spot" somehow meant "drawn in a random spot which will never be the right spot." ]]] except that I'm a little puzzled by the way that the log only shows one data URI, rather than the two that I think I've seen the few times I've actually paid any attention to the details of reftest failures.

Why do we want to have this test marked as random? It doesn't sound to fail THAT much...

I think you mean: it doesn't seem to *pass* that much. :) It's nearly-perma-failing on Windows XP -- so it marked as a failing -- but it occasionally randomly passes. (hence this bug) IMHO, flagging it as random-on-windows-XP would be accurate and appropriate in this case.

Though it also would be interesting to find out if the test ever started (reliably) passing, so I can see the case for leaving the annotation as-is, too. (given the relatively low level of orange-noise that it's causing now)

Oh, my bad... I didn't see it was changing fails-if to random-if, I was too focused on the random-if.

(In reply to comment #8) > Though it also would be interesting to find out if the test ever started > (reliably) passing, so I can see the case for leaving the annotation as-is, > too. (given the relatively low level of orange-noise that it's causing now) I agree. The fact it's currently marked fails-if only reverse the issue: we should only mark something as orange if we really have no other choices.

If someone wants to write that test, which isn't this bug, and which ought to be named something like bug_421587_unfixed and ought to be in bug 421587, I'd think that all they would need to do would be to have something like four iframes with rotated text in them. Since we're only accidentally getting it right four out of several thousand times, making sure we never get four right at once ought to be enough to tell us that bug 421587 still exists.

The changes in bug 598482 will likely make this happen more, because the test will paint fewer times. That could be "fixed" by putting the actual rotation off on a requestAnimationFrame callback or three, I suspect.

I suspect that bug 525095 has much reduced the amount of randomness resulting in this test passing much more frequently than it used to.

(In reply to Robert Longson from comment #77) > I suspect that bug 525095 has much reduced the amount of randomness > resulting in this test passing much more frequently than it used to. Yeah, it seems to be passing half the time, and failing half the time. I can't see any visual difference between the failing image and the reference, so this seems low priority. For now I marked the test as random: https://hg.mozilla.org/integration/mozilla-inbound/rev/eea95e86541f

Test annotation landed: https://hg.mozilla.org/mozilla-central/rev/eea95e86541f Leaving this bug open for now, but you can resolve it if there's no more work planned here.

Any work on why this is random would take place in bug 421587.

(and to be clear, we shouldn't be getting any more oranges from this, since it was marked random in comment 167)

Nice try, but it's not that easy to get out of filing new failures.