Closed Bug 632636 Opened 14 years ago Closed 14 years ago

"REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)"

Categories

(Testing :: Reftest, defect)

All
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: cjones, Assigned: mattwoodrow)

References

Details

Attachments

(2 files)

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1297211639.1297213256.8940.gz#err0 Rev3 MacOSX Leopard 10.5.8 mozilla-central debug test reftest on 2011/02/08 16:33:59 s: talos-r3-leopard-006 REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==) REFTEST IMAGE: data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAXLElEQVR4nO3dTUuVaxuA4f1b+wNRv6EdFGGCNgn6oAZOzAwdN8mIigikgiIiqKiJfaGhht7v6A029eJb6SntdRxwDRbPs+B64B6sk8Vi/TUAAAAifx32AgAAwOQQIAAAQEaAAAAAGQECAABkBAgAAJARIAAAQEaAAAAAmT0D5MiRI+Po0aPj2LFj383x48eN+an50Tk6evToOHLkSHHeAQA4ZHsGyMzMzFhZWRn37t374dy/f9+Y/2v+1xlaWVkZMzMzxXkHAOCQ7RkgCwsLY21tbXz58sWYA5m1tbWxsLBQnHcAAA7ZngGytLQ0NjY2il2YUBsbG2Npaemw1wAAILBngCwvLwsQDtTGxsZYXl4+7DUAAAgIEA6dAAEAmBy/FCCbm5vj7t2748aNG+Pt27cHthyTQYAAAEyOXwqQW7dujcXFxfHmzZvx8ePHA1uOySBAAAAmxy8FyNWrV8ejR48ObCkmiwABAJgcPx0gKysr4++//x7T09Pj+vXrY4wxnj9/Ps6dOzemp6fH5cuXx9ra2rf7b9++Paanp7/dv7W1dQCPwZ9MgAAATI5f+gbk0qVL48mTJ99eT01NjZcvX47d3d1x69atce3atTHGGO/fvx9TU1Pj06dPY2tra8zNzY0HDx7s8yPwpxMgAACT47cD5NWrV2N6evrbtc+fP4+TJ0+Or1+/fvutyH+trq6OK1eu7Nfu/EsIEACAyfHbAfLixYsxOzv77drGxsY4ceLE2NzcHDdv3vzHB8vHjx+PCxcu7Nfu/EsIEACAyfHbAbKzszNOnz49Pnz4MMYY4+HDh2Nubm6MMca7d+/G7Ozs2N7eHru7u2NxcXHcuXNnv5+BP5wAAQCYHPvyG5CHDx+OU6dOjfPnz4+zZ8+O169f/+P9Z86cGTMzM+PixYv+1JDvCBAAgMmxb/+EvrOzM9bX1394bXt7e3z58uXnt2MiCBAAgMmxbwECv0qAAABMDgHCoRMgAACTQ4Bw6AQIAMDkECAcOgECADA59gyQpaWlsb6+PnZ2dr6b3d1dY35qfnSO1tfXx9LSUnHeAQA4ZHsGyPz8/FhdXR1Pnz79bp49e2bMT82PztHq6uqYn58vzjsAAIdszwABAADYLwIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADI/AdYJemjM7UXFwAAAABJRU5ErkJggg== REFTEST INFO | Loading a blank page This first appeared on http://hg.mozilla.org/mozilla-central/rev/f34826051dd5, which disabled a test in the same manifest. That change looks correct. The relevant part of the manifest is fails-if(cocoaWidget) == 586683-1.html 586683-1-ref.html == 589615-1a.xhtml 589615-1-ref.html == 589615-1b.html 589615-1-ref.html == 589672-1.html 589672-1-ref.html == 593544-1.html 593544-1-ref.html == 594333-1.html 594333-1-ref.html == 594624-1.html 594624-1-ref.html == 594737-1.html 594737-1-ref.html == 597721-1.html 597721-1-ref.html needs-focus == 598726-1.html 598726-1-ref.html Here's the relevant part of reftest.js // whether the comparison result matches what is in the manifest var test_passed = (equal == (gURLs[0].type == TYPE_REFTEST_EQUAL)); // what is expected on this platform (PASS, FAIL, or RANDOM) var expected = gURLs[0].expected; output = outputs[expected][test_passed]; ++gTestResults[output.n]; var result = "REFTEST " + output.s + " | " + gURLs[0].prettyPath + " | "; // the URL being tested switch (gURLs[0].type) { case TYPE_REFTEST_NOTEQUAL: result += "image comparison (!=) "; break; case TYPE_REFTEST_EQUAL: result += "image comparison (==) "; break; } gDumpLog(result + "\n"); if (!test_passed && expected == EXPECTED_PASS || test_passed && expected == EXPECTED_FAIL) { if (!equal) { gDumpLog("REFTEST IMAGE 1 (TEST): " + gCanvas1.toDataURL() + "\n"); gDumpLog("REFTEST IMAGE 2 (REFERENCE): " + gCanvas2.toDataURL() + "\n"); gDumpLog("REFTEST number of differing pixels: " + differences + "\n"); } else { gDumpLog("REFTEST IMAGE: " + gCanvas1.toDataURL() + "\n"); } } if (!test_passed && expected == EXPECTED_PASS) { FlushTestLog(); } The error message includes "image comparison (==)", so we know the harness thought |gURLs[0].type == TYPE_REFTEST_EQUAL|. The error message also includes a single image "IMAGE: ...", so we know the harness thought |!!equal|. Working backwards, we have to assume that it deduced |test_passed == true|, from the first line of code. This means the harness must have thought |expected == EXPECTED_FAIL|. For that to be true, it must have found a |fails| condition on the line in the manifest. I have no idea what's going on here.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1297213589.1297215254.17902.gz Rev3 MacOSX Snow Leopard 10.6.2 mozilla-central debug test reftest s: talos-r3-snow-012 REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==) PROCESS-CRASH | Main app process exited normally | application crashed (minidump found) Thread 0 (crashed) PROCESS-CRASH | Main app process exited normally | application crashed (minidump found) Thread 0 (crashed) PROCESS-CRASH | Main app process exited normally | application crashed (minidump found) Thread 0 (crashed) PROCESS-CRASH | Main app process exited normally | application crashed (minidump found) Thread 0 (crashed)
Hardware: x86 → All
Summary: Intermittent, bizarre "REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)" → "REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)"
Attachment #510878 - Flags: review?(jones.chris.g)
Comment on attachment 510878 [details] [diff] [review] Initialize cond when we have needs-focus. This code makes me squeamish but this shouldn't make anything worse.
Attachment #510878 - Flags: review?(jones.chris.g) → review+
Blocks: 615386
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Attached patch Test (deleted) — Splinter Review
Fails without the patch, passes with it. I'll wait to land this until I have other stuff to push.
It might make more sense to just initialize cond where it's declared, and remove the various cond = false. But this is fine. The patch is needed, since |var| in JS is function-scope, and thus |cond| has the value from the previous line. (Really, the whole point of |cond| is to share code that's common across fails and fails-if, etc.)
Depends on: 632707
No longer depends on: 632707
Test: http://hg.mozilla.org/mozilla-central/rev/1bb9a9e03483 (argh, messed up the bug# in the commit message)
Assignee: nobody → matt.woodrow+bugzilla
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: