Closed
Bug 632636
Opened 14 years ago
Closed 14 years ago
"REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)"
Categories
(Testing :: Reftest, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: cjones, Assigned: mattwoodrow)
References
Details
Attachments
(2 files)
(deleted),
patch
|
cjones
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
Details | Diff | Splinter Review |
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1297211639.1297213256.8940.gz#err0
Rev3 MacOSX Leopard 10.5.8 mozilla-central debug test reftest on 2011/02/08 16:33:59
s: talos-r3-leopard-006
REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)
REFTEST IMAGE: data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAXLElEQVR4nO3dTUuVaxuA4f1b+wNRv6EdFGGCNgn6oAZOzAwdN8mIigikgiIiqKiJfaGhht7v6A029eJb6SntdRxwDRbPs+B64B6sk8Vi/TUAAAAifx32AgAAwOQQIAAAQEaAAAAAGQECAABkBAgAAJARIAAAQEaAAAAAmT0D5MiRI+Po0aPj2LFj383x48eN+an50Tk6evToOHLkSHHeAQA4ZHsGyMzMzFhZWRn37t374dy/f9+Y/2v+1xlaWVkZMzMzxXkHAOCQ7RkgCwsLY21tbXz58sWYA5m1tbWxsLBQnHcAAA7ZngGytLQ0NjY2il2YUBsbG2Npaemw1wAAILBngCwvLwsQDtTGxsZYXl4+7DUAAAgIEA6dAAEAmBy/FCCbm5vj7t2748aNG+Pt27cHthyTQYAAAEyOXwqQW7dujcXFxfHmzZvx8ePHA1uOySBAAAAmxy8FyNWrV8ejR48ObCkmiwABAJgcPx0gKysr4++//x7T09Pj+vXrY4wxnj9/Ps6dOzemp6fH5cuXx9ra2rf7b9++Paanp7/dv7W1dQCPwZ9MgAAATI5f+gbk0qVL48mTJ99eT01NjZcvX47d3d1x69atce3atTHGGO/fvx9TU1Pj06dPY2tra8zNzY0HDx7s8yPwpxMgAACT47cD5NWrV2N6evrbtc+fP4+TJ0+Or1+/fvutyH+trq6OK1eu7Nfu/EsIEACAyfHbAfLixYsxOzv77drGxsY4ceLE2NzcHDdv3vzHB8vHjx+PCxcu7Nfu/EsIEACAyfHbAbKzszNOnz49Pnz4MMYY4+HDh2Nubm6MMca7d+/G7Ozs2N7eHru7u2NxcXHcuXNnv5+BP5wAAQCYHPvyG5CHDx+OU6dOjfPnz4+zZ8+O169f/+P9Z86cGTMzM+PixYv+1JDvCBAAgMmxb/+EvrOzM9bX1394bXt7e3z58uXnt2MiCBAAgMmxbwECv0qAAABMDgHCoRMgAACTQ4Bw6AQIAMDkECAcOgECADA59gyQpaWlsb6+PnZ2dr6b3d1dY35qfnSO1tfXx9LSUnHeAQA4ZHsGyPz8/FhdXR1Pnz79bp49e2bMT82PztHq6uqYn58vzjsAAIdszwABAADYLwIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADICBAAACAjQAAAgIwAAQAAMgIEAADI/AdYJemjM7UXFwAAAABJRU5ErkJggg==
REFTEST INFO | Loading a blank page
This first appeared on http://hg.mozilla.org/mozilla-central/rev/f34826051dd5, which disabled a test in the same manifest. That change looks correct. The relevant part of the manifest is
fails-if(cocoaWidget) == 586683-1.html 586683-1-ref.html
== 589615-1a.xhtml 589615-1-ref.html
== 589615-1b.html 589615-1-ref.html
== 589672-1.html 589672-1-ref.html
== 593544-1.html 593544-1-ref.html
== 594333-1.html 594333-1-ref.html
== 594624-1.html 594624-1-ref.html
== 594737-1.html 594737-1-ref.html
== 597721-1.html 597721-1-ref.html
needs-focus == 598726-1.html 598726-1-ref.html
Here's the relevant part of reftest.js
// whether the comparison result matches what is in the manifest
var test_passed = (equal == (gURLs[0].type == TYPE_REFTEST_EQUAL));
// what is expected on this platform (PASS, FAIL, or RANDOM)
var expected = gURLs[0].expected;
output = outputs[expected][test_passed];
++gTestResults[output.n];
var result = "REFTEST " + output.s + " | " +
gURLs[0].prettyPath + " | "; // the URL being tested
switch (gURLs[0].type) {
case TYPE_REFTEST_NOTEQUAL:
result += "image comparison (!=) ";
break;
case TYPE_REFTEST_EQUAL:
result += "image comparison (==) ";
break;
}
gDumpLog(result + "\n");
if (!test_passed && expected == EXPECTED_PASS ||
test_passed && expected == EXPECTED_FAIL) {
if (!equal) {
gDumpLog("REFTEST IMAGE 1 (TEST): " + gCanvas1.toDataURL() + "\n");
gDumpLog("REFTEST IMAGE 2 (REFERENCE): " + gCanvas2.toDataURL() + "\n");
gDumpLog("REFTEST number of differing pixels: " + differences + "\n");
} else {
gDumpLog("REFTEST IMAGE: " + gCanvas1.toDataURL() + "\n");
}
}
if (!test_passed && expected == EXPECTED_PASS) {
FlushTestLog();
}
The error message includes "image comparison (==)", so we know the harness thought |gURLs[0].type == TYPE_REFTEST_EQUAL|. The error message also includes a single image "IMAGE: ...", so we know the harness thought |!!equal|. Working backwards, we have to assume that it deduced |test_passed == true|, from the first line of code. This means the harness must have thought |expected == EXPECTED_FAIL|. For that to be true, it must have found a |fails| condition on the line in the manifest.
I have no idea what's going on here.
Reporter | ||
Comment 1•14 years ago
|
||
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1297213589.1297215254.17902.gz
Rev3 MacOSX Snow Leopard 10.6.2 mozilla-central debug test reftest
s: talos-r3-snow-012
REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)
PROCESS-CRASH | Main app process exited normally | application crashed (minidump found)
Thread 0 (crashed)
PROCESS-CRASH | Main app process exited normally | application crashed (minidump found)
Thread 0 (crashed)
PROCESS-CRASH | Main app process exited normally | application crashed (minidump found)
Thread 0 (crashed)
PROCESS-CRASH | Main app process exited normally | application crashed (minidump found)
Thread 0 (crashed)
Hardware: x86 → All
Summary: Intermittent, bizarre "REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)" → "REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/bugs/598726-1.html | image comparison (==)"
Assignee | ||
Comment 2•14 years ago
|
||
Attachment #510878 -
Flags: review?(jones.chris.g)
Reporter | ||
Comment 3•14 years ago
|
||
Comment on attachment 510878 [details] [diff] [review]
Initialize cond when we have needs-focus.
This code makes me squeamish but this shouldn't make anything worse.
Attachment #510878 -
Flags: review?(jones.chris.g) → review+
Reporter | ||
Comment 4•14 years ago
|
||
Reporter | ||
Comment 5•14 years ago
|
||
Fails without the patch, passes with it. I'll wait to land this until I have other stuff to push.
Comment 6•14 years ago
|
||
It might make more sense to just initialize cond where it's declared, and remove the various cond = false. But this is fine.
The patch is needed, since |var| in JS is function-scope, and thus |cond| has the value from the previous line.
(Really, the whole point of |cond| is to share code that's common across fails and fails-if, etc.)
Reporter | ||
Comment 7•14 years ago
|
||
Test: http://hg.mozilla.org/mozilla-central/rev/1bb9a9e03483 (argh, messed up the bug# in the commit message)
Updated•13 years ago
|
Assignee: nobody → matt.woodrow+bugzilla
You need to log in
before you can comment on or make changes to this bug.
Description
•