Closed
Bug 1383682
Opened 7 years ago
Closed 7 years ago
Intermittent /css/css-backgrounds-3/background-size-031.html | Testing http://web-platform.test:8000/css/css-backgrounds-3/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds-3/reference/background-size-031-ref.xht
Categories
(Core :: Graphics: ImageLib, defect, P3)
Core
Graphics: ImageLib
Tracking
()
RESOLVED
FIXED
mozilla60
Tracking | Status | |
---|---|---|
firefox-esr52 | --- | unaffected |
firefox58 | --- | wontfix |
firefox59 | --- | wontfix |
firefox60 | --- | fixed |
People
(Reporter: intermittent-bug-filer, Assigned: aosmond)
References
(Blocks 1 open bug)
Details
(Keywords: intermittent-failure, Whiteboard: [gfx-noted][stockwell unknown])
Attachments
(3 files, 5 obsolete files)
(deleted),
patch
|
tnikkel
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
tnikkel
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
tnikkel
:
review+
|
Details | Diff | Splinter Review |
Filed by: cbook [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=116659502&repo=mozilla-central https://queue.taskcluster.net/v1/task/YMAnvc_9Rze_A-qlcD2pWQ/runs/0/artifacts/public/logs/live_backing.log https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/YMAnvc_9Rze_A-qlcD2pWQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 5•7 years ago
|
||
this has started failing a bit more this past week, specifically on linux32-stylo debug: https://brasstacks.mozilla.com/orangefactor/index.html?display=Bug&bugid=1383682 :xidorn, could you look at this and help set a priority?
Flags: needinfo?(xidorn+moz)
Comment 6•7 years ago
|
||
Isn't that just because we didn't start testing linux32-stylo until this past week? I don't believe this is related to stylo given it fails on normal builds as well. As I mentioned in bug 1369957 comment 13, it may seem to happen more often on stylo builds simply because tests run more frequently on stylo builds than normal builds. We may want to set up SETA data for linux stylo builds for saving machine resources and reduce noises on intermittents.
Flags: needinfo?(xidorn+moz)
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
![]() |
||
Updated•7 years ago
|
![]() |
||
Updated•7 years ago
|
![]() |
||
Updated•7 years ago
|
![]() |
||
Comment 14•7 years ago
|
||
The see-also bugs fail the same way as this one: slight differences (I can't see them) around the "image" word. Collectively, these bugs account for about 100 failures / week. :jmaher -- What's the best way to get some attention for these failures?
Flags: needinfo?(jmaher)
Whiteboard: [stockwell needswork]
Comment 15•7 years ago
|
||
starting with the right bugzilla component: $ ./mach file-info bugzilla-component testing/web-platform/tests//css/css-backgrounds-3/background-size-031.html Core :: CSS Parsing and Computation testing/web-platform/tests/css/css-backgrounds-3/background-size-031.html
Component: web-platform-tests → CSS Parsing and Computation
Flags: needinfo?(jmaher)
Product: Testing → Core
Version: Version 3 → unspecified
Comment 16•7 years ago
|
||
:jet, we have a series of failures here all seem to be related to the same root tests of background-size-xx.html- in total the failures are extremely high- can you help find someone to look at these failures, I assume there is a single root cause.
Flags: needinfo?(bugs)
Comment 17•7 years ago
|
||
Those are just images, and image showing something unstable (rather than, e.g. wrong position) sounds more like issue in ImageLib or Graphics, though it is also possible that Layout sends some subpixel position to them. Anyway I don't think it's related to style system. Maybe the easiest way is to add some fuzzy, but we cannot do that for wpt reftest I suppose?
Component: CSS Parsing and Computation → ImageLib
Comment 18•7 years ago
|
||
:jgraham, in wpt reftests is there a way to add fuzzy conditions?
Flags: needinfo?(james)
Comment hidden (Intermittent Failures Robot) |
Comment 20•7 years ago
|
||
No. It has been discussed, but there's no cross-vendor agreement that it's a good idea. But we will likely need something at some point if webrender makes exact references unworkable.
Flags: needinfo?(james)
Comment 21•7 years ago
|
||
Given the fact there is no immediate support for fuzzy in wpt-reftests, do you have suggestions of how to resolve this?
Flags: needinfo?(xidorn+moz)
Comment 22•7 years ago
|
||
I have no idea. Maybe ask people working on Graphics or ImageLib and see if they have any idea what may be happening here.
Flags: needinfo?(xidorn+moz)
![]() |
||
Updated•7 years ago
|
![]() |
||
Comment 23•7 years ago
|
||
:milan (when you get back) -- Do you have an idea of who could investigate these frequent intermittent test failures?
Flags: needinfo?(milan)
Comment hidden (Intermittent Failures Robot) |
I'll leave needinfo on me, but chances are we won't get to the intermittents until 57 is out the door.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 30•7 years ago
|
||
There have been 31 failures in the last week. Number of failures per platform: - 11 linux - 11 linux32-stylo-disabled - 5 linux64-ccov - 4 Linux x64 Number of failures per buid type: - 26 debug - 5 opt I see a peak in number of failures on November 24th and from that point on the number of failures is decreasing. Here is an example of a recent log: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=147635673&lineNumber=13605 and a snippet from it: [task 2017-11-25T22:55:00.041Z] 22:55:00 INFO - PID 5511 | 1511650500033 Marionette INFO Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht 13602 [task 2017-11-25T22:55:00.117Z] 22:55:00 INFO - PID 5511 | ++DOMWINDOW == 83 (0xdf4c4000) [pid = 5618] [serial = 83] [outer = 0xe7997040] 13603 [task 2017-11-25T22:55:00.259Z] 22:55:00 INFO - PID 5511 | ++DOMWINDOW == 84 (0xdf4c4c00) [pid = 5618] [serial = 84] [outer = 0xe7997040] 13604 [task 2017-11-25T22:55:00.356Z] 22:55:00 INFO - PID 5511 | 1511650500351 Marionette INFO Found 2388 pixels different, maximum difference per channel 22 13605 [task 2017-11-25T22:55:00.400Z] 22:55:00 INFO - TEST-UNEXPECTED-FAIL | /css/css-backgrounds/background-size-031.html | Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht 13606 [task 2017-11-25T22:55:00.404Z] 22:55:00 INFO - REFTEST IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3dbaxkdX3A8f+9LLDbsourialN+kKMBhSfqm21b3ajqcE0Sompxm66iMgLNDERH1DTV6Y21Gr0Bbyw1bKEriy4y0IRUgJlEQUEFRcwKqHLg92KLS4Li3vdpzu/vph7997dvXPnnJnfnDn37ueTTMLOnDnnf87MnPly7rnnlgAAIFUZ9wAAAJYbgQUAkExgAQAkE1gAAMkEFgBAMoEFAJBMYAEAJBNYAADJBBYAQDKBBQCQTGABACQTWAAAyQQWAEAygQUAkExgAQAkE1gAAMkEFgBAMoG1TJTipZxlWwAw3zi+FyotsZTS9zaSwfmirKzXtqr6OvV7/lIyivGOcxssxddgUHXer4Psg0Y17VJ2sqwny0/dz/Ni8xjF56ByYNX5dxYf+uHUeZ0Gfexk0IYvoHEvvwlV36+9Xo+qr1OdbXkybPeIk2c9WX6Gee+OumUE1jKW9UVyMr8Os+s+7m0w7uU3Ydj/E606D4F1opNlPVl+MgMr20Bz7zeoKofc+h3ir/ujyPmPD/pjhkHGWXWazOX1es6g8+m3rbO3a9XXapBDt8O+RovNdxCZ74uFntNrXr3u6/X6Dvta9ZP1GaqynKrTjHq7H//YKLdbneUsNF2dcTX1+c5ctyrvuUHXv87r3uuzt9i61x131vavs66jHGe/eVb9zPdb/0H3OYsud6An9dkA/e6r+oYadGc06JgGGecw6zLodllI1fWu8/zFxjDMtl5snnXn02u8gyy/zjwHHccg74s69y02bdZ7vtd9vQy7PlXVGU+T273q8/tNV+fLuO46juK1rPP5bnrdBn1dht3fZ7+f6m63uvOuMt+FjPv1rTKmXvdlSg2sfl/SdZ9fd+WrLr/qNFXGOcw0g8yrynzqPK/KtBmv1zDTVn2s7rJGGViZ74te047ii37Y5dSZZtjP++xz6uxkM8Y0is9DVeMcTxOvZdVljmPfU3U8dec/7P6p7r6/ie+jqs9r8rN0UgXWsB/oYZZfdZoq46w6TZXH6n5R9FuGwJr797AfpMz3YpVpBFZ9w+xo64xJYA23rDrzb9O+Z/bxrP19v+nbEFiZATLuz9KyCKzFbotNW2c5dce10GNVxlhnnHXmMex2qbJ+ve6r8/x+j1Xdpr3Wc7HlVZ1P1fEOul2rjHfQ6QVWbmA1Nabsz0PV+xfbvwy6LlXHPsjzF9OmdRtk/zLImHs9VvdzPMx2W2we2d9H43p960yXsc9ZdJkDPWmIHe5i8xx2h1v1RRj2C6PqzqPOfIdZXq9l1P1A1H1smNcr4wMzzPMH2WGMYgxVH8vYMWfet9j9VacZ9vM+rjGN4vMw6DKyxtPkaznudcv+Uq2zvx/FOmZEXlVVv/+q3i+wFnrSADvcQQJn3IFVZbphphl0XFWnaVNgZX1gBtkWGf9DkPVeHHRHLLAGm+coxtTEl0Ibv6ROhsDKeF2WcmBlfB+N+/WtM92SCqxejy20sapOV3W5dec76H3Hz2/Qdak6rl7PrTLNMAEx6Adt2C/tYZ7Ta7phtusg0y42jirzrXPfMOvaRGBlLGeh6Zp6flPbPXsZdZbda77DznOxeYx73TJfl4WeW3cso/yOWsgov4/a8PoOM89Mtec+u0JVPkCLTXf8NFWmqzK2QZc//7664xxkXYbdLr2eU2dZdefTb9qqy646tqx1WGheg8yvzvPqrEOd7VXluVVew2Hu6zemYdex7nt+kHHUHVOV8WatW51lZH0+m34tm163Qdajl6rvvWHH0m/dq2y3xdZrkHUY5j0yzvfuQvMc9PWvY7T51rBRbSQAlqY2fS+0aSyM3rJ6tb15AU5OvY54tEnbxsNoLZtXe9SH+gBotyZ+7DOoto6L0fFKAwAkE1gAAMkEFgBAspMysPwMnEF43wBQVeVvjH7XyVhKltp426bq9VWGnX+d66NkWuwaLONYblu0fXwnI6+Jk8dpr6EDq99jLD+jDqzF5tHUjnRc7+m2f5baPr6TkddEaNJOAovaxn0USWCNT9vHdzLymggs2qnSO3Khy+Uff8n5haarOr9ej/ebdphL+FdZx6rzGnb96+wYspY56LyrvBeqLmOQo2CLLa/uWBYbU6/3Wr/nLva8qq9B1TH2enyx+6qOo9d0dT9ng7wfeq1Hr+dWnW+v51Z5/ep+TjPH0G+eVcaVtd+osw3qLLPOcqq8Z6AN0o5g1flCqnrfQvOsO59e4x1k+b3mVXf9q06bucwq9/Wbd9XxZo6jyv39vmgXU+e1H/R9k/FZyJpf9mdyIYN8nvsto87rO+xnsc68exnF/mCY12T+tG3ZVw2ynH5xBm2SFlhV7q8zj0GW1++xusuqE1hV7u8XAnV3lMPMp864606TsS4Z9w8SV4s9Nuj7Jut9X3d8x99XdRzZn6+MZQzzGa07z2EeyxpD1v7w+OnatK+qu5ysMUFTBFYM9+Fs206r3//hHX8bZByD7jQzxtG2wBomOHrNs4pBIqbJwBr1tq8znrrzHOaxrDEILIHF0td4YFX9kq3yoaz6ZV1lvHXncfzzqtw/jsDKCpg609RZ9+z7q4ZCv8erfhlWed8MG1h13uf9AqvK5ybzy3yx16vOMups26rba9T7mGHGUHd/WEXb9lXzH6/6vswYEzSlNUewMpaXMe/50wwzncBqNrCOf6zqa1fnsUHfD8MEVt1oW2z967y/Bnms3zyHibhh9zEZkVdVxhhOtsAadl4CizZqTWBl7VAG+WKt+yV2/DRt2mkJrOGOZC722KDvmyYDa/axYYIi+/OVsYw2BVaVz8ewYxBYc49njwmaMnBgVdlZVt2h1N35DfqcXtMN8oVU5bkZO7PFllllPk0FVr8dYROBVXX8vcZdZdmjnK7qc4fZBsN+Jof9TAy6jGHe74N+vjP3CXXHkP2a9BvDqPdVg46rzpiG3T6QrdY7b6E3cr8v+8Xe9IM+Xne6KuM6fl51dqSLbZMq61d3BzD/OVXms9DyBhn3sOs+7DjqrGsdC813kPd5vy+KOu+LuuNbaHl11rfqdIN+Nqq+Nxcbf9Xx1H09626DKrLeU4Nug6rrUec9mbENqk476Puy17aEcfDOW4LsNBbXtm3T9Hjatv51LfXxA0QIrCVJYB3r+P8jb5smxtTv6M9SshzWAcCebIkZ9HD9ctbmbdLUuNq6/nUtl/UAsBcDAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI1jewSln6t3Xn7GhgUwIAdC37wFp3zo6IzQ7UAQDNWdaBdTSuBBYA0KBlG1jHxJXAAgAatCwD64S4ElgAQIOWXWAtGFcCCwBo0LIKrJ5xJbAAgAYtycA666xd8dnP/kNcccWnY926HbFmzfOLx5XAAgAatMQCqxOXXPL12Ldvdeza9cr4+c/Pjr17XxL33/O2iG8vElcCCwBo0BIKrE5s3HhNTE2tii1bPhAve9mzsXLlVFx77YbuajwksACAdlgygbVixaG4+uoLY/fuP4zTTjsQk5NH4jOfuqK7Ck/NRNQNAgsAGL8lE1innnowdu58Y2zatDFOOeVwfO7yL0YcKRFPlIjrS8R1M6F1m8ACAMar5YHViYmJztH/vv32v4jnn18Tn/jEV7pDf2IuoDpbS0SnRHxPYAEA49XqwPrwh78R11zzt7F69QtRSsTrX/9I/Pa3v9cNqSdnjlxtLt0T3HeXiP0lYqvAAgDGq7WBdcYZ++LLX74sOp0S27f/VUxOHol3nntHxI9Ld9j/XSLuKREPlohfzxy9uqdHXAksAKBBLQysTlx88TfiwQffGs8995LYt291TE9PxG3bz5s7YrVz5mhVZ+b2Yom4d5G4ElgAQINaF1gbNlwbU1OrYuvW98X69TvizW/+cXzzqxd1T2if/2PBbSXi7hJxV+l/DSyBBQA0qFWBddppB+LWW98djz326njpS/dEKfP+/M2PSne4T1eIKYEFAIxRywLrYOzadVZcddWlUUrn2D9/860S8ZuZyJp/JEtgAQAt06rAKqUTDz/8hnjooTfFeX9824mR9GTpnm81e5K7wAIAWqhlgRXx+c//fRw6tCLi4ZmjVrOB9O2ZuHqoRDxaIo6U6NwksACA9hlrYE1OHonTT/9dTE4eOXrfu99ya8RjJWK6RPyiRNxRupdf+J8ScbBEfKdE3F4iDpeI+wUWANA+YwusSy+9Ku699+3R6ZTYu/fM2LRpY7z3bTfNBdHjpTu86dK9FMNUmbsUw/dm7v++wAIA2mcMgdWJSy75ekxNrYpt2y6IjRs3xeWXXxFTUysjnilzV2K/rkTcXCLumwmpG2fu31q6003Nu09gAQAt0nBgdeK8826LiBI33fTeWLHicJTSiU9+8p+6Q+mUiD0l4oYFAulb8+LqYIn4bo24ElgAQIMaDayJiem48sqPxn33vT3WrHk+JiePxOcu/2L3IqJPzPwI8FA59kjW7O3fS8SzJWLvAHElsACABjV8BCviNa/5RXzkI/8c3T+J8y/dITwxL4QOlO59vynH/hbh5hJxS6l//SuBBQA0rPHAmv1R4brX7YjO7HWtZkPqztI9ef0npf/fFhRYAEBLjTiwOrF+/V1xwQXb4uyzfxYrV05FKTN//ua6mR/3PV7mrnO1u0Q8VyK2JMeVwAIAGjTSwPrYx66M/ftXRadTYv/+VXHzze+Jd5x7Z8TmEp3NJeJXpTuE/yrR+d/SPcm9zqUXBBYA0EIjC6xXvOJX8cwzfxDbt58fl132lfjSlz7VvRTDU2XuPKobSvf8q/8r3XOuRhVXAgsAaNAIAqsTpXTioou+Gfff/2dHL8Ww/rV3Rfxw5ojVL8uJJ7CP+gYA0JDkwOrEu971H3HhhVfHxz/+tdiy5f0xMTHdPedqNnR+VLqXZXiyDP4bgQILAGixxMCau0L79PREHDhwejzyyLnxN+/cfGLsPDDvSJbAAgCWmbTAOvXUg/H0038U11//13H++dvjgx/cHC+++Pvd3wrctkDw/LCM9pwrgQUAjElCYHVi48Zr4gtf+Lt49NHXxdq1z81diuE/S/fCoc+Wen83UGABAEtY7cB6+ct/HWeeuffov1evfiF+8IM/jenpiXjggT+JU045fOw5V22JLACAhtQKrFWr9sfdd6+LW275y1i7ds/R+1/1qsdj5843xJ49a+MzF//jiXFze4k4XCJeKM3/9qDAAgAaVvsI1oYN/xb79p2xYGT97KdnR/yuRNyxQODcWSLucQQLAFj+agdWr8had86OiJtL96T2XpE1zhsAQEMGCqxSIj70oX89GlkX/PmNcyHT1sgCAGjIwIE1/0hW7C7dP9Z8fGS9WCK2tiCuBBYA0KChAmvdOTu651V1ysKRdVcLwkpgAQANGziwjrkUw/dLxKEFIqtNNwCAhgwUWMfE1fGR9VQZ36UYBBYA0AK1A2vBuJq93VfGeykGgQUAtECtwFo0rtp+AwBoSOXAWtJxJbAAgAZVCqwlH1cCCwBoUN/yWBZxJbAAgAZVOoK11G/rztnRwKYEAOha9oF19AgcAEBDlnVgHfPjTQCAhizbwDrh3DEAgIYsy8Ba8MR8AICGLLvA6vlbjwAADVlWgbXoJSUAABqyJAPrrLN2xWc/+w9xxRWfjnXrdsSaNc/3v14XAEBDllhgdeKSS74e+/atjl27Xhk///nZsXfvS+L+e94W8e1F4kpgAQANWkKB1YmNG6+JqalVsWXLB+JlL3s2Vq6cimuv3dBdjYcEFgDQDksmsFasOBRXX31h7N79h3HaaQdicvJIfOZTV3RX4amZiLpBYAEA47dkAuvUUw/Gzp1vjE2bNsYppxyOz13+xYgjJeKJEnF9ibhuJrRuE1gAwHi1PLA6MTHROfrft9/+F/H882viE5/4SnfoT8wFVGdrieiUiO8JLABgvFodWB/+8Dfimmv+NlavfiFKiXj96x+J3/7297oh9eTMkavNpXuC++4Ssb9EbBVYAMB4tTawzjhjX3z5y5dFp1Ni+/a/isnJI/HOc++I+HHpDvu/S8Q9JeLBEvHrmaNX9/SIK4EFADSohYHViYsv/kY8+OBb47nnXhL79q2O6emJuG37eXNHrHbOHK3qzNxeLBH3LhJXAgsAaFDrAmvDhmtjampVbN36vli/fke8+c0/jm9+9aLuCe3zfyy4rUTcXSLuKv2vgSWwAIAGtSqwTjvtQNx667vjscdeHS996Z4oZd6fv/lR6Q736QoxJbAAgDFqWWAdjF27zoqrrro0Sukc++dvvlUifjMTWfOPZAksAKBlWhVYpXTi4YffEA899KY4749vOzGSnizd861mT3IXWABAC7UssCI+//m/j0OHVkQ8PHPUajaQvj0TVw+ViEdLxJESnZsEFgDQPmMNrMnJI3H66b+LyckjR+9791tujXisREyXiF+UiDtK9/IL/1MiDpaI75SI20vE4RJxv8ACANpnbIF16aVXxb33vj06nRJ7954ZmzZtjPe+7aa5IHq8dIc3XbqXYpgqc5di+N7M/d8XWABA+4whsDpxySVfj6mpVbFt2wWxceOmuPzyK2JqamXEM2XuSuzXlYibS8R9MyF148z9W0t3uql59wksAKBFGg6sTpx33m0RUeKmm94bK1YcjlI68clP/lN3KJ0SsadE3LBAIH1rXlwdLBHfrRFXAgsAaFCjgTUxMR1XXvnRuO++t8eaNc/H5OSR+NzlX+xeRPSJmR8BHirHHsmavf17iXi2ROwdIK4EFgDQoIaPYEW85jW/iI985J+j+ydx/qU7hCfmhdCB0r3vN+XY3yLcXCJuKfWvfyWwAICGNR5Ysz8qXPe6HdGZva7VbEjdWbonr/+k9P/bggILAGipEQdWJ9avvysuuGBbnH32z2LlyqkoZebP31w38+O+x8vcda52l4jnSsSW5LgSWABAg0YaWB/72JWxf/+q6HRK7N+/Km6++T3xjnPvjNhcorO5RPyqdIfwXyU6/1u6J7nXufSCwAIAWmhkgfWKV/wqnnnmD2L79vPjssu+El/60qe6l2J4qsydR3VD6Z5/9X+le87VqOJKYAEADRpBYHWilE5cdNE34/77/+zopRjWv/auiB/OHLH6ZTnxBPZR3wAAGpIcWJ1417v+Iy688Or4+Me/Flu2vD8mJqa751zNhs6PSveyDE+WwX8jUGABAC2WGFhzV2ifnp6IAwdOj0ceOTf+5p2bT4ydB+YdyRJYAMAykxZYp556MJ5++o/i+uv/Os4/f3t88IOb48UXf7/7W4HbFgieH5bRnnMlsACAMUkIrE5s3HhNfOELfxePPvq6WLv2ublLMfxn6V449NlS7+8GCiwAYAmrHVgvf/mv48wz9x799+rVL8QPfvCnMT09EQ888CdxyimHjz3nqi2RBQDQkFqBtWrV/rj77nVxyy1/GWvX7jl6/6te9Xjs3PmG2LNnbXzm4n88MW5uLxGHS8QLpfnfHhRYAEDDah/B2rDh32LfvjMWjKyf/fTsiN+ViDsWCJw7S8Q9jmABAMtf7cDqFVnrztkRcXPpntTeK7LGeQMAaMhAgVVKxIc+9K9HI+uCP79xLmTaGlkAAA0ZOLDmH8mK3aX7x5qPj6wXS8TWFsSVwAIAGjRUYK07Z0f3vKpOWTiy7mpBWAksAKBhAwfWMZdi+H6JOLRAZLXpBgDQkIEC65i4Oj6ynirjuxSDwAIAWqB2YC0YV7O3+8p4L8UgsACAFqgVWIvGVdtvAAANqRxYSzquBBYA0KBKgbXk40pgAQAN6lseyyKuBBYA0KBKR7CW+m3dOTsa2JQAAF3LPrCOHoEDAGjIsg6sY368CQDQkGUbWCecOwYA0JBlGVgLnpgPANCQZRdYPX/rEQCgIcsqsBa9pAQAQEOWZGCdddau+Oxn/yGuuOLTsW7djliz5vn+1+sCAGjIEgusTlxyyddj377VsWvXK+PnPz879u59Sdx/z9sivr1IXAksAKBBSyiwOrFx4zUxNbUqtmz5QLzsZc/GypVTce21G7qr8ZDAAgDaYckE1ooVh+Lqqy+M3bv/ME477UBMTh6Jz3zqiu4qPDUTUTcILABg/JZMYJ166sHYufONsWnTxjjllMPxucu/GHGkRDxRIq4vEdfNhNZtAgsAGK+WB1YnJiY6R//79tv/Ip5/fk184hNf6Q79ibmA6mwtEZ0S8T2BBQCMV6sD68Mf/kZcc83fxurVL0QpEa9//SPx29/+Xjeknpw5crW5dE9w310i9peIrQILABiv1gbWGWfsiy9/+bLodEps3/5XMTl5JN557h0RPy7dYf93ibinRDxYIn49c/Tqnh5xJbAAgAa1MLA6cfHF34gHH3xrPPfcS2LfvtUxPT0Rt20/b+6I1c6Zo1WdmduLJeLeReJKYAEADWpdYG3YcG1MTa2KrVvfF+vX74g3v/nH8c2vXtQ9oX3+jwW3lYi7S8Rdpf81sAQWANCgVgXWaacdiFtvfXc89tir46Uv3ROlzPvzNz8q3eE+XSGmBBYAMEYtC6yDsWvXWXHVVZdGKZ1j//zNt0rEb2Yia/6RLIEFALRMqwKrlE48/PAb4qGH3hTn/fFtJ0bSk6V7vtXsSe4CCwBooZYFVsTnP//3cejQioiHZ45azQbSt2fi6qES8WiJOFKic5PAAgDaZ6yBNTl5JE4//XcxOXnk6H3vfsutEY+ViOkS8YsScUfpXn7hf0rEwRLxnRJxe4k4XCLuF1gAQPuMLbAuvfSquPfet0enU2Lv3jNj06aN8d633TQXRI+X7vCmS/dSDFNl7lIM35u5//sCCwBonzEEVicuueTrMTW1KrZtuyA2btwUl19+RUxNrYx4psxdif26EnFzibhvJqRunLl/a+lONzXvPoEFALRIw4HVifPOuy0iStx003tjxYrDUUonPvnJf+oOpVMi9pSIGxYIpG/Ni6uDJeK7NeJKYAEADWo0sCYmpuPKKz8a99339liz5vmYnDwSn7v8i92LiD4x8yPAQ+XYI1mzt38vEc+WiL0DxJXAAgAa1PARrIjXvOYX8ZGP/HN0/yTOv3SH8MS8EDpQuvf9phz7W4SbS8Qtpf71rwQWANCwxgNr9keF6163Izqz17WaDak7S/fk9Z+U/n9bUGABAC014sDqxPr1d8UFF2yLs8/+WaxcORWlzPz5m+tmftz3eJm7ztXuEvFcidiSHFcCCwBo0EgD62MfuzL2718VnU6J/ftXxc03vyfece6dEZtLdDaXiF+V7hD+q0Tnf0v3JPc6l14QWABAC40ssF7xil/FM8/8QWzffn5cdtlX4ktf+lT3UgxPlbnzqG4o3fOv/q90z7kaVVwJLACgQSMIrE6U0omLLvpm3H//nx29FMP6194V8cOZI1a/LCeewD7qGwBAQ5IDqxPvetd/xIUXXh0f//jXYsuW98fExHT3nKvZ0PlR6V6W4cky+G8ECiwAoMUSA2vuCu3T0xNx4MDp8cgj58bfvHPzibHzwLwjWQILAFhm0gLr1FMPxtNP/1Fcf/1fx/nnb48PfnBzvPji73d/K3DbAsHzwzLac64EFgAwJgmB1YmNG6+JL3zh7+LRR18Xa9c+N3cphv8s3QuHPlvq/d1AgQUALGG1A+vlL/91nHnm3qP/Xr36hfjBD/40pqcn4oEH/iROOeXwsedctSWyAAAaUiuwVq3aH3ffvS5uueUvY+3aPUfvf9WrHo+dO98Qe/asjc9c/I8nxs3tJeJwiXihNP/bgwILAGhY7SNYGzb8W+zbd8aCkfWzn54d8bsScccCgXNnibjHESwAYPmrHVi9ImvdOTsibi7dk9p7RdY4bwAADRkosEqJ+NCH/vVoZF3w5zfOhUxbIwsAoCEDB9b8I1mxu3T/WPPxkfViidjagrgSWABAg4YKrHXn7OieV9UpC0fWXS0IK4EFADRs4MA65lIM3y8RhxaIrDbdAAAaMlBgHRNXx0fWU2V8l2IQWABAC9QOrAXjavZ2XxnvpRgEFgDQArUCa9G4avsNAKAhlQNrSceVwAIAGlQpsJZ8XAksAKBBfctjWcSVwAIAGqQ8AACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCXfUFR4AAASaSURBVCwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI9v/NeAG0AATMwwAAAABJRU5ErkJggg== 13607 [task 2017-11-25T22:55:00.409Z] 22:55:00 INFO - REFTEST IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3de4ylZWHH8eecmcUsoqQY29jYGGIMsVVJbU1sNfFWtN5rxVi0oCm6Ypv6V5MGNRqt8V8T65XKXtllh+UmKAreQQui0ZhYqxRFRKkrAq5Cd3dmzvvrH+/Z3bOzOzPnzDxz5uzy+SQnzM55z3s51y/P+857SgAAqKqs9woAAJxsBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBNZJohQP5SHuCwAGrcfnwlBLLKUse1mTlfNBObTF7qthH6flbn8iWYv1Xc/74ER8DFZqlOfrSt6D1mraE9kjZTs5+Yz6el5qHmvxOhg6sEb5dy1e9KszyuO00useCSbhA2i9lz8Owz5fF3s8hn2cRrkvHwn3e/LI2U5OPqt57q51ywisk1itD5JH8uNwaNvX+z5Y7+WPw2r/T3TYeQisYz1StpOTT83Aqm1Fc19upYYZcltuiH/UXZGD1690N8NK1nPYaWoub7HbrHQ+y93Xte/XYR+rlQzdrvYxWmq+K1HzeXG82yw2r8V+t9jju9rHajm1XkPDLGfYadb6fl943Vreb6Ms53jTjbJe43p919y2YZ5zK93+UR73xV57S237qOtd6/4fZVvXcj2Xm+ewr/nltn+l7zlLLndFN1rmDljud8M+oVb6ZrTSdVrJeq5mW1Z6vxzPsNs9yu2XWofV3NdLzXPU+Sy2vitZ/ijzXOl6rOR5Mcrvlpq21nN+sd8tZrXbM6xR1mec9/uwt19uulE+jEfdxrV4LEd5fY9721b6uKz2/b7282nU+23UeQ8z3+NZ78d3mHVa7Hc1VQ2s5T6kR739qBs/7PKHnWaY9VzNNCuZ1zDzGeV2w0xb4/FazbTDXjfqstYysGo+Lxabdi0+6Fe7nFGmWe3r/dBtRnmTrbFOa/F6GNZ6rs84Hsthl7ke7z3Drs+o81/t+9Oo7/3j+Dwa9nbjfC09ogJrtS/o1Sx/2GmGWc9hpxnmulE/KJZbhsA68u/VvpBqPheHmUZgjW41b7SjrJPAWt2yRpn/JL33HLq+1vv9ctNPQmDVDJD1fi2dFIG11GWpaUdZzqjrdbzrhlnHUdZzlHms9n4ZZvsW+90ot1/uumHv08W2c6nlDTufYdd3pffrMOu70ukFVt3AGtc61X49DPv7pd5fVrotw677Sm6/lEnatpW8v6xknRe7btTX8Wrut6XmUfvzaL0e31Gmq/Ges+QyV3SjVbzhLjXP1b7hDvsgrPYDY9g3j1Hmu5rlLbaMUV8Qo163mserxgtmNbdfyRvGWqzDsNfVeGOu+bulfj/sNKt9va/XOq3F62Gly6i1PuN8LNd722p/qI7yfr8W21gj8oY17OffsL8XWMe70QrecFcSOOsdWMNMt5ppVrpew04zSYFV6wWzkvuixv8Q1HourvSNWGCtbJ5rsU7j+FCYxA+pR0Jg1XhcTuTAqvF5tN6P7yjTnVCBtdh1x7uzhp1u2OWOOt+V/m7h/Fa6LcOu12K3HWaa1QTESl9oq/3QXs1tFptuNffrSqZdaj2Gme8ov1vNto4jsGos53jTjev247rfay9jlGUvNt/VznOpeaz3ttV8XI5321HXZS0/o45nLT+PJuHxXc08axp57oc2aJgX0FLTLZxmmOmGWbeVLn/wd6Ou50q2ZbX3y2K3GWVZo85nuWmHXfaw61ZrG443r5XMb5TbjbINo9xfw9x2mMdwNb9bbp1Wu42jPudXsh6jrtMw61tr20ZZRq3X57gfy3Fv20q2YzHDPvdWuy7Lbfsw99tS27WSbVjNc2Q9n7vHm+dKH/9RrG2+jdla3UkAnJgm6XNhktaFtXdSPdqevACPTIuNeEySSVsf1tZJ82iv9VAfAJNtHLt9VmpS14u145EGAKhMYAEAVCawAAAqe0QGln3grITnDQDDGvoTY7nzZJxITrT1nTTDnl9ltfMf5fwoNS11Dpb1WO6kmPT1eyTymDh4nMm16sBa7jpOPmsdWEvNY1xvpOv1nJ7019Kkr98jkcdEaDKZBBYjW+9RJIG1fiZ9/R6JPCYCi8k01DPyeKfLX3jK+eNNN+z8Frt+uWlXcwr/YbZx2HmtdvtHeWOotcyVznuY58Kwy1jJKNhSyxt1XZZap8Wea8vddqnbDfsYDLuOi12/1O+GXY/Fphv1dbaS58Ni27HYbYed72K3HebxG/V1WnMdlpvnMOtV631jlPtglGWOspxhnjMwCaqNYI3ygTTs7443z1Hns9j6rmT5i81r1O0fdtqayxzmd8vNe9j1rbkew/x+uQ/apYzy2K/0eVPjtVBrfrVfk8ezktfzcssY5fFd7WtxlHkvZi3eD1bzmAxOOynvVStZznJxBpOkWmAN8/tR5rGS5S133ajLGiWwhvn9ciEw6hvlauYzynqPOk2Nbanx+5XE1VLXrfR5U+t5P+r6LfzdsOtR+/VVYxmreY2OOs/VXFdrHWq9Hy6cbpLeq0ZdTq11gnERWFndi3PS3rSW+z+8hZeVrMdK3zRrrMekBdZqgmOxeQ5jJREzzsBa6/t+lPUZdZ6rua7WOggsgcWJb+yBNeyH7DAvymE/rIdZ31HnsfB2w/x+PQKrVsCMMs0o217798OGwnLXD/thOMzzZrWBNcrzfLnAGuZ1U/PDfKnHa5RljHLfDnt/rfV7zGrWYdT3w2FM2nvV4PXDPi9rrBOMy8SMYNVYXo15D06zmukE1ngDa+F1wz52o1y30ufDagJr1GhbavtHeX6t5Lrl5rmaiFvte0yNyBtWjXV4pAXWauclsJhEExNYtd5QVvLBOuqH2MJpJulNS2CtbiRzqetW+rwZZ2Adum41QVH79VVjGZMUWMO8Pla7DgLryPW11wnGZcWBNcyb5bBvKKO++a30NotNt5IPpGFuW+PNbKllDjOfcQXWcm+E4wisYdd/sfUeZtlrOd2wt13NfbDa1+RqXxMrXcZqnu8rfX3XfE8YdR1qPybLrcNav1etdL1GWafV3j9Q20jPvOM9kZf7sF/qSb/S60edbpj1WjivUd5Il7pPhtm+Ud8ABm8zzHyOt7yVrPdqt3216zHKto7iePNdyfN8uQ+KUZ4Xo67f8ZY3yvYOO91KXxvDPjeXWv9h12fUx3PU+2AYtZ5TK70Pht2OUZ6TNe6DYadd6fNysfsS1oNn3gnIm8bSJu2+Gff6TNr2j+pEX3+ARGCdkATW0Rb+H/mkGcc6LTf6cyI5GbYBwDvZCWalw/Uns0m+T8a1XpO6/aM6WbYDwLsYAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlywZWKSfapelfkqnubEpJ/upPbhzHfQkAkOSkC6wjcTXdj6uXn31d9u86ZRz3JQBAkpMqsAbj6mBKSV76jM9kfqaT3uXLbiYAQDUnSWAdG1cve8b1mZvpprerk972zjjuSwCAJCdcYDULgmrwskhc7egkO41gAQDjc4IE1pGIKqVJp9PL9PRspqdnM9WdTynN4WOujhm52lnSXCawAIDxOQECqznq506nd8w0nTK/dFwZwQIAxugECKzBEaw2rs466wd597vflw996B159auuTuk2+eunfTZzM1PHjyuBBQCM0QkQWIfiqh2letvbPpb9+0/J3r2Pz0/v+qM8+JvH5pavPCf7d5+SXF6OH1cCCwAYowkNrKMPZu9251JK8prXXJmkZOfOv8vjHverlJLs2vn6JCXzt3fSu7SNqWYwrAQWADBmExhYzYKfm3S78+l05jMzc27uvfcPsmHDgZSS/Ou/fDBzvanM3jGV3mWdZHdJs6M/emUECwBYJxMYWAtHsHoppcn09IHcffcTs2fP36aU5N0Xvy9JSfPjflDtKsldJbmpJNuMYAEA62eCAmvwnFa9Y0aypqcP5nM3vDh33/3EvPPif0tSMn9nN73tJdla0lxd0qQk/1nSbBZYAMD6mZDAGoyp3uFjrjZt+ni2bjs/Gzc+nFKSpzz1h9m377QkJb0fdtJsLcnmkubykubnJc3+klxV0mwXWADA+pmQwBocwWpPxbBhw4FccsmFSUqumHltSklecfanc+B7GzLXdNP8oiRfKck3SnJfaTfl5oHdg47BAgDWyYQE1qGwauPqTW/anO9972m5554/zAP3n56k5IarXpLffurRyY6S3ndLcrDdJdikJA+XNN84smvwmFM0CCwAYIwmILDakatOp90teO65M2maks997sV5wfO/lKc89UfZ/bHXJSlp7urH07aS5pqSfK0/irWnHH1gu8ACANbRBARWDo9enXLKgXzpS8/PXXc9KY897cGUkrzwrC/k1584Iwe/OZ1eOsld7TFWzbbSHoO1tT01wzFBJbAAgHUyAYF15LirjRsfyi/3Pj5bNl+QUpJXPvPa9PaUZHdJb1tJ7i3tKv+ktKdlOHR6hqVGrgQWADBm6xBYzVGX9sube+l0epmens0Pvn9Wvn7rX+Q1z7oys5dPpbe7tCcR3d7fRfhwSZqS5u7+yNVli5xYVGABAOtkjIF1KKran7vd+aOun54+mG5nPm//53/PXLppvluS7SW9LZ1kS0kzU5IDJc23S/Ld/gHu15f2eKzldg8KLABgjNZlBKsdtWr/fcYZ9+XUU393+N+ve/buzP24m/l00vvvktxY0nylJL8syXxJri3JDf2fby/JlgW7CQUWALDOxhRYC3cJJhdcsCW33/7nmZ2dyn33nZEPffgdueBFW3Jw63SaHe2JRHPoNAxpR69ya0nzHyX5WknTlDRfL+1B7kawAIAJMuZdhG1cbdr08STtqRje+MbteefFH2hX5f6SZk//rwN3ljSfaaMqXy/JNSXZXJLdJdnbD66r292Iyx5/JbAAgDEa2wjWoa+/OeeczycpmZk5N1NT7e/+6R8/nIOzGzKXbnq/aiOq2d6Pp6393YCX9YNqb0l6JfnqCHElsACAMVrDwGoG/ttLpzOfUpps2/b3ueOOJ+fR/eOu3vuu9yQp6d1R0ru5tH8heG9JZkqyo7THV+0o7bFXvyzJvn5cHYquYeJKYAEAY7RGgXX0Xwt2u/OZmppNKU2e85ybc+GFl6SU5KJ/+GiSkrkfddPb2sZUs6+0q/WrfkANXj7bD6/lztousACAdbQGgXUkrgb/WnDh5eVPvy6zv5jO3G+76e0syaUlzRdLsr8k/1WS2/q7AAdDalt/NGvUuBJYAMAYVQ6swd2C7c/Pe96Xc955l+Xss7+Tqe5sSmnP0D53ZTe9/SW9O0vyqZJmV0nu7Y9gXbZgF+DxQmqUuBJYAMAYVQyswZGr9iSiF1300f4i2sul296cl//pdeldXtLbVTL/0/6pGP6npPl1O01zy8AuwMGIWuxngQUATJgKgXX0GdqnpmYzNTWXM8+8Mw89tDHXX/+yvG3Tx/KBD16c2WYquae0gbWlk8yUNHeW5MGS3F/a0zFsWWVICSwAYJ1VHcEaPObqrW/9RL7znbNzyvSBlJK84MlfzMO3b2xPxfCT0n634PaS7Gi/U7DZ0f/dWsSVwAIAxmgVgdUsuLRx9ZKX3JBNmz6R97//Xfnkx9+SUpJX/dk16e3ppNlW0ru9vyvwxwMhtfDLmmvHlcACAMZohMA69gD29tLL1NRsOp1eLrzwkgwec/WNbz47b3npJzO3fSrNTElvR7sLsLmtP83P2hGsNY8rgQUAjNGQgXUkqDqdXn9XYHNUbE1NzWbv3sfl2mtfmVe/6pq84fwd+c2+xyQPlTRXliPnudpZ2q+8+XZJvlWOPqfVWl4AAMZkxF2Eg6E13x+9mssb3rAjH/nI2/Otbz0zZ/zefSklee6TvpqHvnhqZg9OpXdfSfaUNIMxta20X4OzcPegwAIATnAjjGC1x1g94Qk/z+mnP3A4sk499Xe57bZnJSn5wo0vTCnJK5756TRXtQex924qyVz/NAxXlqNPHjrOCwDAmAwZWO0uwUc9an9uueUvc9NNL8ppp+07fN2ZZ96ZH3z/rDyw7/S856L35uDmDWl2l/T6JwxtbiztFzQ/WI49zmoco1cCCwAYo5FGsLrd+bz5zZcmKbn++pflMY/Zd3iX4e8/4X9zxw+fnKQNqmZzGzbNzjay8uWS3FqOPajdCBYAcJIZMrCO3k143nmX5VBkHdpdeM4ffz4PXHF6Dt4/nd6BTnJTSTP4dTfby5FjruwiBABOYiOMYLX/7XbnUkpy/vlbk5TMXHVuXv/cXeldXpJdJb2rSvKbkuZgSb7QD6uFUWUECwA4iY04gnV0ZP3Na6/K/x18VLK3tMdcbW+Dqrm6JPtK8nBJrujvFlzPuBJYAMAYjRhYbWRNTx1MKcmLnnJTHrp5Y+Yyld7PSnJ5/ytvtpXkupJ8tYzvNAwCCwCYEEMG1kBcddu4etkzrk/viv7X39xckqakuacku/tBdSi0JiGuBBYAMEZDBNbx42puppveroGvv7m5tLO7e8EuwfXcLSiwAIB1MGRgHS+uOult7yQ7B77+5taS3FaOPrB9EuJKYAEAYzREYCXT3dnF42owoLaVydotKLAAgHWwbHlM9ePqpc/4zOJxdbygmrTIAgAYk6FGsF5+9nWZPxRXO44zcrUwqCYtrgQWADBGy5bHOU//fA7u3JDe5eX4uwVPlAsAwJgMNYJ1Yl2O/NXjod2bf/UnN47jvgQASHLSBdbgKSVmD+/e3L/rlHHclwAASU6qwDr2fF0vfcZnMj/Tab8nEQBgTE6SwFrqZKj9v3oEABiTEyywmoH/LrwsElf9v3oEABiXEySwjkRUKU06nV6mp2czPT2bqe58SmmWPxkqAMCYnACB1Rz1c6fTO2aaTplfOq6MYAEAY3QCBNbgCFYbV2ed9YO8+93vy4c+9I68+lVXp3Sb/PXTPpu5manFv8YHAGBMToDAOhRX7SjV2972sezff0r27n18fnrXH+XB3zw2t3zlOdm/+5RkqZOhAgCMyYQG1tEHs3e7cyklec1rrkxSsnPn3+Vxj/tVSkl27Xx9kpL52zvpXdrGVONM7gDAOprAwGoW/Nyk251PpzOfmZlzc++9f5ANGw6klORf/+WDmetNZfaOqfQu6yS7S5od/dErI1gAwDqZwMBaOILVSylNpqcP5O67n5g9e/42pSTvvvh9SUqaH/eDaldJ7irJTSXZZgQLAFg/ExRYg+e06h0zkjU9fTCfu+HFufvuJ+adF/9bkpL5O7vpbS/J1pLm6pImJfnPkmazwAIA1s+EBNZgTPUOH3O1adPHs3Xb+dm48eGUkjzlqT/Mvn2nJSnp/bCTZmtJNpc0l5c0Py9p9pfkqpJmu8ACANbPhATW4AhWeyqGDRsO5JJLLkxScsXMa1NK8oqzP50D39uQuaab5hcl+UpJvlGS+0q7KTcP7B50DBYAsE4mJLAOhVUbV2960+Z873tPyz33/GEeuP/0JCU3XPWS/PZTj052lPS+W5KD7S7BJiV5uKT5xpFdg8ecokFgAQBjNAGB1Y5cdTrtbsFzz51J05R87nMvzgue/6U85ak/yu6PvS5JSXNXP562lTTXlORr/VGsPeXoA9sFFgCwjiYgsHJ49OqUUw7kS196fu6660l57GkPppTkhWd9Ib/+xBk5+M3p9NJJ7mqPsWq2lfYYrK3tqRmOCSqBBQCskwkIrCPHXW3c+FB+uffx2bL5gpSSvPKZ16a3pyS7S3rbSnJvaVf5J6U9LcOh0zMsNXIlsACAMVuHwGqOurRf3txLp9PL9PRsfvD9s/L1W/8ir3nWlZm9fCq93aU9iej2/i7Ch0vSlDR390euLlvkxKICCwBYJ2MMrENR1f7c7c4fdf309MF0O/N5+z//e+bSTfPdkmwv6W3pJFtKmpmSHChpvl2S7/YPcL++tMdjLbd7UGABAGO0LiNY7ahV++8zzrgvp576u8P/ft2zd2fux93Mp5Pef5fkxpLmKyX5ZUnmS3JtSW7o/3x7SbYs2E0osACAdTamwFq4SzC54IItuf32P8/s7FTuu++MfOjD78gFL9qSg1un0+xoTySaQ6dhSDt6lVtLmv8oyddKmqak+XppD3I3ggUATJAx7yJs42rTpo8naU/F8MY3bs87L/5Auyr3lzR7+n8duLOk+UwbVfl6Sa4pyeaS7C7J3n5wXd3uRlz2+CuBBQCM0dhGsA59/c0553w+ScnMzLmZmmp/90//+OEcnN2QuXTT+1UbUc32fjxt7e8GvKwfVHtL0ivJV0eIK4EFAIzRGgZWM/DfXjqd+ZTSZNu2v88ddzw5j+4fd/Xed70nSUnvjpLezaX9C8F7SzJTkh2lPb5qR2mPvfplSfb14+pQdA0TVwILABijNQqso/9asNudz9TUbEpp8pzn3JwLL7wkpSQX/cNHk5TM/aib3tY2ppp9pV2tX/UDavDy2X54LXfWdoEFAKyjNQisI3E1+NeCCy8vf/p1mf3FdOZ+201vZ0kuLWm+WJL9JfmvktzW3wU4GFLb+qNZo8aVwAIAxqhyYA3uFmx/ft7zvpzzzrssZ5/9nUx1Z1NKe4b2uSu76e0v6d1Zkk+VNLtKcm9/BOuyBbsAjxdSo8SVwAIAxqhiYA2OXLUnEb3ooo/2F9FeLt325rz8T69L7/KS3q6S+Z/2T8XwPyXNr9tpmlsGdgEORtRiPwssAGDCVAiso8/QPjU1m6mpuZx55p156KGNuf76l+Vtmz6WD3zw4sw2U8k9pQ2sLZ1kpqS5syQPluT+0p6OYcsqQ0pgAQDrrOoI1uAxV2996yfyne+cnVOmD6SU5AVP/mIevn1jeyqGn5T2uwW3l2RH+52CzY7+79YirgQWADBGqwisZsGljauXvOSGbNr0ibz//e/KJz/+lpSSvOrPrklvTyfNtpLe7f1dgT8eCKmFX9ZcO64EFgAwRiME1rEHsLeXXqamZtPp9HLhhZdk8Jirb3zz2XnLSz+Zue1TaWZKejvaXYDNbf1pftaOYK15XAksAGCMhgysI0HV6fT6uwKbo2Jramo2e/c+Ltde+8q8+lXX5A3n78hv9j0meaikubIcOc/VztJ+5c23S/KtcvQ5rdbyAgAwJiPuIhwMrfn+6NVc3vCGHfnIR96eb33rmTnj9+5LKclzn/TVPPTFUzN7cCq9+0qyp6QZjKltpf0anIW7BwUWAHCCG2EEqz3G6glP+HlOP/2Bw5F16qm/y223PStJyRdufGFKSV7xzE+nuao9iL13U0nm+qdhuLIcffLQcV4AAMZkyMBqdwk+6lH7c8stf5mbbnpRTjtt3+Hrzjzzzvzg+2flgX2n5z0XvTcHN29Is7uk1z9haHNjab+g+cFy7HFW4xi9ElgAwBiNNILV7c7nzW++NEnJ9de/LI95zL7Duwx//wn/mzt++OQkbVA1m9uwaXa2kZUvl+TWcuxB7UawAICTzJCBdfRuwvPOuyyHIuvQ7sJz/vjzeeCK03Pw/un0DnSSm0qawa+72V6OHHNlFyEAcBIbYQSr/W+3O5dSkvPP35qkZOaqc/P65+5K7/KS7CrpXVWS35Q0B0vyhX5YLYwqI1gAwElsxBGsoyPrb157Vf7v4KOSvaU95mp7G1TN1SXZV5KHS3JFf7fgesaVwAIAxmjEwGoja3rqYEpJXvSUm/LQzRszl6n0flaSy/tfebOtJNeV5KtlfKdhEFgAwIQYMrAG4qrbxtXLnnF9elf0v/7m5pI0Jc09JdndD6pDoTUJcSWwAIAxGiKwjh9XczPd9HYNfP3NzaWd3d0Ldgmu525BgQUArIMhA+t4cdVJb3sn2Tnw9Te3luS2cvSB7ZMQVwILABijIQIrme7OLh5XgwG1rUzWbkGBBQCsg2XLY6ofVy99xmcWj6vjBdWkRRYAwJgMNYL18rOvy/yhuNpxnJGrhUE1aXElsACAMVq2PM55+udzcOeG9C4vx98teKJcAADGZKgRrBPrcuSvHg/t3vyrP7lxHPclAECSky6wBk8pMXt49+b+XaeM474EAEhyUgXWsefreukzPpP5mU77PYkAAGNykgTWUidD7f/VIwDAmJxggdUM/HfhZZG46v/VIwDAuJwggXUkokpp0un0Mj09m+np2Ux151NKs/zJUAEAxuQECKzmqJ87nd4x03TK/NJxZQQLABijEyCwBkew2rg666wf5N3vfl8+9KF35NWvujql2+Svn/bZzM1MLf41PgAAY3ICBNahuGpHqd72to9l//5Tsnfv4/PTu/4oD/7msbnlK8/J/t2nJEudDBUAYEwmNLCOPpi9251LKclrXnNlkpKdO/8uj3vcr1JKsmvn65OUzN/eSe/SNqYaZ3IHANbRBAZWs+DnJt3ufDqd+czMnJt77/2DbNhwIKUk//ovH8xcbyqzd0yld1kn2V3S7OiPXhnBAgDWyQQG1sIRrF5KaTI9fSB33/3E7NnztykleffF70tS0vy4H1S7SnJXSW4qyTYjWADA+pmgwBo8p1XvmJGs6emD+dwNL87ddz8x77z435KUzN/ZTW97SbaWNFeXNCnJf5Y0mwUWALB+JiSwBmOqd/iYq02bPp6t287Pxo0Pp5TkKU/9YfbtOy1JSe+HnTRbS7K5pLm8pPl5SbO/JFeVNNsFFgCwfiYksAZHsNpTMWzYcCCXXHJhkpIrZl6bUpJXnP3pHPjehsw13TS/KMlXSvKNktxX2k25eWD3oGOwAIB1MiGBdSis2rh605s253vfe1ruuecP88D9pycpueGql+S3n3p0sqOk992SHGx3CTYpycMlzTeO7Bo85hQNAgsAGKMJCKx25KrTaXcLnnvuTJqm5HOfe3Fe8Pwv5SlP/VF2f+x1SUqau/rxtK2kuaYkX+uPYlXoaWwAAA4zSURBVO0pRx/YLrAAgHU0AYGVw6NXp5xyIF/60vNz111PymNPezClJC886wv59SfOyMFvTqeXTnJXe4xVs620x2BtbU/NcExQCSwAYJ1MQGAdOe5q48aH8su9j8+WzReklOSVz7w2vT0l2V3S21aSe0u7yj8p7WkZDp2eYamRK4EFAIzZOgRWc9Sl/fLmXjqdXqanZ/OD75+Vr9/6F3nNs67M7OVT6e0u7UlEt/d3ET5ckqakubs/cnXZIicWFVgAwDoZY2Adiqr25253/qjrp6cPptuZz9v/+d8zl26a75Zke0lvSyfZUtLMlORASfPtkny3f4D79aU9Hmu53YMCCwAYo3UZwWpHrdp/n3HGfTn11N8d/vfrnr07cz/uZj6d9P67JDeWNF8pyS9LMl+Sa0tyQ//n20uyZcFuQoEFAKyzMQXWwl2CyQUXbMntt/95Zmenct99Z+RDH35HLnjRlhzcOp1mR3si0Rw6DUPa0avcWtL8R0m+VtI0Jc3XS3uQuxEsAGCCjHkXYRtXmzZ9PEl7KoY3vnF73nnxB9pVub+k2dP/68CdJc1n2qjK10tyTUk2l2R3Sfb2g+vqdjfissdfCSwAYIzGNoJ16Otvzjnn80lKZmbOzdRU+7t/+scP5+Dshsylm96v2ohqtvfjaWt/N+Bl/aDaW5JeSb46QlwJLABgjNYwsJqB//bS6cynlCbbtv197rjjyXl0/7ir977rPUlKeneU9G4u7V8I3luSmZLsKO3xVTtKe+zVL0uyrx9Xh6JrmLgSWADAGK1RYB3914Ld7nympmZTSpPnPOfmXHjhJSkluegfPpqkZO5H3fS2tjHV7Cvtav2qH1CDl8/2w2u5s7YLLABgHa1BYB2Jq8G/Flx4efnTr8vsL6Yz99tuejtLcmlJ88WS7C/Jf5Xktv4uwMGQ2tYfzRo1rgQWADBGlQNrcLdg+/PznvflnHfeZTn77O9kqjubUtoztM9d2U1vf0nvzpJ8qqTZVZJ7+yNYly3YBXi8kBolrgQWADBGFQNrcOSqPYnoRRd9tL+I9nLptjfn5X96XXqXl/R2lcz/tH8qhv8paX7dTtPcMrALcDCiFvtZYAEAE6ZCYB19hvapqdlMTc3lzDPvzEMPbcz1178sb9v0sXzggxdntplK7iltYG3pJDMlzZ0lebAk95f2dAxbVhlSAgsAWGdVR7AGj7l661s/ke985+ycMn0gpSQvePIX8/DtG9tTMfyktN8tuL0kO9rvFGx29H+3FnElsACAMVpFYDULLm1cveQlN2TTpk/k/e9/Vz758beklORVf3ZNens6abaV9G7v7wr88UBILfyy5tpxJbAAgDEaIbCOPYC9vfQyNTWbTqeXCy+8JIPHXH3jm8/OW176ycxtn0ozU9Lb0e4CbG7rT/OzdgRrzeNKYAEAYzRkYB0Jqk6n198V2BwVW1NTs9m793G59tpX5tWvuiZvOH9HfrPvMclDJc2V5ch5rnaW9itvvl2Sb5Wjz2m1lhcAgDEZcRfhYGjN90ev5vKGN+zIRz7y9nzrW8/MGb93X0pJnvukr+ahL56a2YNT6d1Xkj0lzWBMbSvt1+As3D0osACAE9wII1jtMVZPeMLPc/rpDxyOrFNP/V1uu+1ZSUq+cOMLU0ryimd+Os1V7UHsvZtKMtc/DcOV5eiTh47zAgAwJkMGVrtL8FGP2p9bbvnL3HTTi3LaafsOX3fmmXfmB98/Kw/sOz3vuei9Obh5Q5rdJb3+CUObG0v7Bc0PlmOPsxrH6JXAAgDGaKQRrG53Pm9+86VJSq6//mV5zGP2Hd5l+PtP+N/c8cMnJ2mDqtnchk2zs42sfLkkt5ZjD2o3ggUAnGSGDKyjdxOed95lORRZh3YXnvPHn88DV5yeg/dPp3egk9xU0gx+3c32cuSYK7sIAYCT2AgjWO1/u925lJKcf/7WJCUzV52b1z93V3qXl2RXSe+qkvympDlYki/0w2phVBnBAgBOYiOOYB0dWX/z2qvyfwcflewt7TFX29ugaq4uyb6SPFySK/q7BdczrgQWADBGIwZWG1nTUwdTSvKip9yUh27emLlMpfezklze/8qbbSW5riRfLeM7DYPAAgAmxJCBNRBX3TauXvaM69O7ov/1NzeXpClp7inJ7n5QHQqtSYgrgQUAjNEQgXX8uJqb6aa3a+Drb24u7ezuXrBLcD13CwosAGAdDBlYx4urTnrbO8nOga+/ubUkt5WjD2yfhLgSWADAGA0RWMl0d3bxuBoMqG1lsnYLCiwAYB0sWx5T/bh66TM+s3hcHS+oJi2yAADGZKgRrJeffV3mD8XVjuOMXC0MqkmLK4EFAIzRsuVxztM/n4M7N6R3eTn+bsET5QIAMCbKAwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBU9v8G7vocVHkgKQAAAABJRU5ErkJggg== 13608 [task 2017-11-25T22:55:00.409Z] 22:55:00 INFO - TEST-INFO took 369ms 13609 [task 2017-11-25T22:55:00.553Z] 22:55:00 INFO - PID 5511 | --DOCSHELL 0xc72c1c00 == 3 [pid = 5511] [id = {cb9fef26-5b86-4dfc-86dc-514e6e4f09bd}] 13610 [task 2017-11-25T22:55:00.553Z] 22:55:00 INFO - PID 5511 | 13611 [task 2017-11-25T22:55:00.554Z] 22:55:00 INFO - PID 5511 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost 13612 [task 2017-11-25T22:55:00.554Z] 22:55:00 INFO - PID 5511 | 13613 [task 2017-11-25T22:55:00.556Z] 22:55:00 INFO - PID 5511 | [Child 5618, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13614 [task 2017-11-25T22:55:00.657Z] 22:55:00 INFO - PID 5511 | [Parent 5511, Main Thread] WARNING: NS_ENSURE_TRUE(weak) failed: file /builds/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 339 13615 [task 2017-11-25T22:55:00.941Z] 22:55:00 INFO - PID 5511 | [Parent 5511, Main Thread] WARNING: NS_ENSURE_TRUE(weak) failed: file /builds/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 339 13616 [task 2017-11-25T22:55:00.979Z] 22:55:00 INFO - PID 5511 | [Child 5561, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13617 [task 2017-11-25T22:55:01.156Z] 22:55:01 INFO - PID 5511 | --DOMWINDOW == 3 (0xe7933800) [pid = 5561] [serial = 3] [outer = (nil)] [url = about:blank] 13618 [task 2017-11-25T22:55:01.158Z] 22:55:01 INFO - PID 5511 | --DOMWINDOW == 2 (0xe792b400) [pid = 5561] [serial = 2] [outer = (nil)] [url = about:blank] 13619 [task 2017-11-25T22:55:01.158Z] 22:55:01 INFO - PID 5511 | --DOCSHELL 0xe676a000 == 0 [pid = 5561] [id = {d6f15acc-d59d-4955-9e3f-47866903ea62}]
Whiteboard: [stockwell unknown] → [stockwell needswork]
Comment hidden (Intermittent Failures Robot) |
Comment 32•7 years ago
|
||
this bug also appears to be blocking bug 1383233 (13 failures last week), bug 1383676 (17 failures last week), bug 1382969 (23 failures last week), bug 1383230 (9 failures last week), bug 1383061 (46 failures last week). In total that is 141 failures that we believe are related to this bug. :milan, can we get some traction on here now that 57 has been released?
Comment hidden (Intermittent Failures Robot) |
Assignee | ||
Updated•7 years ago
|
Flags: needinfo?(milan) → needinfo?(aosmond)
Assignee | ||
Comment 34•7 years ago
|
||
I have been able to reproduce this locally with some effort. Will continue to investigate.
Assignee: nobody → aosmond
Assignee | ||
Updated•7 years ago
|
Flags: needinfo?(aosmond)
Priority: -- → P3
Whiteboard: [stockwell needswork:owner] → [gfx-noted][stockwell needswork:owner]
Assignee | ||
Updated•7 years ago
|
Flags: needinfo?(aosmond)
Assignee | ||
Comment 35•7 years ago
|
||
From my reproductions, it appears the imgRequest for the image under test is validated, and purged from the cache. However before it is purged, the layout code checks imgRequestProxy::GetImageStatus which says the image is fully decoded. This messes with the invalidation and causes us to exit nsDOMWindowUtils::UpdateLayerTree without drawing with the downscale-on-decoded size, and instead the native size. imgRequestProxy::GetImageStatus should not return any state if it has a validator, similar to how imgRequestProxy::Clone defers notifications until the validator has completed.
Status: NEW → ASSIGNED
Flags: needinfo?(aosmond)
Assignee | ||
Comment 36•7 years ago
|
||
There is a second issue with imgRequestProxy::StartDecodingWithResult. Callers also use this as a way to indirectly get the state of the image, as it will return true if a Draw call will succeed with either DrawResult::SUCCESS or DrawResult::WRONG_SIZE. This implies LOAD_COMPLETE, SIZE_AVAILABLE, FRAME_COMPLETE, and DECODE_COMPLETE are set. As with imgRequestProxy::GetImageStatus, if we are validating, the result can be misleading if we throw out the cache. As such it should continue to return false, even if it has everything, until validation has completed.
Flags: needinfo?(bugs)
Assignee | ||
Comment 37•7 years ago
|
||
try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=08a094796acfbc2ccfb461712f4cddab359fdbb0 A consequence of this change is that it flashes red as it switches between the reference and the test displays in this test case. This is because when it switches to validating the cache, layout now believes the image to be incomplete and doesn't render any image at all. In theory this could be solved by keeping a previous imgIContainer reference around, so that if we switch from frame complete to frame incomplete, then we render what we used to have. In practice I'm not sure how important this is -- this would only affect new requests created via imgLoader::LoadImage which ended up causing a validate, or requests cloned from *that* request via imgRequestProxy::Clone. Existing imgRequestProxy objects will continue using the old RasterImage, even if the cache gets purged. As such, I believe this is much more likely to be observable when doing specific tests rather than general use.
Assignee | ||
Comment 38•7 years ago
|
||
I seem to be running into a similar root cause as to bug 1413378 by asserting notifications are deferred in GetImageStatus. Not all paths of validation will trip the assert in bug 1413378 so presumably I'm hitting the silent majority. I'll take out the assert for now, and see about cornering the other problem as part of another bug.
Attachment #8934961 -
Attachment is obsolete: true
Assignee | ||
Comment 39•7 years ago
|
||
try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=95fe2908beb9ba4587ab1a2a9ab511d831ab8884
Assignee | ||
Comment 40•7 years ago
|
||
Okay, this requires more thought given the sheer number of failures. I believe what we need is to distinguish between requests which were created before the validator, and requests created after. The former get grandfathered in and don't change state if and only if the validator evicts the cache. The latter don't get any state until the validator completes. The trouble with the attached patch (and why I tripped the assert in the first place) is that it does not distinguish between the two, and so all requests don't get any state anymore until validation completes.
Assignee | ||
Comment 41•7 years ago
|
||
The patch got a lot bigger. At this point, the only tests that should be failing are some intermittent accessiblity mochitests. I need to confirm whether or not I caused it to permafail due to timing, or an actual regression. try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9bf2752358cd3079d5d1cb0f11b69be5a3574dd0
Attachment #8934986 -
Attachment is obsolete: true
Assignee | ||
Comment 42•7 years ago
|
||
accessible/tests/mochitest/elm/test_HTMLSpec.html and accessible/tests/mochitest/tree/test_txtcntr.html are failing very consistently and block landing of this patch.
Depends on: 1408636
Assignee | ||
Comment 43•7 years ago
|
||
(In reply to Andrew Osmond [:aosmond] from comment #42) > accessible/tests/mochitest/elm/test_HTMLSpec.html and > accessible/tests/mochitest/tree/test_txtcntr.html are failing very > consistently and block landing of this patch. It appears that if the image is not displayed at the time of onload, there is an extra text leaf containing a single space. This breaks both tests. If I remove the spaces in the <figure> block of test_HTMLSpec.html, it passes. This is why it failed intermittently before -- it usually (wrong) reused an unvalidated cache, which then expired but only after the test completed. With my changes, it consistently uses a fresh entry, but nothing is decoded, and we see this extra text leaf artifact breaking the test.
Comment hidden (Intermittent Failures Robot) |
Assignee | ||
Comment 45•7 years ago
|
||
Combined with the test fixes in bug 1408636, we should be green: try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=dab24372c1067032d885f891a5ec44bed4411aed
Attachment #8935473 -
Attachment is obsolete: true
Assignee | ||
Comment 46•7 years ago
|
||
Comment on attachment 8936648 [details] [diff] [review] 0001-Bug-1383682-imgRequestProxy-should-avoid-leaking-sta.patch, v4 I'm still figuring out the (new) accessibility test failures, but I think they are all related to the whitespace optimizations, rather than a problem with this patch.
Attachment #8936648 -
Flags: review?(tnikkel)
Comment hidden (Intermittent Failures Robot) |
Comment 48•7 years ago
|
||
Comment on attachment 8936648 [details] [diff] [review] 0001-Bug-1383682-imgRequestProxy-should-avoid-leaking-sta.patch, v4 Review of attachment 8936648 [details] [diff] [review]: ----------------------------------------------------------------- Would it make more sense to split this into two patches, one that splits the two notions of deferring notifications, and one that avoids leaking state via GetImageStatus etc? ::: image/imgRequestProxy.h @@ +116,1 @@ > virtual bool NotificationsDeferred() const override I think it would be better if we worked on the naming here a bit more. It's confusing/surprising to have NotificationsDeferred() to return something other than mDeferNotifications.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 53•7 years ago
|
||
There are 40 failures in the past 7 days on Linux/ Linux x64 debug, linux32-stylo-disabled debug, linux64-ccov opt, and linux64-stylo-disabled debug. Most recent log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=154938634 Relevant part of the log: [task 2018-01-09T05:14:41.488Z] 05:14:41 INFO - PID 5515 | 1515474881482 Marionette INFO Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht 13622 [task 2018-01-09T05:14:41.503Z] 05:14:41 INFO - PID 5515 | ++DOMWINDOW == 83 (0x7f28e6423c00) [pid = 5609] [serial = 83] [outer = 0x7f28f057a660] 13623 [task 2018-01-09T05:14:41.565Z] 05:14:41 INFO - PID 5515 | ++DOMWINDOW == 84 (0x7f28e6425400) [pid = 5609] [serial = 84] [outer = 0x7f28f057a660] 13624 [task 2018-01-09T05:14:41.621Z] 05:14:41 INFO - PID 5515 | 1515474881616 Marionette INFO Found 2388 pixels different, maximum difference per channel 22 13625 [task 2018-01-09T05:14:41.661Z] 05:14:41 INFO - TEST-UNEXPECTED-FAIL | /css/css-backgrounds/background-size-031.html | Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht 13626 [task 2018-01-09T05:14:41.667Z] 05:14:41 INFO - REFTEST IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3dbaxkdX3A8f+9LLDbsourialN+kKMBhSfqm21b3ajqcE0Sompxm66iMgLNDERH1DTV6Y21Gr0Bbyw1bKEriy4y0IRUgJlEQUEFRcwKqHLg92KLS4Li3vdpzu/vph7997dvXPnnJnfnDn37ueTTMLOnDnnf87MnPly7rnnlgAAIFUZ9wAAAJYbgQUAkExgAQAkE1gAAMkEFgBAMoEFAJBMYAEAJBNYAADJBBYAQDKBBQCQTGABACQTWAAAyQQWAEAygQUAkExgAQAkE1gAAMkEFgBAMoG1TJTipZxlWwAw3zi+FyotsZTS9zaSwfmirKzXtqr6OvV7/lIyivGOcxssxddgUHXer4Psg0Y17VJ2sqwny0/dz/Ni8xjF56ByYNX5dxYf+uHUeZ0Gfexk0IYvoHEvvwlV36+9Xo+qr1OdbXkybPeIk2c9WX6Gee+OumUE1jKW9UVyMr8Os+s+7m0w7uU3Ydj/E606D4F1opNlPVl+MgMr20Bz7zeoKofc+h3ir/ujyPmPD/pjhkHGWXWazOX1es6g8+m3rbO3a9XXapBDt8O+RovNdxCZ74uFntNrXr3u6/X6Dvta9ZP1GaqynKrTjHq7H//YKLdbneUsNF2dcTX1+c5ctyrvuUHXv87r3uuzt9i61x131vavs66jHGe/eVb9zPdb/0H3OYsud6An9dkA/e6r+oYadGc06JgGGecw6zLodllI1fWu8/zFxjDMtl5snnXn02u8gyy/zjwHHccg74s69y02bdZ7vtd9vQy7PlXVGU+T273q8/tNV+fLuO46juK1rPP5bnrdBn1dht3fZ7+f6m63uvOuMt+FjPv1rTKmXvdlSg2sfl/SdZ9fd+WrLr/qNFXGOcw0g8yrynzqPK/KtBmv1zDTVn2s7rJGGViZ74te047ii37Y5dSZZtjP++xz6uxkM8Y0is9DVeMcTxOvZdVljmPfU3U8dec/7P6p7r6/ie+jqs9r8rN0UgXWsB/oYZZfdZoq46w6TZXH6n5R9FuGwJr797AfpMz3YpVpBFZ9w+xo64xJYA23rDrzb9O+Z/bxrP19v+nbEFiZATLuz9KyCKzFbotNW2c5dce10GNVxlhnnHXmMex2qbJ+ve6r8/x+j1Xdpr3Wc7HlVZ1P1fEOul2rjHfQ6QVWbmA1Nabsz0PV+xfbvwy6LlXHPsjzF9OmdRtk/zLImHs9VvdzPMx2W2we2d9H43p960yXsc9ZdJkDPWmIHe5i8xx2h1v1RRj2C6PqzqPOfIdZXq9l1P1A1H1smNcr4wMzzPMH2WGMYgxVH8vYMWfet9j9VacZ9vM+rjGN4vMw6DKyxtPkaznudcv+Uq2zvx/FOmZEXlVVv/+q3i+wFnrSADvcQQJn3IFVZbphphl0XFWnaVNgZX1gBtkWGf9DkPVeHHRHLLAGm+coxtTEl0Ibv6ROhsDKeF2WcmBlfB+N+/WtM92SCqxejy20sapOV3W5dec76H3Hz2/Qdak6rl7PrTLNMAEx6Adt2C/tYZ7Ta7phtusg0y42jirzrXPfMOvaRGBlLGeh6Zp6flPbPXsZdZbda77DznOxeYx73TJfl4WeW3cso/yOWsgov4/a8PoOM89Mtec+u0JVPkCLTXf8NFWmqzK2QZc//7664xxkXYbdLr2eU2dZdefTb9qqy646tqx1WGheg8yvzvPqrEOd7VXluVVew2Hu6zemYdex7nt+kHHUHVOV8WatW51lZH0+m34tm163Qdajl6rvvWHH0m/dq2y3xdZrkHUY5j0yzvfuQvMc9PWvY7T51rBRbSQAlqY2fS+0aSyM3rJ6tb15AU5OvY54tEnbxsNoLZtXe9SH+gBotyZ+7DOoto6L0fFKAwAkE1gAAMkEFgBAspMysPwMnEF43wBQVeVvjH7XyVhKltp426bq9VWGnX+d66NkWuwaLONYblu0fXwnI6+Jk8dpr6EDq99jLD+jDqzF5tHUjnRc7+m2f5baPr6TkddEaNJOAovaxn0USWCNT9vHdzLymggs2qnSO3Khy+Uff8n5haarOr9ej/ebdphL+FdZx6rzGnb96+wYspY56LyrvBeqLmOQo2CLLa/uWBYbU6/3Wr/nLva8qq9B1TH2enyx+6qOo9d0dT9ng7wfeq1Hr+dWnW+v51Z5/ep+TjPH0G+eVcaVtd+osw3qLLPOcqq8Z6AN0o5g1flCqnrfQvOsO59e4x1k+b3mVXf9q06bucwq9/Wbd9XxZo6jyv39vmgXU+e1H/R9k/FZyJpf9mdyIYN8nvsto87rO+xnsc68exnF/mCY12T+tG3ZVw2ynH5xBm2SFlhV7q8zj0GW1++xusuqE1hV7u8XAnV3lMPMp864606TsS4Z9w8SV4s9Nuj7Jut9X3d8x99XdRzZn6+MZQzzGa07z2EeyxpD1v7w+OnatK+qu5ysMUFTBFYM9+Fs206r3//hHX8bZByD7jQzxtG2wBomOHrNs4pBIqbJwBr1tq8znrrzHOaxrDEILIHF0td4YFX9kq3yoaz6ZV1lvHXncfzzqtw/jsDKCpg609RZ9+z7q4ZCv8erfhlWed8MG1h13uf9AqvK5ybzy3yx16vOMups26rba9T7mGHGUHd/WEXb9lXzH6/6vswYEzSlNUewMpaXMe/50wwzncBqNrCOf6zqa1fnsUHfD8MEVt1oW2z967y/Bnms3zyHibhh9zEZkVdVxhhOtsAadl4CizZqTWBl7VAG+WKt+yV2/DRt2mkJrOGOZC722KDvmyYDa/axYYIi+/OVsYw2BVaVz8ewYxBYc49njwmaMnBgVdlZVt2h1N35DfqcXtMN8oVU5bkZO7PFllllPk0FVr8dYROBVXX8vcZdZdmjnK7qc4fZBsN+Jof9TAy6jGHe74N+vjP3CXXHkP2a9BvDqPdVg46rzpiG3T6QrdY7b6E3cr8v+8Xe9IM+Xne6KuM6fl51dqSLbZMq61d3BzD/OVXms9DyBhn3sOs+7DjqrGsdC813kPd5vy+KOu+LuuNbaHl11rfqdIN+Nqq+Nxcbf9Xx1H09626DKrLeU4Nug6rrUec9mbENqk476Puy17aEcfDOW4LsNBbXtm3T9Hjatv51LfXxA0QIrCVJYB3r+P8jb5smxtTv6M9SshzWAcCebIkZ9HD9ctbmbdLUuNq6/nUtl/UAsBcDAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI1jewSln6t3Xn7GhgUwIAdC37wFp3zo6IzQ7UAQDNWdaBdTSuBBYA0KBlG1jHxJXAAgAatCwD64S4ElgAQIOWXWAtGFcCCwBo0LIKrJ5xJbAAgAYtycA666xd8dnP/kNcccWnY926HbFmzfOLx5XAAgAatMQCqxOXXPL12Ldvdeza9cr4+c/Pjr17XxL33/O2iG8vElcCCwBo0BIKrE5s3HhNTE2tii1bPhAve9mzsXLlVFx77YbuajwksACAdlgygbVixaG4+uoLY/fuP4zTTjsQk5NH4jOfuqK7Ck/NRNQNAgsAGL8lE1innnowdu58Y2zatDFOOeVwfO7yL0YcKRFPlIjrS8R1M6F1m8ACAMar5YHViYmJztH/vv32v4jnn18Tn/jEV7pDf2IuoDpbS0SnRHxPYAEA49XqwPrwh78R11zzt7F69QtRSsTrX/9I/Pa3v9cNqSdnjlxtLt0T3HeXiP0lYqvAAgDGq7WBdcYZ++LLX74sOp0S27f/VUxOHol3nntHxI9Ld9j/XSLuKREPlohfzxy9uqdHXAksAKBBLQysTlx88TfiwQffGs8995LYt291TE9PxG3bz5s7YrVz5mhVZ+b2Yom4d5G4ElgAQINaF1gbNlwbU1OrYuvW98X69TvizW/+cXzzqxd1T2if/2PBbSXi7hJxV+l/DSyBBQA0qFWBddppB+LWW98djz326njpS/dEKfP+/M2PSne4T1eIKYEFAIxRywLrYOzadVZcddWlUUrn2D9/860S8ZuZyJp/JEtgAQAt06rAKqUTDz/8hnjooTfFeX9824mR9GTpnm81e5K7wAIAWqhlgRXx+c//fRw6tCLi4ZmjVrOB9O2ZuHqoRDxaIo6U6NwksACA9hlrYE1OHonTT/9dTE4eOXrfu99ya8RjJWK6RPyiRNxRupdf+J8ScbBEfKdE3F4iDpeI+wUWANA+YwusSy+9Ku699+3R6ZTYu/fM2LRpY7z3bTfNBdHjpTu86dK9FMNUmbsUw/dm7v++wAIA2mcMgdWJSy75ekxNrYpt2y6IjRs3xeWXXxFTUysjnilzV2K/rkTcXCLumwmpG2fu31q6003Nu09gAQAt0nBgdeK8826LiBI33fTeWLHicJTSiU9+8p+6Q+mUiD0l4oYFAulb8+LqYIn4bo24ElgAQIMaDayJiem48sqPxn33vT3WrHk+JiePxOcu/2L3IqJPzPwI8FA59kjW7O3fS8SzJWLvAHElsACABjV8BCviNa/5RXzkI/8c3T+J8y/dITwxL4QOlO59vynH/hbh5hJxS6l//SuBBQA0rPHAmv1R4brX7YjO7HWtZkPqztI9ef0npf/fFhRYAEBLjTiwOrF+/V1xwQXb4uyzfxYrV05FKTN//ua6mR/3PV7mrnO1u0Q8VyK2JMeVwAIAGjTSwPrYx66M/ftXRadTYv/+VXHzze+Jd5x7Z8TmEp3NJeJXpTuE/yrR+d/SPcm9zqUXBBYA0EIjC6xXvOJX8cwzfxDbt58fl132lfjSlz7VvRTDU2XuPKobSvf8q/8r3XOuRhVXAgsAaNAIAqsTpXTioou+Gfff/2dHL8Ww/rV3Rfxw5ojVL8uJJ7CP+gYA0JDkwOrEu971H3HhhVfHxz/+tdiy5f0xMTHdPedqNnR+VLqXZXiyDP4bgQILAGixxMCau0L79PREHDhwejzyyLnxN+/cfGLsPDDvSJbAAgCWmbTAOvXUg/H0038U11//13H++dvjgx/cHC+++Pvd3wrctkDw/LCM9pwrgQUAjElCYHVi48Zr4gtf+Lt49NHXxdq1z81diuE/S/fCoc+Wen83UGABAEtY7cB6+ct/HWeeuffov1evfiF+8IM/jenpiXjggT+JU045fOw5V22JLACAhtQKrFWr9sfdd6+LW275y1i7ds/R+1/1qsdj5843xJ49a+MzF//jiXFze4k4XCJeKM3/9qDAAgAaVvsI1oYN/xb79p2xYGT97KdnR/yuRNyxQODcWSLucQQLAFj+agdWr8had86OiJtL96T2XpE1zhsAQEMGCqxSIj70oX89GlkX/PmNcyHT1sgCAGjIwIE1/0hW7C7dP9Z8fGS9WCK2tiCuBBYA0KChAmvdOTu651V1ysKRdVcLwkpgAQANGziwjrkUw/dLxKEFIqtNNwCAhgwUWMfE1fGR9VQZ36UYBBYA0AK1A2vBuJq93VfGeykGgQUAtECtwFo0rtp+AwBoSOXAWtJxJbAAgAZVCqwlH1cCCwBoUN/yWBZxJbAAgAZVOoK11G/rztnRwKYEAOha9oF19AgcAEBDlnVgHfPjTQCAhizbwDrh3DEAgIYsy8Ba8MR8AICGLLvA6vlbjwAADVlWgbXoJSUAABqyJAPrrLN2xWc/+w9xxRWfjnXrdsSaNc/3v14XAEBDllhgdeKSS74e+/atjl27Xhk///nZsXfvS+L+e94W8e1F4kpgAQANWkKB1YmNG6+JqalVsWXLB+JlL3s2Vq6cimuv3dBdjYcEFgDQDksmsFasOBRXX31h7N79h3HaaQdicvJIfOZTV3RX4amZiLpBYAEA47dkAuvUUw/Gzp1vjE2bNsYppxyOz13+xYgjJeKJEnF9ibhuJrRuE1gAwHi1PLA6MTHROfrft9/+F/H882viE5/4SnfoT8wFVGdrieiUiO8JLABgvFodWB/+8Dfimmv+NlavfiFKiXj96x+J3/7297oh9eTMkavNpXuC++4Ssb9EbBVYAMB4tTawzjhjX3z5y5dFp1Ni+/a/isnJI/HOc++I+HHpDvu/S8Q9JeLBEvHrmaNX9/SIK4EFADSohYHViYsv/kY8+OBb47nnXhL79q2O6emJuG37eXNHrHbOHK3qzNxeLBH3LhJXAgsAaFDrAmvDhmtjampVbN36vli/fke8+c0/jm9+9aLuCe3zfyy4rUTcXSLuKv2vgSWwAIAGtSqwTjvtQNx667vjscdeHS996Z4oZd6fv/lR6Q736QoxJbAAgDFqWWAdjF27zoqrrro0Sukc++dvvlUifjMTWfOPZAksAKBlWhVYpXTi4YffEA899KY4749vOzGSnizd861mT3IXWABAC7UssCI+//m/j0OHVkQ8PHPUajaQvj0TVw+ViEdLxJESnZsEFgDQPmMNrMnJI3H66b+LyckjR+9791tujXisREyXiF+UiDtK9/IL/1MiDpaI75SI20vE4RJxv8ACANpnbIF16aVXxb33vj06nRJ7954ZmzZtjPe+7aa5IHq8dIc3XbqXYpgqc5di+N7M/d8XWABA+4whsDpxySVfj6mpVbFt2wWxceOmuPzyK2JqamXEM2XuSuzXlYibS8R9MyF148z9W0t3uql59wksAKBFGg6sTpx33m0RUeKmm94bK1YcjlI68clP/lN3KJ0SsadE3LBAIH1rXlwdLBHfrRFXAgsAaFCjgTUxMR1XXvnRuO++t8eaNc/H5OSR+NzlX+xeRPSJmR8BHirHHsmavf17iXi2ROwdIK4EFgDQoIaPYEW85jW/iI985J+j+ydx/qU7hCfmhdCB0r3vN+XY3yLcXCJuKfWvfyWwAICGNR5Ysz8qXPe6HdGZva7VbEjdWbonr/+k9P/bggILAGipEQdWJ9avvysuuGBbnH32z2LlyqkoZebP31w38+O+x8vcda52l4jnSsSW5LgSWABAg0YaWB/72JWxf/+q6HRK7N+/Km6++T3xjnPvjNhcorO5RPyqdIfwXyU6/1u6J7nXufSCwAIAWmhkgfWKV/wqnnnmD2L79vPjssu+El/60qe6l2J4qsydR3VD6Z5/9X+le87VqOJKYAEADRpBYHWilE5cdNE34/77/+zopRjWv/auiB/OHLH6ZTnxBPZR3wAAGpIcWJ1417v+Iy688Or4+Me/Flu2vD8mJqa751zNhs6PSveyDE+WwX8jUGABAC2WGFhzV2ifnp6IAwdOj0ceOTf+5p2bT4ydB+YdyRJYAMAykxZYp556MJ5++o/i+uv/Os4/f3t88IOb48UXf7/7W4HbFgieH5bRnnMlsACAMUkIrE5s3HhNfOELfxePPvq6WLv2ublLMfxn6V449NlS7+8GCiwAYAmrHVgvf/mv48wz9x799+rVL8QPfvCnMT09EQ888CdxyimHjz3nqi2RBQDQkFqBtWrV/rj77nVxyy1/GWvX7jl6/6te9Xjs3PmG2LNnbXzm4n88MW5uLxGHS8QLpfnfHhRYAEDDah/B2rDh32LfvjMWjKyf/fTsiN+ViDsWCJw7S8Q9jmABAMtf7cDqFVnrztkRcXPpntTeK7LGeQMAaMhAgVVKxIc+9K9HI+uCP79xLmTaGlkAAA0ZOLDmH8mK3aX7x5qPj6wXS8TWFsSVwAIAGjRUYK07Z0f3vKpOWTiy7mpBWAksAKBhAwfWMZdi+H6JOLRAZLXpBgDQkIEC65i4Oj6ynirjuxSDwAIAWqB2YC0YV7O3+8p4L8UgsACAFqgVWIvGVdtvAAANqRxYSzquBBYA0KBKgbXk40pgAQAN6lseyyKuBBYA0KBKR7CW+m3dOTsa2JQAAF3LPrCOHoEDAGjIsg6sY368CQDQkGUbWCecOwYA0JBlGVgLnpgPANCQZRdYPX/rEQCgIcsqsBa9pAQAQEOWZGCdddau+Oxn/yGuuOLTsW7djliz5vn+1+sCAGjIEgusTlxyyddj377VsWvXK+PnPz879u59Sdx/z9sivr1IXAksAKBBSyiwOrFx4zUxNbUqtmz5QLzsZc/GypVTce21G7qr8ZDAAgDaYckE1ooVh+Lqqy+M3bv/ME477UBMTh6Jz3zqiu4qPDUTUTcILABg/JZMYJ166sHYufONsWnTxjjllMPxucu/GHGkRDxRIq4vEdfNhNZtAgsAGK+WB1YnJiY6R//79tv/Ip5/fk184hNf6Q79ibmA6mwtEZ0S8T2BBQCMV6sD68Mf/kZcc83fxurVL0QpEa9//SPx29/+Xjeknpw5crW5dE9w310i9peIrQILABiv1gbWGWfsiy9/+bLodEps3/5XMTl5JN557h0RPy7dYf93ibinRDxYIn49c/Tqnh5xJbAAgAa1MLA6cfHF34gHH3xrPPfcS2LfvtUxPT0Rt20/b+6I1c6Zo1WdmduLJeLeReJKYAEADWpdYG3YcG1MTa2KrVvfF+vX74g3v/nH8c2vXtQ9oX3+jwW3lYi7S8Rdpf81sAQWANCgVgXWaacdiFtvfXc89tir46Uv3ROlzPvzNz8q3eE+XSGmBBYAMEYtC6yDsWvXWXHVVZdGKZ1j//zNt0rEb2Yia/6RLIEFALRMqwKrlE48/PAb4qGH3hTn/fFtJ0bSk6V7vtXsSe4CCwBooZYFVsTnP//3cejQioiHZ45azQbSt2fi6qES8WiJOFKic5PAAgDaZ6yBNTl5JE4//XcxOXnk6H3vfsutEY+ViOkS8YsScUfpXn7hf0rEwRLxnRJxe4k4XCLuF1gAQPuMLbAuvfSquPfet0enU2Lv3jNj06aN8d633TQXRI+X7vCmS/dSDFNl7lIM35u5//sCCwBonzEEVicuueTrMTW1KrZtuyA2btwUl19+RUxNrYx4psxdif26EnFzibhvJqRunLl/a+lONzXvPoEFALRIw4HVifPOuy0iStx003tjxYrDUUonPvnJf+oOpVMi9pSIGxYIpG/Ni6uDJeK7NeJKYAEADWo0sCYmpuPKKz8a99339liz5vmYnDwSn7v8i92LiD4x8yPAQ+XYI1mzt38vEc+WiL0DxJXAAgAa1PARrIjXvOYX8ZGP/HN0/yTOv3SH8MS8EDpQuvf9phz7W4SbS8Qtpf71rwQWANCwxgNr9keF6163Izqz17WaDak7S/fk9Z+U/n9bUGABAC014sDqxPr1d8UFF2yLs8/+WaxcORWlzPz5m+tmftz3eJm7ztXuEvFcidiSHFcCCwBo0EgD62MfuzL2718VnU6J/ftXxc03vyfece6dEZtLdDaXiF+V7hD+q0Tnf0v3JPc6l14QWABAC40ssF7xil/FM8/8QWzffn5cdtlX4ktf+lT3UgxPlbnzqG4o3fOv/q90z7kaVVwJLACgQSMIrE6U0omLLvpm3H//nx29FMP6194V8cOZI1a/LCeewD7qGwBAQ5IDqxPvetd/xIUXXh0f//jXYsuW98fExHT3nKvZ0PlR6V6W4cky+G8ECiwAoMUSA2vuCu3T0xNx4MDp8cgj58bfvHPzibHzwLwjWQILAFhm0gLr1FMPxtNP/1Fcf/1fx/nnb48PfnBzvPji73d/K3DbAsHzwzLac64EFgAwJgmB1YmNG6+JL3zh7+LRR18Xa9c+N3cphv8s3QuHPlvq/d1AgQUALGG1A+vlL/91nHnm3qP/Xr36hfjBD/40pqcn4oEH/iROOeXwsedctSWyAAAaUiuwVq3aH3ffvS5uueUvY+3aPUfvf9WrHo+dO98Qe/asjc9c/I8nxs3tJeJwiXihNP/bgwILAGhY7SNYGzb8W+zbd8aCkfWzn54d8bsScccCgXNnibjHESwAYPmrHVi9ImvdOTsibi7dk9p7RdY4bwAADRkosEqJ+NCH/vVoZF3w5zfOhUxbIwsAoCEDB9b8I1mxu3T/WPPxkfViidjagrgSWABAg4YKrHXn7OieV9UpC0fWXS0IK4EFADRs4MA65lIM3y8RhxaIrDbdAAAaMlBgHRNXx0fWU2V8l2IQWABAC9QOrAXjavZ2XxnvpRgEFgDQArUCa9G4avsNAKAhlQNrSceVwAIAGlQpsJZ8XAksAKBBfctjWcSVwAIAGqQ8AACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCXfUFR4AAASaSURBVCwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI9v/NeAG0AATMwwAAAABJRU5ErkJggg== 13627 [task 2018-01-09T05:14:41.674Z] 05:14:41 INFO - REFTEST IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3de4ylZWHH8eecmcUsoqQY29jYGGIMsVVJbU1sNfFWtN5rxVi0oCm6Ypv6V5MGNRqt8V8T65XKXtllh+UmKAreQQui0ZhYqxRFRKkrAq5Cd3dmzvvrH+/Z3bOzOzPnzDxz5uzy+SQnzM55z3s51y/P+857SgAAqKqs9woAAJxsBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBNZJohQP5SHuCwAGrcfnwlBLLKUse1mTlfNBObTF7qthH6flbn8iWYv1Xc/74ER8DFZqlOfrSt6D1mraE9kjZTs5+Yz6el5qHmvxOhg6sEb5dy1e9KszyuO00useCSbhA2i9lz8Owz5fF3s8hn2cRrkvHwn3e/LI2U5OPqt57q51ywisk1itD5JH8uNwaNvX+z5Y7+WPw2r/T3TYeQisYz1StpOTT83Aqm1Fc19upYYZcltuiH/UXZGD1690N8NK1nPYaWoub7HbrHQ+y93Xte/XYR+rlQzdrvYxWmq+K1HzeXG82yw2r8V+t9jju9rHajm1XkPDLGfYadb6fl943Vreb6Ms53jTjbJe43p919y2YZ5zK93+UR73xV57S237qOtd6/4fZVvXcj2Xm+ewr/nltn+l7zlLLndFN1rmDljud8M+oVb6ZrTSdVrJeq5mW1Z6vxzPsNs9yu2XWofV3NdLzXPU+Sy2vitZ/ijzXOl6rOR5Mcrvlpq21nN+sd8tZrXbM6xR1mec9/uwt19uulE+jEfdxrV4LEd5fY9721b6uKz2/b7282nU+23UeQ8z3+NZ78d3mHVa7Hc1VQ2s5T6kR739qBs/7PKHnWaY9VzNNCuZ1zDzGeV2w0xb4/FazbTDXjfqstYysGo+Lxabdi0+6Fe7nFGmWe3r/dBtRnmTrbFOa/F6GNZ6rs84Hsthl7ke7z3Drs+o81/t+9Oo7/3j+Dwa9nbjfC09ogJrtS/o1Sx/2GmGWc9hpxnmulE/KJZbhsA68u/VvpBqPheHmUZgjW41b7SjrJPAWt2yRpn/JL33HLq+1vv9ctNPQmDVDJD1fi2dFIG11GWpaUdZzqjrdbzrhlnHUdZzlHms9n4ZZvsW+90ot1/uumHv08W2c6nlDTufYdd3pffrMOu70ukFVt3AGtc61X49DPv7pd5fVrotw677Sm6/lEnatpW8v6xknRe7btTX8Wrut6XmUfvzaL0e31Gmq/Ges+QyV3SjVbzhLjXP1b7hDvsgrPYDY9g3j1Hmu5rlLbaMUV8Qo163mserxgtmNbdfyRvGWqzDsNfVeGOu+bulfj/sNKt9va/XOq3F62Gly6i1PuN8LNd722p/qI7yfr8W21gj8oY17OffsL8XWMe70QrecFcSOOsdWMNMt5ppVrpew04zSYFV6wWzkvuixv8Q1HourvSNWGCtbJ5rsU7j+FCYxA+pR0Jg1XhcTuTAqvF5tN6P7yjTnVCBtdh1x7uzhp1u2OWOOt+V/m7h/Fa6LcOu12K3HWaa1QTESl9oq/3QXs1tFptuNffrSqZdaj2Gme8ov1vNto4jsGos53jTjev247rfay9jlGUvNt/VznOpeaz3ttV8XI5321HXZS0/o45nLT+PJuHxXc08axp57oc2aJgX0FLTLZxmmOmGWbeVLn/wd6Ou50q2ZbX3y2K3GWVZo85nuWmHXfaw61ZrG443r5XMb5TbjbINo9xfw9x2mMdwNb9bbp1Wu42jPudXsh6jrtMw61tr20ZZRq3X57gfy3Fv20q2YzHDPvdWuy7Lbfsw99tS27WSbVjNc2Q9n7vHm+dKH/9RrG2+jdla3UkAnJgm6XNhktaFtXdSPdqevACPTIuNeEySSVsf1tZJ82iv9VAfAJNtHLt9VmpS14u145EGAKhMYAEAVCawAAAqe0QGln3grITnDQDDGvoTY7nzZJxITrT1nTTDnl9ltfMf5fwoNS11Dpb1WO6kmPT1eyTymDh4nMm16sBa7jpOPmsdWEvNY1xvpOv1nJ7019Kkr98jkcdEaDKZBBYjW+9RJIG1fiZ9/R6JPCYCi8k01DPyeKfLX3jK+eNNN+z8Frt+uWlXcwr/YbZx2HmtdvtHeWOotcyVznuY58Kwy1jJKNhSyxt1XZZap8Wea8vddqnbDfsYDLuOi12/1O+GXY/Fphv1dbaS58Ni27HYbYed72K3HebxG/V1WnMdlpvnMOtV631jlPtglGWOspxhnjMwCaqNYI3ygTTs7443z1Hns9j6rmT5i81r1O0fdtqayxzmd8vNe9j1rbkew/x+uQ/apYzy2K/0eVPjtVBrfrVfk8ezktfzcssY5fFd7WtxlHkvZi3eD1bzmAxOOynvVStZznJxBpOkWmAN8/tR5rGS5S133ajLGiWwhvn9ciEw6hvlauYzynqPOk2Nbanx+5XE1VLXrfR5U+t5P+r6LfzdsOtR+/VVYxmreY2OOs/VXFdrHWq9Hy6cbpLeq0ZdTq11gnERWFndi3PS3rSW+z+8hZeVrMdK3zRrrMekBdZqgmOxeQ5jJREzzsBa6/t+lPUZdZ6rua7WOggsgcWJb+yBNeyH7DAvymE/rIdZ31HnsfB2w/x+PQKrVsCMMs0o217798OGwnLXD/thOMzzZrWBNcrzfLnAGuZ1U/PDfKnHa5RljHLfDnt/rfV7zGrWYdT3w2FM2nvV4PXDPi9rrBOMy8SMYNVYXo15D06zmukE1ngDa+F1wz52o1y30ufDagJr1GhbavtHeX6t5Lrl5rmaiFvte0yNyBtWjXV4pAXWauclsJhEExNYtd5QVvLBOuqH2MJpJulNS2CtbiRzqetW+rwZZ2Adum41QVH79VVjGZMUWMO8Pla7DgLryPW11wnGZcWBNcyb5bBvKKO++a30NotNt5IPpGFuW+PNbKllDjOfcQXWcm+E4wisYdd/sfUeZtlrOd2wt13NfbDa1+RqXxMrXcZqnu8rfX3XfE8YdR1qPybLrcNav1etdL1GWafV3j9Q20jPvOM9kZf7sF/qSb/S60edbpj1WjivUd5Il7pPhtm+Ud8ABm8zzHyOt7yVrPdqt3216zHKto7iePNdyfN8uQ+KUZ4Xo67f8ZY3yvYOO91KXxvDPjeXWv9h12fUx3PU+2AYtZ5TK70Pht2OUZ6TNe6DYadd6fNysfsS1oNn3gnIm8bSJu2+Gff6TNr2j+pEX3+ARGCdkATW0Rb+H/mkGcc6LTf6cyI5GbYBwDvZCWalw/Uns0m+T8a1XpO6/aM6WbYDwLsYAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlywZWKSfapelfkqnubEpJ/upPbhzHfQkAkOSkC6wjcTXdj6uXn31d9u86ZRz3JQBAkpMqsAbj6mBKSV76jM9kfqaT3uXLbiYAQDUnSWAdG1cve8b1mZvpprerk972zjjuSwCAJCdcYDULgmrwskhc7egkO41gAQDjc4IE1pGIKqVJp9PL9PRspqdnM9WdTynN4WOujhm52lnSXCawAIDxOQECqznq506nd8w0nTK/dFwZwQIAxugECKzBEaw2rs466wd597vflw996B159auuTuk2+eunfTZzM1PHjyuBBQCM0QkQWIfiqh2letvbPpb9+0/J3r2Pz0/v+qM8+JvH5pavPCf7d5+SXF6OH1cCCwAYowkNrKMPZu9251JK8prXXJmkZOfOv8vjHverlJLs2vn6JCXzt3fSu7SNqWYwrAQWADBmExhYzYKfm3S78+l05jMzc27uvfcPsmHDgZSS/Ou/fDBzvanM3jGV3mWdZHdJs6M/emUECwBYJxMYWAtHsHoppcn09IHcffcTs2fP36aU5N0Xvy9JSfPjflDtKsldJbmpJNuMYAEA62eCAmvwnFa9Y0aypqcP5nM3vDh33/3EvPPif0tSMn9nN73tJdla0lxd0qQk/1nSbBZYAMD6mZDAGoyp3uFjrjZt+ni2bjs/Gzc+nFKSpzz1h9m377QkJb0fdtJsLcnmkubykubnJc3+klxV0mwXWADA+pmQwBocwWpPxbBhw4FccsmFSUqumHltSklecfanc+B7GzLXdNP8oiRfKck3SnJfaTfl5oHdg47BAgDWyYQE1qGwauPqTW/anO9972m5554/zAP3n56k5IarXpLffurRyY6S3ndLcrDdJdikJA+XNN84smvwmFM0CCwAYIwmILDakatOp90teO65M2maks997sV5wfO/lKc89UfZ/bHXJSlp7urH07aS5pqSfK0/irWnHH1gu8ACANbRBARWDo9enXLKgXzpS8/PXXc9KY897cGUkrzwrC/k1584Iwe/OZ1eOsld7TFWzbbSHoO1tT01wzFBJbAAgHUyAYF15LirjRsfyi/3Pj5bNl+QUpJXPvPa9PaUZHdJb1tJ7i3tKv+ktKdlOHR6hqVGrgQWADBm6xBYzVGX9sube+l0epmens0Pvn9Wvn7rX+Q1z7oys5dPpbe7tCcR3d7fRfhwSZqS5u7+yNVli5xYVGABAOtkjIF1KKran7vd+aOun54+mG5nPm//53/PXLppvluS7SW9LZ1kS0kzU5IDJc23S/Ld/gHu15f2eKzldg8KLABgjNZlBKsdtWr/fcYZ9+XUU393+N+ve/buzP24m/l00vvvktxY0nylJL8syXxJri3JDf2fby/JlgW7CQUWALDOxhRYC3cJJhdcsCW33/7nmZ2dyn33nZEPffgdueBFW3Jw63SaHe2JRHPoNAxpR69ya0nzHyX5WknTlDRfL+1B7kawAIAJMuZdhG1cbdr08STtqRje+MbteefFH2hX5f6SZk//rwN3ljSfaaMqXy/JNSXZXJLdJdnbD66r292Iyx5/JbAAgDEa2wjWoa+/OeeczycpmZk5N1NT7e/+6R8/nIOzGzKXbnq/aiOq2d6Pp6393YCX9YNqb0l6JfnqCHElsACAMVrDwGoG/ttLpzOfUpps2/b3ueOOJ+fR/eOu3vuu9yQp6d1R0ru5tH8heG9JZkqyo7THV+0o7bFXvyzJvn5cHYquYeJKYAEAY7RGgXX0Xwt2u/OZmppNKU2e85ybc+GFl6SU5KJ/+GiSkrkfddPb2sZUs6+0q/WrfkANXj7bD6/lztousACAdbQGgXUkrgb/WnDh5eVPvy6zv5jO3G+76e0syaUlzRdLsr8k/1WS2/q7AAdDalt/NGvUuBJYAMAYVQ6swd2C7c/Pe96Xc955l+Xss7+Tqe5sSmnP0D53ZTe9/SW9O0vyqZJmV0nu7Y9gXbZgF+DxQmqUuBJYAMAYVQyswZGr9iSiF1300f4i2sul296cl//pdeldXtLbVTL/0/6pGP6npPl1O01zy8AuwMGIWuxngQUATJgKgXX0GdqnpmYzNTWXM8+8Mw89tDHXX/+yvG3Tx/KBD16c2WYquae0gbWlk8yUNHeW5MGS3F/a0zFsWWVICSwAYJ1VHcEaPObqrW/9RL7znbNzyvSBlJK84MlfzMO3b2xPxfCT0n634PaS7Gi/U7DZ0f/dWsSVwAIAxmgVgdUsuLRx9ZKX3JBNmz6R97//Xfnkx9+SUpJX/dk16e3ppNlW0ru9vyvwxwMhtfDLmmvHlcACAMZohMA69gD29tLL1NRsOp1eLrzwkgwec/WNbz47b3npJzO3fSrNTElvR7sLsLmtP83P2hGsNY8rgQUAjNGQgXUkqDqdXn9XYHNUbE1NzWbv3sfl2mtfmVe/6pq84fwd+c2+xyQPlTRXliPnudpZ2q+8+XZJvlWOPqfVWl4AAMZkxF2Eg6E13x+9mssb3rAjH/nI2/Otbz0zZ/zefSklee6TvpqHvnhqZg9OpXdfSfaUNIMxta20X4OzcPegwAIATnAjjGC1x1g94Qk/z+mnP3A4sk499Xe57bZnJSn5wo0vTCnJK5756TRXtQex924qyVz/NAxXlqNPHjrOCwDAmAwZWO0uwUc9an9uueUvc9NNL8ppp+07fN2ZZ96ZH3z/rDyw7/S856L35uDmDWl2l/T6JwxtbiztFzQ/WI49zmoco1cCCwAYo5FGsLrd+bz5zZcmKbn++pflMY/Zd3iX4e8/4X9zxw+fnKQNqmZzGzbNzjay8uWS3FqOPajdCBYAcJIZMrCO3k143nmX5VBkHdpdeM4ffz4PXHF6Dt4/nd6BTnJTSTP4dTfby5FjruwiBABOYiOMYLX/7XbnUkpy/vlbk5TMXHVuXv/cXeldXpJdJb2rSvKbkuZgSb7QD6uFUWUECwA4iY04gnV0ZP3Na6/K/x18VLK3tMdcbW+Dqrm6JPtK8nBJrujvFlzPuBJYAMAYjRhYbWRNTx1MKcmLnnJTHrp5Y+Yyld7PSnJ5/ytvtpXkupJ8tYzvNAwCCwCYEEMG1kBcddu4etkzrk/viv7X39xckqakuacku/tBdSi0JiGuBBYAMEZDBNbx42puppveroGvv7m5tLO7e8EuwfXcLSiwAIB1MGRgHS+uOult7yQ7B77+5taS3FaOPrB9EuJKYAEAYzREYCXT3dnF42owoLaVydotKLAAgHWwbHlM9ePqpc/4zOJxdbygmrTIAgAYk6FGsF5+9nWZPxRXO44zcrUwqCYtrgQWADBGy5bHOU//fA7u3JDe5eX4uwVPlAsAwJgMNYJ1Yl2O/NXjod2bf/UnN47jvgQASHLSBdbgKSVmD+/e3L/rlHHclwAASU6qwDr2fF0vfcZnMj/Tab8nEQBgTE6SwFrqZKj9v3oEABiTEyywmoH/LrwsElf9v3oEABiXEySwjkRUKU06nV6mp2czPT2bqe58SmmWPxkqAMCYnACB1Rz1c6fTO2aaTplfOq6MYAEAY3QCBNbgCFYbV2ed9YO8+93vy4c+9I68+lVXp3Sb/PXTPpu5manFv8YHAGBMToDAOhRX7SjV2972sezff0r27n18fnrXH+XB3zw2t3zlOdm/+5RkqZOhAgCMyYQG1tEHs3e7cyklec1rrkxSsnPn3+Vxj/tVSkl27Xx9kpL52zvpXdrGVONM7gDAOprAwGoW/Nyk251PpzOfmZlzc++9f5ANGw6klORf/+WDmetNZfaOqfQu6yS7S5od/dErI1gAwDqZwMBaOILVSylNpqcP5O67n5g9e/42pSTvvvh9SUqaH/eDaldJ7irJTSXZZgQLAFg/ExRYg+e06h0zkjU9fTCfu+HFufvuJ+adF/9bkpL5O7vpbS/J1pLm6pImJfnPkmazwAIA1s+EBNZgTPUOH3O1adPHs3Xb+dm48eGUkjzlqT/Mvn2nJSnp/bCTZmtJNpc0l5c0Py9p9pfkqpJmu8ACANbPhATW4AhWeyqGDRsO5JJLLkxScsXMa1NK8oqzP50D39uQuaab5hcl+UpJvlGS+0q7KTcP7B50DBYAsE4mJLAOhVUbV2960+Z873tPyz33/GEeuP/0JCU3XPWS/PZTj052lPS+W5KD7S7BJiV5uKT5xpFdg8ecokFgAQBjNAGB1Y5cdTrtbsFzz51J05R87nMvzgue/6U85ak/yu6PvS5JSXNXP562lTTXlORr/VGsPeXoA9sFFgCwjiYgsHJ49OqUUw7kS196fu6660l57GkPppTkhWd9Ib/+xBk5+M3p9NJJ7mqPsWq2lfYYrK3tqRmOCSqBBQCskwkIrCPHXW3c+FB+uffx2bL5gpSSvPKZ16a3pyS7S3rbSnJvaVf5J6U9LcOh0zMsNXIlsACAMVuHwGqOurRf3txLp9PL9PRsfvD9s/L1W/8ir3nWlZm9fCq93aU9iej2/i7Ch0vSlDR390euLlvkxKICCwBYJ2MMrENR1f7c7c4fdf309MF0O/N5+z//e+bSTfPdkmwv6W3pJFtKmpmSHChpvl2S7/YPcL++tMdjLbd7UGABAGO0LiNY7ahV++8zzrgvp576u8P/ft2zd2fux93Mp5Pef5fkxpLmKyX5ZUnmS3JtSW7o/3x7SbYs2E0osACAdTamwFq4SzC54IItuf32P8/s7FTuu++MfOjD78gFL9qSg1un0+xoTySaQ6dhSDt6lVtLmv8oyddKmqak+XppD3I3ggUATJAx7yJs42rTpo8naU/F8MY3bs87L/5Auyr3lzR7+n8duLOk+UwbVfl6Sa4pyeaS7C7J3n5wXd3uRlz2+CuBBQCM0dhGsA59/c0553w+ScnMzLmZmmp/90//+OEcnN2QuXTT+1UbUc32fjxt7e8GvKwfVHtL0ivJV0eIK4EFAIzRGgZWM/DfXjqd+ZTSZNu2v88ddzw5j+4fd/Xed70nSUnvjpLezaX9C8F7SzJTkh2lPb5qR2mPvfplSfb14+pQdA0TVwILABijNQqso/9asNudz9TUbEpp8pzn3JwLL7wkpSQX/cNHk5TM/aib3tY2ppp9pV2tX/UDavDy2X54LXfWdoEFAKyjNQisI3E1+NeCCy8vf/p1mf3FdOZ+201vZ0kuLWm+WJL9JfmvktzW3wU4GFLb+qNZo8aVwAIAxqhyYA3uFmx/ft7zvpzzzrssZ5/9nUx1Z1NKe4b2uSu76e0v6d1Zkk+VNLtKcm9/BOuyBbsAjxdSo8SVwAIAxqhiYA2OXLUnEb3ooo/2F9FeLt325rz8T69L7/KS3q6S+Z/2T8XwPyXNr9tpmlsGdgEORtRiPwssAGDCVAiso8/QPjU1m6mpuZx55p156KGNuf76l+Vtmz6WD3zw4sw2U8k9pQ2sLZ1kpqS5syQPluT+0p6OYcsqQ0pgAQDrrOoI1uAxV2996yfyne+cnVOmD6SU5AVP/mIevn1jeyqGn5T2uwW3l2RH+52CzY7+79YirgQWADBGqwisZsGljauXvOSGbNr0ibz//e/KJz/+lpSSvOrPrklvTyfNtpLe7f1dgT8eCKmFX9ZcO64EFgAwRiME1rEHsLeXXqamZtPp9HLhhZdk8Jirb3zz2XnLSz+Zue1TaWZKejvaXYDNbf1pftaOYK15XAksAGCMhgysI0HV6fT6uwKbo2Jramo2e/c+Ltde+8q8+lXX5A3n78hv9j0meaikubIcOc/VztJ+5c23S/KtcvQ5rdbyAgAwJiPuIhwMrfn+6NVc3vCGHfnIR96eb33rmTnj9+5LKclzn/TVPPTFUzN7cCq9+0qyp6QZjKltpf0anIW7BwUWAHCCG2EEqz3G6glP+HlOP/2Bw5F16qm/y223PStJyRdufGFKSV7xzE+nuao9iL13U0nm+qdhuLIcffLQcV4AAMZkyMBqdwk+6lH7c8stf5mbbnpRTjtt3+Hrzjzzzvzg+2flgX2n5z0XvTcHN29Is7uk1z9haHNjab+g+cFy7HFW4xi9ElgAwBiNNILV7c7nzW++NEnJ9de/LI95zL7Duwx//wn/mzt++OQkbVA1m9uwaXa2kZUvl+TWcuxB7UawAICTzJCBdfRuwvPOuyyHIuvQ7sJz/vjzeeCK03Pw/un0DnSSm0qawa+72V6OHHNlFyEAcBIbYQSr/W+3O5dSkvPP35qkZOaqc/P65+5K7/KS7CrpXVWS35Q0B0vyhX5YLYwqI1gAwElsxBGsoyPrb157Vf7v4KOSvaU95mp7G1TN1SXZV5KHS3JFf7fgesaVwAIAxmjEwGoja3rqYEpJXvSUm/LQzRszl6n0flaSy/tfebOtJNeV5KtlfKdhEFgAwIQYMrAG4qrbxtXLnnF9elf0v/7m5pI0Jc09JdndD6pDoTUJcSWwAIAxGiKwjh9XczPd9HYNfP3NzaWd3d0Ldgmu525BgQUArIMhA+t4cdVJb3sn2Tnw9Te3luS2cvSB7ZMQVwILABijIQIrme7OLh5XgwG1rUzWbkGBBQCsg2XLY6ofVy99xmcWj6vjBdWkRRYAwJgMNYL18rOvy/yhuNpxnJGrhUE1aXElsACAMVq2PM55+udzcOeG9C4vx98teKJcAADGZKgRrBPrcuSvHg/t3vyrP7lxHPclAECSky6wBk8pMXt49+b+XaeM474EAEhyUgXWsefreukzPpP5mU77PYkAAGNykgTWUidD7f/VIwDAmJxggdUM/HfhZZG46v/VIwDAuJwggXUkokpp0un0Mj09m+np2Ux151NKs/zJUAEAxuQECKzmqJ87nd4x03TK/NJxZQQLABijEyCwBkew2rg666wf5N3vfl8+9KF35NWvujql2+Svn/bZzM1MLf41PgAAY3ICBNahuGpHqd72to9l//5Tsnfv4/PTu/4oD/7msbnlK8/J/t2nJEudDBUAYEwmNLCOPpi9251LKclrXnNlkpKdO/8uj3vcr1JKsmvn65OUzN/eSe/SNqYaZ3IHANbRBAZWs+DnJt3ufDqd+czMnJt77/2DbNhwIKUk//ovH8xcbyqzd0yld1kn2V3S7OiPXhnBAgDWyQQG1sIRrF5KaTI9fSB33/3E7NnztykleffF70tS0vy4H1S7SnJXSW4qyTYjWADA+pmgwBo8p1XvmJGs6emD+dwNL87ddz8x77z435KUzN/ZTW97SbaWNFeXNCnJf5Y0mwUWALB+JiSwBmOqd/iYq02bPp6t287Pxo0Pp5TkKU/9YfbtOy1JSe+HnTRbS7K5pLm8pPl5SbO/JFeVNNsFFgCwfiYksAZHsNpTMWzYcCCXXHJhkpIrZl6bUpJXnP3pHPjehsw13TS/KMlXSvKNktxX2k25eWD3oGOwAIB1MiGBdSis2rh605s253vfe1ruuecP88D9pycpueGql+S3n3p0sqOk992SHGx3CTYpycMlzTeO7Bo85hQNAgsAGKMJCKx25KrTaXcLnnvuTJqm5HOfe3Fe8Pwv5SlP/VF2f+x1SUqau/rxtK2kuaYkX+uPYlXoaWwAAA4zSURBVO0pRx/YLrAAgHU0AYGVw6NXp5xyIF/60vNz111PymNPezClJC886wv59SfOyMFvTqeXTnJXe4xVs620x2BtbU/NcExQCSwAYJ1MQGAdOe5q48aH8su9j8+WzReklOSVz7w2vT0l2V3S21aSe0u7yj8p7WkZDp2eYamRK4EFAIzZOgRWc9Sl/fLmXjqdXqanZ/OD75+Vr9/6F3nNs67M7OVT6e0u7UlEt/d3ET5ckqakubs/cnXZIicWFVgAwDoZY2Adiqr25253/qjrp6cPptuZz9v/+d8zl26a75Zke0lvSyfZUtLMlORASfPtkny3f4D79aU9Hmu53YMCCwAYo3UZwWpHrdp/n3HGfTn11N8d/vfrnr07cz/uZj6d9P67JDeWNF8pyS9LMl+Sa0tyQ//n20uyZcFuQoEFAKyzMQXWwl2CyQUXbMntt/95Zmenct99Z+RDH35HLnjRlhzcOp1mR3si0Rw6DUPa0avcWtL8R0m+VtI0Jc3XS3uQuxEsAGCCjHkXYRtXmzZ9PEl7KoY3vnF73nnxB9pVub+k2dP/68CdJc1n2qjK10tyTUk2l2R3Sfb2g+vqdjfissdfCSwAYIzGNoJ16Otvzjnn80lKZmbOzdRU+7t/+scP5+Dshsylm96v2ohqtvfjaWt/N+Bl/aDaW5JeSb46QlwJLABgjNYwsJqB//bS6cynlCbbtv197rjjyXl0/7ir977rPUlKeneU9G4u7V8I3luSmZLsKO3xVTtKe+zVL0uyrx9Xh6JrmLgSWADAGK1RYB3914Ld7nympmZTSpPnPOfmXHjhJSkluegfPpqkZO5H3fS2tjHV7Cvtav2qH1CDl8/2w2u5s7YLLABgHa1BYB2Jq8G/Flx4efnTr8vsL6Yz99tuejtLcmlJ88WS7C/Jf5Xktv4uwMGQ2tYfzRo1rgQWADBGlQNrcLdg+/PznvflnHfeZTn77O9kqjubUtoztM9d2U1vf0nvzpJ8qqTZVZJ7+yNYly3YBXi8kBolrgQWADBGFQNrcOSqPYnoRRd9tL+I9nLptjfn5X96XXqXl/R2lcz/tH8qhv8paX7dTtPcMrALcDCiFvtZYAEAE6ZCYB19hvapqdlMTc3lzDPvzEMPbcz1178sb9v0sXzggxdntplK7iltYG3pJDMlzZ0lebAk95f2dAxbVhlSAgsAWGdVR7AGj7l661s/ke985+ycMn0gpSQvePIX8/DtG9tTMfyktN8tuL0kO9rvFGx29H+3FnElsACAMVpFYDULLm1cveQlN2TTpk/k/e9/Vz758beklORVf3ZNens6abaV9G7v7wr88UBILfyy5tpxJbAAgDEaIbCOPYC9vfQyNTWbTqeXCy+8JIPHXH3jm8/OW176ycxtn0ozU9Lb0e4CbG7rT/OzdgRrzeNKYAEAYzRkYB0Jqk6n198V2BwVW1NTs9m793G59tpX5tWvuiZvOH9HfrPvMclDJc2V5ch5rnaW9itvvl2Sb5Wjz2m1lhcAgDEZcRfhYGjN90ev5vKGN+zIRz7y9nzrW8/MGb93X0pJnvukr+ahL56a2YNT6d1Xkj0lzWBMbSvt1+As3D0osACAE9wII1jtMVZPeMLPc/rpDxyOrFNP/V1uu+1ZSUq+cOMLU0ryimd+Os1V7UHsvZtKMtc/DcOV5eiTh47zAgAwJkMGVrtL8FGP2p9bbvnL3HTTi3LaafsOX3fmmXfmB98/Kw/sOz3vuei9Obh5Q5rdJb3+CUObG0v7Bc0PlmOPsxrH6JXAAgDGaKQRrG53Pm9+86VJSq6//mV5zGP2Hd5l+PtP+N/c8cMnJ2mDqtnchk2zs42sfLkkt5ZjD2o3ggUAnGSGDKyjdxOed95lORRZh3YXnvPHn88DV5yeg/dPp3egk9xU0gx+3c32cuSYK7sIAYCT2AgjWO1/u925lJKcf/7WJCUzV52b1z93V3qXl2RXSe+qkvympDlYki/0w2phVBnBAgBOYiOOYB0dWX/z2qvyfwcflewt7TFX29ugaq4uyb6SPFySK/q7BdczrgQWADBGIwZWG1nTUwdTSvKip9yUh27emLlMpfezklze/8qbbSW5riRfLeM7DYPAAgAmxJCBNRBX3TauXvaM69O7ov/1NzeXpClp7inJ7n5QHQqtSYgrgQUAjNEQgXX8uJqb6aa3a+Drb24u7ezuXrBLcD13CwosAGAdDBlYx4urTnrbO8nOga+/ubUkt5WjD2yfhLgSWADAGA0RWMl0d3bxuBoMqG1lsnYLCiwAYB0sWx5T/bh66TM+s3hcHS+oJi2yAADGZKgRrJeffV3mD8XVjuOMXC0MqkmLK4EFAIzRsuVxztM/n4M7N6R3eTn+bsET5QIAMCbKAwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBU9v8G7vocVHkgKQAAAABJRU5ErkJggg== 13628 [task 2018-01-09T05:14:41.674Z] 05:14:41 INFO - TEST-INFO took 173ms 13629 [task 2018-01-09T05:14:41.752Z] 05:14:41 INFO - PID 5515 | 13630 [task 2018-01-09T05:14:41.753Z] 05:14:41 INFO - PID 5515 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost 13631 [task 2018-01-09T05:14:41.754Z] 05:14:41 INFO - PID 5515 | 13632 [task 2018-01-09T05:14:41.755Z] 05:14:41 INFO - PID 5515 | [Child 5609, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13633 [task 2018-01-09T05:14:41.997Z] 05:14:41 INFO - PID 5515 | 13634 [task 2018-01-09T05:14:41.998Z] 05:14:41 INFO - PID 5515 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost 13635 [task 2018-01-09T05:14:41.999Z] 05:14:41 INFO - PID 5515 | 13636 [task 2018-01-09T05:14:42.006Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13637 [task 2018-01-09T05:14:42.007Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13638 [task 2018-01-09T05:14:42.008Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13639 [task 2018-01-09T05:14:42.197Z] 05:14:42 INFO - PID 5515 | --DOMWINDOW == 4 (0x7f96a0384400) [pid = 5570] [serial = 4] [outer = (nil)] [url = about:blank] 13640 [task 2018-01-09T05:14:42.198Z] 05:14:42 INFO - PID 5515 | --DOMWINDOW == 3 (0x7f96a0378800) [pid = 5570] [serial = 3] [outer = (nil)] [url = about:blank] 13641 [task 2018-01-09T05:14:42.198Z] 05:14:42 INFO - PID 5515 | --DOMWINDOW == 2 (0x7f96a00d4400) [pid = 5570] [serial = 2] [outer = (nil)] [url = about:blank] 13642 [task 2018-01-09T05:14:42.199Z] 05:14:42 INFO - PID 5515 | --DOCSHELL 0x7f96a024c000 == 0 [pid = 5570] [id = {1a452fd4-8ae2-4c98-beba-16212037e470}] 13643 [task 2018-01-09T05:14:42.481Z] 05:14:42 INFO - PID 5515 | --DOMWINDOW == 1 (0x7f96a1679660) [pid = 5570] [serial = 1] [outer = (nil)] [url = http://web-platform.test:8000/testharness_runner.html] 13644 [task 2018-01-09T05:14:42.481Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13645 [task 2018-01-09T05:14:42.481Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13646 [task 2018-01-09T05:14:42.481Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13647 [task 2018-01-09T05:14:42.481Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13648 [task 2018-01-09T05:14:42.481Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13649 [task 2018-01-09T05:14:42.482Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13650 [task 2018-01-09T05:14:42.482Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13651 [task 2018-01-09T05:14:42.482Z] 05:14:42 INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403 13652 [task 2018-01-09T05:14:42.482Z] 05:14:42 INFO - PID 5515 | --DOMWINDOW == 0 (0x7f96a16eb400) [pid = 5570] [serial = 5] [outer = (nil)] [url = http://web-platform.test:8000/testharness_runner.html] 13653 [task 2018-01-09T05:14:42.489Z] 05:14:42 INFO - PID 5515 | --DOCSHELL 0x7fca6abbd800 == 3 [pid = 5515] [id = {c2e31cf2-1aba-4842-9463-5baa761268a5}] 13654 [task 2018-01-09T05:14:42.506Z] 05:14:42 INFO - PID 5515 | --DOMWINDOW == 10 (0x7fca92bd8800) [pid = 5515] [serial = 2] [outer = (nil)] [url = about:blank] 13655 [task 2018-01-09T05:14:42.506Z] 05:14:42 INFO - PID 5515 | --DOCSHELL 0x7fcab037a000 == 2 [pid = 5515] [id = {ac05895b-a43c-48d7-be75-be7a5bd04d7a}] 13656 [task 2018-01-09T05:14:42.528Z] 05:14:42 INFO - PID 5515 | nsStringStats 13657 [task 2018-01-09T05:14:42.528Z] 05:14:42 INFO - PID 5515 | => mAllocCount: 17761 13658 [task 2018-01-09T05:14:42.528Z] 05:14:42 INFO - PID 5515 | => mReallocCount: 112 13659 [task 2018-01-09T05:14:42.528Z] 05:14:42 INFO - PID 5515 | => mFreeCount: 17761 13660 [task 2018-01-09T05:14:42.529Z] 05:14:42 INFO - PID 5515 | => mShareCount: 8823 13661 [task 2018-01-09T05:14:42.530Z] 05:14:42 INFO - PID 5515 | => mAdoptCount: 1526 13662 [task 2018-01-09T05:14:42.530Z] 05:14:42 INFO - PID 5515 | => mAdoptFreeCount: 1526 13663 [task 2018-01-09T05:14:42.530Z] 05:14:42 INFO - PID 5515 | => Process ID: 5570, Thread ID: 140285391853376 13664 [task 2018-01-09T05:14:42.530Z] 05:14:42 INFO - PID 5515 | --DOCSHELL 0x7fca877a4000 == 1 [pid = 5515] [id = {3207091e-0a5a-4a6d-ac69-5d8b3f6a949e}] 13665 [task 2018-01-09T05:14:42.530Z] 05:14:42 INFO - PID 5515 | --DOCSHELL 0x7fca9203c800 == 0 [pid = 5515] [id = {8546835e-dccd-4073-ab9a-fdd0ee82567b}] 13666 [task 2018-01-09T05:14:42.545Z] 05:14:42 INFO - PID 5515 | 13667 [task 2018-01-09T05:14:42.546Z] 05:14:42 INFO - PID 5515 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost 13668 [task 2018-01-09T05:14:42.546Z] 05:14:42 INFO - PID 5515 | 13669 [task 2018-01-09T05:14:42.547Z] 05:14:42 INFO - PID 5515 | [Child 5667, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791 13670 [task 2018-01-09T05:14:42.547Z] 05:14:42 INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399 13671 [task 2018-01-09T05:14:42.568Z] 05:14:42 INFO - PID 5515 | --DOCSHELL 0x7f28ef2df000 == 0 [pid = 5609] [id = {754b921c-17e1-40e4-ae24-59fad7e2a669}] 13672 [task 2018-01-09T05:14:42.670Z] 05:14:42 INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399 13673 [task 2018-01-09T05:14:42.671Z] 05:14:42 INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399 13674 [task 2018-01-09T05:14:42.692Z] 05:14:42 INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399 13675 [task 2018-01-09T05:14:42.728Z] 05:14:42 INFO - PID 5515 | nsStringStats :milan Could you please take a look?
Flags: needinfo?(milan)
Comment 54•7 years ago
|
||
This bug is assigned and there are patches just waiting for review comments to be addressed.
Flags: needinfo?(milan)
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 57•7 years ago
|
||
We have 30 failures in the last 7 days. They occur on Linux (enabled and disabled), Linux x64 (ccov, enabled and disabled). The affected builds type are debug and opt. Reftest analyzer: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/Kw-MqtXuSlyuYXUzfXfBNQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1 A recent log example: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=158760609&lineNumber=13633
Comment hidden (Intermittent Failures Robot) |
Comment 59•7 years ago
|
||
:tnikkel, there is a patch here for review from 2+ months ago- we continue to have a very high rate of failures- can you please take a look at this patch soon so we can land it or the patch can be updated to pass review and then land?
Flags: needinfo?(tnikkel)
Comment 60•7 years ago
|
||
I'm waiting for review comments to be addressed, see comment 48, comment 54.
Flags: needinfo?(tnikkel)
Comment 61•7 years ago
|
||
Comment on attachment 8936648 [details] [diff] [review] 0001-Bug-1383682-imgRequestProxy-should-avoid-leaking-sta.patch, v4 :aosmond, can you address the feedback from comment 48?
Flags: needinfo?(aosmond)
Attachment #8936648 -
Flags: review?(tnikkel)
Assignee | ||
Comment 62•7 years ago
|
||
This patch mostly splits off the validation state tracking from regular notification deferral.
Attachment #8936648 -
Attachment is obsolete: true
Flags: needinfo?(aosmond)
Attachment #8947568 -
Flags: review?(tnikkel)
Assignee | ||
Comment 63•7 years ago
|
||
This part renames SetNotificationsDeferred to Mark/ClearPendingNotify to make things more clear.
Attachment #8947569 -
Flags: review?(tnikkel)
Assignee | ||
Comment 64•7 years ago
|
||
This part prevents StartDecoding/StartDecodingWithResult/GetImageStatus from leaking state information when validating a request.
Attachment #8947570 -
Flags: review?(tnikkel)
Comment 65•7 years ago
|
||
Comment on attachment 8947568 [details] [diff] [review] 0001-Bug-1383682-Part-1.-Split-off-imgRequestProxy-notifi.patch, v1 Review of attachment 8947568 [details] [diff] [review]: ----------------------------------------------------------------- ::: image/imgLoader.cpp @@ +2947,5 @@ > + > +void > +imgCacheValidator::UpdateProxies() > +{ > + for (auto& proxy : mProxies) { ChangeOwner can SyncNotify too (if mCancelled) so shouldn't we make a copy of the mProxies array in case it is changed? I guess the old code didn't do this, so is that a bug in the old code? ::: image/imgRequestProxy.cpp @@ +270,2 @@ > > // If we'd previously requested a synchronous decode, request a decode on the Why move the decode requesting into ClearValidating? It adds an ordering requirement for the ChangeOwner/ClearValidating call for what benefit?
Updated•7 years ago
|
Attachment #8947569 -
Flags: review?(tnikkel) → review+
Updated•7 years ago
|
Attachment #8947570 -
Flags: review?(tnikkel) → review+
Comment hidden (Intermittent Failures Robot) |
Assignee | ||
Comment 67•7 years ago
|
||
(In reply to Timothy Nikkel (:tnikkel) from comment #65) > Comment on attachment 8947568 [details] [diff] [review] > 0001-Bug-1383682-Part-1.-Split-off-imgRequestProxy-notifi.patch, v1 > > Review of attachment 8947568 [details] [diff] [review]: > ----------------------------------------------------------------- > > ::: image/imgLoader.cpp > @@ +2947,5 @@ > > + > > +void > > +imgCacheValidator::UpdateProxies() > > +{ > > + for (auto& proxy : mProxies) { > > ChangeOwner can SyncNotify too (if mCancelled) so shouldn't we make a copy > of the mProxies array in case it is changed? I guess the old code didn't do > this, so is that a bug in the old code? > Grrr, you are right. I missed this. This is a bug in the old code. Maybe I should make imgCacheValidator::RemoveProxy dispatch. It should only be called rarely. > ::: image/imgRequestProxy.cpp > @@ +270,2 @@ > > > > // If we'd previously requested a synchronous decode, request a decode on the > > Why move the decode requesting into ClearValidating? It adds an ordering > requirement for the ChangeOwner/ClearValidating call for what benefit? I wanted to call StartDecoding if deferred even if ChangeOwner was not called.
Assignee | ||
Comment 68•7 years ago
|
||
(In reply to Andrew Osmond [:aosmond] from comment #67) > (In reply to Timothy Nikkel (:tnikkel) from comment #65) > > Comment on attachment 8947568 [details] [diff] [review] > > 0001-Bug-1383682-Part-1.-Split-off-imgRequestProxy-notifi.patch, v1 > > > > Review of attachment 8947568 [details] [diff] [review]: > > ----------------------------------------------------------------- > > > > ::: image/imgLoader.cpp > > @@ +2947,5 @@ > > > + > > > +void > > > +imgCacheValidator::UpdateProxies() > > > +{ > > > + for (auto& proxy : mProxies) { > > > > ChangeOwner can SyncNotify too (if mCancelled) so shouldn't we make a copy > > of the mProxies array in case it is changed? I guess the old code didn't do > > this, so is that a bug in the old code? > > > > Grrr, you are right. I missed this. This is a bug in the old code. Maybe I > should make imgCacheValidator::RemoveProxy dispatch. It should only be > called rarely. > Actually while ChangeOwner can call SyncNotify, it will be deferred as I haven't cleared the validation flag yet: https://searchfox.org/mozilla-central/rev/f80722d4f3bfb722c5ec53880c4a7efb71285676/image/ProgressTracker.cpp#305 This strikes me as sort of strange, but it is *not* a change in behaviour as the original code does the same thing: https://searchfox.org/mozilla-central/rev/f80722d4f3bfb722c5ec53880c4a7efb71285676/image/imgLoader.cpp#3065 I did however move the array onto the stack, just in case. I think the SyncNotifyListener in the second loop is still unsafe for similar reasons... try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=35ba549bbf86515591d0bb2dd9034eae2498a376
Attachment #8947568 -
Attachment is obsolete: true
Attachment #8947568 -
Flags: review?(tnikkel)
Attachment #8948670 -
Flags: review?(tnikkel)
Updated•7 years ago
|
Attachment #8948670 -
Flags: review?(tnikkel) → review+
Comment 69•7 years ago
|
||
Pushed by aosmond@gmail.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/c657d87a0011 Part 1. Split off imgRequestProxy notification deferrals for validation. r=tnikkel https://hg.mozilla.org/integration/mozilla-inbound/rev/62497967f092 Part 2. Rename IProgressObserver::SetNotificationsDeferred to make purpose clear. r=tnikkel https://hg.mozilla.org/integration/mozilla-inbound/rev/30ee5a21a6a1 Part 3. Prevent imgRequestProxy from leaking the current state when validating. r=tnikkel
Comment 70•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c657d87a0011 https://hg.mozilla.org/mozilla-central/rev/62497967f092 https://hg.mozilla.org/mozilla-central/rev/30ee5a21a6a1
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox60:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
Updated•7 years ago
|
status-firefox58:
--- → wontfix
status-firefox59:
--- → affected
status-firefox-esr52:
--- → unaffected
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Updated•7 years ago
|
You need to log in
before you can comment on or make changes to this bug.
Description
•