We're starting to see reports of crashes on OS X Sierra from building with an older SDK (e.g. https://bugzilla.mozilla.org/show_bug.cgi?id=1320048#c40) We should make the new SDK (macOS 10.12) available on the Mac builders so we can start building against it.

mshal: do you recall what we had to do to make the previous SDK available for cross-compilation?

ted set that up originally, but I think we just need to zip up the sdk and put it in tooltool: https://dxr.mozilla.org/mozilla-central/rev/567894f026558e6dada617a3998f29aed06ac7d8/browser/config/tooltool-manifests/macosx64/cross-releng.manifest#24 And then point the mozconfigs to the new directory. ted, do any of the other cross tools need to be rebuilt against a newer SDK? (cctools, clang, etc)

I forgot to mention, but the SDK needs to be private in tooltool.

(In reply to Michael Shal [:mshal] from comment #2) > ted, do any of the other cross tools need to be rebuilt against a newer SDK? > (cctools, clang, etc) They're not actually built against the SDK, so they should continue to be fine.

I uploaded the 10.12 SDK from my local XCode 8.2 install to tooltool, here's the manifest.

Andrei or Alin, can you investigate this bug?

I did the following changes and pushed to try: - point to the new SDK here: https://dxr.mozilla.org/mozilla-central/source/build/macosx/cross-mozconfig.common#24 - add the SDK to all tooltoolmanifests: https://dxr.mozilla.org/mozilla-central/source/browser/config/tooltool-manifests/macosx64 - add an "ac_add_options --with-macos-sdk=$topsrcdir/MacOSX10.12.sdk" option in https://dxr.mozilla.org/mozilla-central/source/build/macosx/local-mozconfig.common However, both builds are failing: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9168fbd57964d20fbb882ce85c8a1b312b22aa97 @Ted: any thoughts on what's needed here?

We also need to fix bug 1285754 before we can make the switch, I think.

(In reply to Markus Stange [:mstange] from comment #8) > We also need to fix bug 1285754 before we can make the switch, I think. The comments on that bug seem to indicate that it's about running codesign on 10.12, not using the 10.12 SDK. Is that accurate? (In reply to Alin Selagea [:aselagea][:buildduty] from comment #7) > @Ted: any thoughts on what's needed here? At some point Apple changed their SDKs to include 'tbd' files to link against instead of .dylibs. I think the linker version we're using is too old to support these. For the cross-mac builds we have a cctools package in tooltool, we'd likely have to just build a newer version. From some poking around, it looks like we'd need to update to at least ld64 253.3 to get support for tbd files: https://github.com/tpoechtrager/cctools-port/commits/master/cctools/ld64/src/ld/parsers/textstub_dylib_file.cpp For the non-cross-compile builds, the most-supported way would be to update to a newer Xcode, but that is pretty invasive. I expect we could rebuild a native ld64 binary, put it in tooltool and point the builds at it like we do for the cross-builds, but I have never done that so I don't have a great idea of where to start.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #9) > (In reply to Markus Stange [:mstange] from comment #8) > > We also need to fix bug 1285754 before we can make the switch, I think. > > The comments on that bug seem to indicate that it's about running codesign > on 10.12, not using the 10.12 SDK. Is that accurate? You're right, I think. The fact that signing on 10.12 after building with the 10.11 SDK still failed seems to indicate that.

froydnj: in bug 1289847 you were testing builds with a newer linker. Presumably that's still in tooltool and we could try that patch on top of the patches in the try push in comment 7, right?

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #11) > froydnj: in bug 1289847 you were testing builds with a newer linker. > Presumably that's still in tooltool and we could try that patch on top of > the patches in the try push in comment 7, right? That linker is still in tooltool, but note that said linker is compiled for OS X, rather than Linux. It looks like that linker is ld64 version 264.mumble, so it should be new enough. Building cctools-port on OS X was rather painless; I suspect it ought to be equally painless on Linux. I think we even have a taskcluster job to do the Linux build: http://dxr.mozilla.org/mozilla-central/source/taskcluster/scripts/misc/build-cctools.sh#4 I guess that uses crosstool-ng rather than cctools-port, but ideally the experience should be roughly similar?

ehsan might wind up updating our cctools as part of bug 1331957, which should unblock this.

The new cctools that I have built is ready to use in tooltool in case someone wants to give it a try. This gives you the tooltool manifest magic: https://hg.mozilla.org/try/rev/2ca1b2bbe29d5d44e1ffa6ebc510ad9b14d42b55 Two things to note: 1. I had to change -mlinker-version=136 to -mlinker-version=137 in order to get clang to translate -rdynamic to -dynamic_export when constructing the linker command line. 2. This cctools toolchain targets x86_64-apple-darwin11 and not x86_64-apple-darwin10, so to use it you'd have to make some small changes like this: https://hg.mozilla.org/try/rev/933280ec43d9625580a38852cbce7b590fd72eb1

(BTW I also have a TC job to build this which I haven't checked in yet, so if this works for you don't waste time on those bits.)

Kim: now that this is causing crashes, is there an ETA for when buildduty will be able to get to it?

Alin: Are you able to look at this bug since you worked on it before?

I'm currently dealing with several security bugs, so, if possible, I'd prefer someone else take a look at this. I'm happy to provide details related to my previous work on this bug though.

Andrei, is this bug something that you can take a look at?

Coop, how do the requirements for this bug change now that we are going the cross compile route? We have a project branch setup up to test tc migration including cross compile builds. Do we just need to enable the builds and try them running the new sdk?

(In reply to Kim Moir [:kmoir] from comment #20) > Coop, how do the requirements for this bug change now that we are going the > cross compile route? We have a project branch setup up to test tc migration > including cross compile builds. Do we just need to enable the builds and > try them running the new sdk? I think comment #2 still stands, and mshal can add more cross-compile-specific detail if necessary.

Now that all Mac builds on mozilla-central etc are building as cross-compiles, this shouldn't require any RelEng/RelOps work. It's just a matter of someone taking the SDK I previously uploaded to tooltool (attachment 8820705 [details]), updating the in-tree manifests and mozconfigs, and pushing that to try to see if it all works.

Did the push on try but the builds still fail: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c902efbcaf2334b1648f34d4194514de6d9f44bd I did the following changes before the push: - point to the new SDK here: https://dxr.mozilla.org/mozilla-central/source/build/macosx/cross-mozconfig.common - add the SDK to all tooltoolmanifests: https://dxr.mozilla.org/mozilla-central/source/browser/config/tooltool-manifests/macosx64 - add an "ac_add_options --with-macos-sdk=$topsrcdir/MacOSX10.12.sdk" option in https://dxr.mozilla.org/mozilla-central/source/build/macosx/local-mozconfig.common

I thought we had gotten updated to a new-enough cctools that the linker would support tbd files, but I guess not?

Per my comment 3 we need at least ld64 253.3.

Our build script says we're building from https://github.com/tpoechtrager/cctools-port/commit/8e9c3f2506b51cf56725eaa60b6e90e240e249ca : https://dxr.mozilla.org/mozilla-central/rev/211d4dd61025c0a40caea7a54c9066e051bdde8c/taskcluster/scripts/misc/build-cctools-port.sh#11 ...and that version of ld64 has code to support tbd files: https://github.com/tpoechtrager/cctools-port/blob/8e9c3f2506b51cf56725eaa60b6e90e240e249ca/cctools/ld64/src/ld/parsers/textstub_dylib_file.cpp

Oh, ugh: https://github.com/tpoechtrager/cctools-port/issues/26#issuecomment-256915520 "The 10.0 SDK requires a newer not yet open sourced linker." So maybe we can't actually use the 10.12 SDK yet?

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #27) > So maybe we can't actually use the 10.12 SDK yet? Is there an SDK version released between 10.7 and 10.12 that we *could* use, i.e. in what Xcode version does the linker change happen?

It looks like 10.9 was the last SDK to ship with actual binaries: https://github.com/phracker/MacOSX-SDKs/tree/master/MacOSX10.9.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A 10.10 and 10.11 have tbd files: https://github.com/phracker/MacOSX-SDKs/blob/master/MacOSX10.10.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation.tbd https://github.com/phracker/MacOSX-SDKs/blob/master/MacOSX10.11.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation.tbd however, those seem to be a slightly different format than the files in the 10.12 SDK, which start with the line `--- !tapi-tbd-v2`. It's possible the latest open source Apple linker can deal with these files, per my comment 26. I can track down a 10.11 SDK and put it in tooltool and we can try it out.

I extracted the 10.11 SDK from the Xcode 7.3.1 dmg, tarred it up and uploaded it to tooltool.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #31) > https://treeherder.mozilla.org/#/ > jobs?repo=try&revision=d7e5dbf0f306d5879706a272e3e46714ab214ff3 Oops, this broke because I screwed up my try syntax. Second try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=69b2bae7fe7b222fff3391f163a9108eaea4a927

I think this broke due to this sccache bug: https://github.com/mozilla/sccache/issues/163

ted: what's the ETA on this? This is blocking an important issue... are you taking this? thanks

glandium is working on the sccache issue I linked in comment 33, so hopefully once he has a fix there we can update the sccache in use in automation and that will unblock updating to the 10.11 SDK.

Comment on attachment 8897957 [details] bug 1324892 - update Mac builds to use the OS X 10.11 SDK. https://reviewboard.mozilla.org/r/169266/#review174596

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #38) > https://treeherder.mozilla.org/#/ > jobs?repo=try&revision=6280e74b992e02a93e1c2b4e39b406dcabc7c2f7 The builds are all green, I'm just waiting to see test results to make sure nothing breaks in any odd way.

The try push looks pretty good, but there are mochitest-chrome-3 failures on both the opt and stylo-opt tests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f2a8e24310a53969d402f2198d7118e2c80eae36&selectedJob=123666321 https://treeherder.mozilla.org/#/jobs?repo=try&revision=f2a8e24310a53969d402f2198d7118e2c80eae36&selectedJob=123666275 These look like they could be legitimate failures: 15:56:38 INFO - TEST-UNEXPECTED-FAIL | toolkit/content/tests/chrome/test_largemenu.xul | panel movement (1, 1) y - got 23, expected 1 15:56:38 INFO - SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:312:5 15:56:38 INFO - is@chrome://mochitests/content/chrome/toolkit/content/tests/chrome/window_largemenu.xul:164:24 15:56:38 INFO - testPopupMovement@chrome://mochitests/content/chrome/toolkit/content/tests/chrome/window_largemenu.xul:331:3 15:56:38 INFO - onpopupshown@chrome://mochitests/content/chrome/toolkit/content/tests/chrome/window_largemenu.xul:1:1

spohl: Can you look at this failure?

Yes, looking into it.

This fixes the failing test and try is now green (see comment 45). With the 10.11 SDK it looks like we need to adjust the expected Y value for the panel test too, not just the movement test. Markus, are you familiar with this test and is this acceptable, or is this revealing a greater issue?

I'm not sure; does this mean that the large popup now overlaps the menubar, and it didn't before?

Yes, I believe this is what's going on. This screenshot shows the behavior when built with the 10.12 SDK (which causes the same failure as we've seen on try). Unfortunately, when building locally with the 10.7 SDK, I get the same behavior and failure, so I'm unable to take a screenshot to compare the behavior before and after the SDK switch.

Comment on attachment 8898929 [details] [diff] [review] Update test Review of attachment 8898929 [details] [diff] [review]: ----------------------------------------------------------------- Sorry, I still don't really understand what's going on here. What are the values of screen.availTop before and after the SDK change, and what's the menu's position before and after the SDK change?

(In reply to Markus Stange [:mstange] from comment #49) > Comment on attachment 8898929 [details] [diff] [review] > Update test > > Review of attachment 8898929 [details] [diff] [review]: > ----------------------------------------------------------------- > > Sorry, I still don't really understand what's going on here. What are the > values of screen.availTop before and after the SDK change, and what's the > menu's position before and after the SDK change? Before the SDK change, screen.availTop is 1. After the SDK change, screen.availTop is 23. I can confirm that with the 10.12 SDK, the menu does not overlap the native menu bar at the top of the screen, which would explain the value of 23 (the menu bar is 22px). Unfortunately, I haven't been able to confirm that the 10.7 SDK overlaps the menu bar. When I build with the 10.7 SDK locally, I actually get a value of 23 for screen.availTop as well and the menu does not overlap the menu bar either. I'm not sure if this has to do with the fact that I'm compiling/testing this on macOS 10.12, or if there is something else that's different on try.

(In reply to Markus Stange [:mstange] from comment #47) > I'm not sure; does this mean that the large popup now overlaps the menubar, > and it didn't before? I misspoke in comment 48: It's actually the opposite. I have to assume that it used to overlap when built with the 10.7 SDK (although I can't reproduce when building locally with the 10.7 SDK) and now it no longer overlaps the native menu bar.

(In reply to Stephen A Pohl [:spohl] from comment #50) > Before the SDK change, screen.availTop is 1. After the SDK change, > screen.availTop is 23. I can confirm that with the 10.12 SDK, the menu does > not overlap the native menu bar at the top of the screen, which would > explain the value of 23 (the menu bar is 22px). Unfortunately, I haven't > been able to confirm that the 10.7 SDK overlaps the menu bar. When I build > with the 10.7 SDK locally, I actually get a value of 23 for screen.availTop > as well and the menu does not overlap the menu bar either. I'm not sure if > this has to do with the fact that I'm compiling/testing this on macOS 10.12, > or if there is something else that's different on try. Ah, I see. Thanks! Yes, it probably has to do with the fact that you're testing this on macOS 10.12. Sometimes, Apple changes behavior in two stages: macOS version A has old behavior regardless of SDK version macOS version B has old behavior with old SDK version and new behavior with new SDK version macOS version C has new behavior regardless of SDK version So they keep compatibility code around for one or two releases (which kicks in depending on which SDK you built with), and then they remove that compatibility code at some point.

I think now there's just one part that I don't understand yet: why "isPanelTest" was part of the condition before. Do we apply different rules when placing panels vs when placing menus? If so, is the code that makes that distinction no longer working?

(In reply to Markus Stange [:mstange] from comment #53) > I think now there's just one part that I don't understand yet: why > "isPanelTest" was part of the condition before. Do we apply different rules > when placing panels vs when placing menus? If so, is the code that makes > that distinction no longer working? I was finally able to track this down to the fact that [mWindow frame].origin.y in nsCocoaWindow::UpdateBounds()[1] returns different values depending on the SDK used. Try runs show that with a screen height of 1200: 1. the 10.7 SDK will show 1171 for [mWindow frame].origin.y. With a popup height of 28 and doing the math, we get to the value of 1 that the test expects: 1200 - (1171 + 28) = 1 2. the 10.11 SDK will show 1149 for [mWindow frame].origin.y. With a popup height of 28 and doing the math, we get to the new value of 23: 1200 - (1149 + 28) = 23 nsCocoaWindow::UpdateBounds() is called by nsCocoaWindow::ReportMoveEvent() before we call NotifyWindowMoved(), which ultimately positions the popup as can be seen in the callstack below[2]. So, the difference appears to be in native code and we don't appear to be handling menus and panels differently ourselves. Menus appear to have had the "proper" value (one that accounts for the native menu bar) all along. [1] https://dxr.mozilla.org/mozilla-central/rev/1b4c59eef820b46eb0037aca68f83a15088db45f/widget/cocoa/nsCocoaWindow.mm#1674 [2] * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 * frame #0: 0x000000011076e208 XUL`nsMenuPopupFrame::MoveTo(this=0x0000000163510c08, aPos=0x00007fff5f07b820, aUpdateAttrs=false) at nsMenuPopupFrame.cpp:2417 frame #1: 0x000000011078fa49 XUL`nsXULPopupManager::PopupMoved(this=0x0000000104945080, aFrame=0x0000000163510c08, aPnt=nsIntPoint @ 0x00007fff5f07b888) at nsXULPopupManager.cpp:528 frame #2: 0x000000010fe2cd46 XUL`nsView::WindowMoved(this=0x00000001359d3900, aWidget=0x00000001059e1800, x=2, y=46) at nsView.cpp:996 frame #3: 0x000000010fe3e808 XUL`nsBaseWidget::NotifyWindowMoved(this=0x00000001059e1800, aX=2, aY=46) at nsBaseWidget.cpp:1759 frame #4: 0x000000010ff39df6 XUL`nsCocoaWindow::ReportMoveEvent(this=0x00000001059e1800) at nsCocoaWindow.mm:1972 frame #5: 0x000000010ff3cf7e XUL`::-[WindowDelegate windowDidMove:](self=0x000000010592d900, _cmd="windowDidMove:", aNotification="NSWindowDidMoveNotification") at nsCocoaWindow.mm:2769 frame #6: 0x00007fff7dba845c CoreFoundation`__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ + 12 frame #7: 0x00007fff7dba835b CoreFoundation`_CFXRegistrationPost + 427 frame #8: 0x00007fff7dba80c2 CoreFoundation`___CFXNotificationPost_block_invoke + 50 frame #9: 0x00007fff7db65523 CoreFoundation`-[_CFXNotificationRegistrar find:object:observer:enumerator:] + 1827 frame #10: 0x00007fff7db6455c CoreFoundation`_CFXNotificationPost + 604 frame #11: 0x00007fff7f589677 Foundation`-[NSNotificationCenter postNotificationName:object:userInfo:] + 66 frame #12: 0x00007fff7bf226b7 AppKit`_NSWindowSendWindowDidMove + 63 frame #13: 0x00007fff7b758216 AppKit`-[NSWindow _setFrameCommon:display:stashSize:] + 2680 frame #14: 0x00007fff7b757791 AppKit`-[NSWindow _setFrame:display:allowImplicitAnimation:stashSize:] + 222 frame #15: 0x00007fff7b7576ac AppKit`-[NSWindow setFrame:display:] + 67 frame #16: 0x000000010ff386b9 XUL`nsCocoaWindow::DoResize(this=0x00000001059e1800, aX=1, aY=1, aWidth=91, aHeight=27, aRepaint=true, aConstrainToCurrentScreen=false) at nsCocoaWindow.mm:1623 frame #17: 0x000000010ff38a4f XUL`nsCocoaWindow::Resize(this=0x00000001059e1800, aX=1, aY=1, aWidth=91, aHeight=27, aRepaint=true) at nsCocoaWindow.mm:1634 frame #18: 0x000000010fe3de21 XUL`nsBaseWidget::ResizeClient(this=0x00000001059e1800, aX=1, aY=1, aWidth=91, aHeight=27, aRepaint=true) at nsBaseWidget.cpp:1553 frame #19: 0x000000010fe2a0c2 XUL`nsView::DoResetWidgetBounds(this=0x00000001359d3900, aMoveOnly=false, aInvalidateChangedSize=true) at nsView.cpp:368 frame #20: 0x000000010fe29c9c XUL`nsView::ResetWidgetBounds(this=0x00000001359d3900, aRecurse=false, aForceSync=true) at nsView.cpp:216 frame #21: 0x000000010fe2f22d XUL`nsViewManager::ProcessPendingUpdatesForView(this=0x000000015e70b380, aView=0x0000000162392c80, aFlushDirtyRegion=false) at nsViewManager.cpp:399 frame #22: 0x000000010fe3142b XUL`nsViewManager::UpdateWidgetGeometry(this=0x000000015e70b380) at nsViewManager.cpp:1108 frame #23: 0x000000011039034a XUL`mozilla::PresShell::DoFlushPendingNotifications(this=0x00000001359ae000, aFlush=(mFlushType = Layout, mFlushAnimations = true)) at PresShell.cpp:4223 frame #24: 0x0000000110341970 XUL`nsIPresShell::FlushPendingNotifications(this=0x00000001359ae000, aType=(mFlushType = Layout, mFlushAnimations = true)) at nsIPresShell.h:571 frame #25: 0x000000011038fa0b XUL`mozilla::PresShell::DoFlushPendingNotifications(this=0x00000001359ae000, aType=Layout) at PresShell.cpp:4048 frame #26: 0x000000010d301767 XUL`nsIPresShell::FlushPendingNotifications(this=0x00000001359ae000, aType=Layout) at nsIPresShell.h:562 frame #27: 0x000000010d6f6953 XUL`nsDocument::FlushPendingNotifications(this=0x0000000135986000, aType=Layout) at nsDocument.cpp:8334 frame #28: 0x000000010d579b32 XUL`mozilla::dom::Element::GetPrimaryFrame(this=0x00000001634eaa60, aType=Layout) at Element.cpp:2247 frame #29: 0x000000010d57ae58 XUL`mozilla::dom::Element::GetBoundingClientRect(this=0x00000001634eaa60) at Element.cpp:1035 frame #30: 0x000000010e536a58 XUL`mozilla::dom::ElementBinding::getBoundingClientRect(cx=0x0000000105581000, obj=Handle<JSObject *> @ 0x00007fff5f07d030, self=0x00000001634eaa60, args=0x00007fff5f07d098) at ElementBinding.cpp:2193 frame #31: 0x000000010e8495fb XUL`mozilla::dom::GenericBindingMethod(cx=0x0000000105581000, argc=0, vp=0x0000000105a911f8) at BindingUtils.cpp:3053 frame #32: 0x0000000113787f84 XUL`js::CallJSNative(cx=0x0000000105581000, native=(XUL`mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) at BindingUtils.cpp:3026), args=0x00007fff5f07f550)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&) at jscntxtinlines.h:293 frame #33: 0x0000000113787ce7 XUL`js::InternalCallOrConstruct(cx=0x0000000105581000, args=0x00007fff5f07f550, construct=NO_CONSTRUCT) at Interpreter.cpp:469 frame #34: 0x000000011378832d XUL`InternalCall(cx=0x0000000105581000, args=0x00007fff5f07f550) at Interpreter.cpp:514 frame #35: 0x000000011378812d XUL`js::CallFromStack(cx=0x0000000105581000, args=0x00007fff5f07f550) at Interpreter.cpp:520 frame #36: 0x000000011377c6eb XUL`Interpret(cx=0x0000000105581000, state=0x00007fff5f080678) at Interpreter.cpp:3065 frame #37: 0x00000001137714e4 XUL`js::RunScript(cx=0x0000000105581000, state=0x00007fff5f080678) at Interpreter.cpp:409 frame #38: 0x0000000113787de5 XUL`js::InternalCallOrConstruct(cx=0x0000000105581000, args=0x00007fff5f080870, construct=NO_CONSTRUCT) at Interpreter.cpp:487 frame #39: 0x000000011378832d XUL`InternalCall(cx=0x0000000105581000, args=0x00007fff5f080870) at Interpreter.cpp:514 frame #40: 0x00000001137883a6 XUL`js::Call(cx=0x0000000105581000, fval=JS::HandleValue @ 0x00007fff5f0807f0, thisv=JS::HandleValue @ 0x00007fff5f0807e8, args=0x00007fff5f080870, rval=JS::MutableHandleValue @ 0x00007fff5f0807e0) at Interpreter.cpp:533 frame #41: 0x0000000113dd4aa5 XUL`JS::Call(cx=0x0000000105581000, thisv=JS::HandleValue @ 0x00007fff5f080860, fval=JS::HandleValue @ 0x00007fff5f080858, args=0x00007fff5f080990, rval=JS::MutableHandleValue @ 0x00007fff5f080850) at jsapi.cpp:2951 frame #42: 0x000000010e584a23 XUL`mozilla::dom::Function::Call(this=0x00000001058d44c0, cx=0x0000000105581000, aThisVal=Handle<JS::Value> @ 0x00007fff5f080a38, arguments=0x0000000130947270, aRetVal=MutableHandle<JS::Value> @ 0x00007fff5f080a30, aRv=0x00007fff5f080e08) at FunctionBinding.cpp:36 frame #43: 0x000000010d4e33fb XUL`void mozilla::dom::Function::Call<nsCOMPtr<nsISupports> >(this=0x00000001058d44c0, thisVal=0x00007fff5f080e28, arguments=0x0000000130947270, aRetVal=MutableHandle<JS::Value> @ 0x00007fff5f080b90, aRv=0x00007fff5f080e08, aExecutionReason="setTimeout handler", aExceptionHandling=eReportExceptions, aCompartment=0x0000000000000000) at FunctionBinding.h:72 frame #44: 0x000000010d4e2c15 XUL`nsGlobalWindow::RunTimeoutHandler(this=0x0000000133cdb000, aTimeout=0x000000010591e4e0, aScx=0x000000015e70b600) at nsGlobalWindow.cpp:13196 frame #45: 0x000000010d649b1e XUL`mozilla::dom::TimeoutManager::RunTimeout(this=0x00000001624848e0, aNow=0x00007fff5f081408, aTargetDeadline=0x00007fff5f081410) at TimeoutManager.cpp:868 frame #46: 0x000000010d62dd7c XUL`mozilla::dom::TimeoutExecutor::MaybeExecute(this=0x000000015e0ce150) at TimeoutExecutor.cpp:167 frame #47: 0x000000010d62e0cb XUL`mozilla::dom::TimeoutExecutor::Notify(this=0x000000015e0ce150, aTimer=0x00000001058d3240) at TimeoutExecutor.cpp:235 frame #48: 0x000000010d62e10c XUL`non-virtual thunk to mozilla::dom::TimeoutExecutor::Notify(this=0x000000015e0ce150, aTimer=0x00000001058d3240) at TimeoutExecutor.cpp:0 frame #49: 0x000000010b321bbb XUL`nsTimerImpl::Fire(this=0x00000001059156b0, aGeneration=2) at nsTimerImpl.cpp:517 frame #50: 0x000000010b3216ad XUL`nsTimerEvent::Run(this=0x00000001061e2408) at TimerThread.cpp:286 frame #51: 0x000000010b32b658 XUL`nsThread::ProcessNextEvent(this=0x0000000103f1ba00, aMayWait=false, aResult=0x00007fff5f081d73) at nsThread.cpp:1040 frame #52: 0x000000010b328d4c XUL`NS_ProcessPendingEvents(aThread=0x0000000103f1ba00, aTimeout=10) at nsThreadUtils.cpp:463 frame #53: 0x000000010fe8f1be XUL`nsBaseAppShell::NativeEventCallback(this=0x0000000105aa9a10) at nsBaseAppShell.cpp:99 frame #54: 0x000000010ff2bbf7 XUL`nsAppShell::ProcessGeckoEvents(aInfo=0x0000000105aa9a10) at nsAppShell.mm:409 frame #55: 0x00007fff7dbb2321 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17 frame #56: 0x00007fff7db9321d CoreFoundation`__CFRunLoopDoSources0 + 557 frame #57: 0x00007fff7db92716 CoreFoundation`__CFRunLoopRun + 934 frame #58: 0x00007fff7db92114 CoreFoundation`CFRunLoopRunSpecific + 420 frame #59: 0x00007fff7d0f2ebc HIToolbox`RunCurrentEventLoopInMode + 240 frame #60: 0x00007fff7d0f2cf1 HIToolbox`ReceiveNextEventCommon + 432 frame #61: 0x00007fff7d0f2b26 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 71 frame #62: 0x00007fff7b68ba54 AppKit`_DPSNextEvent + 1120 frame #63: 0x00007fff7be077ee AppKit`-[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796 frame #64: 0x000000010ff2a5d4 XUL`::-[GeckoNSApplication nextEventMatchingMask:untilDate:inMode:dequeue:](self=0x0000000100eee980, _cmd="nextEventMatchingMask:untilDate:inMode:dequeue:", mask=18446744073709551615, expiration=4001-01-01 00:00:00 UTC, mode="kCFRunLoopDefaultMode", flag=YES) at nsAppShell.mm:131 frame #65: 0x00007fff7b6803db AppKit`-[NSApplication run] + 926 frame #66: 0x000000010ff2c817 XUL`nsAppShell::Run(this=0x0000000105aa9a10) at nsAppShell.mm:688 frame #67: 0x000000011311242b XUL`nsAppStartup::Run(this=0x000000010570fd30) at nsAppStartup.cpp:288 frame #68: 0x0000000113273043 XUL`XREMain::XRE_mainRun(this=0x00007fff5f083fe8) at nsAppRunner.cpp:4646 frame #69: 0x00000001132745e4 XUL`XREMain::XRE_main(this=0x00007fff5f083fe8, argc=5, argv=0x00007fff5f084670, aConfig=0x00007fff5f0841b8) at nsAppRunner.cpp:4810 frame #70: 0x0000000113274d5c XUL`XRE_main(argc=5, argv=0x00007fff5f084670, aConfig=0x00007fff5f0841b8) at nsAppRunner.cpp:4905 frame #71: 0x0000000113288257 XUL`mozilla::BootstrapImpl::XRE_main(this=0x0000000100e13150, argc=5, argv=0x00007fff5f084670, aConfig=0x00007fff5f0841b8) at Bootstrap.cpp:45 frame #72: 0x0000000100b7c0e6 firefox`do_main(argc=5, argv=0x00007fff5f084670, envp=0x00007fff5f0846a0) at nsBrowserApp.cpp:236 frame #73: 0x0000000100b7bbc1 firefox`main(argc=5, argv=0x00007fff5f084670, envp=0x00007fff5f0846a0) at nsBrowserApp.cpp:309 frame #74: 0x0000000100b7bad4 firefox`start + 52

Tracking for 56. We don't know the release date for Sierra, it seems likely to happen while Firefox 56 is on the release channel this fall (after Sept. 15).

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #55) > Tracking for 56. We don't know the release date for Sierra, it seems likely > to happen while Firefox 56 is on the release channel this fall (after Sept. > 15). Just to be clear, this is a current problem with Sierra (10.12.x) that is already out. It is High Sierra (10.13) that will most likely be shipping this fall.

I'm going to reassign this to spohl since he's the one actively working on fixing the test failure that's blocking this from landing. spohl: feel free to land both patches once you get your test fix reviewed. Thanks for digging into that!

Comment on attachment 8898929 [details] [diff] [review] Update test Thanks for the thorough investigation! This change looks good to me, then.

Thanks, Markus! Are we cleared to land this, or did you want to hold off due to the regressions that the 10.11 SDK causes on 10.13?

Let's land this now; that gives us more time to shake out any other regressions. And we're not going to uplift this, so we have a bit of time to deal with the 10.13 fallout.

https://hg.mozilla.org/integration/mozilla-inbound/rev/ea1de92a83a1dc0f20972940013b62089cd06fde Bug 1324892: Update Mac builds to use the OS X 10.11 SDK. r=mshal https://hg.mozilla.org/integration/mozilla-inbound/rev/ac1381f38c3f7cd0f548d55cea7840f9351fccf5 Bug 1324892: Fix tests for switch to macOS SDK 10.11. r=mstange

Backed out for toolchain bustage: https://hg.mozilla.org/integration/mozilla-inbound/rev/81de05d441e4bdb2b37b0344f3f88de5076c5edb https://hg.mozilla.org/integration/mozilla-inbound/rev/c1d35cfb410bfeacc3a30e838c792ae27aa551d0 Push with bustage: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=ac1381f38c3f7cd0f548d55cea7840f9351fccf5&filter-resultStatus=exception&filter-resultStatus=usercancel&filter-resultStatus=runnable&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=retry Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=129626412&repo=mozilla-inbound [task 2017-09-08T17:39:26.431688Z] -- Check for working C compiler: /builds/worker/workspace/build/src/clang/bin/clang [task 2017-09-08T17:39:26.467991Z] -- Check for working C compiler: /builds/worker/workspace/build/src/clang/bin/clang -- broken [task 2017-09-08T17:39:26.468221Z] CMake Error at /usr/local/share/cmake-3.7/Modules/CMakeTestCCompiler.cmake:51 (message): [task 2017-09-08T17:39:26.468253Z] The C compiler "/builds/worker/workspace/build/src/clang/bin/clang" is not [task 2017-09-08T17:39:26.468274Z] able to compile a simple test program. [task 2017-09-08T17:39:26.468284Z] [task 2017-09-08T17:39:26.468303Z] It fails with the following output: [task 2017-09-08T17:39:26.468313Z] [task 2017-09-08T17:39:26.468334Z] Change Dir: /builds/slave/moz-toolchain/build/stage1/build/CMakeFiles/CMakeTmp [task 2017-09-08T17:39:26.468346Z] [task 2017-09-08T17:39:26.468355Z] [task 2017-09-08T17:39:26.468363Z] [task 2017-09-08T17:39:26.468383Z] Run Build Command:"/usr/local/bin/ninja-build" "cmTC_4df33" [task 2017-09-08T17:39:26.468396Z] [task 2017-09-08T17:39:26.468415Z] [1/2] Building C object CMakeFiles/cmTC_4df33.dir/testCCompiler.c.o [task 2017-09-08T17:39:26.468428Z] [task 2017-09-08T17:39:26.468444Z] clang-3.9: warning: no such sysroot directory: [task 2017-09-08T17:39:26.468464Z] '/builds/worker/workspace/build/src/MacOSX10.10.sdk' [task 2017-09-08T17:39:26.468475Z] [task 2017-09-08T17:39:26.468490Z] [2/2] Linking C executable cmTC_4df33 [task 2017-09-08T17:39:26.468501Z] [task 2017-09-08T17:39:26.468521Z] FAILED: : && /builds/worker/workspace/build/src/clang/bin/clang -target [task 2017-09-08T17:39:26.468539Z] x86_64-apple-darwin11 -mlinker-version=137 -B [task 2017-09-08T17:39:26.468561Z] /builds/worker/workspace/build/src/cctools/bin -isysroot [task 2017-09-08T17:39:26.468581Z] /builds/worker/workspace/build/src/MacOSX10.10.sdk [task 2017-09-08T17:39:26.468599Z] -I/builds/worker/workspace/build/src/MacOSX10.10.sdk/usr/include [task 2017-09-08T17:39:26.468609Z] -iframework [task 2017-09-08T17:39:26.468629Z] /builds/worker/workspace/build/src/MacOSX10.10.sdk/System/Library/Frameworks [task 2017-09-08T17:39:26.468648Z] -arch x86_64 -Wl,-search_paths_first -Wl,-headerpad_max_install_names [task 2017-09-08T17:39:26.468666Z] -Wl,-syslibroot,/builds/worker/workspace/build/src/MacOSX10.10.sdk [task 2017-09-08T17:39:26.468685Z] -Wl,-dead_strip CMakeFiles/cmTC_4df33.dir/testCCompiler.c.o -o cmTC_4df33 [task 2017-09-08T17:39:26.468693Z] && : [task 2017-09-08T17:39:26.468700Z] [task 2017-09-08T17:39:26.468715Z] clang-3.9: warning: no such sysroot directory: [task 2017-09-08T17:39:26.468731Z] '/builds/worker/workspace/build/src/MacOSX10.10.sdk' [task 2017-09-08T17:39:26.468738Z] [task 2017-09-08T17:39:26.468752Z] ld: warning: directory not found for option [task 2017-09-08T17:39:26.468772Z] '-F/builds/worker/workspace/build/src/MacOSX10.10.sdk/System/Library/Frameworks' [task 2017-09-08T17:39:26.468781Z] [task 2017-09-08T17:39:26.468790Z] [task 2017-09-08T17:39:26.468813Z] ld: warning: directory not found for option '-F/System/Library/Frameworks/' [task 2017-09-08T17:39:26.468825Z] [task 2017-09-08T17:39:26.468839Z] ld: library not found for -lSystem [task 2017-09-08T17:39:26.468853Z] [task 2017-09-08T17:39:26.468873Z] clang-3.9: error: linker command failed with exit code 1 (use -v to see [task 2017-09-08T17:39:26.468886Z] invocation) [task 2017-09-08T17:39:26.468897Z] [task 2017-09-08T17:39:26.468911Z] ninja: build stopped: subcommand failed. [task 2017-09-08T17:39:26.468922Z] [task 2017-09-08T17:39:26.468934Z] [task 2017-09-08T17:39:26.468942Z] [task 2017-09-08T17:39:26.468953Z] [task 2017-09-08T17:39:26.468960Z] [task 2017-09-08T17:39:26.468980Z] CMake will not be able to correctly generate this project. [task 2017-09-08T17:39:26.468995Z] Call Stack (most recent call first): [task 2017-09-08T17:39:26.469011Z] CMakeLists.txt:48 (project) [task 2017-09-08T17:39:26.469019Z] [task 2017-09-08T17:39:26.469027Z] [task 2017-09-08T17:39:26.471514Z] -- Configuring incomplete, errors occurred! [task 2017-09-08T17:39:26.471548Z] See also "/builds/slave/moz-toolchain/build/stage1/build/CMakeFiles/CMakeOutput.log". [task 2017-09-08T17:39:26.471574Z] See also "/builds/slave/moz-toolchain/build/stage1/build/CMakeFiles/CMakeError.log". [task 2017-09-08T17:39:26.473857Z] Traceback (most recent call last): [task 2017-09-08T17:39:26.473882Z] File "./build-clang.py", line 544, in <module> [task 2017-09-08T17:39:26.473959Z] build_type, assertions, python_path, gcc_dir, libcxx_include_dir) [task 2017-09-08T17:39:26.473984Z] File "./build-clang.py", line 220, in build_one_stage [task 2017-09-08T17:39:26.474015Z] build_package(build_dir, cmake_args) [task 2017-09-08T17:39:26.474038Z] File "./build-clang.py", line 67, in build_package [task 2017-09-08T17:39:26.474059Z] run_in(package_build_dir, ["cmake"] + cmake_args) [task 2017-09-08T17:39:26.474081Z] File "./build-clang.py", line 43, in run_in [task 2017-09-08T17:39:26.474096Z] check_run(args) [task 2017-09-08T17:39:26.474113Z] File "./build-clang.py", line 36, in check_run [task 2017-09-08T17:39:26.474135Z] assert r == 0 [task 2017-09-08T17:39:26.474149Z] AssertionError [taskcluster 2017-09-08 17:39:27.772Z] === Task Finish

Looks like we need to update a directory in our toolchain. Ted, can you look into this?

This updates all 10.10 paths to 10.11. Waiting for try to tell us if this was all that was needed here (comment 66).

Comment on attachment 8906068 [details] [diff] [review] Update toolchain paths Looks like this does it.

Comment on attachment 8906068 [details] [diff] [review] Update toolchain paths Review of attachment 8906068 [details] [diff] [review]: ----------------------------------------------------------------- Ah, oops! I think all of those toolchain task scripts were landed after I wrote that initial patch.

https://hg.mozilla.org/integration/mozilla-inbound/rev/643c70cb9158f57712042853b3265b1e4ec8d474 Bug 1324892: Update Mac builds to use the OS X 10.11 SDK. r=mshal https://hg.mozilla.org/integration/mozilla-inbound/rev/096247c952ea54152e8a0faa360dc8470d892214 Bug 1324892: Fix tests for switch to macOS SDK 10.11. r=mstange https://hg.mozilla.org/integration/mozilla-inbound/rev/8a67124d36d9ed0c6fa9111ae33620aeddc4ce4d Bug 1324892: Update toolchain paths to point to the macOS 10.11 SDK. r=ted

https://hg.mozilla.org/mozilla-central/rev/643c70cb9158 https://hg.mozilla.org/mozilla-central/rev/096247c952ea https://hg.mozilla.org/mozilla-central/rev/8a67124d36d9

This has totally messed up the title and tab bar on macOS… 3:48.55 INFO: Last good revision: a85670ee5e959299556b6ad0ef1370743debc43d 3:48.55 INFO: First bad revision: 8a67124d36d9ed0c6fa9111ae33620aeddc4ce4d 3:48.55 INFO: Pushlog: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=a85670ee5e959299556b6ad0ef1370743debc43d&tochange=8a67124d36d9ed0c6fa9111ae33620aeddc4ce4d

It seems that is a known issue (bug 1391790) but please consider a backout until this is fixed because it makes tab switching difficult. You have to click the bottom line of the tabs where the title overlaps the titles…

Pushed by mozilla@jorgk.com: https://hg.mozilla.org/comm-central/rev/b0d8d878f627 Keep build files in sync (Port bug 1324892: Update Mac builds to use the OS X 10.11 SDK). rs=bustage-fix DONTBUILD

(In reply to Sören Hentzschel from comment #73) > It seems that is a known issue (bug 1391790) but please consider a backout > until this is fixed because it makes tab switching difficult. You have to > click the bottom line of the tabs where the title overlaps the titles… Are you on the 10.13 Beta, or the released 10.12?

oh, sorry for the missing information. I am on 10.13 Beta.

Per comment 60.

https://hg.mozilla.org/integration/mozilla-inbound/rev/0f15faa4f0fab46833bf9f8ca8c48c5f482f22b3 Backout f89ae3c450ce (bug 1324892) for busting Thunderbird. r=me

https://hg.mozilla.org/mozilla-central/rev/0f15faa4f0fa (Backout)

Ignore that, that's the backout for bug 1398582.