Writing an automated test for bug 403232 is proving to be very difficult. This an OS-specific focus bug (it's specific to Cocoa widgets on OS X), and the problems I've encountered may effect all tests for OS-specific focus bugs (or perhaps all tests for focus bugs of any kind) ... but I'm not yet sure about this. The STR for bug 403232 requires mouse-clicks to be synthesized on objects outside of the DOM (for example you need to use the mouse to switch tabs). So neither a mochitest nor a chrome test will work. In principle a browser chrome test should work (since it allows the test to access non-DOM browser elements that (for security reasons) aren't accessible from a web page). But Martijn and I tried two different ways to check whether focus was correctly set or not, and neither works properly. (In later comments I will post Martijn's browser chrome test and the changes I made to it.) Martijn's test uses document.commandDispatcher.focusedElement to find the currently focused element. But this only shows Gecko's notion of what the focused element should be, and appears to be too high-level to catch bug 403232 -- Martijn's test always passes, whether or not bug 403232 has been fixed. Seeing that document.commandDispatcher.focusedElement didn't work, I tried adding onblur and onfocus handlers to the <input> control or <textbox> element inside the web page (accessible via the "contentDocument"). But this doesn't work either, and in fact is very fragile: My test always passes when the "Browser chrome tests" window is behind the window where the tests are run, and always fails when the "Browser chrome tests" window is above the window where the tests are run. At this point I'm not sure it's possible to write an automated test for bug 403232. But over time I will keep trying, and would appreciate any help I can get.

Thanks very much for your help, Martijn! This would have been even more painful (and time consuming) without it.

Would it be possible (and not too hard) to mix a "regular" test together with native (cocoa) code to make what you want happen? For example using a ObjC++ xpcom component that does whatever high-level stuff you want, when called from the JS.

(In reply to comment #4) I don't know (I'm new to the Mozilla automated tests and don't know much about them). But I suspect detecting low-level (OS-level) focus errors will be difficult no matter how you do it.

If you know some low-level code you want to run, you can just wrap it in an XPCOM component, and call it from the javascript. Let me know if you want any help in showing that. The XPCOM component could do a test (click somewhere? see if something then has the right focus status?) and return a result to the JS, which would propagate the result to the test suite. This is only useful if you know some specific native code you want to run that would make this test possible though. I just read comment 0 - an idea: Maybe you can test for this bug by synthesizing some key events and then checking that they went into the textbox (when you are in the "broken" state)?

Håkan, thanks for the suggestions. I'll follow up on them when I have time. But the goal is to write some kind of automated test (one that will work with existing test harnesses, or perhaps newly designed/written ones). Would your strategy of writing an XPCOM component and calling it from JS work with any of Mozilla.org's current test harnesses, or with any others that you're aware of? If so, please let me know.

Feel free to move this into testing:Mochitest if the test harness is lacking something you need.

Steven, any update to this bug?

This bug depends (at least) on bug 470845.

It'd also be nice (though probably not essential) to have bug 418475 and bug 418859 fixed.