In Mozmill, it is currently impossible to access system level dialogs. This affects several test cases. Virtually all of our Mozmill tests flagged "not doable" are in this state because of this issue. There are 11 Smoketests and 26 BFTs currently flagged "not doable".

Mikeal posted something related to this in the newsgroups. I'm copying it here: > The restart testing framework in mozmill just got a feature that lets you > > trigger Python callbacks from JavaScript. This could conceivably be used to > > support access to system-level dialogs the way people using windmill have > > been testing file uploads and other things that require OS dialog support. We should move this to the Mozmill class so not only restart tests will have this feature. We can follow-up on that with the new implementation bug. Anthony can you file that on? > > There is a pretty good amount of documentation for different Python tools > > that can automate OS level widgets for Windows and Linux, not sure about Mac > > tho. Sounds great. Let's find out.

You have forgotten the most important information from his posting. So lets re-cite this part: > Example: > http://groups.google.com/group/windmill-dev/msg/0655c9822e03f313 > > There is a pretty good amount of documentation for different Python tools > that can automate OS level widgets for Windows and Linux, not sure about Mac > tho. > > http://pywinauto.openqa.org > https://fedorahosted.org/dogtail/ Looks like we have to find some tool for OS X now.

And lets add the callback feature as dependency which could be helpful.

Python callbacks are in. Does anyone wanna take a shot at using one of these OS libraries.

I'm not sure what else there is to resolve this ticket. Python callbacks are in, so you can use OS specific libraries now. I don't think it will be feasible to wrap the OS platform libraries in a way that could abstract them sufficiently for you to generalize an API for accessing certain windows. In the end, I think every time you interact with an OS dialog you'll need to write new code in one of the previously mentioned platform libraries available to Python. I think we can mark this ticket resolved and you guys can start taking a crack at using this stuff.

Sorry when I reopen the bug but there are still some open issues which probably need some more discussion. I'm not really happy with the current solution because... * When we go that way and only allow calling system level functions via the Python callback function we will make the extension useless for normal use. Means users who only install the extension will not be able to run our suite of tests for Firefox. We always force them to use the pyPI package. * Python callbacks are implemented asynchronously which doesn't give you a way to return any result from the tests which are ran against the system level controls. * Right now there is no dependency list of needed external modules to access the system level elements on each of the platforms. Everyone would have to install those manually - which is really bad IMO. Given those facts I believe we should better implement it as a component which is included inside the Mozmill extension. There is PyXPCOM out there (especially the available extension) which looks really promising. Here some links: * https://developer.mozilla.org/en/Building_PyXPCOM * http://pyxpcomext.mozdev.org/ Is there any Python library out there for accessing system elements which is platform independent?

I don't think it's technically feasible at this time to build a system that accesses system level dialogs directly from platform javascript. While Python callbacks don't allow you to send an event back over the bridge they do allow you to trigger a test failure which is sufficient for many cases. Most of the use cases for accessing a system level dialog from within a test will be easy to wait for on the javascript side. Sure, these tests can't run with only the extension but this just puts them in the same place that the restart tests are in. I agree that the dependencies might be a problem, it sucks to be telling people to install extra modules by hand in to their local Python. PyXPCOM requires a local Python and relevant packages to be installed so I don't see how this provides us a way around the issues you've described. There are libraries out there for **creating** desktop applications in a platform independent manor but simulating native OS events on arbitrary application is not possible with any library unless it was written for it (wxPython for example).

(In reply to comment #6) > Sorry when I reopen the bug but there are still some open issues which probably > need some more discussion. I'm not really happy with the current solution > because... > > * When we go that way and only allow calling system level functions via the > Python callback function we will make the extension useless for normal use. > Means users who only install the extension will not be able to run our suite of > tests for Firefox. We always force them to use the pyPI package. > > * Python callbacks are implemented asynchronously which doesn't give you a way > to return any result from the tests which are ran against the system level > controls. > > * Right now there is no dependency list of needed external modules to access > the system level elements on each of the platforms. Everyone would have to > install those manually - which is really bad IMO. > > Given those facts I believe we should better implement it as a component which > is included inside the Mozmill extension. There is PyXPCOM out there > (especially the available extension) which looks really promising. Here some > links: I disagree. The only way to completely solve the problems you mention above would be to implement common C++ XPCOM scriptable wrappers for the APIs that you are interested in. Then these would be available from the extension, but you'd have a separate version of the extension for each target operating system. I think the PyXPCOM stuff is too new to depend on, and like you mentioned by continuing to use Python for this, you end up pushing people into Python-module-installing-hell, which really blows regardless of what OS you're on. If we choose to do that, we're looking at about a month or so of dedicated work depending on the number and type of APIs that you want to be exposed. This will also have the annoying side effect of making Mozmill one of those dreaded "binary" extensions, so we'll have a much harder time staying current and in sync with the changing platform (we can mitigate that somewhat by using only Frozen interfaces, but for things like E10S we could be up the creek without a paddle). I'm really not much in favor of this solution, but that's the only other option I see. What do you want to do? This isn't a place where you get to have your cake and eat it too, unfortunately.

Given by our testdev meeting today jgriffin will try to put an example together with the help of the python callbacks.

Some notes: I've been investigating this on Windows. It appears that the call which displays the system dialogs (at least the Save As dialog) is blocking, since mozmill scripts are completely suspended while the dialog is visible. Thus, no javascript solution is possible and the python callbacks seem to be the only viable path. Second, because of the above, you cannot cause the dialog to appear in the mozmill script, but have to cause it to appear using the python callback. That is, to invoke the Save As dialog, you cannot use something like: controller.click(new elementslib.Elem(controller.menus["file-menu"].menuSave)); and then trigger a python callback to deal with it. Instead, you have to manage to invoke the "Save As" dialog from within the callback. Third, we already knew this would be messy, since we'd have different python dependencies on different OS's, and different code. But system-level dialogs differ also based on OS-version, so code that works on Windows 7 won't necessarily work on Windows XP, or Windows 2000, ad infinitum. So I'm afraid we'll have to identify which versions of which OS's we want to support with this, and then write code for each, and maintain them all. Yuck. This also means each time a new test is written that uses new OS-level code, we'd have to test that test on each version of each supported OS. Double yuck. Here's a sample test that illustrates how this could work. It works on Windows 7 only (and maybe Vista, see above!), and uses the pywinauto library. It must be run as a mozmill-restart test, because the python callbacks don't currently work in "regular" mozmill. I'll file a bug about that and fix it. If you want to run this test, you need to install: PIL - http://www.pythonware.com/products/pil/index.htm SendKeys - http://www.rutherfurd.net/python/sendkeys/index.html PyWinAuto - http://sourceforge.net/projects/pywinauto/files/ in that order. If we agree to continue on this path, I'll find OS-level libraries for Mac and Linux (hopefully), and can work with Whimboo on identifying which system-level dialogs we need to use, so we can write some python libraries to abstract some of the details and make future maintenance easier.

(In reply to comment #10) > Second, because of the above, you cannot cause the dialog to appear in the > mozmill script, but have to cause it to appear using the python callback. Oh, that should be possible. We could call the python callback before we trigger the event which opens the system level dialog or interacts in any way with the system level. Then we would have to wait for the dialog to appear in the backend function. That way we can really simulate all possible paths to invoke those system level actions.

> We could call the python callback before we > trigger the event which opens the system level dialog or interacts in any way > with the system level. This doesn't seem to be possible. The callback appears to be synchronous, so once you call the callback, mozmill script execution is suspended until the callback finishes.

Updated example which runs as a regular mozmill test, per the changes in bug 538058. See comment #10 in this bug for pre-requisites. This version also differs in that it deals with the asynchronous nature of the pywinauto functions, so it waits between each action for the next expected window before continuing. Of course we'd probably want to abstract this all into a SaveAs class, which could be called from any test's callback.

(In reply to comment #13) > This doesn't seem to be possible. The callback appears to be synchronous, so > once you call the callback, mozmill script execution is suspended until the > callback finishes. Are you sure? As given by Mikeal, Python callbacks will be executed asynchronously. So the Mozmill script should not stop the execution after we called the callback.

> As given by Mikeal, Python callbacks will be executed > asynchronously. Oh yes you're quite correct after all. The original test I used to verify this was buggy. :(

This version works on both Windows XP and Windows 7. It has abstracted the details of dealing with the dialog to a class, so the code in the callback itself is quite simple. In order to test this, you have to put SystemLevelCallbacks.py somewhere in your sys.path, like python/lib.

Jonathan, could you attach your work as a patch? That would be awesome and will make it much easier to handle.

Here's a newer version of the example. Apply the mozmill patch to mozmill, then unzip this example anywhere and run it. Tested on WinXP and Windows 7. The mozmill patch allows callbacks to import modules from files in the same directory. That works for this example, but going forward we probably want the module in some central location.

I agree that we probably want it in some central location going forward, but I really like the the approach here. It looks simple and pretty straight forward.

let's consider this for 1.4.2 if it works and is nearly there.

Jonathan, do you have a branch setup for this work on Github?

> Jonathan, do you have a branch setup for this work on Github? I haven't put it in github, since I hadn't yet worked out the python path issues from comment #20. The remaining issues are: - work out python path issues (how should mozmill find the callback file? the relative difference in their paths is not known ahead of time.) - need to add support for Linux and Mac if desired. I briefly tried to find a Mac equivalent for pywinauto but was unable to. - add support for more dialog types (currently only have code for the Save As dialog) - the asynchronous nature of the callbacks, combined with the requirement to use mozmill to interact with the UI, leads to confusing flow in tests, I believe. Re the last point: the current tests's flow is something like this: Mozmill test: 1. invoke python callback asynchronously to handle Save As dialog 2. invoke Save As dialog 3. wait until some condition is met so that we can tell the python callback terminated (in this case, maybe it's that the Download dialog appears) Python callback: 4. wait until Save As dialog appears 5. handle Save As dialog I don't think this is an ideal model, but I'm not familiar enough with mozmill's guts to know what a better model might look like. It might be to encapsulate things on the mozmill side with a synchronous function that wouldn't return until the python callback had notified mozmill it was done (something that isn't supported atm).

This looks like it's too big for 1.4.2, let's consider it for .next.

Is there any reason this shouldn't be built into mozmill (at least the handles necessary to interact with system level diaglogues). Using python callbacks relies on each set of tests having a large amount of python code to do this. If this is generally desired functionality, we should build it into mozmill, in my opinion, rather than trying to tie it into python callbacks that the tests have to implement their handlers themselves

Absolutely, I think that it should be part of mozmill. It's just quite fuzzy to me exactly how we're going to do that.

Mozmill will reach its end of life soon. We are currently working on getting all the tests for Firefox ported to Marionette. For status updates please see bug 1080766.