As talked over on bug 799406 it would be very helpful to have a suite of ui tests for the NSIS installers we support. Those are specifically the full and stub installers. As Rob answered on that bug: >> Regarding testing of NSIS installers, is there a test framework out there we >> could leverage to test the UI? I can remember that I have looked out for one >> more than a year ago but didn't found something. > > I used MS Visual Test back in the 90's for this type of testing and I would be > highly surprised if there wasn't something similar today. For quick, one off > tests I've used vbscript though I don't think it can check UI elements. Once we find the time we should check what we can do here, given that Windows is the major platform for us and having no ui tests makes me nervous.

Not blocking ship for v1 of the English deployment, but this is most certainly a want.

I haven't checked the details of Cobra but it could be an option if there is no other framework dedicated to NSIS installers is available.

I got in contact with the NSIS team yesterday and asked if there is an ui testing framework available for NSIS. But there isn't. So we have to find an alternative solution.

What's the status and priority of this bug? We'd like to get it resolved as it blocks getting automated coverage for a recent stub installer regression (bug 910459).

Might be a good thing to have a bouncer specific test since that has been very consistently where the problem has happened.

(In reply to Robert Strong [:rstrong] (do not email) from comment #5) > Might be a good thing to have a bouncer specific test since that has been > very consistently where the problem has happened. Who can take care of that? Is that a RelEng responsibility?

Not sure but perhaps Laura Thomson

Robert, I have a side-question. Would it be possible to add at least a silent mode to the stub installer? That way we could at least install Firefox and check if the correct build has been selected.

Possibly at some point in the future but not likely anytime soon.

Oh, I missed that there was bug 799406, which has been marked as wontfix at some point. Looks like it was not clear why we need that mode. I will reopen the other bug.

(In reply to Henrik Skupin (:whimboo) from comment #10) > Oh, I missed that there was bug 799406, which has been marked as wontfix at > some point. Looks like it was not clear why we need that mode. I will reopen > the other bug. Specifically, any organization needing to deploy using a silent installer should be using the full installer. It would be really a great thing if there were a simple test of bouncer vs. adding code to the stub (which we really try to avoid due to size and complexity) for a silent option that deployments should never use so it can be automated with mozmill. Ideally, an automation framework that can interact with UI would be better especially since that also tests the UI (you and I spoke of visual test which I used in the 90's for this purpose and I am sure there are newer ones though I don't have time to investigate and evaluate such items)