Many of the new tests use tags like: skip-if.os = mac run-if.config = ipc but it seems these tags are not being respected. The testing machines still run because the #ifdef mechanism in the makefile still keeps wrong-os tests out of the test directory, but I was not able to skip tests solely from within xpcshell.ini (am I doing something wrong?). Also, what config tags are supposed to be supported? I saw ipc,mozcrashreporter and a few others, but I would need a config tag that tells me if MOZ_CALENDAR is set or not.

To be more specific, I need these lines to work: [include:calendar/test/xpcshell.ini] run-if.config = calendar which means it should run if MOZ_CALENDAR is set in comm-central's autoconfig.mk.

Jeff and Joel are working on enabling this support in the parser.

I'm happy to help get this working but I'm not sure of the action item here. The integration of mozmill with e.g. xpcshell is described at https://wiki.mozilla.org/Projects/ManifestDestiny/Integration What this means in terms of xpcshell is that the mozinfo dictionary from writemozinfo.py is passed verbatim to TestManifest: http://mxr.mozilla.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#82 The format has changed a bit; `run-if.config = calendar` would now be `run-if = config == 'calendar'`; but there is no config variable (that's a pretty strange name for it anyway). I'm not sure how the xpcshell tests are run on comm-central or what the best solution is. If you wrap this at all, you can either override build_test_list method, or add to mozinfo, or augment what writemozinfo.py does. Or we can morph this bug to having writemozinfo.py to output something to indicate the app you're on (not sure how/if this works with multiple --enable-application lines). Does any of that make sense? Not sure what the actionability is here or what to do with this bug. I'm in #ateam if you want to interact more directly

I believe the xpcshell tests are run almost the same on comm-central as they are on mozilla-central. I don't want to do too much specialcasing for Calendar since it will likely stay unmaintained. When building Lightning, --enable-application will always be mail, so that doesn't really help. Ideally there would be a way to map the autoconf.mk variables to something that can be used in xpcshell.ini. I'll have a look at the mozinfo stuff tomorrow.

Ok, writemozinfo could be a solution for this issue. The problem is that there needs to be an easy way for comm-central to extend the current mozinfo.json. From looking at the current code in mozilla/configure.in, the file is just overwritten. Maybe this could be solved by making writemozinfo.py read the current json file, apply its changes, and then write it back out. This way, adding a script in comm-central that extends mozinfo.json won't conflict. What do you think?

On IRC, jhammel suggested that you have it backwards. You should, after running m-c's configure script, read in the mozinfo.json file in c-c's configure and add any data you deem necessary at that point.

Is this still an issue?

I'd rather ask Standard8/jcranmer here what the best way to proceed is. If we want to add custom data to mozinfo.json, this would totally work out for Lightning and might also be a good idea for other parts of c-c.

Since the elimination of the master manifest, conditional inclusion of xpcshell tests in moz.build files ought to work well enough for most of the uses of mozinfo.json. The more interesting problem is that Lightning doesn't get installed into packaged-tests, so tests will work fine locally but not on tbpl--something that isn't easily solved with a mozinfo.json file. Another issue is that c-c's current build process doesn't let us overwrite mozinfo.json during configure. I think there may be some benefit to adding things to mozinfo.json in the long run, but the current one appears to be working fine for us at the moment.

In that case its probably fine to close this bug and create a new one when the need comes up again. Thanks for the info!