Reftest manifests currently support a default-preferences identifier that can be used to set prefs once that then get used by all subsequent tests in a manifest. I need the ability to also specify manifest level skip-if identifiers.

I propose that rather than adding a new default-skip-if tag, we instead implement a generic defaults line that can contain any of the test modifiers. So, e.g:

defaults pref(some.pref,true) skip-if(someCondition) http
== test-foo.html test-foo-ref.html
!= test-bar.html test-bar-ref.html

// clears defaults
defaults
== test-baz.html test-baz-ref.html

For context, I need this because we are going to start scheduling tests more granularly (i.e by passing in paths to manifests). One of the blockers around getting reftest supported is the fact that we have a single root manifest that registers all the other manifests. This root manifest contains a few skip-if keys that would no longer be picked up if we were to start passing the child manifests into the harness directly. So instead, we need to encode these skip-if annotations directly into the included manifests themselves.

Depends on D63246

Needinfo'ing you because if you're anything like me, you never see those phabricator mentions:
https://phabricator.services.mozilla.com/D63247#inline-381237

Are you planning to remove the ability to have a skip/skip-if() default across an include line after you've done this? (Separate bug makes sense, but it would probably be good to have on file.)

I can answer the question to James since I'm usually the one to run that script (and will once I submit the phab review) -- it's fine.

I wasn't initially planning to (there are sub-manifests that have a skip on includes that won't get in the way of anything), but I can certainly do this if you like.

It is nice to have skips encoded directly in the manifest that needs them. That way tests will be skipped even when passing those manifests directly into the harness (rather than the parent).

I'll be filing another bug to deprecate the root manifest and can remove the ability to skip includes as part of that bug.

(In reply to Andrew Halberstadt [:ahal] from comment #6)

I wasn't initially planning to (there are sub-manifests that have a skip on includes that won't get in the way of anything), but I can certainly do this if you like.

It is nice to have skips encoded directly in the manifest that needs them. That way tests will be skipped even when passing those manifests directly into the harness (rather than the parent).

I'll be filing another bug to deprecate the root manifest and can remove the ability to skip includes as part of that bug.

Actually there are some skip-if(...) includes in the jstests.list manifests that look like they are skipping generated manifests. So we'll need to do something about those prior to removing the ability to skip includes. I'll file an independent bug to do this, though this work is getting a bit out of scope for my current project.

https://hg.mozilla.org/mozilla-central/rev/3eb51d4d8499
https://hg.mozilla.org/mozilla-central/rev/a183c3e2eacf

WPT import in https://github.com/web-platform-tests/wpt/pull/21926 .