Open Bug 1688043 Opened 4 years ago Updated 2 years ago

test-verify-wpt's errorsummary.log don't contain the group names that run in the task

Categories

(Testing :: web-platform-tests, defect, P3)

Default
defect

Tracking

(Not tracked)

People

(Reporter: marco, Unassigned)

References

(Blocks 1 open bug)

Details

James, can you please provide some info for this one, thank you!

Flags: needinfo?(james)

Is this still a problem?

Generally wptrunner doesn't know what the right group name for a test is; that logic got moved to the task selection. It could guess based on the directory name, but that might get out of sync with any changes in the test selection logic. So maybe the right solution here is for mozharness to pass in the correct group name for each test in the verify run via a groups JSON file, like we do for full test runs.

Severity: -- → S3
Flags: needinfo?(james)
Priority: -- → P3

Yes it seems to still be happening, e.g. https://firefox-ci-tc.services.mozilla.com/tasks/OnW8lgw2R0yDREyG6ggz0w/runs/0/logs/public/test_info/wpt-test1_errorsummary.log and https://firefox-ci-tc.services.mozilla.com/tasks/ExobDrDeQw22DOh5Nzq2ZA/runs/0/logs/public/test_info/wpt-test11_errorsummary.log.

The first has lines such as {"groups": ["reftest"], "action": "test_groups", "line": 33}, the second has lines such as {"groups": ["testharness"], "action": "test_groups", "line": 33}.

You need to log in before you can comment on or make changes to this bug.