Closed Bug 1554493 Opened 6 years ago Closed 5 years ago

Fission tests shouldn't run with try syntax

Categories

(Firefox Build System :: Task Configuration, defect)

defect
Not set
normal

Tracking

(Fission Milestone:M4, firefox-esr60 unaffected, firefox-esr68 unaffected, firefox68 unaffected, firefox69 fixed)

RESOLVED FIXED
mozilla69
Fission Milestone M4
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox68 --- unaffected
firefox69 --- fixed

People

(Reporter: rmaries, Assigned: ahal)

References

(Regression)

Details

(Keywords: regression)

Attachments

(3 files)

Flags: needinfo?(jmaher)
Flags: needinfo?(ahal)

From IRC:

nika> apavel|sheriffduty: hey - if you're seeing Fission tests they're not supposed to be visible or on by default
5:36 PM They're super orange
5:36 PM (or red(
<nika> There may have been a mistake adding them which makes them visible.

Pushed by archaeopteryx@coole-files.de: https://hg.mozilla.org/mozilla-central/rev/893e1d6cad95 Don't run fission tests on Try because they are permafailing. a=cleaner-try-pushes

I'm very confused at how those tasks appeared on that central push (and equally confused why removing the "try" project seems to have fixed it). If I run mach taskgraph target locally (which uses mozilla-central's parameters by default), I don't see fission tasks listed. This means that "fission" isn't targeted for central (as expected) and shouldn't show up on any central pushes.

Adding them to try was intentional since the only way to select them would have been to use mach try fuzzy --full or mach try chooser --full. In theory, this shouldn't have any impact on whether or not they run on central, but clearly something is afoot.

Flags: needinfo?(ahal)

As far as I understood, they got manually added on central (triggered by jya in comment 0) to check if the failures on jya's Try push were from his push or also occur on central.

The issue from my POV is that Try pushes with -u all or a try syntax requesting wpt jobs will yield permafailing W-fis jobs.

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #4)

As far as I understood, they got manually added on central (triggered by jya in comment 0) to check if the failures on jya's Try push were from his push or also occur on central.

Yes, I confirm.

Following the massive presence of orange, and having failed to realise those were fission tests I ran the same test on central to determine if my changes were the cause or if it was already present on central and missed.

The issue from my POV is that Try pushes with -u all or a try syntax requesting wpt jobs will yield permafailing W-fis jobs.

indeed.

While their are known to fail, they should only be enabled if explicitly asked for it.

Ok, this makes sense.

Fwiw they do only show up when explicitly requested when using one of the more modern try selectors (fuzzy, chooser, etc). But try syntax doesn't have this ability. The proper solution is to fix bug 1400295 and bring mach try syntax in-line with the other selectors. But I don't have time to work on that at the moment, so in the meantime I can add "yet another exception" to the try syntax parser. This means it won't be possible to schedule fission tasks with try syntax, but that's probably fine.

Flags: needinfo?(jmaher)
Assignee: nobody → ahal
Status: NEW → ASSIGNED
Summary: Massive Fission failures on Windows 10 x64 & Linux x64 → Fission tests shouldn't run with -u all
Summary: Fission tests shouldn't run with -u all → Fission tests shouldn't run with try syntax

Depends on D32701

(In reply to Andrew Halberstadt [:ahal] from comment #3)

Adding them to try was intentional since the only way to select them would have been to use mach try fuzzy --full or mach try chooser --full.

Note that doing this doesn't currently affect what is selectable via mach try fuzzy, as that uses a mozilla-central graph. It would affect try syntax, but if it is going to be filtered out there anyway, it won't have an effect


One thing I would have thought would caused these tests to not run via syntax is this logic to skip tests that are not tier-1 if they are other variants that are, but given that -fis is added to the name, all the tests are going to be tier 3.

Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5925360fcc6c [ci] Improve some formatting in try_option_syntax.py, r=jmaher https://hg.mozilla.org/integration/autoland/rev/c584c415ca35 [ci] Don't run 'fission' tests with try syntax, r=jmaher https://hg.mozilla.org/integration/autoland/rev/4caaa2fcdf35 [ci] Re-enable fission tests on try, r=jmaher

FYI, "-u all" still shows these broken tests:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b9e2e628cf8e0eb9fa38a001b43d9bc1430809f
It's rather confusing when a whitespace change shows up as orange across the board...

The fix hasn't merged to central yet.

Ah, I misread "integration" in the push links above as "inbound".
OK, I'll try again later after it's merged...

Not sure why leave-open was set, but I don't think there's anything else to do here.

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Keywords: leave-open
Resolution: --- → FIXED
Target Milestone: --- → mozilla69

Retroactively moving fixed bugs whose summaries mention "Fission" (or other Fission-related keywords) but are not assigned to a Fission Milestone to an appropriate Fission Milestone.

This will generate a lot of bugmail, so you can filter your bugmail for the following UUID and delete them en masse:

0ee3c76a-bc79-4eb2-8d12-05dc0b68e732

Fission Milestone: --- → M4
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: