Closed Bug 1532809 Opened 6 years ago Closed 6 years ago

There should be Try syntax for all of the debugger tests

Categories

(DevTools :: Debugger, enhancement, P3)

enhancement

Tracking

(firefox67 fixed)

RESOLVED FIXED
Firefox 67
Tracking Status
firefox67 --- fixed

People

(Reporter: jlast, Assigned: jlast)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

I currently run

./mach try -b do -p linux64,macosx64,win32,win64 -u mochitest-clipboard-e10s,mochitest-e10s-dt,mochitest-chrome -t none

this catches all of the mochitests that are likely to be hit,

it does not catch the server unit tests

it also does not catch the new jest and flow tests

./mach try fuzzy -q "debugger-test"

it also doesn't catch the talos tests

./mach try -b o -p linux64 -u none -t damp-e10s --rebuild-talos 6

We should try to have a single try run that ideally runs all of these tests or all but the talos tests if they can be expensive.

Some options are:

  1. switching over to fuzzy
  2. using the new presets feature

This might work:

./mach try fuzzy -q "mochitest-clipboard-e10s" -q "mochitest-e10s-dt" -q "mochitest-chrome" -q "debugger-tests"

...

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3e852eda82327f4579fe6ead234670f91206406b

It would also be nice if using '-u all' would run the jest/flow debugger tests.

It'd be interesting if we could make something work with ./mach try chooser as well.

At one time you could pass --tag to try syntax, however that appears broken now.

(In reply to Mark Banner (:standard8) from comment #4)

At one time you could pass --tag to try syntax, however that appears broken now.

Yes, we should get this working again. In the meantime...

I just recently landed in-tree presets:
https://firefox-source-docs.mozilla.org/tools/try/presets.html#editing-and-sharing-presets

This means you can add a preset to tools/tryselect/try_presets.yml and it will be available to everyone with:

$ ./mach try --preset <name>

There are still a couple of follow-up series in the process of landing to make these work a bit better, so I haven't advertised them yet, but feel free to go ahead and add one.

Note: I strongly recommend creating presets with |mach try fuzzy|, even if you use |mach try syntax| locally. Not only because I prefer fuzzy, but because:

A) It is well structured
B) It is more powerful
C) People will be able to post-filter the preset selection (once bug 1531360 lands)
D) You can select tasks much more granularly

If you aren't familiar with fuzzy, please ping me and I can help you craft a preset.

Here are some notes on the fuzzy queries... https://firefox-source-docs.mozilla.org/tools/try/selectors/fuzzy.html

I'll try and set up the right fuzzy query that we can then turn into a preset.

Priority: -- → P3

Okay, i think i have something that is pretty good

debugger-tests:
  selector: fuzzy
  description: >-
    Runs all of the relevant debugger tests
  query:
   -q "debugger"
   -q "'mozlint-eslint"
   -q "!asan !pgo devtools"
   -q "!android !asan !pgo 'mochitest-chrome"
   -q "!android !asan !pgo 'mochitest-clipboard-e10s"

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3c8a067b2b82f1bf566b8d7ceabd31b93fac67f6

There are some really nice things here:

  1. it includes the debugger test suite (flow, jest)
  2. it includes webreplay
  3. it does not run unnecessary jobs
  4. it probably would be good for most devtoolers as it is not really debugger specific

You can play with it here:

./mach try fuzzy -q "debugger" -q "'mozlint-eslint" -q "!asan !pgo devtools" -q "!android !asan !pgo 'mochitest-chrome" -q "!android !asan !pgo 'mochitest-clipboard-e10s"

Nice! I would recommend using the ' prefix on debugger and devtools just in case. I'd also combine those last two queries like this:

!android !asan !pgo 'mochitest-chrome | 'mochitest-clipboard-e10s

Pushed by jlaster@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7f67893c6896
There should be Try preset debugger tests. r=ahal
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67
Assignee: nobody → jlaster
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: