in any given week we have upwards of 20 unique intermittent failure bugs related to marionette test cases.

For other test harnesses we assume that any failure of a given test case (filename) falls into a bucket and we track them all together. So an example would be:
test_foo.html | timed out
test_foo.html | failed to set widget bar
test_foo.html | assertion count 1 is great than 0

so we would then have:
test_foo.html | single tracking bug

and then collect all the failures together. This is beneficial as a developer will be editing test_foo.html to work on one failure, and could determine if it is worth the time to fix a secondary error. i.e. we don't lose track of the other stuff, but instead we can see it all in one place.

For Marionette tests, I see a batch of bug summaries that looks like this:

Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_minimum_select_one_character2_textarea2 | AssertionError: u'F' != u'\r\n'
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_minimum_select_one_character_textarea | AssertionError: 'A' != ''
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_minimum_select_one_character_textarea_disabled | AssertionError: 'A' != 'GHI JKL MNO PQR'
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_minimum_select_one_character_textarea_rtl | marionette_driver.errors.JavascriptException: [Exception... "Unexpected error"  nsresult
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_minimum_select_one_character_textarea_disabled | AssertionError: 'A' != ''
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_minimum_select_one_character_textarea_disabled | marionette_driver.errors.JavascriptException: [Exception... "Unexpected error" nsr
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py AccessibleCaretSelectionModeTestCase.test_carets_not_show_after_key_scroll_down_and_up | AssertionError: 'AAAAA\nBBBBB' == 'AAAAA\nBBBBB'

would it make sense to have a single bug:
Intermittent layout/base/tests/marionette/test_accessiblecaret_selection_mode.py | single tracking bug

or is there a different layer of granularity that makes sense for most of the marionette tests?

:whimboo, I would really appreciate your input on what would make sense here

If the failures are happening all within the same file I think that we can apply the same rules as for other test failures that cause single tracking bugs to be created. Then the triage owner will have to check if parts of the failures or all can be fixed. Usually we file new bugs for fixes that block the single failure bugs which means that we can keep the latter open way longer and no new such bug has to be filed.

So go ahead and apply these rules and we will adapt our work process. It's better to have general rules as test harness specific ones.

Actually let us discuss this in the next triage meeting on Monday. And we can give a team-wide feedback on that.

sure; I will stay tuned!

Joel, this proposal sounds fine to us. So lets go ahead and allow sheriffs to file single tracking bugs.

Regarding the handling of those bugs do we have some guidelines for developers and triage owners? If not I wonder if it might make sense to add something to eg. firefox-source-docs. Something like I mentioned in my comment 2 works actually pretty fine for us and allow for re-using the same bug multiple times. I'm personally not a fan of having multiple bugs with the same bug summary filed. What do you think?