tree-listbox.js uses the role=listbox but has the full functionality of a tree. It is used as a list in the addressbook, but as a full tree in the work-in-progress about:3pane.

It makes sense that they are sharing code, but I would like to distinguish between listbox and tree for accessibility purposes. We could use the similar mixin constructor, and just make it more fine grained.

We would have to stop using tree-listbox as a custom element name since it confuses the issue. Any ideas for other names? Would is=tree and is=listbox be ok, or are hyphened names still needed for custom extensions?

Links (for my future self and others):

https://www.w3.org/TR/wai-aria-1.1/#listbox
https://www.w3.org/TR/wai-aria-1.1/#tree
https://www.w3.org/TR/wai-aria-practices/#keyboard-interaction-24

@darktrojan I would like your general opinion on this direction to make sure this isn't working against something you have planned.

More specifically, are you planning on using TreeViewListbox in a purely listbox way, or will it always be functionally a tree? If the second, then we could change it to a role=tree, else we can do a similar mixin. What do you think about changing the name to just TreeView (and ListView)?

is="something" is for customized built-in version of Custom Elements (custom element that extends a built in element), so needs to have a name with hypen. Non-hypened names are not allowed for Custom Elements, though there is an temporary exception for chrome ones.

It's not clear whether you're talking about TreeListbox or TreeViewListbox, both get used as a tree in some places and a list in others. I don't recall exactly how I ended up deciding that the role should be listbox and not tree, I was having a pretty big fight with my screen reader at the time so it's possible that I just gave up once things started working as intended.

AFAICT, the major difference is whether the descendants are option or treeitem (potentially inside group). Is that a fair assessment? I think a tidy way to achieve this would be only setting the widget's role in connectedCallback if it didn't already have one (so it could be explicitly set in the HTML) and using the role to decide what the descendants are.

(In reply to Geoff Lankow (:darktrojan) from comment #3)

It's not clear whether you're talking about TreeListbox or TreeViewListbox, both get used as a tree in some places and a list in others.

I'm talking about both (all the classes in tree-listbox.js). I know TreeListbox is functionally used as both a tree and a listbox, but I wasn't sure if TreeViewListbox will ever used as a strict listbox (it cannot contain sub-lists).

AFAICT, the major difference is whether the descendants are option or treeitem (potentially inside group). Is that a fair assessment?

Yes. The other major difference is that a listbox wouldn't have sub-lists: this is the key structural difference. So a listbox element would only be along the lines of

<ul role="listbox">
  <li role="option">Item 1</li> 
  <li role="option">Item 2</li>
</ul>

whilst a tree element could be along the lines of

<ul role="tree">
  <li role="treeitem">
    Node 1
    <ul role="group" aria-expanded="true">
      <li role="treeitem">Leaf 1</li>
      <li role="treeitem">Leaf 2</li>
    </ul>
  </li>
</ul>

Due to the lack of sub-lists, a listbox won't need aria-expanded nor the keyboard controls for expanding and collapsing.

I think a tidy way to achieve this would be only setting the widget's role in connectedCallback if it didn't already have one (so it could be explicitly set in the HTML) and using the role to decide what the descendants are.

I think we should make them separate custom elements so developers are explicit if they want a tree or a listbox. Whilst a tree can structurally look like a listbox, and they share some keyboard controls, they are still different. A tree is more complex and tends to be for navigation, a listbox is for selecting items and can be used in a form.

A TreeViewListbox is structurally a flat list, but it can look and behave like a tree. That only happens for the threaded view in the mail tab, and it's not even really behaving like a tree, the collapsing/expanding action operates on everything below the top-level message. Also if we have to have group elements around the descendants that's going to be very difficult.

(In reply to Geoff Lankow (:darktrojan) from comment #5)

A TreeViewListbox is structurally a flat list

I see, I didn't realise this. I think aria-level might help for this https://www.w3.org/TR/wai-aria-1.1/#aria-level.

Actually, after more research, using role=treegrid https://www.w3.org/TR/wai-aria-1.1/#treegrid might be better for the email list. This has the same role as the current xul:tree in the accessibility tree as well, so will probably work quite well. Note, this was prompted by the example for an email inbox https://w3c.github.io/aria-practices/examples/treegrid/treegrid-1.html

So to represent

v Subject 1               friend@server.org
|-  Re: Subject 1         me@server.org
|-v Um actually...        bad.actor@server.org
  |- Re: Um actually...   friend@server.org

this would be

<tree-view-listbox role="treegrid">
  <tree-view-listrow role="row"
                     aria-level="1"
                     aria-posinset="&inbox.pos;"
                     aria-setsize="&inbox.size;"
                     aria-expanded="true">
    <div role="gridcell">Subject 1</div><div role="gridcell">friend@server.org</div>
  </tree-view-listrow>
  <tree-view-listrow role="row"
                     aria-level="2"
                     aria-posinset="1"
                     aria-setsize="2" >
    <div role="gridcell">Re: Subject 1</div><div role="gridcell">me@server.org</div>
  </tree-view-listrow>
  <tree-view-listrow role="row"
                     aria-level="2"
                     aria-posinset="2"
                     aria-setsize="2"
                     aria-expanded="true">
    <div role="gridcell">Um actually...</div><div role="gridcell">bad.actor@server.org</div>
  </tree-view-listrow>
  <tree-view-listrow role="row"
                     aria-level="3"
                     aria-posinset="1"
                     aria-setsize="1" >
    <div role="gridcell">Re: Um actually...</div><div role="gridcell">friend.actor@server.org</div>
  </tree-view-listrow>
</tree-view-listbox>

So it is mostly a matter of setting the right aria-level, aria-expanded, aria-setsize and aria-posinset and giving each field a role="gridcell".

So overall, it seems we need:

• <ul is="tree-list" role="tree"> for trees that are simple (no focusable children within the tree node), used for navigation and selectable. So the folder tree and the addressbook tree.
• <ol is="orderable-tree" role="tree"> for the same function, but also re-orderable. Used for the account manager tree.
• <ul is="selection-list" role="listbox"> for lists whose items can be selected. Basically a replacement for xul:richlistbox.
• <ol is="orderable-list" role="listbox"> for the same function, but also re-orderable. Used for the calendar list.
• <list-view role="listbox"> based off the current tree-view-listbox for long lists with selectable items. Used for the contact list.
• <tree-view role="tree-grid"> based off the current tree-view-listbox. Used for email list (threaded or not).

@darktrojan As part of this, I'm going to replace aria-activedescendant with a roving tabindex. This has the advantage of:

1. Allowing for focusable elements within a row.
2. No longer requiring an id for each row.

On the latter point, I'm going to be removing ids from elements that no longer need them. However, it can be hard to track id usage (e.g. in tests). Of all the tree-listbox elements where you have used an id for rows, do you know if the id is used for anything else other than aria-activedescendant?

(In reply to Henry Wilkes [:henry] from comment #7)

Of all the tree-listbox elements where you have used an id for rows, do you know if the id is used for anything else other than aria-activedescendant?

• It's used in the agenda but could easily be replaced by a Map.
• It's used in this one place in about:3pane but that's not important and probably not worth spending any time on.
• It's used in this one place in the account manager which could be done a different way, maybe also with a Map.

I think that covers it.

(In reply to Henry Wilkes [:henry] from comment #6)

Actually, after more research, using role=treegrid https://www.w3.org/TR/wai-aria-1.1/#treegrid might be better for the email list. This has the same role as the current xul:tree in the accessibility tree as well, so will probably work quite well. Note, this was prompted by the example for an email inbox https://w3c.github.io/aria-practices/examples/treegrid/treegrid-1.html

I wish I had seen that example a long time ago when I started out creating these new custom elements. It's pretty much exactly what I was trying to do for focussing elements inside a row.

So overall, it seems we need:

• <ul is="tree-list" role="tree"> for trees that are simple (no focusable children within the tree node), used for navigation and selectable. So the folder tree and the addressbook tree.
• <ol is="orderable-tree" role="tree"> for the same function, but also re-orderable. Used for the account manager tree.
• <ul is="selection-list" role="listbox"> for lists whose items can be selected. Basically a replacement for xul:richlistbox.
• <ol is="orderable-list" role="listbox"> for the same function, but also re-orderable. Used for the calendar list.
• <list-view role="listbox"> based off the current tree-view-listbox for long lists with selectable items. Used for the contact list.
• <tree-view role="tree-grid"> based off the current tree-view-listbox. Used for email list (threaded or not).

That's quite a collection but seems reasonable. As a part of this we should split the two major types into separate files. They were put together originally because they are/were very similar conceptually, but as things get more complicated it's becoming a bit of a mess.

I did some research into what we'll need from the tree or listbox elements eventually, and I've come up with a semi-detailed plan. This will also address bug 1751978. Anyone interested, please read the details below and give feedback.

(Multi-)Selectable widgets

Trees and listboxes are widgets whose items can be selected, sometimes multiple items can be selected. Another example of a widget that allows multi-selection is the recipient pills in the compose window.

All these should share the same focus and selection controls. I'm going to base this off the current XUL:tree controls.

Multi-selection and focus model

This is a "selection follows focus, by default" model. In the following "Click" is a primary button click. We assume the widget has a vertical layout for now.

Focus is controlled by:

• Click: Focus the clicked item.
• UpArrow: Move focus to previous item, if there is one.
• DownArrow, Home, End: Same as above, except move focus to the next item, first item and last item, respectively.

How this changes the selection is controlled by modifiers:

• No modifiers: Only the newly focused item is selected, and everything else is unselected.
• Ctrl: If clicking, this toggles the selection state of the newly focused item. If using arrow keys, it leaves the selection state unchanged. Instead, Ctrl+Space will toggle the selection state of the focused item.
• Shift: Select all the items between the newly focused item and a "selection origin" item, and nothing else. If the "selection origin" item is cleared, we use the previously focused item or the first item. This "selection origin" is cleared whenever the selection state of an item is changed by some other method.

If the list is a tree with collapsable child items, then we have additional controls:

• RightArrow or LeftArrow. This will try to expand or collapse the focused item, depending on the display direction. Expanded (or collapsed) items are unselected. If this would collapse the focused item, but it is already collapsed or cannot be collapsed, then focus is moved to the parent instead and the selection is changed to just the parent (regardless of modifiers). Similarly, it may move focus to the first child if it cannot be expanded.
• Clicking a twisty icon does the same on the clicked item. This replaces the above focus and selection behaviour. In particular, focus is not moved to the item.

Some further controls that seem to be common:

• Context menu: If the clicked or focused item is selected, do nothing special. Otherwise, temporarily only select the single item, and restore the old selection when the context menu closes.
• Delete: Delete the selected items. I feel like an exception should be made when the focused item is not part of the selection.

Finally, if the widget is layed out horizontally, then we swap the vertical arrow keys for horizontal arrow keys.

Alternative behaviour

GTK

I also tested the default controls for gtk4's tree views, and it has similar behaviour except the "selection origin" item is initially set to the last item that was selected or unselected. Moreover, the expand/collapse controls require the Shift modifier.

WAI-ARIA recommended

The model differs from the "Recommended selection model" on https://www.w3.org/TR/wai-aria-practices/#listbox_kbd_interaction . Specifically, the recommended model does not require a modifier to be pressed in order to do multi-selection. However, this means that selection does not follow focus by default. In most situations in thunderbird, even though multi-selection may be possible, the primary usage is single-selection, and it is mostly sufficient. In these cases, I think the selection following the focus is more desirable.

However, if there are widgets where mutli-selection is the primary usage, or "selection follows focus" is inappropriate, then the above model should not be used. Instead, selection should be unchanged with moving focus, and be toggled with "Space". I can't think of an example in thunderbird where this would be the case though.

Note that the model defined above is similar to the "Alternative selection model" defined in the link. The main difference is

• Shift + Down Arrow: Moves focus to and toggles the selection state of the next option.

This doesn't make much sense to me because if you start focused on a single selected item and press Shift+Down,Shift+Down,Shift+Up, then you end up with "selected, un-selected, selected", rather than "selected, selected, un-selected". I feel like this isn't the intended behaviour.

Implementation

The above behaviour is fairly complex, and still doesn't cover every detail. It also gets more complicated if we need to ensure that at least one item is always selected. So I wouldn't want each widget with multi-selection behaviour to re-implement this, and they could diverge. For example, XUL richlistbox has similar controls to XUL tree, but there are edge cases that cause them to differ (probably a bug). Therefore, I would like all these widgets to share some common code.

The main complication to sharing code is that it would have to work with TreeViewListbox. I think a shared class doesn't make sense because I wouldn't want each widget to have to share public methods, or have to distinguish between private and "protected" methods.

Instead, I think a common interface or trait-like class makes more sense. E.g. something like

class SelectionWidgetController {
  constructor(methdos) {
    this.#methods = methods;
  }

  handleEvent(event) {
    if (event.type == "click") {
      let clickedIndex = this.#methods.getIndexFromClickTarget(event.target);
      this.#methods.setFocusItem(clickedIndex);
      /* ... */
    }
  }
}

class TreeViewListbox {
  connectedCallback() {
    /* ... */
    let this.#selectionWidgetController = new SelectionWidgetController({
      getIndexFromClickTarget: node => /* ... */,
      setFocusItem: index => /* ... */,
      /* ... */
    });
    this.addEventListner("click", event => this.#selectionWidgetController.handleEvent(event))
  }
}

Obviously it would be more complicated than this, and I might end up shifting more into the SelectionWidgetController to avoid boilerplate. But the general idea is that SelectionWidgetController itself would hold almost no data on the tree or list composition, but would track items using an index and use the provided methods to find out what it needs to. This means that it won't scale with the number of rows. Basically, it would be similar to nsITreeSelection but would also control focus, handle events, and it would be independent of both nsITreeView and the widget.

I'm not 100% set on this approach, and it is not particularly "elegant" within javascript. So if anyone knows of a better approach, let me know.

Specific widgets

Long lists or trees

When performance is an issue, not every entry needs a corresponding element. As such, we cannot replicate the tree or list data in the DOM tree, so we need to use aria-level, aria-setsize and aria-posinset (see comment 6) to convey this information. Basically, I'm going to modify TreeViewListbox:

• Add a means to select whether the role of the widget is a listbox or a treegrid (and maybe tree if we ever need it).
• We need to ensure that the focused item is never removed from the DOM. Right now, if you scroll then the tree will loose track of the focused element as it goes out of view. We need to keep it present in the appropriate place.
• Drop the nsITreeSelection. Otherwise this will duplicate what the SelectionWidgetController is already tracking. We'll need to ensure that the nsITreeViews in use request this selection information through the widget itself. Hopefully this'll also help address the current three-way-entangled nature of nsITreeView, nsITreeSelection and the tree widget, and this should make it easier to replace nsITreeView in the future.
• I might rename it to something else since the name is a bit confusing. I would like just TreeView, but it is kind of taken by nsITreeView, so maybe I'll choose TreeDisplay or TreeViewWidget or TreeViewElement.

Other lists and trees.

For other elements, I think overall we should prefer using the DOM structuring to imply semantic relations, rather than relying on aria-level, aria-setsize and aria-posinset. This covers all the elements that currently use the TreeListboxMixin class constructor.

I'm planning on making bigger changes to these. I'm going to make them more opaque and stricter about structuring. My general aim is to make these simple for a developer to use (less effort than setting up an nsITreeView), and hard for them to make mistakes if they stick to the public methods.

Overall, I think we'll need three distinct structures.

1. Basic lists.

Most of the time, where we would have used a XUL richlistbox, we can instead use

<ul is="selection-list" role="listbox">
  <li role="option"><!--Item--></li>
</ul>

This structure would be enforced within the class itself, rather than for each usage (as is done now), instead a developer would use a public "API" to add an item (they provide what goes into <!--Item--> above), remove an item, etc.

For lists that can be reordered, we need a live region to inform a screen reader user of the reordering that has taken place. E.g.

<span aria-live="polite">Moved to position 2</span>

Also, since there are no standard keyboard controls for reordering, we also need to expose another way to reorder the selected item. E.g. in the account settings tree, a menuitem in the account actions menu with "move account up" and another for "move account down".

2. Grouped lists.

For lists whose items are grouped under headings, we can basically use a tree with depth 2, where the depth-0 items are non-focusable headings. But we can still expose it as a listbox:

<ul is="grouped-list" role=listbox">
  <li role="none">
    <span id="heading1"><!--Group heading--></span>
    <ul role="group" aria-labeledby="heading1">
      <li role="option"><!--Item--></li>
    </ul>
  </li>
</ul>

This would be used in a few places. Currently it would be used in the agenda list (where the headings are the dates). In the future we might use it in the multiday views with the events grouped by all-day and not-all-day.

3. Trees

For basic trees, which are normally used for navigation, we can use the following structure.

<ul is="selection-tree" role="tree">
  <li role="treeitem">
    <img src="twisty" alt=""/><span><!--Parent--></span>
    <ul role="group" aria-expanded="true">
      <li role="treeitem"><!--Leaf--></li>
    </ul>
  </li>
</ul>

In particular, the "twisty" icon would be owned and controlled by the class, rather than per usage.

Method and Priorities

Rather than try and replace TreeListboxMixin in a single patch, I'm going to create these other widgets along side it and then one-by-one convert widgets from TreeListboxMixin to the new widgets. Once this is complete, TreeListboxMixin will be removed.

I'm going to prioritise the widgets that are needed in the new addressbook tab:

• A list view for the list of contacts. This can be very long, so would use the modified TreeViewListbox with role="listbox".
• A tree widget for the list of addressbooks and mailing lists. This will require a basic single-select non-reorderable tree widget.

Pushed by geoff@darktrojan.net:
https://hg.mozilla.org/comm-central/rev/232c6f063635
Create the SelectionWidgetController class. r=darktrojan

This model is for situations where selection has no side effect beyond what a change in focus would do.

Pushed by thunderbird@calypsoblue.org:
https://hg.mozilla.org/comm-central/rev/b04d37e07895
Add the "focus" SelectionWidgetController model. r=darktrojan

I just want to point out some other known issues with the current implementation. They would be fixed automatically by the changes in this bug, but if my work takes too long to be uplifted to 102, we can address these in a more targeted way (in a separate bug):

• aria-keyshortcuts is set incorrectly. E.g. https://searchfox.org/comm-central/rev/c58077d4d43dc09ef98a60b54cfa723c782a733a/mail/base/content/widgets/tree-listbox.js#53
• aria-posinset is set incorrectly. It counts from 0, when the attribute should count from 1. https://searchfox.org/comm-central/rev/c58077d4d43dc09ef98a60b54cfa723c782a733a/mail/base/content/widgets/tree-listbox.js#1593

We also adjust the previous behaviour when the focused item is removed. Previously we would only select the new focused item if the selection would otherwise be empty and the previous focus was selected. Now we only require the condition that the selection would otherwise be empty.

Depends on D147768

Depends on D152659

Depends on D152660

We allow the default mousedown handlers to run so that dragging and text selection is possible if desired. This means that the focus may be moved to the clicked element, so we need to restore focus when this does not align with the #focusIndex.

A benefit of this approach is that the default focus handler will distinguish between ":focus-visible" and just ":focus" styling. It will also capture cases where the edge cases where focus is triggered by a secondary or middle mouse button click.

Depends on D153571

We delay the selection of a single item to the "click" event so that dragging a multi selection may occur first.

Depends on D153625

We rename the method and now use a callback to remove the items. This allows the controller to store information just prior to the removal of the items, and makes the implementation simpler and more direct.

Depends on D153626

Depends on D154121

setItemSelected is needed for multi-selection widgets that have more complex selection behaviour.

itemIsSelected is needed for widgets that do not keep track of the selection state themselves, or only for some items.

Depends on D154122

Depends on D154123

We ensure that when a context menu is opened that the event target is selected. We also make sure that when the event target is lost that we force the context menu to close early.

Depends on D155197

Since I'm leaving I won't be able to finish the work here. This area is fairly complex, so I've compiled together all my research, plans and thoughts together to help whoever takes over complete this bug, and any adjacent bugs. Some of this re-explains what I already wrote above or in other places, but these notes take priority over anything I said before.

Since these notes are so long, I couldn't really proof read it thoroughly. So there might be a few typos or some mistakes in the example code I give.

SelectionWidgetController

This class is basically complete for one-dimensional list widgets, like "listbox" widgets. Note that I haven't landed the patches yet since they aren't needed in the application yet (In hindsight, I should have done this with the first two patches as well, but they already landed), but they are complete and their tests run fine locally, and have a successful try run for an older version https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=2e81f200db9814498c2910b54a95f80bf1ec55a9

There are also two work-in-progress patches that add additional features: notifying the widget of a change in selected items, and support for context menus. These need to be taken over by someone else to verify my implementation (not tested) and to add tests.

I suggest applying all the patches locally (including the work-in-progress ones) and reading the inline documentation for SelectionWidgetController and its public methods, as well as how it is initialized and used in mail/base/test/browser/files/selectionWidget.js. The notes below will refer to the internals of SelectionWidgetController as they are at the current tip of these patches, including the work-in-progress ones which include some name changes.

Still needed

There are two major features that are still needed.

Support for grids

We want to allow for multiple columns and a header row. For grid widgets, the rows themselves act as items and can receive focus, but the user may also move focus between its cells. This is important for screen reader users: when the row itself has focus the entire row will be read back, which can be useful initially but if they want specific information or to interact with a specific cell they need to be able to focus the individual cells.

Also note that for SelectionWidgetController the entire row is selected, rather than individual cells. And the styling should reflect this. As such, this is not suitable for a spreadsheet-like pattern, which would require separate controls. This is more intended as an extension of a basic list to allow each item to contain multiple and possibly interactive parts.

When SelectionWidgetController is constructed, one of the arguments should specify whether the widget is a grid. The SelectionWidgetController would need to keep track of four new properties:

• #isGrid tracks whether the controller was initialized as a grid. It should not be possible for a widget to dynamically change this (similar to the selection model).
• #numColumns tracks the number of columns and initializes as 0 for all widgets.
• #columnIndex tracks the column that has focus and initializes to null. If it is null then the entire item/row has focus. Otherwise it should be on a single cell specified by the index between 0 and #numColumns - 1.
• #canEnterHeader tracks whether the user can enter the header area. This should be set to true if the header row will be visible and interactive. Like #isGrid this should be set in the constructor and it should not be possible for a widget to dynamically change this.

In addition, we need to modify setFocusableItem to take an additional argument. Basically, every time the #columnIndex changes we call this.#methods.setFocusableItem(this.#focusableItem.index, this.#columnIndex, this.#focusInWidget()). Widgets that are not grids can ignore the columnIndex (which will always be null), whilst grid widgets would move the roving tabindex to the corresponding column, or the row. E.g. if columnIndex is null, the widget should change the state of the index row to

<div role="row" tabindex="0">
  <span tabindex="-1">Cell 1</span>
  <span tabindex="-1">Cell 2<span>
</div>

and if it is 0, change it to

<div role="row" tabindex="1">
  <span tabindex="0">Cell 1</span>
  <span tabindex="-1">Cell 2<span>
</div>

We also add another widget method activateHeader(columnIndex). This will tell the grid to activate the given header. Normally this is used for sorting.

We also supply two public APIs:

• addColumns(index, number). This adds number additional columns at the given index, and increases #numColumns. The #columnIndex will need to be adjusted if it was at index or higher.
• removeColumns(index, number). This removes number columns at the given index, and decreases #numColumns. The #columnIndex will need to be adjusted if it was after index + number. If it was in a removed column, we move it to the next available column, or the last column if there is none, or null if there are no columns left. If we move column in this way, we need to call setFocusableItem to ensure that the focus moves as well.

We also modify indexFromTarget to return an object rather than a number. The object should have the index property that it currently returns, plus a columnIndex that corresponds to the column the user clicked on. For non-grid widgets, they can always return null for this. grid widgets may also return null if the user clicked a part of the row that is not associated with a cell. During #handleMouseDown we should move the #columnIndex to match the clicked column.

In #handleKeyDown for a widget whose rows are laid out vertically, if and only if #isGrid is true, we call event.preventDefault() and event.stopPropagation() when the user presses "ArrowRight" or "ArrowLeft". It is important that we do not do this for the widgets where #isGrid is false to allow them to implement their own responses to these arrow keys. If #isGrid is true and the user presses the "ArrowRight" or "ArrowLeft" keys without any modifiers this should increase or decrease the #columnIndex depending on the writing direction (right-to-left or left-to-right).

• If the #columnIndex is 0 and the user tries to decrease it, it will change to null. I.e. we focus the entire row again.
• If the #columnIndex is null and the user tries to decrease it, it will remain as null.
• If the #columnIndex is null and the user tries to increase it, and #numColumns > 0 it will change to 0. I.e. we focus the first column.
• If the #columnIndex is (#numColumns - 1) and the user tries to increase it, it remains the same. In particular, we do not focus the row, or do any wrap around.

If the widget is laid out horizontally we do the same, but with "ArrowUp" (decrease the #columnIndex) and "ArrowDown".

If the user presses "Space" to select a row, or "Ctrl+Space" to toggle a row, it should only work when #columnIndex == null. This would allow the user to still activate interactive cells.

Finally, we also add support for a single "header" row (we could support multiple header rows, but I don't think we need this). If #isGrid is true and #canEnterHeader is true, we assume there is a "header" row in the widget as long as #numColumns >= 1. The column headers in this row can also receive focus, but the header row cannot be selected. As such, we can keep track of whether the focus is within the header by setting the #focusableItem.index to -1.

• If the user clicks the header, we focus the column header but we do not change the selection. We also call this.#methods.activateHeader(clickedColumn) for the clicked column.
• The "Home" key should still take us to the first item, rather than the header.
• "PageUp" should similarly not take us into the header.
• Pressing "ArrowUp" when focus is on the first item should take us into the header, but should not change the selection. This should happen even in the #focusIsSelected model.
• If #columnIndex is null when we navigate into the header, we should change it to be 0 instead. There is no need to change it back when we return to the items.
• Note, for a keyboard user, generally they will need to press "Home" and then "ArrowUp" to get into the header. This is fine, but we should consider adding some shortcut to do the same in one step, but I don't know what the standard controls would be.
• Pressing "ArrowDown" when focus is in the header should take us to the first item.
• Pressing "Home" when focus is in the header should still take us to the first item, and "End" should take us to the last item.
• Pressing "PageUp" when focus is in the header should take us to the first visible item on the page, and "PageDown" to the last visible page. This is the same as when #focusableItem.index == null.
• If focus is in the header and the user presses "Enter" or "Space", we call this.#methods.activateHeader(this.#columnIndex). Note that #columnIndex should be non-null since we are in the header.
• Note it is possible for the selection to be lost whilst the focus is in the header if the selected row is removed. Even if #focusIsSelected is true. However, on returning to the items, a new row will be selected again.
• In general, in most cases where we test for #focusableItem.index == null in the code we should also test for #focusableItem.index == -1, and possibly handle it differently.
• In the rare case where #numColumns becomes 0 when #focusableItem.index == -1 we should move the focus either to the first selected item, or the first item (in this case we should also #selectSingle the first item), or the widget itself if it has no items.
• Currently, if the widget calls the selectSingleItem API, this will both focus and select the given item. However, if #focusableItem.index == -1 we should only select the item and not focus it because we want to avoid forcing the user out of the header.
• Normally, when #numItems becomes 0 we move focus to the widget itself, but if we can enter the header row we should move the focus to the header instead. This is important if we want to allow the user to sort the rows by activating the column header. If we follow my recommended approach below, the widget will be emptied of all items and then they will be re-added in their new order. During this operation we want the user to be able to remain in the column header.

Support for trees

Trees allow for items to be nested below other items. Some of these items may also be expanded or collapsed, but this is not necessary.

The SelectionWidgetController can support interacting with tree structures, but it will not keep track of the tree data, such as which item is a child of which, or whether an item is expanded or collapsed. That is left up to the implementation.

Note that whilst an item is collapsed, all of its descendants are not considered "selectable items" by the SelectionWidgetController. Therefore, they do not have an index and should be skipped over when working with the SelectionWidgetController. Moreover, every time an item is collapsed the widget must call removeSelectableItems for the visible items that are now hidden, and similarly addSelectableItems whenever the item is expanded. Note that using these methods, collapsed items will loose their selection state and expanded items will not be initially selected, regardless of their parents selection state: this is the desired behaviour.

We need to keep track of one new property:

• #isTree tracks whether the controller was initialized as a tree. It should not be possible for a widget to dynamically change this.

We also require an additional widget method getTreeDetails(index). This should return an object that includes:

• parentIndex - The index of the parent item, or null if it is toplevel.
• numSelectableDescendants - The number of selectable descendants that are not below a collapsed item, or null if it has no children or it is collapsed.
• canCollapse - Whether the item can collapse.
• isExpanded - Whether the item is expanded (not collapsed).

We also add another widget method setItemExpansionState(index, expanded) which tells the widget to expand or collapse the corresponding item. We choose "Expansion" rather than "Collapse" to match the aria-expanded attribute. Note that this method should not be called when canCollapse of the corresponding item is false.

We also further expand indexFromTarget to also return another property isTwisty, which indicates whether the user clicked a twisty icon. We might also want to give this method a rename.

In #handleMouseDown, we want to do something like

clickDetails = this.#methods.indexFromTarget(event.target);
if (clickDetails.index == null) {
  return;
}
if (!ctrlKey && !shiftKey && this.#isTree && clickDetails.isTwisty) {
  let treeDetails = this.#methods.getTreeDetails(clickDetails.index);
  // Allow for the possibility that we can click the twisty icon
  // when it is non-collapsable. E.g. if `visibility: hidden` was applied.
  if (treeDetails.canCollapse) {
    // Instead of changing focus or columnIndex or selection, we toggle the expansion state.
    this.#methods.setItemExpansionState(clickDetails.index, !treeDetails.isExpanded);
    return;
  }
}
// Continue with changing focus and selection.

We also need to make sure that #handleClick does not select the item if the user clicked the twisty icon.

In #handleKeyDown, similar to the grid we want to respond to Arrow keys in the other direction. Note that a tree may also be a grid. Assuming we have a left-to-right direction and a vertical layout, we want to do something like

if (
  (this.#isGrid || this.#isTree)
  (event.key == "ArrowRight" || event.key == "ArrowLeft")
) {
  event.preventDefault();
  event.stopPropogation();
  let focusIndex = this.#focusableItem.index;
  // We are only interested in the tree details when focus is on
  // the item/row as a whole: #columnIndex == null
  let treeDetails = this.#isTree && this.#columnIndex == null
    ? this.#methods.getTreeDetails(focusIndex)
    : null;
  if (event.key == "ArrowRight") {
    if (treeDetails?.canCollapse && !treeDetails.isExpanded) {
      this.#methods.setItemExpansionState(focusIndex, true);
      return;
    }
    if (this.#isGrid) {
      // Move to the next columnIndex, or `0` if `#columnIndex == null`
      // NOTE: Even if we are already at the end, we always return
      // early if we are a grid.
      // In particular, it is not possible to move to the first child
      // if we are a grid. But this isn't that useful anyway since the
      // user can press ArrowDown to do the same thing.
      return;
    }
    if (treeDetails?.numSelectableDescendants) {
      // We move to the first child, i.e. to `focusIndex + 1`
      // and single-select it
    }
    return;
  }
  if (event.key == "ArrowLeft") {
    if (treeDetails?.canCollapse && treeDetails.isExpanded) {
      this.#methods.setItemExpansionState(focusIndex, false);
      return;
    }
    if (this.#columnIndex != null) {
      // Move to the previous columnIndex, or to null if `#columnIndex == 0`.
      // NOTE: we test for the weaker condition that `#columnIndex != null` rather than
      // #isGrid. This is to still allow users to jump to the parent if they are in a treegrid,
      // which can be useful.
      return;
    }
    if (treeDetails?.parentIndex != null) {
      // We move to treeDetails.parentIndex
      // and single-select it
    }
    return;
  }
}

Basically, if focus is on the row as a whole (#columnIndex is null) then pressing "ArrowLeft" or "ArrowRight" will try to expand or collapse the row. Otherwise, if we are a grid or a treegrid, we try and move column. Else, we move to the first child or the parent node if they exist. This is a combination of the recommended "Right arrow" and "Left arrow" controls for trees and treegrids from WAI: https://www.w3.org/WAI/ARIA/apg/patterns/treeview/#keyboard-interaction-24 and https://www.w3.org/WAI/ARIA/apg/patterns/treegrid/#keyboard-interaction-25. Plus, the current xul:tree also uses this behaviour, modulo moving columns (xul:tree has no controls for this). The three different behaviours depending on the current state is potentially a little bit confusing for users, so would take some getting used to.

Note that I didn't take the modifier states into consideration. Unlike the non-modifier behaviour I don't know of any established controls. Note that the xul:tree does not take the modifiers into account, but unlike xul:tree we want to allow users to enter individual columns. A good starting point would be to do nothing with "ArrowLeft" or "ArrowRight" if any modifier is pressed. Beyond that, some options are:

• We could make "Ctrl+ArrowLeft" always take you to the parent item, if it exists, without changing the selection. Similarly "Ctrl+ArrowRight" takes you to the first child, if it exists, without changing the selection. Simialrly, "Shift+ArrowRight" and "Shift+ArrowLeft" would do a range selection. However, I'm not sure this is particularly useful, especially the "ArrowRight" behaviour.
• Alternatively, we could make "Ctrl+ArrowRight" always take you to the next column without expanding the row, even if #columnIndex == null. This would be useful if users want to read a column but do not want to expand the row. "Ctrl+ArrowLeft" would do the same, which basically means we will not move to the parent row if #columnIndex == null, but instead just remain the same. This is less useful, but would allow the user to hold down "Ctrl+ArrowLeft" to move to the row. Put another way, we ignore the treeDetails if "Ctrl" is pressed.
• We might also want to add controls that "Shift+ArrowLeft" and "Shift+ArrowRight" always expand and collapse the row, even if #columnIndex != null or the row is already expanded or collapsed. Otherwise, if the user is inside a column 1 and they want to expand the row, they would have to press "ArrowLeft", "ArrowLeft" (to focus the row instead) and then "ArrowRight" (to expand the row), which is a little strange.

Note that in GTK, the Shift modifier is required to expand or collapse tree items (ArrowRight and ArrowLeft do nothing otherwise). We could do a similar thing:

if (event.key == "ArrowRight" || event.key == "ArrowLeft") {
  if (event.ctrlKey || event.altKey || event.metaKey) {
    return;
  }
  event.preventDefault();
  event.stopPropogation();
  let focusIndex = this.#focusableItem.index;
  if (event.shiftKey) {
    if (!this.#isTree) {
      return;
    }
    let treeDetails = this.#methods.getTreeDetails(focusIndex);
    if (!treeDetails.canCollapse) {
      return;
    }
    if (event.key == "ArrowRight" && !treeDetails.isExpanded) {
      this.#methods.setItemExpansionState(focusIndex, true);
    } else if (event.key == "ArrowLeft" && treeDetails.isExpanded) {
      this.#methods.setItemExpansionState(focusIndex, false);
    }
    return;
  }
  if (#this.isGrid && event.key == "ArrowRight") {
    // Move to the next columnIndex
    return;
  }
  if (#this.columnIndex != null && event.key == "ArrowLeft") {
    // Move to the previous columnIndex
    return;
  }
  if (this.#isTree) {
    let treeDetails = this.#methods.getTreeDetails(focusIndex);
    if (event.key == "ArrowRight" && treeDetails.numSelectableDescendants) {
      // move to first child `focusIndex + 1` and single-select it.
    } else if (event.key == "ArrowLeft" && treeDetails.parentIndex != null) {
      // move to `treeDetails.parentIndex` and single-select it.
    }
  }
  return;
}

This would remove a lot of the conditionals and simplify the controls at the cost of requiring the user to press a Shift modifier to expand/collapse, and departing from the controls of xul:tree and the WAI recomendations.

In general, the modifier behaviour needs a lot of thought and ideally some screen-reader user feedback.

Finally, we want to make a special consideration for the tree structure in removeSelectableItems. There is already a TODO note about this. When an item is focused and removed, we want to keep the focus within its remaining ancestor. To do this, before the call to removeCallback() we do

let ancestorIndex = null;
let lastDescendant;
if (
  this.#isTree &&
  this.#focusItem.index != null &&
  this.#focusItem.index >= index &&
  this.#focusItem.index < index + number
) {
  // Focus will be lost at the end of the method.
  ancestorIndex = this.#methods.getTreeDetails(this.#focusableItem.index).parentIndex;
  while (ancestorIndex != null) {
    if (ancestorIndex < index) {
      // Ancestor will remain.
      let numDescendants = this.#methods.getTreeDetails(ancestorIndex).numSelectableDescendants;
      if (ancestorIndex + numDescendants < index + number) {
        // End will be cropped away.
        lastDescendant = index - 1;
      } else {
        lastDescendant = ancestorIndex + numDescendants - number;
      }
      break;
    }
    ancestorIndex = this.#methods.getTreeDetails(ancestorIndex).parentIndex;
  }
}

removeCallback();

Then further down, below #adjustIndexOnRemoveItems we want to do

let newFocus = index;
// Adjust for #shiftRangeDirection first so that the restrictions from
// ancestorIndex and lastDescendant take priority.
...
if (ancestorIndex != null) {
  newFocus = Math.max(newFocus, ancestorIndex);
  newFocus = Math.min(newFocus, lastDescendant);
}

Implementing SelectionWidgetController

In general, SelectionWidgetController was built to handle the selection and focus entirely by itself, and any widgets that use it should avoid deviating from the behaviour it establishes. There are two public methods for programatically changing the selection and focus: selectSingleItem and setItemSelected. In general these methods should be avoided because they can be disruptive and confusing. Moreover, the whole point of this controller class is to give the user a consistent experience when using these different widgets so that controls are familiar, and all edge cases are treated the same.

A lot of the widgets that use SelectionWidgetController will also have other things in common, but I decided to not to include these in the implementation of SelectionWidgetController because they were adjacent to selection and focus, and were not too complex or boiler-plate-y to implement in individual widgets. However, if it turns out all approaches do the same thing, there might be room to create another controller class to handle this.

Focus

It is very important that every widget manages its focus correctly. Whilst there is aria-activedescendant, SelectionWidgetController was built around using a roving tabindex because it comes with more supporting features, such as being able to detect :focus-visible.

See the setFocusbaleItem method in mail/base/test/browser/files/selectionWidget.js for how to work with a roving tab index. In general, whatever element semantically contains the items (usually the widget itself) should initially have a "0" tabindex and all of its item children should initially have "-1" tabindex. After this, there should always be exactly one element within a widget that has a tabindex of "0".

In particular, the focusable item should always be present in the DOM, this is currently a bug with the current TreeViewListbox class where the focusable item is removed from the DOM when it is scrolled out of view. This needs to be fixed when transitioning to SelectionWidgetController. It needs to maintain the focusable item even if it is scrolled out of view, once it is no longer focusable (because focus has moved to another item) it can be removed.

Also note that in a few places I used the word "focus" when really it would have been clearer to have written "focusable". Normally the "focused item" is the item that has tabindex="0", but it may or may not be the document.activeElement, and it may or may not match the :focus selector (which requires the window to have focus).

Empty widgets

Currently, SelectionWidgetController is able to handle when a widget has no items. However, in a few situations we will want to show something else when the widget is empty to direct the user. This should be separate from the selection widget though because otherwise its semantics will be broken, which can lead to confusing outputs from a screen reader.

Choosing a selection model

I implemented three selection models: "focus", "browse" and "browse-multi", which have descriptions in the documentation. These models seem to cover most existing use cases in thunderbird. For example, "focus" would be used on the event list in the today pane because there should be no distinction between focus and selection in this case. "browse" would be used on the addressbook tree because we want to be able to move the focus with the selection by default, but a user may also want to move the focus to read the items without triggering their selection. And "browse-multi" would be used for the contact list.

I avoided simply using the word "single" or "multiple" for a selection model because whilst a widget may want single or multiple selection, the existing models may not be appropriate so the name would be misleading.

In particular, there is room to add more models that do not move selection with focus (by default). The "browse" and "browse-multi" models require modifiers to be able to move focus without changing the selection, or to perform multi selection. These can be difficult for some users. In the existing cases I came across within thunderbird, a user could get by ok if selection followed focus always, and they never used multi-selection. So the convenience of "selection follows focus" was worth it.

However, in situations where selecting an item is an expensive operation that would slow down a user, then it would be better to not have focus follows selection (by default). Such a selection model would behave as:

• Arrow keys without any modifier would move focus without changing the selection state. Arrow keys with a modifier would do nothing.
• Space would select just the focused item and nothing else (as it does in the existing models). Space with a modifier would do nothing.
• Clicking would select and focus the clicked item (as it does in the existing models).

In situations where multi-selection is the primary use case, it would be better to have a model that does.

• Arrow keys without any modifier would move focus without changing the selection state.
• Space without any modifier would toggle the selection state of the focussed item.
• Clicking without any modifier would toggle the selection state of the clicked item.

Basically, this would act like a set of checkboxes. With some room to add additional controls using modifiers to select groups or clear the selection.

Note that these last two models essentially follow the WAI recommended controls https://www.w3.org/WAI/ARIA/apg/patterns/listbox/#keyboard-interaction-11. The "browse" and "browse-multi" controls are similar to the "Alternative selection model", although the main distinction being the Shift behaviour, which seems to be a mistake (see comment 9).

Zero, Single and Multi selection

The SelectionWidgetController class only exposes one method for fetching which returns selected indices: getSelectionRanges. The returned object is an array of selection ranges. This was chosen to optimize performance when lists are very long, but there are only ever a handful of distinct selection ranges.

If an implementation is using a single-selection model, then you should expect to either receive an empty array (no selection), or a single range that only covers one item. In particular, single-selection widgets still need to be able to respond to a zero-selection, even if the widget is non-empty.

The SelectionWidgetController specifically does not have a selectedIndex method like nsITreeSelection which returns the first selected index if it exists. The reason for this is to discourage multi-selection widgets from using this. For example, the old folder tree in the 3pane uses selectedIndex to decide which folder to display in the message tree. However, this means that if the user starts to multi-select the folders (which is allowed) the message tree of whatever selected folder is first is shown, rather than something that is responsive to the multi-selection state. Moreover, the user may also select no folder (also allowed) and the UI fails to respond. In general, if we are using a multi-selection model then anything that wants to respond to the selection state must be able to respond to multi and zero selections.

contextmenu event

A lot of widgets will want to respond to the "contextmenu" event. For a selection widget, if the "contextmenu" is triggered by a key press then the event target will be the focused item. If it is triggered by a secondary mouse click, it will be within the clicked item.

A widget should use the openContextMenu method supplied to the SelectionWidgetController to open context menus. This will ensure that the event target is always selected when the context menu is open, which helps avoid confusion over whether a menu item effects the event target or a separate selected item.

A lot of context menu items are related a specific widget item, like "Reply to All". Others can related to selected items, like "Delete Selected Messages". Others are more generic, like "New Event" is shown in the context menu of calendar event boxes. As such, implementations should be careful to only show items that are appropriate to the event target and currently selected items.

Note that it is possible for the selection state to change whilst the context menu is open by items being removed in the background, which may also remove the event target. For example, in a message display the message may be deleted on the server whilst the menu is still open. The SelectionWidgetController will inform the widget to close the menu early before the user can activate its items in the rare instances where the event target is removed or if the implementation calls selectSingleItem or setItemSelectionState whilst the menu is open. However, selected items that are not the event target may still be removed whilst the menu is open, and the index of items may change whilst the menu is open if items are added or removed in the background. As such, when the user activates the item it is important to use a fresh call to the SelectionWidgetController, like getSelectionRanges, to determine which items should be effected by the activation.

selection-changed event

Most widgets will want to implement a custom event that fires whenever the set of selected items changes. Note, we are normally not interested in cases where the selected indices change when items are added, removed or moved. Instead, we basically want to know whether the return of getSelectionRanges points to the same set of items. In this case, widgets should release their "selection-changed" events whenever the SelectionWidgetController calls the selectionStateChanged method. In general, this method should only be invoked by the SelectionWidgetController once per public API call or user generated event, and is set up to avoid false-triggers so it should only fire when really needed. If a widget is performing a series of transactions that call SelectionWidgetController API methods, then it could freeze the release of the event during the sequence.

Delete key event

When the user presses "Delete", if items can be deleted by the user it should remove the focused item. If the focused item is also selected, then the other selected items may also be removed. In particular, implementations should not simply delete whatever is selected, because this may not have focus so would likely be unexpected behaviour for the user. Note that the removal of items should still be handled through the SelectionWidgetController's removeSelectableItems method by calling it on each removed range.

If a deletion is non-reversible or would remove multiple items, it may be better to first warn the user and inform them of what they are deleting.

Enter key or double click event

Double clicking an item or pressing "Enter" is often used to "activate" an item. Normally to "open" the item to see more details. This is distinct from selecting an item, so keep in mind that the activated item may not be selected. Depending on what the activation does, you may want to also select the activated item through selectSingleItem. Having some "activation" behaviour is not essential and would be outside the scope of expected behaviour. In fact, if the selection widget is within a form, having Enter trigger the form submission may be more useful. This is why it was not included in SelectionWidgetController.

Note that if this is implemented for a grid or treegrid widget, then "Enter" should probably only activate the item when the focus is on the whole row, i.e. when columnIndex == null. This would allow uses to still activate individual gridcells.

Interactive grid cells

In a grid or treegrid, we often want a gridcell to contain some interactive element, like a button or a checkbox. If this is the case, then rather than the gridcell receiving focus, it should move to the interactive element inside it. A user should be able to activate the element by pressing "Enter" or "Space" when it has focus.

Note that SelectionWidgetController should be set up so that pressing "Space" only select a row when focus is on the row itself, i.e. when columnIndex == null.

Generally, we should avoid any interactive elements that have arrow key controls since this will disrupt the navigation controls. If such an interactive element is needed, we could require that the user first presses "Enter" to enter the interactive element and give it full keyboard control, and then have them press "Esc" or "Enter" again to leave it. This would require some testing though because it is non-standard behaviour.

Re-ordering

Some widgets want to allow for re-ordering items directly by the user. Any such re-orderings should be done through the moveSelectableItems API, but the exact controls and behaviour are left up to the widget. This could be moved into the SelectionWidgetController if all implementations need to do the same thing.

One way to do this is through drag and drop. The SelectionWidgetController was set up to allow for drag and drop, including dragging multi-selections, so it should be possible to implement drag and drop for moving items. However, drag and drop can be difficult for some users, or not possible through a keyboard. So there need to be other means of re-ordering. Like buttons or through a context menu. Keyboard shortcuts could also be added in addition, and the convention seems to be Alt+Arrow or Alt+Home or Alt+End to move items. However, they are still non-standard controls, so if you have the same action in a context menu you could advertise the Alt+Key shortcut in the shortcut column, and if there is a button that moves items you can set aria-keyshortcuts on it.

Another thing to consider is that if the item is not focused then the change in position will not be noticeable to someone using a screen reader. Even if it does have focus, the change in position might not be obvious or may be unannounced depending on the screen reader configuration. This can make re-ordering seem unresponsive. Therefore, it might be a good idea to include an aria-live region used to announce the re-ordering in a human friendly way, like "moved to position 5". In such a case, the position should count from "1" rather than "0", so should be the index + 1.

The WAI guides provide a good example of reordering https://www.w3.org/WAI/ARIA/apg/example-index/listbox/listbox-rearrangeable.html

Sorting

Many of widgets may want to allow the user to sort its content. In principle you could keep the selection and focus state of items as they get sorted by coupling moveSelectableItems with the sorting algorithm, where it is broken down into a sequence of movements. However, there is a performance cost to doing this. Moreover, the end result can make the selection state far more complex. For example, if you do a Shift-range selection on the old 3pane message tree that covers a lot of messages, this is represented by a single range. But if you then sort by a column, the number of selection ranges can become very large and it can introduce a noticeable performance issue.

Therefore, when sorting, it may be better to empty the widget of all items using removeSelectableItems, do the sorting, and then add the items back with addSelectableItems. This will clear the selection for you and put the widget into a fresh state. When the user returns to the widget, they will automatically select the first (selected) item.

Expand all

For trees, we may wish to allow the user to expand all the tree items. However, similar to sorting this can come with performance problems because each tree is expanded with addSelectableItems, each of which can add another selection range. E.g. select all and then expand all. In this case, if there is a multi-selection, it may be desirable to single-select the focused item before expanding all.

Widgets

This details what general widgets are needed, their semantic structures and any additional accessibility requirements.

In general, a widget should not expose the SelectionWidgetController outside of its class to prevent confusion over which class is being controlled. Instead, I suggest exposing separate APIs that the widget implementation translates into SelectionWidgetController method calls.

The current tree-listbox widget should just be replaced entirely. It requires users of the widget to structure the DOM entirely themselves. A nicer API should be exposed that handles all this structure in a standard way.

The tree-view-listbox should be adapted to the use SelectionWidgetController instead of nsITreeSelection. Really, these should move away from nsITreeView as well, but it should be possible to continue using it with SelectionWidgetController as long as the view is adapted to no longer use nsITreeSelection.

Useful aria- attributes

For each widget role, it is a good idea to look up the their specification on https://w3c.github.io/aria/ but here are some specific attributes that are useful.

For widgets

• aria-label or aria-labelledby - All these widgets need an accessible name. Make sure they are human readable. Try and use aria-labelledby if there is some visible text, like a heading, for the widget. Otherwise use aria-label. This name will basically be announced every time the user tabs into the widget, so should be useful for knowing where they are.
• aria-orientation - This needs to be set to match the keyboard navigation direction, as determined by the getLayoutDirection method.
• aria-multiselectable - This needs to be set to "true" when a multi-selection model is used.
• aria-readonly - This can be used to mark the widget as being readonly. I.e. it is being used to present data. This doesn't mean it is entirely non-interactive though.
• aria-controls - This can be used to indicate that the widget controls some other part of the application. I.e. the contact list in the addressbook controls the contact display pane. However, it seems this attribute has limited support https://a11ysupport.io/tech/aria/aria-controls_attribute

There are also other form-related controls for when the widget is embedded as part of a form, like aria-required, aria-invalid and aria-errormessage.

For items

• aria-selected - All items should have this set to either "true" or "false". The setItemSelectionState method is a good place to set this.
• aria-expanded - This should only be set on tree items or rows that are expandable and collapsible. These items should return canCollapse when getTreeDetails is called on them, and setItemExpansionState is a good place to set this attribute. If a tree item is a leaf, or a parent that cannot be collapsed (it is always expanded) then this attribute should be absent! Do not set it to "true" because this indicates to screen readers that the item could be collapsed.

For column headers

• aria-sort- Whilst the rows are sorted by a "columnheader", this should be set to "ascending" or "descending". If the "columnheader" is not in the the sorting column it should be set to "none".

When missing a full DOM structure

The following attributes should only be used when the DOM structure does not represent the displayed structure. I.e. in the tree view widget where only the visible portion and the focus is present. If you use the full DOM structure there is no need to use them.

For "grid" and "treegrid":

• aria-rowcount - The total number of items. I think for "treegrid", this does not include items below a collapsed item (see https://www.w3.org/2021/12/09-aria-minutes.html#t05). So this would just be the same as the #numItems in the SelectionWidgetController.

For "row" beneath a "grid" or "treegrid":

• aria-rowindex - The (index + 1) of the item.

For "option" under a "listbox":

• aria-setsize - The number of items in the list. Note that this is set on the item rather than the "listbox" widget.
• aria-posinset - The (index + 1) of the item.

For "treeitem" under a "tree", or "row" under a "treegrid":

• aria-level - The depth of a tree item, starting from "1".
• aria-setsize - The number of siblings: items under the same parent at the same level, including itself.
• aria-posinset - The position amongst the siblings: items under the same parent at the same level.

Note: for a "treegrid" there is some confusion on whether to set aria-rowcount and aria-rowindex when we already set aria-setsize and aria-posinset. See https://www.w3.org/2021/12/09-aria-minutes.html#t05 and https://github.com/w3c/aria/issues/1303#issuecomment-902808416. But I think aria-rowcount and aria-rowindex are distinct in that they count all the rows, not just the ones at the same level below the same parent. So we should try setting them and see how screen readers respond. Generally, the position amongst the siblings is more useful information than the overall row position, and is all you would get in a role="tree" widget.

listbox

This basically covers the listbox pattern. See https://www.w3.org/WAI/ARIA/apg/patterns/listbox for a good discussion. And these should replace xul:richlistbox where appropriate.

Note on the WAI recommendations for listbox

It should be noted that the aria specification and WAI recommendations hints at a usage that is more restrictive than what we use it for. The WAI-ARIA specification implies its functionality is for selection of items, as if it is just html:select with more complex items. And the WAI-ARIA practices implies the items should not contain interactive elements. Instead, the recommendation pushes you to use a grid pattern for these cases.

However, I check in with jamie from the firefox accessibility team on matrix and overall:

I think the ARIA spec is trying to push people towards usage of grid for complex cases like this, even one-dimensional cases, but I don't think there's been much buy-in there.

And I asked some questions:

Q: Is "listbox" still appropriate if the user can "activate" an item with "Enter", delete it from the list with "Delete", or open a context menu on the item? Or would that be considered too interactive?
A: I think that's perfectly acceptable. This maps pretty closely to the way list boxes behave in simple native desktop apps, which is what ARIA listbox was modelled on.

Q: Is "listbox" appropriate to use for navigation? I.e. depending on what is selected, a different "document" is shown elsewhere.
A: I think this is reasonable also. It's perhaps a bit non-standard.\00\00

Q: Is "listbox" appropriate if the widget is self-contained, but selection still plays a role? For example, above an email message we show a list of recipients. This is primarily just to display each email as a separate item, but the user can also select multiple items and open a context menu to compose a new message to all of the selected.
A: Again, seems reasonable to me. One catch here is that screen readers, when using "browse mode" or equivalent (where the arrow keys read through a document), only render a placeholder for the list box; the user has to press enter to "interact" with the list box. This might feel a bit weird to some users if they can only view the recipients of a message by "interacting". This is only an issue if the control is inside the same "document" as the body of the message, which it may not be in Thunderbird anyway [This is indeed true, the screen reader would be in "focus" mode].

Q: Is "listbox" appropriate if the widget is self-contained and selection plays no role? For example, if we are just displaying a list of items and the focused item can be interacted with through "Enter" or context menu. There is no side-effect to selecting an item. In this case, if I did use "listbox" it I would assume a strict selection-follows-focus model.
A: That's fine; there are many list boxes where focus and selection are more or less synonymous. However, again there's the browse mode interaction issue.

So overall, it seems safe to use listbox to present 1-dimensional information with some room for interaction. As ever though, it should be tested on a screen reader!

Basic lists

Snap shot (without other needed aria attributes):

<ul role="listbox">
  <li role="option">Item 1</li>
  <li role="option">Item 2</li>
  ...
</ul>

This is very basic. For the API, instead of using an index, which may vary for the same item, it might be more useful to refer to an id for the item, or to the actual <li> element.

For re-orderable lists, we can still use <ul> rather than <ol>. I'm pretty sure the difference between <ul> and <ol> does not make a difference to screen reader output, especially when using a role.

Grouped lists

Snap shot:

<ul role="listbox">
  <li role="none">
    <span id="heading1">Group 1</span>
    <ul role="group" aria-labeledby="heading1">
      <li role="option">Item 1</li>
      <li role="option">Item 2</li>
    </ul>
  </li>
  <li role="none">
    <span id="heading1">Group 2</span>
    <ul role="group" aria-labeledby="heading1">
      <li role="option">Item 3</li>
      <li role="option">Item 4</li>
    </ul>
  </li>
</ul>

Note that, unlike a tree, the group titles are not items. They should not be selectable or focusable and should have no interactivity! They should just be labels. In the above snap shot, "Item 2" should have an index of "1" and "Item 3" should have an index of "2" when interacting with the SelectionWidgetController. I.e. they are adjacent in index.

When using a screen reader, when you navigate from "Item 2" to "Item 3" it will read out that you are entering a new group and the name of the group.

Again, WAI provides a good example https://www.w3.org/WAI/ARIA/apg/example-index/listbox/listbox-grouped.html

In terms of API, we can use a set of ids for the group and a set of ids for the items. When we add an item we add it at a specified position under an existing group.

Note that if you want to support re-ordering items between groups then you should distinguish between lying at the end of a group and at the start of the next group. So in the above snap shot, if "Item 1" has focus and the user presses "Alt+DownArrow" the item would move to the end of "Group 1" with a new index of "1". If they press "Alt+DownArrow" again, it will move to the start of "Group 2" with the same index of "1". Similar to re-ordering in the basic case, it might be a good idea to include an aria-live region to announce both a change in group and a change in position (relative to the start of the group). For example, "moved to Group 2 position 1".

tree

See the tree pattern https://www.w3.org/WAI/ARIA/apg/patterns/treeview/ These are normally used for navigation.

Snap shot

<ul role="tree">
  <li role="treeitem" aria-expanded="true">
    <img src="twisty" alt=""/><span>Parent 1</span>
    <ul role="group">
      <li role="treeitem">Leaf 1</li>
      <li role="treeitem">Leaf 2</li>
    </ul>
  </li>
  <li role="treeitem" aria-expanded="true">
    <img src="twisty" alt=""/><span>Parent 2</span>
    <ul role="group">
      <li role="treeitem">Leaf 3</li>
      <li role="treeitem">
        <span>Non-expandable Parent</span>
        <ul role="group">
          <li role="treeitem">Leaf 4</li>
        </ul>
      </li>
    </ul>
  </li>
</ul>

Note in particular that a tree item does not need to be expandable to be considered a parent. If it is expandable and collapsible, we set the aria-expanded attribute appropriately, otherwise the attribute is absent.

In terms of API, we can again use an id or the <li> elements themselves to reference an item. When we add an item, we specify the parent we want it to lie directly under.

grid

See the grid pattern https://www.w3.org/WAI/ARIA/apg/patterns/grid/

A grid is actually very generic, but here we want to use them as a means to select rows rather than cells.

Snap shot

<table role="grid">
  <thead>
    <tr role="row">
      <th>Person</th>
      <th>Email</th>
    </tr>
  </thead>
  <tbody>
    <tr role="row">
      <td>Alice</td>
      <td>alice@sever.org</td>
    </tr>
    <tr role="row">
      <td>Amy</td>
      <td>amy@sever.org</td>
    </tr>
  </tbody>
</table>

Note that for a grid, each <tr role="row"> in the <tbody> acts as a selectable item in SelectionWidgetController. The <thead> row is not a selectable item!

I found that even though <tr> has an implied role of "row" I had to set it explicitly for it to use aria-selected properly and for its accessible name to default to its text content (with whitespace between columns). I think the <td> elements have an implied gridcell role, but these may also need to be explicitly set for similar reasons.

Note that if a gridcell contains a focusable item, like a checkbox or a button, then it should receive the focus rather than the gridcell.

Note also that having a header row is important for accessibility so that the columns can be distinguished. Even if #canEnterHeader is false. If you do not want the header row to be hidden, then you can use screen-reader-only on the header text content to hide them visually. Note that applying screen-reader-only to the <th> element itself can destroy some of its semantic meaning in a plain table, so you might want to avoid doing that here as well. See bug 1776644.

treegrid

See the treegrid pattern https://www.w3.org/WAI/ARIA/apg/patterns/treegrid/

In this case it might be better to use a flat DOM structure so we can use a <table> and to keep the structure similar to the grid structure

<table role="treegrid">
  <thead>
    <tr role="row">
      <th>Subject</th>
      <th>Address</th>
    </tr>
  </thead>
  <tbody>
    <tr role="row"
        aria-level="1"
        aria-posinset="1"
        aria-setsize="33"
        aria-expanded="true">
      <div role="gridcell"><img src="twisty" alt=""/>Subject 1</div>
      <div role="gridcell">friend@server.org</div>
    </tr>
    <tr role="row"
        aria-level="2"
        aria-posinset="1"
        aria-setsize="2" >
      <div role="gridcell">Re: Subject 1</div>
      <div role="gridcell">me@server.org</div>
    </tr>
    <tr role="row"
        aria-level="2"
        aria-posinset="2"
        aria-setsize="2"
        aria-expanded="true">
      <div role="gridcell"><img src="twisty" alt=""/>Um actually...</div>
      <div role="gridcell">bad.actor@server.org</div>
    </tr>
    <tr role="row"
        aria-level="3"
        aria-posinset="1"
        aria-setsize="1" >
      <div role="gridcell">Re: Um actually...</div>
      <div role="gridcell">friend.actor@server.org</div>
    </tr>
    ...
  </tbody>
</table>

Note that we need aria-level, aria-posinset and aria-setsize to reproduce a tree structure. However, I don't think we need aria-rowcount or aria-rowindex because all of the rows (that are not beneath a collapsed item) are present in the DOM.

Note that WAI has a good example https://www.w3.org/WAI/ARIA/apg/example-index/treegrid/treegrid-1.html However, one thing the WAI example does is allow you to press "Tab" to move to the first interactive element on a row. After some thought, I would not do this and would instead make the interactive element not part of the tab sequence by default. Usually a user wants to be able to tab into and out of a treegrid widget in one step, so having it stop on cell when they try to leave would be annoying, especially if there are several interactive cells. This would also be inconsistent with how grids behave. The user can still reach the interactive cell by navigating to it with arrow keys when they are within the widget. This is the pattern that SelectionWidgetController would use.

I thought maybe it would be possible to embed the tree structure into the DOM and avoid using aria-level, aria-posinset and aria-setsize. However, all the examples I found use the flat <table> pattern. This is probably because there is no semantic html element that would have the same structure. When I played around with this I struggled to get something to work well in the accessibility tree. So using a <table> is probably still the best approach.

What to use where

Here is some discussion about what to use in specific parts of the UI. These are not exhaustive, and are mostly suggestions based on my understanding of the areas. They should be a useful starting point and help illustrate what each ARIA role can be used for.

Addressbook tree in about:addressbook

Used for selecting addressbooks or their lists. This should use a role="tree" widget with the "browse" selection model.

Contacts table in about:addressbook

In the "tabular" view, it should use role="grid", set #isGrid and #canEnterHeader to true in the SelectionWidgetController, and use the "browse-multi" selection model. Currently the tabular view is not actually a table, and the column headers are just separate buttons: this should be fixed and the headers should be within the widget.

Contacts "list" in about:addressbook

The contact "list" should really be a grid as well because it holds two columns: the "Name" and "Email". I think there are also plans to add more columns. Note, even though they are not laid out as columns, they should still be semantically represented as columns. However, unlike the tabular view, #canEnterHeader should be false. The headings would also need to be visually hidden with screen-reader-only.

Currently the contacts "list" is a listbox and it sets aria-label on each row, which hides the email! This should be converted to a grid and the aria-label should not be set on the rows.

There may be some temptation to combine the "tabular" view with the "list" view into the same widget because they have the same "role". Just note that the specification for grid support that I gave above was designed with #canEnterHeader being fixed. Whilst it could be changed dynamically, it would require adjusting the focus (and possibly the selection) when its value changes if the focus was in the header.

Folder pane in about:3pane

The folder pane in the new 3pane should use a role="tree" widget with a "browse-multi" selection model. The area that normally shows the message tree should be able to respond to when the user selects zero or multiple folders.

Message pane in about:3pane

The message tree in the new 3pane should use role="treegrid" widget with "browse-multi" selection model.

Note that unlike the current message tree in the old 3pane, the new widget should allow keyboard users to interact with the cells. E.g. to toggle the "read" status through a <input type="checkbox" >. It would also allow screen reader users to navigate to individual cells to read their content, rather than rely on the whole row being read as one string.

Similar to the contacts in about:3pane I imagine there will be a tabular and list-like view. In both cases we need headings for the columns, but in the latter we make it visually hidden and set #canEnterHeader to false.

If the view is meant to be non-threaded with no ability to expand or collapse items, then we may want to use role="grid" instead to better represent its capabilities. Whilst it might be tempting to use the same widget for both cases, I think dynamically changing the role could cause issues or be confusing when using a screen reader. This would need some special testing.

Message header recipients

We could group all the recipients into a single role="listbox" widget, which groups items into their corresponding header field: "To", "Cc", "Bcc", etc. This could potentially also include other recipient-like fields that are not "From", like "Reply-To" or "Sender". For this widget, we would use a "horizontal" layout, so we use "ArrowRight" and "ArrowLeft" to navigate.

The advantage of this approach is that it saves on the many tab-stops we currently have. If we use the "browse-multi" selection model, it would also allow a user to select multiple recipients to send a message to or other operations.

The current "MORE" button could be one of the items. Alternatively, we could automatically expand the list when the user navigates to the end of the list. We may want to set aria-setsize to indicate the full size of the list.

Compose recipient pills

The current recipient pill selection controls have a few inconsistencies. See bug 1763362 comment 3. They should use SelectionWidgetController with the "browse-multi" selection model. Each recipient list, for "To", "Cc", etc, could be its own role="listbox" widget with a vertical layout, which could resolve some of the accessibility hacks that the recipient pills currently use. We should probably also make the recipient lists their own separate tab stop from the <input> element to make it easier to navigate into and to make the semantics clearer. We might also want to deselect all of the pills when the list looses focus, as we do now.

Note that currently it is possible to multi-select pills across different fields, but only for mouse users. I'm not really sure what the use case for this is and if this really needs to be supported. In principle, it could be supported by sharing a single SelectionWidgetController between all of the recipient lists. It would require some extra focus management though, and a way to navigate to the next list without the keyboard without loosing the selection state.

Note, we can't really use a grouped listbox like in the message header because we need extra tab stops for the recipient <input> and the "remove field" buttons in between the items. It would not be appropriate to place these distinct interactive elements within a listbox.

Attachment list

This is relevant for bug 1733662. We could use a role="listbox" widget with the "browse-multi" selection model. The layout direction of the current implementation would be "horizontal". But we might want to add alternative layouts where the direction switches to "vertical".

Today pane event agenda

This should use a single role="listbox" widget which groups the items into days, with the "focus" selection model. We use "focus" because there are no side effects to selecting an item.

Currently, each day is a separate listbox, which means you cannot navigate between the days with arrow keys. These should be grouped into a single widget.

Calendar list

The list of calendars could use a role="grid" widget with a "focus" selection model. The columns would be "Name", "Read-Only" and "Hidden". With the latter containing a <input type="checkbox"> to allow the user to toggle the hidden status of the calendar (this checkbox is currently an icon that appears on hover of the row).

This list also supports re-ordering, which should be made more visually obvious and have exposed controls to change it for keyboard users and screen reader users.

Calendar views

We could possibly use one of these widgets for the calendar views, which should help towards making the calendar component keyboard and screen-reader accessible (bug 431076). However, unlike the other widgets above, I less sure about what to do because these components are structurally and functionally quite distinct. It may be that a different "role" or conceptualisation works best to capture their behaviour.

Multiweek views

For the multi-week and month views, we could use a grid where each day is a cell. Note this grid should not use SelectionWidgetController because the concept of selecting or focusing a row (this this case a week of days) is not useful or relevant. Instead you would just need basic arrow key navigation between the days or weeks, and to scroll up/down to the previous or next week.

However, within each day we have a list of events, which could be a listbox. The main issue with this approach is having to switch between navigating the day grid and navigating the event lists. You could require the user to press Enter to move control to the list, and Esc to move back, but this may be cumbersome. Ideally, the accessible name for the day's gridcell would provide some summary of its contents. E.g. "2 events", or something with a few more details. This would require a lot of testing with a screen reader to see what works.

Note that some event items have a lot of parts: an icon to indicate whether it spans onto other days, the start or end time, category, and more icons that represent different states. Given this, it may be more appropriate to use a grid for the list of events instead where these parts are columns. But having a grid (with headings) inside another grid might be over the top and verbose. So alternatively, we could allow the user to remove categories and icons (bug 1754482) to reduce the accessible name to just the key details in a more-or-less human friendly way. E.g. like "11:15am Dentist appointment", or for all-day "My birthday", or for the start of a multi day event "10:30pm Birthday Party, continues to next day", or for the end of a multiday event "1:00am end of Birthday Party".

Note also that for events that span multiple days, we need to sync the selection state. We may also want to allow users to multi-select across different days. Even if each day has a distinct listbox, we may want to have them share a single SelectionWidgetController. This should make the mouse controls very simple, but the keyboard controls may need some hacking because they have to contend with navigating between days as well.

Multiday views.

For the multiday views, we could do a similar thing. But it might make sense to use a single grouped listbox instead. Each day would form a group, and each event would be an item in this list.

Note that bug 1760318 is already open to provide items (which are currently just plain <li> elements) with labels
for the start and end time.

For the all-day events, we might be tempted to group them separately, but I think they could live in the same list and group as the non-all-day events. And they can be identified by the user by the lack or start or end times.

Since the listbox is a 1-dimensional navigation widget, we could also let the user skip to the next day using the other directions. Since the items do not align between days we should just move to the first item on the corresponding day.

Similar to the discussion above, we could try and use a grid to represent all the location, category and icons information, but it might be simpler for users to just have human readable information that is a bit more detailed than in the other view. Like the all-day "My birthday" or "August 15th to August 20th Holiday" or "11:15 am until 12:30pm Dentist appointment", or for an event spanning multiple days and the current day is in the middle "From August 15th 9:00am to August 20th 11:30pm Super Convention".

This would also need to sync the selection state of the items that span multiple days.

One of the stumbling blocks to using this approach is how to handle days with no events. It might be confusing for a user to skip over days when navigating, so we might want to insert a dummy event that says "No events"

All the previously accepted revisions have been rebased, we can land those and I'll take care of finishing the last 2.

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/0c3652426869
Expose a separate SelectionWidgetController API for selecting a single item. r=darktrojan
https://hg.mozilla.org/comm-central/rev/985b5e3cf124
Add multi-selection model to SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/a08ef924923e
Add API for selection ranges to SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/60a49aa7a424
Add support for PageUp and PageDown in SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/784de47a4ae8
Handle focus without calling preventDefault on mousedown in SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/8670e524f214
Delay selecting a single item when part of a multi selection in SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/4716c7afda9a
Replace removingSelectableItems with removeSelectableItems in SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/616d47cfd5c9
Add moveSelectableItems API to SelectionWidgetController. r=darktrojan
https://hg.mozilla.org/comm-central/rev/ffe1519da2a6
Add setItemSelected and itemIsSelected to SelectionWidgetController API. r=darktrojan

add [supernova] to whiteboard 20230217_1525

We're not tackling this for 115 and this is not a supernova effort.
We probably end up not using this approach as a more targeted and solution has been adopted.
Removing the supernova tag.

It was Henry who set blocks sn-folderpane some comments up. But if it's not a supernova bug, it shouldn't block that any more.

(In reply to Alessandro Castellani [:aleca] from comment #30)

We're not tackling this for 115 and this is not a supernova effort.
We probably end up not using this approach as a more targeted and solution has been adopted.

Long gaps between landings (comment 28) can create problems. To avoid confusion for release engineering with future checkins, should we close this partly finished bug and create a new one for Martin for the remaining "targeted" work? Or is the work done so far completely unused code?

We will most likely close this but not yet as we need to sync up on this effort and evaluate if this code is needed or the approach we ended up using allows us to remove what landed.
For sure we're not planning to keep pushing patches from this bug, just leaving it open so it doesn't disappear from our radar.

hello :)

I'm still around in the mozilla universe so feel free to message me for any quick inputs, here on bugzilla or in matrix (under henry-x).

The actual files were unused in any widget (apart from the test widget) when I finished. The idea was you could use the controller to have universal controls for lists, trees, tables, grids, treegrids, etc, to ensure that keyboard (and screen reader) users have consistent controls for all these widgets. But you can also implement the same control behaviours individually for each widget.

But I would recommend going through the notes in comment 26, even though it is a long read, to pick out the stuff that isn't implemented yet. It goes through some widget patterns that haven't been implemented at all, but would be great from improving accessibility, e.g. for bug 1629981.

In the shorter-term, since the mail space has a lot of screen reader users and this is being replaced in 115, you might want to look at the notes about the treegrid pattern and the grid and tree keyboard controls. I opened bug 1840989 to track this specific case.