Closed Bug 1849007 Opened 1 year ago Closed 1 year ago

Accessible text range extents / bounding box are sometimes empty

Categories

(Core :: Disability Access APIs, defect)

defect

Tracking

()

RESOLVED FIXED
119 Branch
Tracking Status
firefox119 --- fixed

People

(Reporter: jdiggs, Assigned: Jamie)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files)

Attached file Listener: range-extents.py (deleted) —

Steps to reproduce:

  1. Launch the attached accessible-event listener in a terminal.
  2. Load (or reload) the attached test case.
  3. Examine the listener output.

Expected results: Extents would never be (0, 0, 0, 0)

Actual results: Extents for the 'B' are (0, 0, 0, 0)

My output:

Examining text range extents for [section | ] '[OBJ]Bar. '
Char '[OBJ]' has extents (99, 245, 49, 27)
Char 'B' has extents (0, 0, 0, 0)
Char 'a' has extents (148, 245, 13, 27)
Char 'r' has extents (161, 245, 11, 27)
Char '.' has extents (172, 245, 7, 27)
Char ' ' has extents (179, 245, 5, 27)

Impact: Orca uses text range extents in its logic to determine line contents. If a substring has empty extents, it will likely be treated as if it's on a separate line.

Attached file test case: range-extents.html (deleted) —

Jamie: I don't think this is a regression. Or if it is, it's been around awhile. But it would be great to have this fixed. Thanks!

This is bizarre. Further distilled:
data:text/html,<div><strong title="a">a </strong>%0abc
The line feed seems to be what triggers the problem here, but I don't really understand why. I guess this is probably another case where the primary text frame is empty or something.

Morgan, does the test case above look any different visually to this one without the line feed?
data:text/html,<div><strong title="a">a </strong>bc

Flags: needinfo?(mreschenberg)
Blocks: boundsa11y, orca
Severity: -- → S3

(In reply to James Teh [:Jamie] from comment #3)

This is bizarre. Further distilled:
data:text/html,<div><strong title="a">a </strong>%0abc
The line feed seems to be what triggers the problem here, but I don't really understand why. I guess this is probably another case where the primary text frame is empty or something.

Morgan, does the test case above look any different visually to this one without the line feed?
data:text/html,<div><strong title="a">a </strong>bc

No these two test cases look the same :(

Flags: needinfo?(mreschenberg)

Here's a frame tree dump for data:text/html,<div><strong title="a">a </strong>%0abc if it helps:

Viewport(-1)@105548020 [view=10eddf500] (x=0, y=0, w=1024, h=720) [cs=10fa495d8:-moz-viewport] <
  HTMLScroll(html)(-1)@105548198 parent=105548020 (x=0, y=0, w=1024, h=720) [content=110408070] [cs=10fa498a8:-moz-viewport-scroll] <
    ScrollbarFrame(scrollbar)(-1)@105548430 parent=105548198 next=105548678 (x=0, y=0, w=0, h=0) [content=110408220] [cs=10fa4a898] <
      SliderFrame(slider)(-1)@105548520 parent=105548430 (x=0, y=0, w=0, h=0) [content=1104083d0] [cs=10fa4a988] <
        Frame(thumb)(0)@105548600 parent=105548520 (x=0, y=0, w=26, h=0) [content=11040c0c0] [cs=10fa4aa78]
      >
    >
    ScrollbarFrame(scrollbar)(-1)@105548678 parent=105548198 next=1055488c0 (x=0, y=0, w=0, h=0) [content=1104082b0] [cs=10fa4ab68] <
      SliderFrame(slider)(-1)@105548768 parent=105548678 (x=0, y=0, w=0, h=0) [content=110408460] [cs=10fa4ac58] <
        Frame(thumb)(0)@105548848 parent=105548768 (x=0, y=0, w=0, h=26) [content=11040c170] [cs=10fa4ad48]
      >
    >
    Frame(scrollcorner)(-1)@1055488c0 parent=105548198 next=1055480c8 (x=1024, y=720, w=0, h=0) [content=110408340] [cs=10fa49b78]
    Canvas(html)(-1)@1055480c8 parent=105548198 (x=0, y=0, w=1024, h=720) [content=110408070] [cs=10fa4a028:-moz-scrolled-canvas] <
      Block(html)(-1)@105548938 parent=1055480c8 (x=0, y=0, w=1024, h=35.2) [content=110408070] [cs=10fa484f8] <
        line@105548ac8 count=1 state=block,clean,prevmarginclean,not-impacted,not-wrapped,no-break,clear-before:none,clear-after:nonebm=8 (x=8, y=8, w=1008, h=19.2) ink-overflow=(x=7.5, y=8, w=1008.5, h=19.2) scr-overflow=(x=8, y=8, w=1008, h=19.2) <
          Block(body)(1)@105548a00 parent=105548938 (x=8, y=8, w=1008, h=19.2) ink-overflow=(x=-0.5, y=0, w=1008.5, h=19.2) scr-overflow=(x=0, y=0, w=1008, h=19.2) [content=110408190] [cs=10fa497b8] <
            line@105548e18 count=1 state=block,clean,prevmarginclean,not-impacted,not-wrapped,no-break,clear-before:none,clear-after:none(x=0, y=0, w=1008, h=19.2) ink-overflow=(x=-0.5, y=0, w=1008.5, h=19.2) scr-overflow=(x=0, y=0, w=1008, h=19.2) <
              Block(div)(0)@105548b18 parent=105548a00 (x=0, y=0, w=1008, h=19.2) ink-overflow=(x=-0.5, y=0, w=1008.5, h=19.2) [content=110408580] [cs=10fa4a2f8] <
                line@105548dc8 count=2 state=inline,clean,prevmarginclean,not-impacted,not-wrapped,no-break,clear-before:none,clear-after:none(x=0, y=0, w=27.1, h=19.2) ink-overflow=(x=-0.5, y=0, w=28.1, h=19.2) scr-overflow=(x=0, y=0, w=27.1, h=19.2) <
                  Inline(strong)(0)@105548be0 parent=105548b18 next=105548d28 (x=0, y=1.6, w=12, h=16) ink-overflow=(x=-0.5, y=0, w=13, h=16) scr-overflow=(x=0, y=0, w=12, h=16) [content=110408610] [cs=10fa4a3e8] <
                    Text(0)"a "@105548c88 parent=105548be0 (x=0, y=0, w=12, h=16) ink-overflow=(x=-0.5, y=0, w=13, h=16) [content=110410080] [cs=10fa4a4d8:-moz-text] [run=10edd2590][0,2,T] 
                  >
                  Text(1)"\nbc"@105548d28 parent=105548b18 (x=12, y=1.6, w=15.1, h=16) ink-overflow=(x=-0.5, y=0, w=16.1, h=16) scr-overflow=(x=0, y=0, w=15.1, h=16) [content=110410100] [cs=10fa4a5c8:-moz-text] [run=10edd2860][0,3,T] 
                >
              >
            >
          >
        >
      >
    >
  >
>

this section of code is where I'd start looking 🤕

Text(1)"\nbc"@105548d28 parent=105548b18 (x=12, y=1.6, w=15.1, h=16) ink-overflow=(x=-0.5, y=0, w=16.1, h=16) scr-overflow=(x=0, y=0, w=15.1, h=16) [content=110410100] [cs=10fa4a5c8:-moz-text] [run=10edd2860][0,3,T]

Oh wow. So this does not get split across two continuations like we thought it might. Now I'm even more confused.

Oh dear. I understand what's going on here.

GetCharacterRectsInRange works with content offsets, not rendered offsets; i.e. these offsets include non-rendered white space. But a11y TextLeafAccessibles use rendered offsets; i.e. they exclude non-rendered white space. We have RenderedToContentOffset and ContentToRenderedOffset in TextLeafRange to deal with this. However, I think using those is going to be way too expensive if we're doing it for every character.

This also means that this is broken:

data:text/html,<p contenteditable>a%20 b%20 c

The double spaces get rendered as a single space, so that means the rects for every offset after the first space are very wrong.

I think gfxSkipCharsIterator allows us to ask whether a character is rendered or not in an iterative way. GetCharacterRectsInRange uses this, but because that function returns an array, we don't have any access to that as it iterates. And we don't want to get every character rect individually because that would be awful for performance.

GetCharacterRectsInRange must be returning an empty rect or at least something that causes us to return an empty rect. Perhaps:

  1. We can assume that an empty rect means the character is non-rendered. I'm not sure whether there are other "rendered" offsets that have 0 rects though; e.g. what about zero-width spaces? Or
  2. We can have an additional check on top of that to see if the character is white space. Or
  3. If we have to, we could call RenderedToContentOffset only when we detect this to see if it's non-rendered.
Assignee: nobody → jteh
Pushed by jteh@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5c56b92baa65 Skip non-rendered characters in a11y cached character rects, since a11y only deals with rendered text. r=morgan
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 119 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: