Open
Bug 887015
Opened 11 years ago
Updated 2 years ago
Huge amounts of CPU time spent in memcpy when using XMLHttpRequest.responseText
Categories
(Core :: DOM: Core & HTML, defect)
Tracking
()
NEW
People
(Reporter: kael, Unassigned)
References
(Blocks 1 open bug, )
Details
(Whiteboard: [games:p?][diamond])
When using XMLHttpRequest to stream large files, .responseText accesses seem to produce tons and tons of reallocations and memcpy operations. In my profiles the memcpy traffic alone accounts for 10-13% of CPU time according to the SPS profile.
The reallocations also create a lot of GC pressure and pauses.
Things that would help here:
Avoid reallocating the responseText string's buffer repeatedly (this has other benefits, in particular reducing the consequences of the responseText leak)
Being able to stream an arraybuffer response (last time I checked this doesn't work?)
Being able to request a string or array buffer representing a subset of the response (instead of getting back a 40MB string representing an entire 40MB file)
Being able to adjust the chunk size the browser uses when notifying you about updates via onreadystatechange.
For reference, this is the library I am using to stream TAR files via XHR:
https://github.com/kevingadd/JSIL/blob/tar/Libraries/multifile.js
Let me know if you need a full test case to test against and I can upload one (it's big, though)
Reporter | ||
Updated•11 years ago
|
Whiteboard: games:p? → [games:p?]
Reporter | ||
Comment 1•11 years ago
|
||
There's a test case up that demonstrates the problem:
http://hildr.luminance.org/bugs/3/Lumberjack/Lumberjack.html?disableSound
Talk to ack -- he has some tar+IndexedDB stuff. Also, yeah, XHR will realloc, though so will arraybuffer. responseText is doubly awful though, because it's ucs2; why use responseText and not an arraybuffer? (you can do chunked arraybuffer)
In the future we also plan to honor content-length and preallocate a buffer if possible.
Comment 3•11 years ago
|
||
> Avoid reallocating the responseText string's buffer repeatedly
Not really doable if JS asks for the .responseText repeatedly, because it's sharing the underlying buffer, which therefore cannot be modified in-place....
> Being able to stream an arraybuffer response (last time I checked this doesn't work?)
chunked-arraybuffer should work, though it's Gecko-only last I checked.
> Being able to request a string or array buffer representing a subset of the response
Does using HTTP Range headers not work? Note that this may not play nice with gzip encoding, though.
> In the future we also plan to honor content-length and preallocate a buffer if possible.
That won't help if the web page polls .responseText on progress, unless the JS GC is very expeditious about dropping those strings; see above.
Updated•11 years ago
|
Blocks: gecko-games
Whiteboard: [games:p?] → [games:p3]
Updated•10 years ago
|
Whiteboard: [games:p3] → [games:p3][diamond]
Whiteboard: [games:p3][diamond] → [games:p?][diamond]
Assignee | ||
Updated•6 years ago
|
Component: DOM → DOM: Core & HTML
Updated•2 years ago
|
Severity: normal → S3
You need to log in
before you can comment on or make changes to this bug.
Description
•