Jeff Walden [:Waldo] Assignee
	Description • 6 years ago

Nothing strictly necessary for UTF-8 parsing, but things that will to one degree or another contribute to getting there.

	Jeff Walden [:Waldo] Assignee
	Comment 1 • 6 years ago

Technically the "drain" aspect of this is not necessary, but it seems preferable to drain immediately than to hold onto unbounded memory we're never going to use.  There are other places that could stand this "drain" treatment, but one thing at a time, as I run across them.

	Jeff Walden [:Waldo] Assignee
	Comment 5 • 6 years ago

Small improvement, nice to not copy stuff (or impose on the compiler to figure out how not to copy, maybe, if we were really lucky before).

	Jeff Walden [:Waldo] Assignee
	Comment 6 • 6 years ago

Fewer lines of code *and* less copying.

	Pulsebot
	Comment 7 • 6 years ago

Pushed by jwalden@mit.edu:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b64aec5cf563
Introduce a drainCharBufferIntoAtom function to clear charBuffer and create an atom from the chars that were in it.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/08cf6e4cccce
Rename atomizeChars to atomizeSourceChars to clarify it's only for atomizing directly from source text.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/481871dcc488
Define TokenStreamAnyChars::updateFlagsForEOL inline in the header, for simplicity.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/d857b06f51ca
Implement a SourceUnits::remaining() function exposing how many code units are left.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/dad92f9be9b7
Avoid copying potential directives in TokenStreamSpecific::getDirective.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/ba24991f7f4d
Add SourceUnits::matchHexDigits and remove SourceUnits::peekCodeUnits used for the same ends (but that also performed a needless copy).  r=arai

	Bogdan Tara[:bogdan_tara | bogdant]
	Comment 8 • 6 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/b64aec5cf563
https://hg.mozilla.org/mozilla-central/rev/08cf6e4cccce
https://hg.mozilla.org/mozilla-central/rev/481871dcc488
https://hg.mozilla.org/mozilla-central/rev/d857b06f51ca
https://hg.mozilla.org/mozilla-central/rev/dad92f9be9b7
https://hg.mozilla.org/mozilla-central/rev/ba24991f7f4d