While working on tests for bug 1152033, I came across an apparent discrepancy between nsCSSScanner and the CSS Level 3 spec. Consider the input "url(http://example.com @". From http://dev.w3.org/csswg/css-syntax/#consume-url-token I would expect this to return a BAD_URL token and consume the " @", due to step #4, and following http://dev.w3.org/csswg/css-syntax/#consume-the-remnants-of-a-bad-url However, in the test case, what actually happens is that the BAD_URL tokenizing drops the " " and then a SYMBOL "@" token is subsequently returned. I couldn't think of a way to make this cause a user-visible bug.

Sounds like a bug in the spec. If that were a (, I'd want it to be separate so that ()-matching happens correctly.

... oh, wait, the spec describes the bad-url token as including to the next ). I think we implement this is an entirely different way that was intended to be equivalent. (Not sure if exactly.) But I'd need to examine much more closely to be sure.

Or it's possible that this is just something that the WG kept going back and forth on (whether there should be [] and {} matching while looking for the closing ) of a bad URL) and we're just not up-to-date with the latest rule.

One other minor oddity is that the firefox tokenizer treats url("stuff") as a URL token. However, the level 3 spec says this should be a function-token (followed by string, etc). In this case I think that nsCSSScanner's behavior is better. As long as the tokenizer is used purely internally (including devtools) I think it's fine to have the odd divergence.

Looks to me like the fix for bug 790997 will effectively fix this bug....