The query SELECT 1 from moz_places WHERE levenshteinDistance(url, title) <= 10 AND url <> '' causes sqlite3.c:46028: failed assertion `pMem1->enc==pMem2->enc If you disable the guts of the levenshteinDistance function (return a dummy result right before the construction of "A" in levenshteinDistanceFunction()) then the even simpler query SELECT 1 from moz_places WHERE levenshteinDistance(url, '123') <= 10 AND url <> '' will cause the assertion failure, but the first query will not! So I don't think it's a problem with the levenshteinDistance function proper, but rather with the "glue" code, either ours or Sqlite's I'm not sure yet. The fact that the first query succeeds when the main levenshteinDistance code runs but fails when it's disabled sounds an awful lot like a memory overwrite to me. I hate memory overwrites.

What version of SQLite are we talking about. What does "SELECT sqlite_version()" say? Can the OP run this in a debugger and tell me what pMem1[0] and pMem2[0] hold? The pMem->enc field records the text encoding of the value. This will be one of UTF-8, UTF-16be, or UTF-16le. But does FF ever use anything other than UTF-8?

(In reply to comment #1) > What version of SQLite are we talking about. What does "SELECT > sqlite_version()" say? 3.6.16 > The pMem->enc field records the text encoding of the value. This will be one > of UTF-8, UTF-16be, or UTF-16le. But does FF ever use anything other than > UTF-8? As far as I know, we don't specify except for a few test functions in our test harness. We should get the default (I think that's UTF-8).

(In reply to comment #1) > What version of SQLite are we talking about. What does "SELECT > sqlite_version()" say? "3.6.16" I won't have time to investigate this further for the next couple of days. However, if the bug is on our side, I strongly suspect it's in this function, most likely in the four statements starting with "int aLen =": void levenshteinDistanceFunction(sqlite3_context *aCtx, int aArgc, sqlite3_value **aArgv) { NS_ASSERTION(2 == aArgc, "Invalid number of arguments!"); // If either argument is a SQL NULL, then return SQL NULL. if (::sqlite3_value_type(aArgv[0]) == SQLITE_NULL || ::sqlite3_value_type(aArgv[1]) == SQLITE_NULL) { ::sqlite3_result_null(aCtx); return; } int aLen = ::sqlite3_value_bytes16(aArgv[0]) / sizeof(PRUnichar); const PRUnichar *a = static_cast<const PRUnichar *>(::sqlite3_value_text16(aArgv[0])); int bLen = ::sqlite3_value_bytes16(aArgv[1]) / sizeof(PRUnichar); const PRUnichar *b = static_cast<const PRUnichar *>(::sqlite3_value_text16(aArgv[1])); ;;;;;; // ::sqlite3_result_int(aCtx, aLen > bLen ? aLen - bLen : bLen - aLen); // return; ;;;;;; // Compute the Levenshtein Distance, and return the result (or error). int distance = -1; const nsDependentString A(a, aLen); const nsDependentString B(b, bLen); int status = levenshteinDistance(A, B, &distance); if (status == SQLITE_OK) { ::sqlite3_result_int(aCtx, distance); } else if (status == SQLITE_NOMEM) { ::sqlite3_result_error_nomem(aCtx); } else { ::sqlite3_result_error(aCtx, "User function returned error code", -1); } }

I suspected as much. I was in the process or writing a test case along the same lines as the function in the previous comment as the previous comment arrived in my mailbox. What SQLite is doing is evaluating the two arguments to the levenshteinDistance() functions and storing those values in "Mem" objects. The text encoding is UTF-8. Then down inside levenshteinDistance(), sqlite3_value_text16() is called which forces a conversion to UTF-16. After the levenshteinDistance() function returns, SQLite attempts to reuse the same Mem object in order to evaluate the url<>'' term, and at that point sees that the encodings no longer match. SQLite was assuming that the parameters to a function do not change - which is mostly true. Their values don't change, but the text encoding might. I'll have to make a few tweaks to SQLite to get it to relax its assumption and not reuse the parameters to functions so aggressively. I'll enter a corresponding bug into the SQLite bug database as soon as I have a test case. I'll follow-up here with a link to that bug report once I have one.

See the SQLite ticket here: http://www.sqlite.org/src/info/2ea2425d34

Problem fixed by this SQLite check-in: http://www.sqlite.org/src/info/f22e388727f0 The fix will appear in the next SQLite release (currently scheduled for 2009-09-14). Let me know if you need me to build an amalgamation for you out of our latest code. A temporary work-around might be to put the url<>'' test in front, like this: SELECT 1 from moz_places WHERE url<>'' AND levenshteinDistance(url, title) <= 10

(In reply to comment #6) > The fix will appear in the next SQLite release (currently scheduled for > 2009-09-14). Let me know if you need me to build an amalgamation for you out > of our latest code. That would be SQLite 3.6.18, yes?

Fixed by bug 524144.