7,500 instances of "NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80630001" emitted from storage/mozStorage(Connection|Service).cpp during linux64 debug testing

Hi, I would like to work on this. Please check the submitted patch.

The fix here doesn't just consist into removing the warning, you should run one (or more) of the above test get a stack of where and why initializeInternal() is returning an error, probably one of the executeSql calls inside it return a native SQLITE error, it would be useful to know which one. It would also be useful to know what's the url passed to openDatabaseWithFileURL

At that point we should be able to evaluate whether this warning is correct and the consumers should be fixed, or wrong, and then we could mute it.

To inspect this locally you'll want to have a debug build, you can enable this by adding the following to your .mozconfig build file:

ac_add_options --enable-debug

One of the tests where this reproduces is the wpt idbobjectstore-rename-store.html test, you can run that locally with:

./mach test testing/web-platform/tests/IndexedDB/idbobjectstore-rename-store.html

Marco would like to know which line is returning a failure in initializeInternal [1]. You can either inspect with a debugger or more simply just add a printf indicating which line is returning. Additionally it would be useful to log error that comes back from sqlite. You'll also want to confirm that we still see this warning

[1] https://searchfox.org/mozilla-central/rev/44a212460990ffffecf50a8e972d3cbde2e7216b/storage/mozStorageConnection.cpp#703

Per https://www.oxymoronical.com/nsresult/:

Results for 0x80630001
    NS_ERROR_STORAGE_BUSY
    SQLite database connection is busy

This likely indicates that an IndexedDB test is attempting to open a database that's currently being checkpointed on another thread. I believe bug 1541550 most specifically covers this, but bug 1541548 could also be involved. Fixing those bugs is likely to be the best way to eliminate this spam.

In general, a connection being busy does indicate a scheduling/coordination problem, so the spam is arguably good, except for how long it's taken to address it. It could make sense to have some kind of open flag that indicates that a busy-wait is expected/possible and the specific error shouldn't be warned. When a bug is filed to fix the underlying issue (as has been done for IDB), the open requests related to that bug could then be marked as expecting the busy-wait to suppress the spam.

Context

The general issue is that there are potentially 3 threads involved in IDB database work:

1. The PBackground thread where IndexedDB coordinates access to databases and decides what thread should handle an IDB requests.
2. The QuotaManager I/O thread where IDB open requests are serviced. This thread will initially open a database to do a version check, and then subsequently close the database so it can be re-opened on the next thread. That's bug 1541548.
3. The IDB connection thread where all of the database transactions are serviced. When the database is closed, write-ahead-log checkpointing may occur which does blocking IO. Bug 1541550 is about IDB doing a better job of tracking/coordinating this closure so QM I/O opens don't get run while this is still happening.