indexedDB broken - UnknownError - Error opening Database
Categories
(Core :: Storage: IndexedDB, defect, P2)
Tracking
()
People
(Reporter: aymeric, Assigned: tt)
References
(Blocks 2 open bugs)
Details
(Keywords: dataloss, DevAdvocacy, Whiteboard: DWS_NEXT)
Attachments
(3 files, 1 obsolete file)
Reporter | ||
Comment 1•11 years ago
|
||
Reporter | ||
Comment 2•11 years ago
|
||
Updated•11 years ago
|
Reporter | ||
Comment 3•11 years ago
|
||
Reporter | ||
Comment 4•11 years ago
|
||
Reporter | ||
Comment 6•11 years ago
|
||
Reporter | ||
Comment 8•11 years ago
|
||
Reporter | ||
Comment 10•11 years ago
|
||
Reporter | ||
Comment 11•11 years ago
|
||
Reporter | ||
Comment 12•10 years ago
|
||
Comment 13•10 years ago
|
||
Comment 14•9 years ago
|
||
Comment 15•9 years ago
|
||
Comment 16•8 years ago
|
||
Comment 17•8 years ago
|
||
Comment 18•8 years ago
|
||
Comment 19•8 years ago
|
||
Comment 20•8 years ago
|
||
Comment 21•7 years ago
|
||
Comment 22•7 years ago
|
||
Comment 23•7 years ago
|
||
Comment 24•7 years ago
|
||
Updated•7 years ago
|
Comment 25•7 years ago
|
||
Comment 27•7 years ago
|
||
Reporter | ||
Comment 28•7 years ago
|
||
Reporter | ||
Comment 29•7 years ago
|
||
Comment 30•7 years ago
|
||
Reporter | ||
Comment 31•7 years ago
|
||
Comment 32•7 years ago
|
||
Comment 33•7 years ago
|
||
Comment 34•7 years ago
|
||
Reporter | ||
Comment 35•7 years ago
|
||
Comment 36•7 years ago
|
||
Comment 37•6 years ago
|
||
Comment 38•6 years ago
|
||
Comment 39•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Comment 40•6 years ago
|
||
Comment 41•6 years ago
|
||
Comment 42•6 years ago
|
||
Comment 43•6 years ago
|
||
Updated•6 years ago
|
Comment 44•6 years ago
|
||
Updated•6 years ago
|
Assignee | ||
Comment 45•6 years ago
|
||
Comment 46•6 years ago
|
||
Comment 47•6 years ago
|
||
Comment 48•6 years ago
|
||
Comment 49•6 years ago
|
||
Assignee | ||
Updated•6 years ago
|
Comment 51•6 years ago
|
||
Work status update for broken indexedDB bug fixes (Bug 944918, Bug 1423917) can be found in Bug 1482662.
Comment 52•6 years ago
|
||
I have a question.
Since October last year I'm getting some reports from my user where IndexedDB fails with message "Error: undefined".
In such case, restarting Firefox or whole PC doesn't help, however reinstalling my add-on helps. Also the error seems to be thrown only for some specific queries which fails every time, rest of them still works fine.
Is this related to this bug?
Also where is some info about "Error: undefined" error? Normally I get a full description, like "Error: An attempt was made to use an object that is not, or is no longer, usable".
Also in October I started to store image Blobs in the IndexedDB (maybe it's related).
Assignee | ||
Comment 54•6 years ago
|
||
(In reply to juraj.masiar from comment #52)
I have a question.
Since October last year I'm getting some reports from my user where IndexedDB fails with message "Error: undefined".
In such case, restarting Firefox or whole PC doesn't help, however reinstalling my add-on helps. Also the error seems to be thrown only for some specific queries which fails every time, rest of them still works fine.Is this related to this bug?
Also where is some info about "Error: undefined" error? Normally I get a full description, like "Error: An attempt was made to use an object that is not, or is no longer, usable".Also in October I started to store image Blobs in the IndexedDB (maybe it's related).
It seems like it's not related to because it's not an unknown error unless the message has been transformed (or, perhaps, checking if there are more information on browser console). The definition of the unknown error is here [1]. I don't think IndexedDB would pump out errors like "Error: undefined". (Should be at least be one type of errors on the list of [1]. e.g. UnknownError @ https://xxx.js:8262)
However, we should make messages of unknown errors better. Most of the time, message which developers can get is a bit vague. Here is the bug for tracking that [2].
Here, I intend to fix the majority of unknown issues on initialization. Hopefully, we can reduce the number of unknown errors for indexedDB here.
[1] https://heycam.github.io/webidl/#unknownerror
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1502077
Assignee | ||
Comment 55•6 years ago
|
||
(In reply to Tom Tung [:tt, :ttung] from comment #54)
It seems like it's not related to because it's not an unknown error unless the message has been transformed (or, perhaps, checking if there are more information on browser console). The definition of the unknown error is here [1]. I don't think IndexedDB would pump out errors like "Error: undefined". (Should be at least be one type of errors on the list of [1]. e.g. UnknownError @ https://xxx.js:8262)
If the error message has been transformed (don't know which kinds of error it is), then it's really hard to say.
Comment 56•6 years ago
|
||
Thank you for the info!
So if I understand it correctly, I should just wait for better error reporting to be implemented in Firefox. Right?
Or is there anything else I can do to improve error reporting in my add-on?
Also the error reporting page on MDN is not very clear, there is even a TODO written in the code example :D
https://developer.mozilla.org/en-US/docs/Web/API/IDBRequest/onerror
objectStoreTitleRequest.onerror = function() {
// If an error occurs with the request, log what it is
console.log("There has been an error with retrieving your data: " + objectStoreTitleRequest.error);
// TODO what about event parameter into onerror()? What will be inside of this event?
};
Assignee | ||
Comment 57•6 years ago
|
||
(In reply to juraj.masiar from comment #56)
Thank you for the info!
So if I understand it correctly, I should just wait for better error reporting to be implemented in Firefox. Right?
Yes, we will definitely improve the error reporting on IndexedDB in the future.
I would also encourage logging more information since there are many possibilities to get "Error: undefined".
For instance, the reason could be in the Javascript level:
function foo(error) {
console.log("Error: " + error);
}
// should pass an argument
foo();
let object = {attrbute1: 1}; // dinfine object.attribut1 but doesn't define object.attribute2.
foo(object.attribute2);
Or is there anything else I can do to improve error reporting in my add-on?
Also the error reporting page on MDN is not very clear, there is even a TODO written in the code example :D
https://developer.mozilla.org/en-US/docs/Web/API/IDBRequest/onerrorobjectStoreTitleRequest.onerror = function() { // If an error occurs with the request, log what it is console.log("There has been an error with retrieving your data: " + objectStoreTitleRequest.error); // TODO what about event parameter into onerror()? What will be inside of this event? };
I guess logging the error message is good enough or you could simply log all the DOM exception like your example from MDN.
So, there are two options in my mind.
- Option one is just like your example from MDN and it would show everything about that DOM exception.
- Option two is to log message attribute (and maybe the name attribute as well) inside and you should get one of the error messages in this list [1] if the error comes from IndexedDB.
[1] https://searchfox.org/mozilla-central/source/dom/base/domerr.msg#60-77
Comment 58•6 years ago
|
||
Regarding the "Error: undefined" - I'm not building this kind of strings :), it must be coming from some browser
API. And from what I can see in my method it can be only one of these these:
browser.runtime.getBackgroundPage()
- that I use to detect container / private windowbrowser.runtime.sendMessage
- that I use to get data from background scriptidbRequest.onerror = reject;
- that I use to catch database errors - and it works fine for many other erros
So there is a chance it came from browser.runtime.sendMessage
, although I'm skeptical.
I'm gonna change my error logging to be sure where was the error created. And I will add the 'name' and 'message' attributes.
Thank you for your help!
Assignee | ||
Comment 59•6 years ago
|
||
(In reply to juraj.masiar from comment #58)
Regarding the "Error: undefined" - I'm not building this kind of strings :), it must be coming from some
browser
API. And from what I can see in my method it can be only one of these these:
browser.runtime.getBackgroundPage()
- that I use to detect container / private windowbrowser.runtime.sendMessage
- that I use to get data from background scriptidbRequest.onerror = reject;
- that I use to catch database errors - and it works fine for many other errosSo there is a chance it came from
browser.runtime.sendMessage
, although I'm skeptical.
I'm gonna change my error logging to be sure where was the error created. And I will add the 'name' and 'message' attributes.Thank you for your help!
No problem! Thank you for providing more information! And, it'd be useful if we can find out any other problem making initialization failure and fix them at once.
Comment 60•6 years ago
|
||
Per request of Luca Grego I am uploading the storage of an add-on of his that got corrupted during my recent upgrade to Firefox 66.0.2 (Mac OS X Mojave version 10.14.4 (18E226)).
Steps to reproduce the issue I am having:
- Download and build Simple Tab Groups from source (although I suspect any add-on would cause the same problem)
- Navigate to
about:debugging
and enable it if needed - Load the built
addon/dist/background.js
using the Load Temporary Add-on button and take note of its UUID - Open a terminal,
cd
to your profile directory, untar the attachment into anEXTRACTED
subdirectory andcd
back to your profile directory rsync
theEXTRACTED
data over that of the temporary add-on with a command likenewuuid=f2ae7be4-3368-0d40-a655-bc8e6fcff78e; rsync -av EXTRACTED/storage/default/moz-extension+++d5b1612f-bb2d-cf46-9278-eb78a95372e0\^userContextId=4294967295/idb storage/default/moz-extension+++${newuuid}\^userContextId=4294967295/; rsync -av EXTRACTED/storage/default/moz-extension+++d5b1612f-bb2d-cf46-9278-eb78a95372e0/idb storage/default/moz-extension+++${newuuid}/
- Back to the
about:debugging
page in Firefox, deactivate and reactivate the temporary add-on.
Expected result: An additional Simple Tab Groups icon shows up in the tab bar.
Actual result: An exception about it not being IndexedDB's fault pops up.
Additional steps to confirm that the problem is with storage.local
and (presumably) not IndexedDB itself:
- Run
sqlite3
on every*.sqlite
file in the attached archive, and at the prompt, runPRAGMA integrity_check;
to rule out an sqlite corruption issue - From the
about:debugging
Firefox tab, clic Debug to start a debugger instance; click OK if prompted - From the JS console of said debugger instance, evaluate
browser.storage.local.get(null).then(console.log)
Expected result: a dump of the storage.local
state for this add-on
Actual result: crashes with the same message as before.
As correctly surmised in up-thread, one cannot backtrack out of that situation by using browser.storage.local
alone, and indeed browser.storage.local.clear()
does not help.
Assignee | ||
Comment 61•6 years ago
|
||
(In reply to Dominique Quatravaux from comment #60)
Thanks for the information, especially the STR! Will check that.
Comment 62•6 years ago
|
||
(In reply to Tom Tung [:tt, :ttung] from comment #61)
Thanks for the information, especially the STR! Will check that.
Hey Tom,
I gave it a look and it seems that the failure is not happening when the IndexedDB is opened, but the storage.local.get()
API call is failing consistently when the IndexedDB cursor, used internally to retrieve all the stored value, reaches the item with key "groups".
I tried to retrieve that item directly by key, and that call is also failing consistently with the error:
IndexedDB UnknownErr: ActorsParent.cpp:565 2
UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code.
(but the values set on other keys can be retrieved just fine).
And so it seems that the issue may be triggered by the value set with that key, which is likely a quite big object when there are a bunch of groups with many tabs stored in it, but I don't know yet why it has been able to be stored and then fails to be retrieved.
Let me know if you notice something suspicious in those IndexedDB storage files that may be able to trigger the error raised when we try to retrieve that particular value, I'll still try to dig into it more (the data related to the storage.local IndexedDB database is in the directory named "moz-extension+++d5b1612f-bb2d-cf46-9278-eb78a95372e0^userContextId=4294967295").
As a side note related to the STR:
Instead of manually renaming the storage files, you could use the following slightly modified STR
(which tweaks the about:config preference to force the installed extension to use the UUID of the broken storage IDB directory):
- create a new blank Firefox profile
- install Simple Tab Group from AMO (https://addons.mozilla.org/en-US/firefox/addon/simple-tab-groups/)
- open about:config in a tab, search for the "extensions.webextensions.uuids" preference, edit it and change the uuid value
of the extension with id "simple-tab-groups@drive4ik" to match the UUID from the attached file (that is "d5b1612f-bb2d-cf46-9278-eb78a95372e0") - close Firefox
- copy all the content of the "storage/default/" dir as it is, from the attached files into the "storage/default/" dir of the newly created profile
- start Firefox on the modified profile
Comment 63•6 years ago
|
||
Well, I digged a bit more into the sqlite file of the database in the "broken value" state and I found something suspicious while inspecting the ".sqlite" file using the sqlite3 cli tool:
there are two files "referenced" by the data stored in this database, and they should be named 904 and 893
sqlite> select * from object_data;
1|0bvupCbdlvqFobcmf|||<
...
1|0hspvqt||.904|4294967296
...
1|0uivncobjmt||.893|4294967296
...
sqlite> select * from file;
893|1
904|1
But the files actually stored in the ".files" dir are not exactly matching what we would expect based on the content of the sqlite file:
> ls test-profile-1/storage/default/moz-extension+++d5b1612f-bb2d-cf46-9278-eb78a95372e0\^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files
893 903 journals
I tried to just copy the file 903 to a file named 904, as expected by the status in the sqlite file, and after that the "storage.local.get" calls are able to complete successfully (both when "groups" value is being retrieved by key or through the cursor).
It is also interesting to notice that the "groups" value wasn't that big, especially compared with the "thunmbnails" key:
> JSON.stringify(await db.get("groups")).length
1181104
> JSON.stringify(await db.get("thumbnails")).length
9183118
And so now the questions are:
- how the IndexedDB can enter this kind of inconsistent state?
- is there any particular reason why the "groups" value entered the inconsistent state but "thumbnails" one didn't?
Comment 64•6 years ago
|
||
Could be a similar problem as in bug 1517145.
Updated•6 years ago
|
Comment 65•6 years ago
|
||
Comment 66•6 years ago
|
||
(In reply to Dominique Quatravaux from comment #60)
Created attachment 9055172 [details]
Corrupt IndexedDB storage possibly caused by this bugPer request of Luca Grego I am uploading the storage of an add-on of his that got corrupted during my recent upgrade to Firefox 66.0.2 (Mac OS X Mojave version 10.14.4 (18E226)).
Thanks a lot for sharing the corrupted file, it has been very useful to inspect the internal state and get more insights about what kind of issue we have to investigate.
I asked to take the file down because in Comment 63 there are now detailed enough information to recover it from the corrupted state, and I wanted to be sure that the file wasn't available anymore on this bug, just as a precaution
(as a side note, you may also use comment 63 to recover your data from that database).
:ttung :jvarga let me know if you need the corrupted file to dig more into the underlying issue.
Assignee | ||
Comment 67•6 years ago
|
||
(In reply to Luca Greco [:rpl] from comment #63)
sqlite> select * from file;
893|1
904|1But the files actually stored in the ".files" dir are not exactly matching what we would expect based on the content of the sqlite file:
ls test-profile-1/storage/default/moz-extension+++d5b1612f-bb2d-cf46-9278-eb78a95372e0^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files
893 903 journals
Hmm, it seems it's a similar issue related to structedClone
Assignee | ||
Comment 68•6 years ago
|
||
Based on comment 60, comment 62, and comment 63, I believe this is the same issue as bug 1519859.
Although :rpl in comment 63 provided a way to recover (thanks!), I would like to find out why the file_id in the sqlite table and the physical file are different as my first step.
(In reply to Luca Greco [:rpl] from comment #63)
I need to investigate more to answer these questions.
And so now the questions are:
- how the IndexedDB can enter this kind of inconsistent state?
- is there any particular reason why the "groups" value entered the inconsistent state but "thumbnails" one didn't?
Assignee | ||
Updated•6 years ago
|
Assignee | ||
Comment 69•6 years ago
|
||
I don't think this is an initialization issue because it won't block the initialization. Since the problem is really serious, I am putting it into another meta bug to track this issue. (I will definitely keep working on this.)
Comment 70•5 years ago
|
||
Could be misplaced sqlite files cause of the "Error: The operation failed because the requested database object could not be found. For example, an object store did not exist but was being opened."?
Comment 71•5 years ago
|
||
(In reply to Andrew Sutherland [:asuth] (he/him) from comment #43)
… The problem is that some code paths which enumerate the
contents of the directory report an effectively fatal error if
they find files that QuotaManager doesn't think should be there. …
bug 1493262 comment 11 re: thumbs.db
bug 1423917 comment 47 re: case insensitivity, elsewhere in 1423917 I see Thumbs.db
but not thumbs.db
– are both cases covered?
- https://docs.microsoft.com/windows/desktop/api/thumbcache/nn-thumbcache-ithumbnailcache#remarks exemplifies
Thumbs.db
- https://docs.microsoft.com/azure/storage/files/storage-files-faq#afs-files-excluded exemplifies lowercase
thumbs.db
and if I recall correctly, I have seen at least one screenshot of Windows Explorer or File Explorer withthumbs.db
From bug 1423917 comment 85:
… let's just remove this constant and use case-insensitive comparison.
– but I can't tell whether the eventual fix compares in that way. (If I'm overlooking something: sorry.)
☑ bug 1423917 re: Desktop.ini
and desktop.ini
☑ bug 1423917 comment 46 re: .desktop
and .DS_Store
☐ Anything else?
Comment 72•5 years ago
|
||
(In reply to Graham Perrin from comment #71)
bug 1423917 comment 47 re: case insensitivity, elsewhere in 1423917 I see
Thumbs.db
but notthumbs.db
– are both cases covered?
No. Thanks for pointing this out and providing the links (below)!
- https://docs.microsoft.com/windows/desktop/api/thumbcache/nn-thumbcache-ithumbnailcache#remarks exemplifies
Thumbs.db
- https://docs.microsoft.com/azure/storage/files/storage-files-faq#afs-files-excluded exemplifies lowercase
thumbs.db
and if I recall correctly, I have seen at least one screenshot of Windows Explorer or File Explorer withthumbs.db
From bug 1423917 comment 85:
… let's just remove this constant and use case-insensitive comparison.
– but I can't tell whether the eventual fix compares in that way. (If I'm overlooking something: sorry.)
That was for desktop.ini and it is implemented that way, yeah.
:ttung, what do you think about making Thumbs.db comparison insensitive (so lower-case the constant and change the comparison to LowerCaseEqualsLiteral too)? QM_INIT_TELEMETRY_ERROR does still seem to be showing a lot of unexpected files.
Assignee | ||
Comment 73•5 years ago
|
||
(In reply to Andrew Sutherland [:asuth] (he/him) from comment #72)
:ttung, what do you think about making Thumbs.db comparison insensitive (so lower-case the constant and change the comparison to LowerCaseEqualsLiteral too)? QM_INIT_TELEMETRY_ERROR does still seem to be showing a lot of unexpected files.
Sure, I will send a patch to you. Thank Graham, and Andrew for the notice! I didn't notice there is "thumbs.db".
BTW, I want to get these filenames by telemetry probes instead of waiting for reports, but still waiting for replies [1] because I'm not so sure if that data would be too sensitive and there is no category for tracking filenames.
Assignee | ||
Comment 74•5 years ago
|
||
(In reply to juraj.masiar from comment #70)
Could be misplaced sqlite files cause of the "Error: The operation failed because the requested database object could not be found. For example, an object store did not exist but was being opened."?
Yes, that's the error messages about not finding a database in the expected place. (Normally, SQLite databases should live inside the "idb" folder with a corresponding folder (the same name as the database file but ends with ".files"))
Assignee | ||
Comment 75•5 years ago
|
||
Assignee | ||
Updated•5 years ago
|
Comment 76•5 years ago
|
||
Comment 77•5 years ago
|
||
bugherder |
Assignee | ||
Comment 78•5 years ago
|
||
Comment on attachment 9069286 [details]
Bug 944918 - Allow thumbs.db in the storage directory;
Beta/Release Uplift Approval Request
- User impact if declined: If their storage directory contains "thumbs.db", the storage initialization would fail.
- Is this code covered by automated tests?: Yes
- Has the fix been verified in Nightly?: Yes
- Needs manual test from QE?: No
- If yes, steps to reproduce:
- List of other uplifts needed: None
- Risk to taking this patch: Low
- Why is the change risky/not risky? (and alternatives if risky): The patch only makes QuotaManager allow files with name "thumbs.db" in the storage directory and is verified with a unit test. So, this is not risky.
- String changes made/needed:
Comment 79•5 years ago
|
||
So the strategy to avoid this bug is to whitelist a number of different files that are "allowed" to be in the DB directory?
Forgive me if this is a stupid question but why should the IndexedDB engine care at all about files that are not part of the database? I mean shouldn't the engine code just read the files that it needs and ignore everything else there is in the directory?
The problem with the whitelisting approach is that there will always be new antivirus programs, OS features etc. that may add their temporary junk to whichever directory. And when that happens, things break and it can take a long time before a Firefox patch is released to fix that?
Assignee | ||
Comment 80•5 years ago
|
||
(In reply to voltron from comment #79)
So the strategy to avoid this bug is to whitelist a number of different files that are "allowed" to be in the DB directory?
Forgive me if this is a stupid question but why should the IndexedDB engine care at all about files that are not part of the database? I mean shouldn't the engine code just read the files that it needs and ignore everything else there is in the directory?
The problem with the whitelisting approach is that there will always be new antivirus programs, OS features etc. that may add their temporary junk to whichever directory. And when that happens, things break and it can take a long time before a Firefox patch is released to fix that?
Thanks for providing opinion/suggestion! It's always good to have viewpoints in any direction. It would be great if you can point any existing files which don't appear on the list and break the IDB/storage.
We have been looking into this issue for a long while, but we don't have enough information on whether the current approach is suitable or not.
The advantage of the current approach (maintaining an allow list) is:
- Easy to catch problems because this approach is more strict.
- Making sure the usage of firefox storage be almost the same as the usage calculated by the OS (For instance, if there is a file created by some applications on the firefox storage directory, then general users might be confused when they check the usage from Firefox and OS) [1].
[1] Also, QuotaManager is responsible for managing quota and usage of Firefox. If a unexpected file which takes a lots of usage and it's ignored, that might be a potential issue for QuotaManager to manage the storage.
The disadvantage of it is just as what you mentioned:
- Comparatively easy to have an unexpected file and thus break the storage initialization/IDB
It is clear that there are not a few unexpected files on the storage/IDB directory and these files haven't been reported yet. I wanted to know the distribution of filenames for these unexpected files before making the next decision (bug 1529016). Like if most of them have the same filename, then we can keep the current approach. If these files are in different names, then we should consider ignoring these unexpected files in the storage directory instead of maintaining a list.
TBH, I don't have a strong opinion of taking the current approach, but maybe it's better to have more evidence (likes there are many unexpected files in different names created by antivirus/OS in the IDB/storage directories) before deciding to change.
Andrew and Jan, please correct me if I misunderstand anything and please elaborate more if you think there is anything unclear. Thanks!
Comment 81•5 years ago
|
||
Tom's understanding matches my own. Elaborating on our future plans, we are going to start erasing origins that are "corrupt" in the interest of being self-healing. The allow-list/allow-heuristics are useful to know what to ignore versus what merits erasing the origin.
I think the interesting question about features adding junk to directories is why/how the junk gets there. The options are:
- Daemon process like A/V. I'd assume A/V would filter out any temporary files it creates, but even if they are transient, at least as long as the A/V cleans up after itself, it should not interfere with QuotaManager. If it is leaving garbage around, it does seem appropriate to clear the origin because no one else is going to clean it up.
- Users browsing into the Quota Manager directories with file managers. It's probably worth noting that the entries we've added to our allow-list stem from the Windows and OS X file managers that date from when the OS's used "dumb" filesystems (or when they were using dumb filesystems, in the case of OS X dealing with FAT) that didn't support journaling or associated metadata. Going forward, I don't think we'd expect new types of metadata detritus to be created by Windows and OS X. For linux, well, we've added IsDotFile() to approve anything that starts with a period. That's been convention forever.
In the latter case, there's the question of why a user would be looking in the directory. Options would seem to be investigating breakage, investigating for privacy reasons, or investigating for disk usage purposes. In the breakage case, if we get confused and wipe the origin, it seems okay. Wiping if they're investigating privacy or disk usage does seem bad, but it's not clear why tools like that would be creating derived data-files everywhere in the tree.
The big thing for us is making sure that we have Telemetry data that tracks the rate of these clearings in general and specifically on an aggregate per-profile basis so that we can tell if these aren't one-off clearings but in fact evidence of a systemic problem. (Which is why bug 1529016 would be nice to get fixed.)
Comment 82•5 years ago
|
||
One unlikely but extremely frustrating scenario would arise if a user finds a database they want to keep more safe, makes a "backup" copy in the same directory - and exactly this action causes Firefox to remove both the original database and the "backup".
Comment 83•5 years ago
|
||
(In reply to Denis Lisov from comment #82)
One unlikely but extremely frustrating scenario would arise if a user finds a database they want to keep more safe, makes a "backup" copy in the same directory - and exactly this action causes Firefox to remove both the original database and the "backup".
I agree this would be unfortunate and really frustrating for the user in question. Noting that IndexedDB databases' schemas are highly specific to the version of JS code that created/used them at the time, so it's not clear that export of the databases would be useful[1]... I think our best hope for any use-case like this would be exporting the data to some weird newline-delimited JSON-ish format. (IndexedDB can store object graphs and things that JSON can't express, though, so it would be lossy.). Bug 1062613 was filed on doing that for localStorage/sessionStorage and one could certainly file a bug on that, but it's unlikely to be the highest priority.
1: In the sense that having the exported database isn't likely to be able to be usefully re-imported. I think it could be handy for analyzing what data a site is storing on your/its behalf. (Although most sinister uses of data would likely be exfiltrated and/or stored on less obviously associated origins/etc.)
Comment 84•5 years ago
|
||
Comment on attachment 9069286 [details]
Bug 944918 - Allow thumbs.db in the storage directory;
I guess this won't hurt. I wish it were in its own bug though, as leave-open makes it painful to track changes.
Approved for 68.0b8.
Assignee | ||
Comment 85•5 years ago
|
||
(In reply to Julien Cristau [:jcristau] from comment #84)
I guess this won't hurt. I wish it were in its own bug though, as leave-open makes it painful to track changes.
Sorry, will file a new bug to make it easier to be tracked next time.
Comment 86•5 years ago
|
||
bugherder uplift |
Updated•5 years ago
|
Comment 87•5 years ago
|
||
Comment on attachment 9069286 [details]
Bug 944918 - Allow thumbs.db in the storage directory;
clearing uplift flag to get this out of the uplift queries.
Comment 88•5 years ago
|
||
Is this related?
PushService: stateChangeProcessEnqueue: Error transitioning state UnknownError PushService.jsm:144
IndexedDB UnknownErr: ActorsParent.cpp:581
IndexedDB UnknownErr: ActorsParent.cpp:581 2
IndexedDB UnknownErr: ActorsParent.cpp:581
Error: IndexedDB list() The operation failed for reasons unrelated to the database itself and not covered by any other error code. kinto-offline-client.js:579:19
IndexedDB UnknownErr: ActorsParent.cpp:581 Error: IndexedDB list() The operation failed for reasons unrelated to the database itself and not covered by any other error code. kinto-offline-client.js:579:19
IndexedDB UnknownErr: ActorsParent.cpp:581 Error: IndexedDB list() The operation failed for reasons unrelated to the database itself and not covered by any other error code. kinto-offline-client.js:579:19
IndexedDB UnknownErr: ActorsParent.cpp:581 Error: IndexedDB list() The operation failed for reasons unrelated to the database itself and not covered by any other error code. kinto-offline-client.js:579:19
IndexedDB UnknownErr: ActorsParent.cpp:581 Error: IndexedDB list() The operation failed for reasons unrelated to the database itself and not covered by any other error code. kinto-offline-client.js:579:19
IndexedDB UnknownErr: ActorsParent.cpp:581 Error: IndexedDB list() The operation failed for reasons unrelated to the database itself and not covered by any other error code. kinto-offline-client.js:579:19
IndexedDB UnknownErr: ActorsParent.cpp:581
Updated•5 years ago
|
Updated•5 years ago
|
Updated•5 years ago
|
Assignee | ||
Comment 89•5 years ago
|
||
The unknown error is a general error when there is no better error code to define the failure.
So far, issues like:
- The errors that cause initialization failure
For instance:- Unexpected files in the profile and thus cause initialization failures
- Long filename issues on Windows
They are tracked in Bug 1482662, and Bug 1619893 for the errors during temporary storage initialization specifically.
The long filename issues should be fixed starting from Nightly 77.
Hopefully, we should get away from unknown files once we fixed bug 1621916, bug 1621920, and bug 1594075.
Note that another future plan is QuotaManager version 4 (bug 1593365). Hopefully, that would help us to avoid any corrupted storage in an origin to block all QuotaManager service.
- Error in structure cloned and thus cause entries in the sqlite database mismatching
We have made a refactoring for the indexedDB code that related to structure clone recently in Bug 1617170. We will see if the problem will still exist or not.
(In reply to gwarser from comment #88)
Sorry for the late reply, but do the problem still exist? If so, I will file another bug for that, and let's figure the problem out there.
I'm going to close this bug because this bug is a bit general. There are too many different issues inside this ticket. If I don't miss something, I think all issues here are tracked except the one in comment #88.
Please let me know if there is an issue in this bug and it is not tracked by another existing ticket. And we should focus on the cause of the issue on another ticket.
Assignee | ||
Comment 90•5 years ago
|
||
(In reply to Tom Tung [:tt, :ttung] from comment #89)
(In reply to gwarser from comment #88)
Sorry for the late reply, but do the problem still exist? If so, I will file another bug for that, and let's figure the problem out there.
needinfo in case the issue is not tracked.
Description
•