Closed Bug 1397163 Opened 7 years ago Closed 7 years ago

Firefox sync wheel spins indefinitely and crashes often

Categories

(Firefox :: Sync, defect)

Product:
Firefox
Component:
Sync
Version:
55 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1339390

People

(Reporter: porcelain_mouse, Unassigned)

Details

Attachments

(2 files)

error-sync-1504602255858.txt
(deleted), text/plain
Details
error-sync-1504917401312.txt
(deleted), text/plain
Details
Attached file error-sync-1504602255858.txt (deleted) — 
User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0
Build ID: 20170824070828

Steps to reproduce:

Sometimes, when I click on the firefox sync wheel, it spins indefinitely.


Actual results:

 firefox sync wheel, it spins indefinitely.


Expected results:

Sync should complete quickly.

Marking as security related just cause the log looks really sensitive, but I can't be sure.  If not, you may make bug public.
Kit, can you take a look please?
Component: Untriaged → Sync
Flags: needinfo?(kit)
Sync is getting stuck during a bookmark Sync, but the logs don't show why.

The title also mentions "crashes" - could you please elaborate on that?

> Marking as security related just cause the log looks really sensitive, 
> but I can't be sure.  If not, you may make bug public.

I don't seem able to change that state, but these logs aren't sensitive.
Unhiding per comment #2.
Group: firefox-core-security
I'd like to support this request.

Experienced this behaviour on 4 different machines with FF 55, 55.0.1-55.0.3.

Sync wheel spinning, then crash of Firefox ("Firefox is not responding"). Tested x32 and x64. No crash reports are generated when this issue occurs.

Upgraded now to Version 56 Beta 8 and (now) Beta 9 and the problem is gone. Syncing now works blazingly fast as it did with 54.0.1. No crashes for 3 days now on all of the machines which were upgraded to the Beta.
(In reply to Mark Hammond [:markh] from comment #2)
> The title also mentions "crashes" - could you please elaborate on that?

Yes, it's very interesting.  I have two FF sessions on different computers using the same FF account.  When I make a bookmark in one session, one or the *other* one will hang.  Once I realized what was happening, it became pretty easy to reproduce.  It happens more than 50% of the time when I make a bookmark.  So, that fits with the bookmark sync issue, and probably has the same root cause.

I also occasionally get crashed when closing, which I assume is also related.  I always let FF report these so there should be several bug reports, somewhere.

Let me know if I can instrument something and get you more data.

Do you think it would be helpful to export, delete, and import bookmarks?
(In reply to Porcelain Mouse from comment #5)
> I also occasionally get crashed when closing, which I assume is also
> related.

Yeah, I suspect it is related.

>  I always let FF report these so there should be several bug
> reports, somewhere.

about:crashes should list some of those crashes - if you can copy one here we can probably confirm it's related to places.

> Let me know if I can instrument something and get you more data.
> 
> Do you think it would be helpful to export, delete, and import bookmarks?

To start, you could try about:support, and near the bottom you will see a "Places" section with a "verify integrity" button.

If you are comfortable with a terminal/command-line, another slightly risky option would be to:

* Open "show all bookmarks" and from the window select "import and export" -> "restore" - that will list all recent automatically created backups of your bookmarks. Verify there is a recent backup - if not, create one. Maybe create one anyway :)

* Exit Firefox and take a copy of your profile, or at least the places.db file in your profile directory, then delete places.db

* Restart the browser. You should find that the most recent backup listed previously was automatically restored, although from "show all bookmarks -> import and export -> restore" can still be used to restore from the backup you explicitly created.

Note that it's risky as we are messing with your profile in ways we probably shouldn't - hence the advice to ensure you have a backup of the profile available. IOW, do the above at your own risk and only if you are comfortable with the instructions.
ack - sorry - in the above comment, everywhere I wrote "places.db" I meant to write "places.sqlite"
Attached file error-sync-1504917401312.txt (deleted) — 
Another error file from other computer.  Again, I added a bookmark on one system and, when I went to the other computer, FF was hung.
Here is a recent crash report that was submitted: https://crash-stats.mozilla.com/report/index/bp-8dfc25d3-6f21-4610-8931-5f4b30170906
I tried clicking on the "Verify Integrity" button, but I didn't notice anything happen.

I will try futzing with places db later.
Okay, on my faster system, "Verify Integrity" produced this output

> Task: checkIntegrity
+ The database is sane
> Task: checkCoherence
+ The database is coherent
> Task: expire
+ Database cleaned up
> Task: vacuum
+ Initial database size is 46080 KiB
+ The database has been vacuumed
+ Final database size is 46080 KiB
> Task: stats
+ Database size is 46080 KiB
+ pragma_user_version is 37
+ pragma_page_size is 32768
+ pragma_cache_size is -2048
+ pragma_journal_mode is wal
+ pragma_synchronous is 0
+ History can store a maximum of 104510 unique pages
+ Table moz_places has 74371 records
+ Table moz_historyvisits has 196336 records
+ Table moz_inputhistory has 186 records
+ Table moz_hosts has 10078 records
+ Table moz_bookmarks has 30645 records
+ Table moz_keywords has 6 records
+ Table sqlite_sequence has 1 records
+ Table moz_favicons has 0 records
+ Table moz_anno_attributes has 12 records
+ Table moz_annos has 5038 records
+ Table moz_items_annos has 3499 records
+ Table sqlite_stat1 has 15 records
+ Table moz_bookmarks_deleted has 0 records
+ Index sqlite_autoindex_moz_inputhistory_1
+ Index sqlite_autoindex_moz_hosts_1
+ Index sqlite_autoindex_moz_keywords_1
+ Index sqlite_autoindex_moz_favicons_1
+ Index sqlite_autoindex_moz_anno_attributes_1
+ Index sqlite_autoindex_moz_bookmarks_deleted_1
+ Index moz_places_faviconindex
+ Index moz_places_hostindex
+ Index moz_places_visitcount
+ Index moz_places_frecencyindex
+ Index moz_places_lastvisitdateindex
+ Index moz_historyvisits_placedateindex
+ Index moz_historyvisits_fromindex
+ Index moz_historyvisits_dateindex
+ Index moz_bookmarks_itemindex
+ Index moz_bookmarks_parentindex
+ Index moz_bookmarks_itemlastmodifiedindex
+ Index moz_places_url_hashindex
+ Index moz_places_guid_uniqueindex
+ Index moz_bookmarks_guid_uniqueindex
+ Index moz_annos_placeattributeindex
+ Index moz_items_annos_itemattributeindex
+ Index moz_keywords_placepostdata_uniqueindex
> Task: _refreshUI
I deleted places.sqlite and when I restarted FF, everything looks fine.  Bookmarks are all there.  I'm not sure what else I'm looking for.

But, after this, the FF session on my other computer hung, again.  I don't know what's going on, but it's troublesome.
I've been avoiding refreshing my profile.  I know that is sometimes recommended to fix problems, but I'm not sure if that is recommended for me.  Let me know if you think it is a good option for this situation.
(In reply to Porcelain Mouse from comment #12)
> But, after this, the FF session on my other computer hung, again.  I don't
> know what's going on, but it's troublesome.

Isn't it that other computer that should have these steps performed on it?


(In reply to Porcelain Mouse from comment #13)
> I've been avoiding refreshing my profile.  I know that is sometimes
> recommended to fix problems, but I'm not sure if that is recommended for me.
> Let me know if you think it is a good option for this situation.

It may reset some preferences and customizations you have made, but otherwise it's generally a safe thing to do.
(In reply to Mark Hammond [:markh] from comment #14)
> (In reply to Porcelain Mouse from comment #12)
> > But, after this, the FF session on my other computer hung, again.  I don't
> > know what's going on, but it's troublesome.
> 
> Isn't it that other computer that should have these steps performed on it?

I don't know.  Both computer FF sessions experience the all the same symptoms.  I cannot tell them apart by behavior.

As I already reported, the other computer produces no output when I click "Verify Integrity".  CPU usage goes to 100% and the sync wheel is spinning.  But, FF isn't hung, yet.  What does this mean?  Is this diagnostic?

When I quit FF at this point, it crashed.

When I deleted places.sqlite on this system and restarted FF, it came up with CPU pegged and, after a while, a notice popped up that places.js script was unresponsive.  I let it continue twice, but I cannot interact with the tab bar or menu.  Tried to close, caused another crash.  (Continued in next post...)

Was the crash I reported before, useful?

> (In reply to Porcelain Mouse from comment #13)
> > I've been avoiding refreshing my profile.  I know that is sometimes
> > recommended to fix problems, but I'm not sure if that is recommended for me.
> > Let me know if you think it is a good option for this situation.
> 
> It may reset some preferences and customizations you have made, but
> otherwise it's generally a safe thing to do.

Unless there is a reasonable expectation it will fix the problem, I'm reluctant to do it.
After the the third restart of FF on the system that wouldn't run "Verify Integrity", I tried it again and it ran.  Here is the result:

> Task: checkIntegrity
+ The database is sane
> Task: checkCoherence
+ The database is coherent
> Task: expire
+ Database cleaned up
> Task: vacuum
+ Initial database size is 10240 KiB
+ The database has been vacuumed
+ Final database size is 10240 KiB
> Task: stats
+ Database size is 10240 KiB
+ pragma_user_version is 37
+ pragma_page_size is 32768
+ pragma_cache_size is -2048
+ pragma_journal_mode is wal
+ pragma_synchronous is 0
+ History can store a maximum of 67074 unique pages
+ Table moz_places has 9401 records
+ Table moz_historyvisits has 2 records
+ Table moz_inputhistory has 0 records
+ Table moz_hosts has 5311 records
+ Table moz_bookmarks has 30047 records
+ Table moz_bookmarks_deleted has 0 records
+ Table moz_keywords has 6 records
+ Table sqlite_sequence has 1 records
+ Table moz_anno_attributes has 6 records
+ Table moz_annos has 320 records
+ Table moz_items_annos has 2921 records
+ Table sqlite_stat1 has 13 records
+ Index sqlite_autoindex_moz_inputhistory_1
+ Index sqlite_autoindex_moz_hosts_1
+ Index sqlite_autoindex_moz_bookmarks_deleted_1
+ Index sqlite_autoindex_moz_keywords_1
+ Index sqlite_autoindex_moz_anno_attributes_1
+ Index moz_places_url_hashindex
+ Index moz_places_hostindex
+ Index moz_places_visitcount
+ Index moz_places_frecencyindex
+ Index moz_places_lastvisitdateindex
+ Index moz_places_guid_uniqueindex
+ Index moz_historyvisits_placedateindex
+ Index moz_historyvisits_fromindex
+ Index moz_historyvisits_dateindex
+ Index moz_bookmarks_itemindex
+ Index moz_bookmarks_parentindex
+ Index moz_bookmarks_itemlastmodifiedindex
+ Index moz_bookmarks_guid_uniqueindex
+ Index moz_keywords_placepostdata_uniqueindex
+ Index moz_annos_placeattributeindex
+ Index moz_items_annos_itemattributeindex
> Task: _refreshUI

Are either of these reports useful?
However, sync wheel is still spinning on this "other" computer.  Rats. I thought it might have fixed itself.
On the other hand, when quitting FF with sync wheel running, it *did not* crash this last time.  Here is the most recent crash reported from this "other" computer.

https://crash-stats.mozilla.com/report/index/bp-d0b624a9-c587-4fb7-adf9-07e780170909
Thanks - these crashes are indeed caused by something going wrong with places. Mak/Mark, do you have any insights into them?
Flags: needinfo?(standard8)
Flags: needinfo?(mak77)
I also exported bookmarks and tried to import them into a new, clean profile and got an error.  I opened another bug for that.
Okay, so this is now a serious issue.  I created a new profiles on both computers and a new FF account and, when I connected the second computer to the new FF account, it started syncing but hasn't finished after two hours.  I don't think this has anything to do with my old profile.  I'm out of options.  Now I don't understand why this isn't happening for everyone.
Regarding the crash, based on the metadata sync is spinning the events loop waiting for Bookmarks.jsm:reorderChildren. That is waiting on an executeTransaction.
The crash just indicates that operation never completed and when you closed the browser we were still waiting for it, so we forced a crash, it could happen if executeTransaction depends on a promise that never resolved.

We should not be fooled by Bookmarks.jsm: updateBookmark, that was a typo that Kit fixed here:
http://searchfox.org/mozilla-central/diff/674b6aad46db143351dd877fe4f2d2d2492c0e47/toolkit/components/places/Bookmarks.jsm#1802

The crash clearly points to reorderChildren.
It may be related to bug 1339390, the fact Sync starts a complete reorder even just for a few bookmarks is an issue, and that bug was indeed about detecting those cases and doing it in a cheaper way.

Do you have thousands bookmarks in a single folder? like under unsorted bookmarks, or the menu, or the toolbar?
What I see here is a query updating something like 7200 bookmarks.
Flags: needinfo?(mak77)
This is also not common at all for our average user, so we're definitely in an edge case not well tested: Table moz_bookmarks has 30645 records
Ah, yes!  A long time ago, I stopped organizing my bookmarks, sometime after tags were implemented.  Since then, I just put new ones in Other Bookmarks, which is the default when you click the button.  (I wish CTRL-J didn't put things in the Toolbar.)  Sorry, I didn't know this could lead to a problem, but I understand there could be limits to node children.

I'm guessing that also explains the new profile issue, as I tried to restore bookmarks to new profile before syncing.

Thank you for all your help!  This is great news.  I await further troubleshooting/instructions/recommendations.
Thanks for the continued help tracking down your problem, and I'm embarrassed I didn't connect the dots to associate this with the reorder problem. Sadly there's no good advice I can offer other than to (a) reorganize the bookmarks into folders of no more than ~3k bookmarks (it doesn't matter if the folders are nested, just that the count in each isn't too large), or (b) wait for Firefox 58 where we expect to have solved this in a different way (via bug 1305563). I understand neither of these are ideal, but I have no other ideas. In the meantime, I'm duping this against the bug where we are tracking the fact that reordering this many bookmarks is known to have issues.
Status: UNCONFIRMED → RESOLVED
Closed: 7 years ago
Flags: needinfo?(standard8)
Flags: needinfo?(kit)
Resolution: --- → DUPLICATE
No problem!  I'm just glad there is a solution.  I didn't know what what the child limit, so, now I can work with it.  No worries; I'm totally satisfied.  And, I only have to make it to FF58, which isn't very long.  Thanks for all your help.  I hope it didn't make things harder by opening two bugs.  Sounds like you got it all figured out, now, though.  Cheers.
Okay, sorry to keep this alive, but I'm still having problems.  I removed the "bad" keyword that was found in Bug 1398536, and I separated all my unsorted bookmarks into subfolders with less than 2000 child nodes.  But, even after the "other" computer has synced the new bookmark order, it will not finish synchronizing...or hanging when I add bookmarks to the first system.
(In reply to Porcelain Mouse from comment #27)

Same here. I am on Windows. No problems while syncing in FF 54.0.1, problems with 55.0.1-55.0.3, (much) less problems with FF 56 Beta 8 and 9, problems back in part with Beta 10.

Followed this thread and organized all bookmarks from "Other Bookmarks" into yearly folders. Yesterday the stalled Firefox (56B10) happened again.

So the reduction of amount of bookmarks per folder did not solve this problem.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: