[1.Description]: [Aries KK v2.5][Flame KK v2.5][Browser] After you cleared browsing history, cookie, and stored data, you can find there is no 'History' and 'Top sites' any more. Time: 20:19 See attachmenats: and Aries_KK v2.5.3gp and logcat_2019.txt [2.Testing Steps]: 1.Launch settings app. 2.Browsing privacy > clear browsing history > clear. 3.Clear cookies and stored data > clear. 4.Launch browser app and visit some websites. 5.Kill the progress and launch brower app again. [3.Expected Result]: 5.There should be 'History' and 'Top sites'. [4.Actual Result]: 5.There is no 'History' and 'Top sites'. [5.Reproduction build]: Aries KK v2.5 (Affected) Build ID 20151019205841 Gaia Revision a87f947366c2e044bd6336e1982419ac45378969 Gaia Date 2015-10-19 15:22:08 Gecko Revision https://hg.mozilla.org/mozilla-central/rev/9605da94e75d61598d3c00f01a12d1b6bc427a6c Gecko Version 44.0a1 Device Name aries Firmware(Release) 4.4.2 Firmware(Incremental) eng.worker.20151019.201730 Firmware Date Mon Oct 19 20:17:38 UTC 2015 Bootloader s1 Flame KK v2.5 (Affected) Build ID 20151019150205 Gaia Revision a87f947366c2e044bd6336e1982419ac45378969 Gaia Date 2015-10-19 15:22:08 Gecko Revision https://hg.mozilla.org/mozilla-central/rev/9605da94e75d61598d3c00f01a12d1b6bc427a6c Gecko Version 44.0a1 Device Name flame Firmware(Release) 4.4.2 Firmware(Incremental) eng.cltbld.20151019.182947 Firmware Date Mon Oct 19 18:29:58 EDT 2015 Firmware Version v18D v4 Bootloader L1TC000118D0 Flame KK v2.2 (Unaffected) Build ID 20151019032501 Gaia Revision 885647d92208fb67574ced44004ab2f29d23cb45 Gaia Date 2015-10-07 13:05:24 Gecko Revision https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/6b4e563acaf9 Gecko Version 37.0 Device Name flame Firmware(Release) 4.4.2 Firmware(Incremental) eng.cltbld.20151019.070114 Firmware Date Mon Oct 19 07:01:25 EDT 2015 Firmware Version v18D v4 Bootloader L1TC000118D0 [6.Reproduction Frequency]: Always Recurrence,10/10 [7.TCID]: Free test

[Blocking Requested - why for this release]: regression issue

Noticing that the the repro in this is about 1 out of every 3 attempts, sometime 1/4. Please disregard the 10/10 recurrence found in the first comment. Currently working on a window for this.

Caused by changes made in Bug 1202770 B2g Inbound Regression Window Last Working Environmental Variables: Device: Flame 2.5 BuildID: 20151015081954 Gaia: 8ea9029190af2ffeb04dcd97b323738125e31a0e Gecko: 234d761e29f96020d9659f1eacdb318d59d5055f Version: 44.0a1 (2.5) Firmware Version: v18D User Agent: Mozilla/5.0 (Mobile; rv:44.0) Gecko/44.0 Firefox/44.0 First Broken Environmental Variables: Device: Flame 2.5 BuildID: 20151015084857 Gaia: aec2c319e898b6cfce2d625217ccf669b5c8f88f Gecko: 2434699f9cad0d3c262ebdc852fdc1763b5c48ef Version: 44.0a1 (2.5) Firmware Version: v18D User Agent: Mozilla/5.0 (Mobile; rv:44.0) Gecko/44.0 Firefox/44.0 Last Working gaia / First Broken gecko - This issue does NOT occur with broken Gecko Gaia: 8ea9029190af2ffeb04dcd97b323738125e31a0e Gecko:2434699f9cad0d3c262ebdc852fdc1763b5c48ef Last Working gecko / First Broken gaia - This issue DOES occur with broken Gaia Gecko: 234d761e29f96020d9659f1eacdb318d59d5055f Gaia: aec2c319e898b6cfce2d625217ccf669b5c8f88f B2g Inbound Pushlog: https://github.com/mozilla-b2g/gaia/compare/8ea9029190af2ffeb04dcd97b323738125e31a0e...aec2c319e898b6cfce2d625217ccf669b5c8f88f

Guillaume, this issue seems to have been caused by the changes for bug 120770. Can you please take a look?

Comment on attachment 8676794 [details] [gaia] gmarty:Bug-1216394-After-cleared-browsing-history-no-History-Top-sites > mozilla-b2g:master Dale, can you take a look at this patch? The remove operation is only called when clearing the browsing history (we don't use clear on the Places datastore because we want to keep the pinned pages). So it's safe enough to wipe out the visits indexedDB on this occasion. This safes us a lot of troubles as we don't have to look for visits to a particular web page and clear them manually.

Reverted merge: https://github.com/mozilla-b2g/gaia/commit/7e02c343af6261bfbaa201d171c33d1f596abf55

This seems like the wrong assumption to be making, we dont call remove in the system app for individual places currently, but we may want to in the future and I am fairly certain history sync can call to remove individual history items without wanting to wipe all visits. Whats the cause of the bug? it may be that I am misunderstanding something

We now remove individual places when clearing browser history. Clear wouldn't work as we want to keep pinned pages (See Bug 1202770). But when clearing the history, we want to clear the visits indexedDB too, and that includes visits made to pinned pages (we are already removing the visits array property of pinned pages). So really what we need is to call clear on the visits indexedDB of the search app. We don't really have a way to let the search app know when the history is being clear, so I rely on individual places being removed. I know it's hacky but it works for now. If in the future we need to remove individual places from the Places datastore outside of clearing the browsing history, then we can revisit this and think of another approach. What do you think?

Not particularly as mentioned sync has already added the ability to remove individual places and it is super confusing to have individual .removes interpreted as a clear(), Adding Sean since he worked on the sync adapter When places_idb_store sees an update with visits = [], or a .remove of an individual place it should do the corresponding update to the visits store and then everything should be kept in sync right?

Hey Dale, Guillaume, Guillaume's patch fixes bug 1211370 as well after testing the patch, and thank you for doing this. :) I provide Firefox Sync/HistoryAdapter information to discuss if there is any issue that Search app and SyncAdapter should cooperate together. Firefox Sync has three ways to remove whole/single visits of a place record: * 'deleted: true' command: SyncAdapter uses datastore.remove(url) to remove a specific place record. * deleting one visit: SyncAdapter uses datastore.update(...) to override the visits with new one. * deleting one visit until visits empty: the same as 'deleted: true' History Adapter provides one-way synchronization (FirefoxService to FxOS TV/Phone) currently, and I think the patch can provide an expected behavior. I am considering two-way case that HistoryAdapter has to know any changes of Places DataStore and push back to Firefox Service. When a place record is removed by an app (I am not sure which app will do that), will it be removed at both IndexedDB and Places DataStore? Thanks for doing this :)

Comment on attachment 8677367 [details] [gaia] gmarty:Bug-1216394-After-cleared-browsing-history-no-History-Top-sites-new > mozilla-b2g:master Thanks Sean for the details. I implemented the solution suggested by Dale. Both the places datastores and search app indexedDB should stay in sync as long as all the transactions are completed. The solution required more work to allow deleting entries from the visits db given a URL and includes an indexedDB version bump + upgrade script. Dale, how does it look now?

Comment on attachment 8677367 [details] [gaia] gmarty:Bug-1216394-After-cleared-browsing-history-no-History-Top-sites-new > mozilla-b2g:master This looks great, thanks a lot, sorry for the hassle

Landed in master: https://github.com/mozilla-b2g/gaia/commit/1c9bec1837676093fdef074bdc2d6de0a83c0238

Hi Jessica, Please verify it, thanks.

Verified on [Flame] Build ID 20151027150240 Gaia Revision a26eadc5e1133d5112b6cbc10badbb7670a1090f Gaia Date 2015-10-27 17:36:52 Gecko Revision https://hg.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d Gecko Version 44.0a1 Device Name flame Firmware(Release) 4.4.2 Firmware(Incremental) eng.cltbld.20151027.184038 Firmware Date Tue Oct 27 18:40:49 EDT 2015 Bootloader L1TC000118D0 [Aries] Build ID 20151027221526 Gaia Revision a26eadc5e1133d5112b6cbc10badbb7670a1090f Gaia Date 2015-10-27 17:36:52 Gecko Revision https://hg.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d Gecko Version 44.0a1 Device Name aries Firmware(Release) 4.4.2 Firmware(Incremental) eng.worker.20151027.213419 Firmware Date Tue Oct 27 21:34:27 UTC 2015 Bootloader s1