Richard Newman [:rnewman] Reporter
	Description • 13 years ago

In particular:

* Don't sync if info/collections doesn't report changes and there are no local changes. (Will require touching the DB… maybe after some time interval? But can advance local time if there are no local changes.)
* Don't upload if we don't have records.
* Don't sync at all if meta/global doesn't mention that engine.

	Richard Newman [:rnewman] Reporter
	Comment 1 • 13 years ago

Two axioms:

* We want to sync as frequently as we can, to ensure that devices appear to be always up to date, and to reduce the possibility of conflicts.

* We want to sync as infrequently as we can, because each sync has a measurable impact on battery life, network charges, and usability of other applications.


From observation, Android typically gives us the opportunity to sync every 35 seconds (Transformer, ICS, wifi):

02-25 20:00:02.628 I/SyncAdapter( 2572): Not syncing: must wait another 283178ms.
02-25 20:00:55.738 I/SyncAdapter( 2572): Not syncing: must wait another 230061ms.
02-25 20:01:30.928 I/SyncAdapter( 2572): Not syncing: must wait another 194869ms.
02-25 20:02:04.618 I/SyncAdapter( 2572): Not syncing: must wait another 161181ms.

To take advantage of this, of course, would be insane. But eventually:

* We'll cache keys and meta/global
* The engines to sync can be computed, rather than fixed -- ("server enabled" intersect ("server has changes" union "local has changes"))

and we will thus have a much better ability to cheaply decide when and what to sync.

Checking for local changes will be as easy as a LIMIT 1 query through a content provider, and checking for remote changes will be as easy as comparing the result of a single authenticated HTTP request to info/collections to some stored timestamps.

At this point we can come up with some schedule and sequence (every five minutes, check remote; every time we're prompted, check the most frequently updated enabled local engine; every minute, check the rest…) and get something close to Instant Sync.

As mentioned in Comment 0, we can also slim down the actual sync operations: don't upload if there are no local changes; don't download if there are no remote changes; don't unbundle our keys unless we'll need them; etc.

Note that we absolutely don't want to mess with our timing so long as we fetch keys, meta/global, and each collection on every sync.

In fact, our current five-minute backoff is probably too costly, given the amount of work we do, but that depends on how frequently Android will really ask us to sync on a constrained device. We also don't want to change that to 30 minutes and accidentally cause syncing to occur every 90 minutes, due to accidents of scheduling.

	Paul [pwd]
	Comment 2 • 13 years ago

Since swapping Sense for CyanogenMod, Fennec's sync is the only thing that locks up my system.

	Nick Alexander :nalexander [he/him] (PTO until Montreallhands August 21, 2023)
	Comment 3 • 13 years ago

(In reply to Richard Newman [:rnewman] from comment #0)

> * Don't upload if we don't have records.

AFAICT, we won't.  See Server11RepositorySession.flush, around

https://github.com/mozilla-services/android-sync/blob/develop/src/main/java/org/mozilla/gecko/sync/repositories/Server11RepositorySession.java#L347

> * Don't sync at all if meta/global doesn't mention that engine.

AFAICT, we won't.  See GlobalSession.engineIsEnabled and ServerSyncStage.isEnabled.

	Richard Newman [:rnewman] Reporter
	Comment 5 • 11 years ago

Could also alter our decisions based on connection type.

	Paul [pwd]
	Comment 6 • 11 years ago

I think that bug 850600 may be worth reconsidering given the newer concerns depending on this bug.

	Richard Newman [:rnewman] Reporter
	Comment 7 • 11 years ago

Additional thoughts:

* Sync onPause, subject to some rate-limiting. That solves the phone => desktop case.
* Zero-out our rate limit in BrowserApp.onCreate, so that we'll sync as soon as Android thinks we should (typically as soon as about:home is drawn), regardless of when we last synced. onCreate rather than onResume avoids some bouncing.
* Alter rate-limiting based on whether data was transferred.

Nick, what are your opinions on these?

	Nick Alexander :nalexander [he/him] (PTO until Montreallhands August 21, 2023)
	Comment 8 • 11 years ago

(In reply to Richard Newman [:rnewman] from comment #7)
> Additional thoughts:
> 
> * Sync onPause, subject to some rate-limiting. That solves the phone =>
> desktop case.

This is clear win.  I'm not sure how much rate-limiting is needed, even: if you request a Sync nicely, Android will Sync you within 30 seconds.  I would tend to schedule a force Sync and rate-limit ourselves in this case.

> * Zero-out our rate limit in BrowserApp.onCreate, so that we'll sync as soon
> as Android thinks we should (typically as soon as about:home is drawn),
> regardless of when we last synced. onCreate rather than onResume avoids some
> bouncing.

Not sure what you mean by "zero-out our rate limit".  This is the same rate-limit that is preventing too many onPause() syncs?  So you want to make sure that the first onPause really triggers a sync, but that we don't get too many?

> * Alter rate-limiting based on whether data was transferred.

This is interesting, and I did add code that surfaces how many records were transferred (but not bytes transferred).  But it's not clear which way the limiting should go!

Do you sync less because you sent a lot of data?  In fact, you could probably stand to sync more, because you're likely to not have data next time.  Or do you sync less because you're not sending anything?  When you actually have data, it's likely to pile up, leading to infrequent big syncs.

I think we should push a simple sync on onResume and onPause, using Android instead of forcing and limiting, and see how that feels.

	Richard Newman [:rnewman] Reporter
	Comment 9 • 11 years ago

(In reply to Nick Alexander :nalexander from comment #8)

> Not sure what you mean by "zero-out our rate limit".  This is the same
> rate-limit that is preventing too many onPause() syncs?

Yes.

Our rate limiting operates outside the scope of Android's scheduling; unless you send a forced sync, we won't sync more than once per N minutes.

My proposal is to pseudo-force a sync onPause (that is, sync unless you're in backoff mode), and to simply remove our rate limit onCreate -- if you were already due a sync, this does nothing different; if you synced a few minutes ago, it'll sync again.

I suspect that the impact of the latter will be fairly small: onCreate should be called infrequently, typically less often than our rate limit. But I think our rate limit is probably broken (Bug 814993).

> sure that the first onPause really triggers a sync, but that we don't get
> too many?

I want to make sure we sync after you're done browsing, and make sure that we sync soon after you start. The rate limit thus serves the purpose of avoiding expensive syncs during browsing, and redundant syncs while your phone is idle.

Android's own rate limiting and CP-watching *should* do this, but we don't fit the mold well enough.

> Do you sync less because you sent a lot of data?  In fact, you could
> probably stand to sync more, because you're likely to not have data next
> time.  Or do you sync less because you're not sending anything?  When you
> actually have data, it's likely to pile up, leading to infrequent big syncs.

And arguably: if you successfully synced a bunch of stuff, we should sync more frequently -- you're likely on cheap, fast, power-friendly wifi, and if there are more records to get we should get them now.

	BMO Automation
	Comment 10 • 4 years ago

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.