Per talking with Jonas, Chris Jones, and Lisa, we need to enforce a max size for packaged apps for a couple reasons: 1. Prevent a DOS attack with uploading a 100 GB packaged app 2. 30MB / month plans in Brazil are much more common 3. The very nature of the limitation of how much space is allowed on a phone 100 MB appears to be initial proposal, but let's wait for final confirmation on this.

500 MB has been decided as the max packaged app size

lol 500 MB is huge. I'd be surprised if anyone can even successfully upload that. it took me over 30 minutes and I got to 19% on a 100 MB upload. To generate a junk packaged app: dd if=/dev/zero of=junk.zip bs=1024 count=102400

500mb is way too big. Besides the fact that you'll blow out those 30mb/month plans by a factor of almost 20 (if you do the math, it'll take over a year to download an app that big...less than one KB/minute), this is going to cause enormous internal problems. Packing and unpacking a zip that big will be entirely too resource intensive. If a file that big gets pushed to the storage API, that's an automatic $0.06 for every time we read that file back out in transfer costs alone (app gets downloaded 100k times? boom $6000.00), not to mention the fact that that the time to transfer that is going to be ridiculous. Packaged apps themselves should not contain any very large resources. If they need resources larger than a megabyte or two, they should be fetched by the app on startup form a remote server and stored locally. What's in the package should be almost exclusively JS, CSS, and HTML. Which meeting can I attend to petition this decision?

Packaged apps could also contain images and fonts (referenced in CSS). But yeah, I agree, a few megabytes sounds about right.

(In reply to Jason Smith [:jsmith] from comment #1) > 500 MB has been decided as the max packaged app size I'd also like to hear more about how this was decided

(In reply to Wil Clouser [:clouserw] from comment #5) > (In reply to Jason Smith [:jsmith] from comment #1) > > 500 MB has been decided as the max packaged app size > > I'd also like to hear more about how this was decided Chris - Can you provide input here? Sounds like 500 MB might be too high. It was just off of a random email thread (excerpt below): For the general marketplace, across desktop and mobile, that's probably too low. There are AAA game companies who say they want to ship hundreds of MB of assets. I'd hate for a too-low marketplace quota stifle that sector. A developer *account* quota might be best of all though. Set the initial quota at 500MB for all apps. If they try to go over, an alarm goes off and negotiations ensue. A big game company could get all the space it wanted. (A DoS launched by 10000 uploads of 10MB apps would be just as bad as 100GB of one.)

It should also be noted that, as andym mentioned on IRC, the game companies are by no means forced to create packaged apps. Games in particular probably don't need all of the features that packaged apps afford (many games don't require camera access, for instance). >For the general marketplace, across desktop and mobile, that's probably too low. There are AAA game companies who say they want to ship hundreds of MB of assets. I'd hate for a too-low marketplace quota stifle that sector. Packaged apps can take the same approach to asset distribution as hosted apps do: only scripts, css, HTML, and potentially fonts and some images need to be stored in the package. The package itself isn't a convenience afforded to the developers, it's a mechanism for us to ensure that the content in the app does what it says it does. It's also a better user experience to make assets remote: if I understand correctly, packaged apps load on a single HTTP connection. Making the large assets remote and pulling them down from within the app itself allows the app to increase the number of connections and pull down multiple assets simultaneously. Encouraging developers to make their assets remote is probably the best way to ensure a good user experience. It also avoids all of the internal issues that we'd have to deal with and places the cost of distributing a crap ton of files on the developer instead of the marketplace. If they've got a 2gb asset bundle that they need pushed around, there's no real barrier to them (right now) distributing that from their own servers.

(In reply to Jason Smith [:jsmith] from comment #6) > For the general marketplace, across desktop and mobile, that's probably too > low. There are AAA game companies who say they want to ship hundreds of MB > of assets. I'd hate for a too-low marketplace quota stifle that sector. Many mobile games today will do a small (5-10MB) install from the app store, and then prompt the user to do a bigger download of the assets on first run. This seems like what we'd want to encourage. Having to rev a 500MB package to change a 1kb script to fix a bug seems like failure.

I don't think that we should say "downloading resources will give a better user experience, setting a low limit is ok". Whether it's better for users if the resources are downloaded dynamically or not will likely vary greatly from app to app. Users generally expect an app to work once installed. A user that installs an app, gets on an airplane and then realizes that the app doesn't work at all because the first thing it needs to do is to download a bunch of resources, is likely going to be very surprised and disappointed. Likewise, if an app runs very slowly at first because its blocked in a bunch of places on resources that are still being download in the background is going to think that the app, or the platform, is very slow. In other instances it's definitely true that it's better if the app can download resources dynamically after initial install. For example if the amount of resources needed is simply larger than would fit on the device the app could allow the user to pick and choose which parts of the app to make available offline. Think a navigation app for example. That said, we could simply have a limit of 100MB, but not make it a hard limit but instead ask the developer what the reason is for going over that limit. All depends on what the marketplace feel that they can build in time.

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #8) > (In reply to Jason Smith [:jsmith] from comment #6) > > For the general marketplace, across desktop and mobile, that's probably too > > low. There are AAA game companies who say they want to ship hundreds of MB > > of assets. I'd hate for a too-low marketplace quota stifle that sector. > > Many mobile games today will do a small (5-10MB) install from the app store, > and then prompt the user to do a bigger download of the assets on first run. > This seems like what we'd want to encourage. I don't know that we want to encourage it. But we should definitely enable it. > Having to rev a 500MB package > to change a 1kb script to fix a bug seems like failure. We should definitely enable downloading "diffs" for updates. This is something we punted on for the initial release, but the long-term plan is definitely to make this possible. The nice thing about zip compressing files separately is that doing a binary diff should enable most unchanged files to not appear in the diff at all.

So I've been reading this bug and thinking about if there is a hard fast rule. My current answer has to be, I think both packaged apps and downloaded assets will happen. In the early days, it's going to be very tough to get people into the store, the minimum amount they need to change their code the better. Just did a quick survey of the iOS store to get a range of sizes. I looked for the extreem largest first and then went to a more common level. Infinity Blade for example comes in at 550+ MB in a single package so clearly, on iOS, this is acceptable. Infinity Blade II clocks in at 1.03 GB. These are extremely high end though so it's unlikely that we will see something like this for some time but still something to think about. They download as a single package. 2D games that we are likely to see such as Angry Birds or Field Runner are all under 20-200MB based on a quick survey. For the initial release, I would expect this to be the norm. I should also mention the users like to download an app and have it run when they are not online in a one click process. Having to open up the app to get the rest of the content is a use case that often causes user irritation if they don't always have access to the net. They open up a game they just installed and on the subway find out it's not playable. Several game developers have commented that they prefer an all up front approach in past conversations. I can follow up and get direct feedback if you guys want more data.

I think we're looking at this from the wrong perspective. It's certainly out of the question to ban "large packages" in general, as they do serve a purpose. I think this bug would be better phrased as "we should set a cap on app size for self-serve app submission". To avoid the issue entirely: if, in the future, it becomes an issue where developers require exceptionally large packages, we can set up mechanisms where Mozilla signs the packages through less-automated means and then in the Marketplace, we redirect the user to the developer's own servers who can serve the file however they like (e.g.: with a CDN or the like). The package doesn't necessarily need to be hosted by us, so long as it has our signature on it. The fact of the matter is that at the moment, we're absurdly underprepared to handle files even remotely close to 500mb. I think having a separate "large package" submission flow for folks that are putting out really huge packaged apps is a decent compromise that wouldn't require a terrible amount of extra work on the marketplace side or the reviewer side and wouldn't cause any undue burden on the developer. Wil/Jonas/cvan: What are your feelings about something like this? We'll probably need fligtar to spend some time thinking about how this would work with refunds and robhudson to think about how to make it actually happen. CCing adora because she's the Queen of Apps.

That sounds good to me. (Another argument for allowing large packages is that it makes it easier for the user to manage when the download happens. I.e. in countries like Brazil where people are often on limited data connections the user can choose not to download a 500MB packaged app. But he'll have a much harder time controlling this if the package is just 2MB but the game starts downloading 498MB of data as soon as it starts).

I'm all for large packages at some point, but would rather not have this distract us from basecamp. I would also recommend setting a really low cap, for example 10 or 20mb for now and then expanding it as we see fit and the marketplace is able. I'd like to see that cap increase and we can file bugs on that to see it increase, but that will require us to make changes to the marketplace infrastructure and we are still in the process of figuring out getting a version 1 of packaged apps working.

That's fair. I'd say lets make the limit clearly larger than what the bulk of apps need, but smaller tharln that it will cause problems for marketplace. How about making the limit 50MB?

Let's do 50MB. I talked to Jeremy and Basta and it sounds like that's a good start.

Awaiting review: https://github.com/mattbasta/zamboni/commit/c8077195759b668da81514fb39264e7ca7ddfe22

Done: https://github.com/mozilla/zamboni/commit/057ac978efd152320b4e4260a45423e6c3df1e4c