Follow-up of Bug 809772. Each time we try to install a packaged app while running OOP we get the 'NS_ERROR_FILE_ACCESS_DENIED' error when the homescreen app tries to get the application icon from the packaged downloaded to '/data/local/webapps/{appId}'. During the installation, the application folder and assets are created with 600 permissions (user and group root:root), so I first thought that this was the reason of the failure since the homescreen is running as the 'app_0' user, but changing the permissions to 644 doesn't help. I am attaching a log with the 'adb logcat' output after changing the permissions.

Actually, it was a permissions issue! \o/ Setting the app directory to 644 wasn't enough since the execute bit is needed to traverse a directory. Changing the permissions of the app dir to 755 fixes the issue.

No dice - this does not work on today's build. I tried installing the packaged app here - http://everlong.org/mozilla/packaged/. First time twice in a row on two cleanly flashed builds - prompt appears to install the packaged app with the right app details, however, when I hit install, I go back to the homescreen with the generic no app icon shown, but no download progress seen, and no app name seen. Any ideas? I'll try a couple of other packaged app examples to see the behavioral differences.

Also tried Fabrice's app here - http://people.mozilla.com/~fdesre/openwebapps/test.html with "install packaged app" - In this example after I attempted to install the packaged app from comment 4, The name appears, the generic missing app icon appears, downloading starts, but it spins infinitely. Logcat that looks useful to look at from installing fabrice's app: 11-21 22:59:07.600: E/GeckoConsole(561): [JavaScript Error: "TypeError: app.manifest is null" {file: "http://people.mozilla.com/~fdesre/openwebapps/test.html" line: 54}] 11-21 22:59:07.640: E/GeckoConsole(109): [JavaScript Error: "TypeError: app.manifest is null" {file: "http://people.mozilla.com/~fdesre/openwebapps/test.html" line: 54}] 11-21 22:59:07.710: E/GeckoConsole(109): [JavaScript Error: "[Exception... "'TypeError: this is undefined' when calling method: [nsIRequestObserver::onStopRequest]" nsresult: "0x8057001c (NS_ERROR_XPC_JS_THREW_JS_OBJECT)" location: "native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0" data: no]"] For the other app: 11-21 22:52:18.060: I/GeckoDump(109): XXX FIXME : Got a mozContentEvent: webapps-install-denied

For everlong packaged app install, more relevant logcat: 11-21 23:37:25.121: E/GeckoConsole(110): Content JS LOG at app://browser.gaiamobile.org/js/places.js:134 in anonymous: error fetching icon: 404 11-21 23:37:28.644: E/GeckoConsole(434): [JavaScript Error: "NS_ERROR_FILE_ACCESS_DENIED: File error: Access denied" {file: "app://homescreen.gaiamobile.org/js/page.js" line: 145}] 11-21 23:37:28.644: E/GeckoConsole(434): [JavaScript Error: "NS_ERROR_FILE_ACCESS_DENIED: File error: Access denied" {file: "app://homescreen.gaiamobile.org/js/page.js" line: 145}]

(In reply to Jason Smith [:jsmith] from comment #4) > No dice - this does not work on today's build. > > I tried installing the packaged app here - > http://everlong.org/mozilla/packaged/. First time twice in a row on two > cleanly flashed builds - prompt appears to install the packaged app with the > right app details, however, when I hit install, I go back to the homescreen > with the generic no app icon shown, but no download progress seen, and no > app name seen. > > Any ideas? I'll try a couple of other packaged app examples to see the > behavioral differences. Thanks Jason. It seems that the issue only happens in production builds :(. The content process is not able to properly set the permissions of the webapp directory: root@android:/data/local/webapps # ls -la drwxr-x--- root root 2012-11-22 11:27 maps drwxr-x--- root root 2012-11-22 11:27 marketplace drwxr-x--- root root 2012-11-22 11:27 marketplace-dev drwxr-x--- root root 2012-11-22 11:27 marketplace-staging -rw-r----- root root 13691 2012-11-22 11:31 webapps.json drwxr-x--- root root 2012-11-22 11:29 {42e8ed2f-30de-4b5e-86e5-a4c1d9a6cc7c} drwxr-x--- root root 2012-11-22 11:31 {6ff81091-18f2-49f9-8f1d-cde7e98c0194} As you can see the permissions are set to 750 while in a development build the permissions are set to 755: root@android:/data/local/webapps # ls -la [...] drwxrwx--x shell shell 2012-11-22 07:26 twittershare.gaiamobile.org drwxrwx--x shell shell 2012-11-22 07:26 uitest.gaiamobile.org drwxrwx--x shell shell 2012-11-22 07:10 video.gaiamobile.org drwxrwx--x shell shell 2012-11-22 07:10 wallpaper.gaiamobile.org -rw-r----- root root 18267 2012-11-22 11:41 webapps.json drwxr-xr-x root root 2012-11-22 11:41 {74a8f751-adbb-463a-9758-9285a1e54e21} I am not familiar enough with the differences between the development and production builds, so any help would be much appreciated here. (In reply to Jason Smith [:jsmith] from comment #5) > Also tried Fabrice's app here - > http://people.mozilla.com/~fdesre/openwebapps/test.html with "install > packaged app" Fabrice's app is intentionally broken to test errors. If you take a look at the manifest [1] you can see that the 'packaged_path' value points to a wrong URL [2]. If you want to try with a different packaged app, you can use 'Packaged app' and 'Big packaged app' from [3]. [1] http://people.mozilla.com/~fdesre/openwebapps/package.manifest [2] http://people.mozilla.com/~fdesre/openwebapps/fbowd.zip2 [3] http://owapps.cloudfoundry.com/

(In reply to Fernando Jiménez Moreno [:ferjm] (PTO until 26th November) from comment #7) > > Thanks Jason. It seems that the issue only happens in production builds :(. > The content process is not able to properly set the permissions of the > webapp directory: s/content/parent/

I've just briefly talked with Chris Jones via IRC and he doesn't like the idea of having the app directories with 755 permissions, so we might need to change that and leave the app dirs as 750. Note that webapps at '/system/b2g/webapps' in a production build are also stored with 755 permissions. Since the child processes won't have access to app directories with 750 permissions, I guess that we would need to move the JAR channel creation in the AppProtocolHandler to the parent. Any thoughts?

Content processes can't touch that part of the file system. The master process needs to do whatever is being done here. /system/b2g/webapps is on a read-only file system ...

(In reply to Chris Jones [:cjones] [:warhammer] from comment #11) > Content processes can't touch that part of the file system. What do you mean by "that part of the file system"? > The master > process needs to do whatever is being done here. > > /system/b2g/webapps is on a read-only file system ... I am afraid that I might have not explained myself properly. Let me summarize the current scenario and the issues that we've found so far: A. On one side, we have the parent process creating the webapps directories during the *installation* of web applications via the webapps API [1]. In both kind of builds (production and development) this creation happens in '/data/local/webapps'. *Core* applications are stored at build time with 755 permissions in '/data/local/webapps' in development builds or also with 755 permissions in '/system/b2g/webapps' in production builds. A.1. *Without* the attached patch applied, the webapps dirs (/data/local/webapps/{appId}/) are created with 700 permissions in both builds. The webapps files are stored with 600 permissions in both builds. A.2. *With* the attached patch applied (landed in m-c), the webapps dirs are created with 755 permissions in a development build and with 750 permissions in a production build. The webapps files are stored with 644 permissions in a development build and with 640 permissions in a production build. B. On the other side, we have child processes trying to access (read) the assets stored in the webapps directories. For example, for the case of a packaged app installation, the homescreen app is trying to access the recently created app folder (remember, located in '/data/local/webapps/{appId}' in both kind of builds) to get the application icon. Since the homescreen app is running in a process owned by the user 'app_0' and the owner of the webapp directory is root:root, it won't be able to traverse the directory (originally 700, see A.1) unless we change its permissions to 755 (A.2). Also, it won't be able to read the application.zip that contains the app icon (originally 600, see A.1) unless we change the permissions of this file to 644 (A.2). As per the above, my questions are: - Chris, do you consider insecure to change the permissions of the webapp directory that is being installed (/data/local/webapps/{appId}) from 700 to 755 and its application.zip (/data/local/webapps/{appId}/application.zip) from 600 to 644? Note that the access to this directories is via the app:// protocol. - In case that you are ok with the permissions change. Do you know about any reason that avoids us to change the permissions in a production build just like we do in a development build? - In case that you are not ok with the permissions change, if I am not wrong, a possible solution would be to change the AppProtocolHandler [2] to open the JAR channel from the parent process. Chris, Fabrice: are you ok with this change? Can you think about an alternative? Sorry for the long comment and thanks in advance. [1] http://mxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#1106 [2] http://mxr.mozilla.org/mozilla-central/source/netwerk/protocol/app/AppProtocolHandler.js#51

I think I might have found out the reason of why this is failing in a production build... The patch is not applied!! Looking at the manifest used to generate the production builds [1] it seems that production builds are based in a 4eaacd3fad7 Gecko revision which doesn't have the patch to change the permissions applied [2]. I have just started a production build with the patch applied to check that it works. I'll let you know as soon as I can test it. Even if it works, I'll leave the bug opened, so Chris can confirm if he is ok with the proposed solution. Sorry for all the noise :| [1] http://stage.mozilla.org/pub/mozilla.org/b2g/manifests/2012-11-26/source_otoro_2012-11-26_eu.xml [2] http://git.mozilla.org/?p=releases/gecko.git;a=blob;f=dom/apps/src/Webapps.jsm;h=4ad9ea74cf2d2c2ebe14b98dbac28a5c7ce96432;hb=4eaacd3fad7b6da220de206c0013dc0f307455c9#l1106

(In reply to Fernando Jiménez Moreno [:ferjm] from comment #12) > (In reply to Chris Jones [:cjones] [:warhammer] from comment #11) > > Content processes can't touch that part of the file system. > > What do you mean by "that part of the file system"? > The /data partition. > > The master > > process needs to do whatever is being done here. > > > > /system/b2g/webapps is on a read-only file system ... > > I am afraid that I might have not explained myself properly. Let me > summarize the current scenario and the issues that we've found so far: > > A. On one side, we have the parent process creating the webapps directories > during the *installation* of web applications via the webapps API [1]. In > both kind of builds (production and development) this creation happens in > '/data/local/webapps'. *Core* applications are stored at build time with 755 > permissions in '/data/local/webapps' That's a bug, yuck :(. > B. On the other side, we have child processes trying to access (read) the > assets stored in the webapps directories. For example, for the case of a > packaged app installation, the homescreen app is trying to access the > recently created app folder (remember, located in > '/data/local/webapps/{appId}' in both kind of builds) to get the application > icon. That's not expected. App processes shouldn't be directly reading /data. > - Chris, do you consider insecure to change the permissions of the webapp > directory that is being installed (/data/local/webapps/{appId}) from 700 to > 755 and its application.zip (/data/local/webapps/{appId}/application.zip) > from 600 to 644? Note that the access to this directories is via the app:// > protocol. > Yes, this weakens the security model. > - In case that you are not ok with the permissions change, if I am not > wrong, a possible solution would be to change the AppProtocolHandler [2] to > open the JAR channel from the parent process. Chris, Fabrice: are you ok > with this change? Can you think about an alternative? This is what I thought was supposed to have been happening all along.

Answering to various points from previous comments here: *Core* applications in VARIANT=user builds are stored in /system/b2g/webapps and are not copied out of this readonly partition. I think we're safe there. Other installed and updatable apps are installed in /data/local/webapps. These are the one we currently fail to read back from the content process. The app: protocol, like the jar: one is not remoted, so the content process needs to have read access to /data/local/webapps/$APP_ID/application.zip currently. I understand this is not ideal, and if someone has cycles to remote this protocol, that would be great. I'm not sure what made this regress since we had successful installation and running of packaged apps before. I don't care too much about the situation with VARIANT=eng where the core apps are in /data/local since it's a just a developer convenience.

After flashing, we have the following permissions: drwxrwx--x system system /data drwxr-x--x root root /data/local drwxr-x--- root root /data/local/webapps That looks right. I filed bug 815523 to remote the app: protocol instead of hacking the permissions.

Setting priority based on triage discussions. Feel free to decrease priority if you disagree.

(In reply to Andrew Overholt [:overholt] from comment #17) > Setting priority based on triage discussions. Feel free to decrease > priority if you disagree. agree this is P1. we need this work landed so we can pass B2G Smoketests (installing packaged apps)

(In reply to Tony Chung [:tchung] from comment #18) > (In reply to Andrew Overholt [:overholt] from comment #17) > > Setting priority based on triage discussions. Feel free to decrease > > priority if you disagree. > > agree this is P1. we need this work landed so we can pass B2G Smoketests > (installing packaged apps) Can we do something ASAP, before we resolve bug 815523, to get install working, even if it means temporarily regressing in some aspect of security, so that we can do the functional testing and so we can get the signature checking stuff tested and landed. If there is a patch we can backout to get things working, let's back that out and then re-land it once we've resolved bug 815523.

(In reply to Brian Smith (:bsmith) from comment #19) > (In reply to Tony Chung [:tchung] from comment #18) > > (In reply to Andrew Overholt [:overholt] from comment #17) > > > Setting priority based on triage discussions. Feel free to decrease > > > priority if you disagree. > > > > agree this is P1. we need this work landed so we can pass B2G Smoketests > > (installing packaged apps) > > Can we do something ASAP, before we resolve bug 815523, to get install > working, even if it means temporarily regressing in some aspect of security, > so that we can do the functional testing and so we can get the signature > checking stuff tested and landed. If there is a patch we can backout to get > things working, let's back that out and then re-land it once we've resolved > bug 815523. Could you disable out of process to test installation?

I agree with Brian, getting the signing stuff landed so we can test is the biggest security priority ATM.

(In reply to Jason Smith [:jsmith] from comment #20) > Could you disable out of process to test installation? That's too big of a change. People are really eager to be testing this stuff on the device, and we don't want to disable OOP on the device (if it is even possible).

(In reply to Brian Smith (:bsmith) from comment #22) > (In reply to Jason Smith [:jsmith] from comment #20) > > Could you disable out of process to test installation? > > That's too big of a change. People are really eager to be testing this stuff > on the device, and we don't want to disable OOP on the device (if it is even > possible). What I was meaning is that you could disable OOP for now to be able to test installation of a packaged app, so that you are not blocked from being able to test a packaged app install (even though that's not the true flow).

Yeah, this is a very bad regression that while it doesn't block most users, it's blocking a lot of development both for signed apps and for pre-installed 3rd party apps. What bug fixed things so that child processes couldn't read the app package files? Can we back it out while we're working on the remoting in bug 815523? Does anyone know the regression range? Alternatively, can we change the permissions on the files and directories that app packages live in such that the child processes can access them again?

Wait a second, we need to back up. > Each time we try to install a packaged app while running OOP we get the > 'NS_ERROR_FILE_ACCESS_DENIED' error when the homescreen app tries to get the application icon > from the packaged downloaded to '/data/local/webapps/{appId}'. This means that the application was *already* installed, right? How could a content process write an app package to /data/local/webapps if it can't read it? So the real issue here is that the homescreen can't read app://foo.app.on.data.partition/some/resource, right? Updating title. So there are two issues - if an app is installed to /data/local/webapps, which works fine, the homescreen can't read its icon - if we were to try to launch an app installed to /data/local/webapps, it would not be able to open its own application package The "real" solution for this is - applications with webapps-manage (or whatever the permission) can get an fd for any application package in /data/local/webapps - applications can always get an fd for their own app package (duh!) The only change that might possibly have regressed this is umask'ing the b2g process correctly. Because that change is a core, required part of the security model, I'm *extremely* loath to back that out, even temporarily. That change also landed months ago. (In reply to Jonas Sicking (:sicking) from comment #24) > Alternatively, can we change the permissions on the files and directories > that app packages live in such that the child processes can access them > again? This is a fundamental change to the security model, not something to be bandied about lightly. It will give the false impression that the feature is done when it's actually just broken in another blocking way.

(In reply to Chris Jones [:cjones] [:warhammer] from comment #25) > Wait a second, we need to back up. > > > Each time we try to install a packaged app while running OOP we get the > > 'NS_ERROR_FILE_ACCESS_DENIED' error when the homescreen app tries to get the application icon > > from the packaged downloaded to '/data/local/webapps/{appId}'. > > This means that the application was *already* installed, right? How could a > content process write an app package to /data/local/webapps if it can't read > it? The installation is done by the parent process, not the content one.

(In reply to Chris Jones [:cjones] [:warhammer] from comment #25) > (In reply to Jonas Sicking (:sicking) from comment #24) > > Alternatively, can we change the permissions on the files and directories > > that app packages live in such that the child processes can access them > > again? > > This is a fundamental change to the security model, not something to be > bandied about lightly. I'm definitely not taking it lightly. However the current state of affairs is that both the signing work and the preinstalled 3rd part app work is unable to move forward with things as they are now. We need to unblock development and I don't believe that we can get a fix for bug 815523. The remoting work there is likely going to take the better part of next week. This is a smoketest blocker and I don't believe that we can leave it unfixed for that long. > It will give the false impression that the feature > is done when it's actually just broken in another blocking way. I agree this is a problem. But it's a smaller problem than losing developer productivity. Especially when app signing is already very late.

Also, this didn't regress a couple of months ago. According to smoketest logs this regressed between the 11-28 and 11-29 builds. I have no idea what we changed in that timeframe :(

(In reply to Jonas Sicking (:sicking) from comment #28) > Also, this didn't regress a couple of months ago. According to smoketest > logs this regressed between the 11-28 and 11-29 builds. I have no idea what > we changed in that timeframe :( If a regression range would be helpful, then I can look into this. We recently just added packaged apps to our smoke test list, so I don't think that date range is likely when we actually regressed this functionality.

That range is what the sheet at [1] is indicating. But I guess that could be wrong if we just added the test. [1] https://docs.google.com/a/sicking.cc/spreadsheet/ccc?key=0AqDJxnTd7VDLdDRKOE1uanE3M3FtMDBOVjRIV253S1E#gid=41

(In reply to Jonas Sicking (:sicking) from comment #30) > That range is what the sheet at [1] is indicating. But I guess that could be > wrong if we just added the test. > > [1] > https://docs.google.com/a/sicking.cc/spreadsheet/ > ccc?key=0AqDJxnTd7VDLdDRKOE1uanE3M3FtMDBOVjRIV253S1E#gid=41 I believe it is. There was some confusion early on about whether that test was passing or not (it's flagged incorrectly on some days). I need to get that process problem fixed. I'll try to get a regression range soon to help narrow down the breaking patch.

(In reply to Fabrice Desré [:fabrice] from comment #26) > (In reply to Chris Jones [:cjones] [:warhammer] from comment #25) > > Wait a second, we need to back up. > > > > > Each time we try to install a packaged app while running OOP we get the > > > 'NS_ERROR_FILE_ACCESS_DENIED' error when the homescreen app tries to get the application icon > > > from the packaged downloaded to '/data/local/webapps/{appId}'. > > > > This means that the application was *already* installed, right? How could a > > content process write an app package to /data/local/webapps if it can't read > > it? > > The installation is done by the parent process, not the content one. Yep, that's why I changed the bug title :). This was a rhetorical question.

(In reply to Jonas Sicking (:sicking) from comment #27) > (In reply to Chris Jones [:cjones] [:warhammer] from comment #25) > > (In reply to Jonas Sicking (:sicking) from comment #24) > > > Alternatively, can we change the permissions on the files and directories > > > that app packages live in such that the child processes can access them > > > again? > > > > This is a fundamental change to the security model, not something to be > > bandied about lightly. > > I'm definitely not taking it lightly. However the current state of affairs > is that both the signing work and the preinstalled 3rd part app work is > unable to move forward with things as they are now. > The development of those isn't blocked on this.

This is not the information I'm getting from Brian who is the developer of app signing and Karen who is the person working with partners for 3rd party apps.

As we discussed on IRC, this bug can be worked around in several ways - disable content processes. There's a gaia setting for that, even. "Device Information" -> "More Information" -> "Developer" -> "Disable out-of-process". You need to restart after flipping this setting. (It can also be disabled with a gecko pref but that requires a rebuild.) - manually do |adb shell chmod [whatever] /data/local/webapps| to make things work Neither is great and that's unfortunate, but they're not difficult to pull off. We discussed a deadline for 3rd party apps on IRC that I'd like to understand better.

FYI the work on preloaded 3rd party apps is tracked in bug 812198

This patch changes the permissions from within gecko, and could be reverted easily when landing bug 815523. That would be easier for QA than other methods.

(In reply to Jonas Sicking (:sicking) from comment #27) > We need to unblock development and I don't believe that we can get a fix for > bug 815523. The remoting work there is likely going to take the better part > of next week. Development should not be blocked on this. Apart from the workarounds mentioned in comment #35, if I am not wrong, the attached patch is landed in m-c [1] and working for development builds (VARIANT=eng). [1] https://hg.mozilla.org/mozilla-central/rev/e977d28f1243

Mass Modify: All un-milestoned, unresolved blocking-basecamp+ bugs are being moved into the C3 milestone. Note that the target milestone does not mean that these bugs can't be resolved prior to 12/10, rather C2 bugs should be prioritized ahead of C3 bugs.

Hey, we need to land a workaround patch here. More and more people are running into this, most recently a partner got the impression that all packaged apps had to be signed because they couldn't get their packed apps working. Telling people to disable out-of-process seems very scary since it means testing against a dramatically different runtime. This could easily result in missing bugs in our OOP code for the various features. I'll ensure that bug 815523 remains a blocker so that we can fix the security model right and back out whatever workaround we land here.

(In reply to Jonas Sicking (:sicking) from comment #40) > Hey, we need to land a workaround patch here. More and more people are > running into this, most recently a partner got the impression that all > packaged apps had to be signed because they couldn't get their packed apps > working. I submitted a workaround patch. I use it to work on 3rd party preinstalled packaged apps and this works fine. Feel free to review...

Comment on attachment 687621 [details] [diff] [review] followup patch Review of attachment 687621 [details] [diff] [review]: ----------------------------------------------------------------- Let's land this if nobody disagrees. r=me ::: dom/apps/src/Webapps.jsm @@ +301,5 @@ > this.installSystemApps(onAppsLoaded); > else > onAppsLoaded(); > + > + // XXX: To be removed as soon as the app:// protocol is remoted. Add Bug 819061 here, please.

Thanks Fabrice and Fernando!

https://hg.mozilla.org/releases/mozilla-aurora/rev/56af6830834a https://hg.mozilla.org/releases/mozilla-aurora/rev/9cda25b6c0a7 https://hg.mozilla.org/releases/mozilla-beta/rev/36e43acd12cb https://hg.mozilla.org/releases/mozilla-beta/rev/b5e7b7833da8

Verified a few days ago that I'm able to install packaged apps again.