Closed Bug 1549259 Opened 6 years ago Closed 3 years ago

Download part files should be stored in some user-level folder (e.g. "local" appdata or similar folder) to avoid permission/fscrypt issues

Categories

(Firefox :: Downloads Panel, defect, P3)

67 Branch
defect

Tracking

()

RESOLVED FIXED
97 Branch
Tracking Status
firefox97 --- fixed

People

(Reporter: info, Unassigned)

References

Details

Attachments

(2 files)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0

Steps to reproduce:

Set Up New ubuntu 18.04 LTS on Dell XPS 13 Laptop. Installed Firefox. Everything works fine for the first user (user id 1001) which was created in ubuntu upon installation. Specifically, when downloading files, pictures, videos everything works as expected.

Actual results:

Once we created second user on the system (user id 1000), and started firefox then problems with downloads started. They all fail.

Firefox displays everything well inside the browser, but as soon as we attempt to "save as" to download pictures, zip, documents, etc ... an animation indicates that download starts but then fails immediately. Retry button will initiate download sometimes but many retries are needed to complete a simple 2 MB download. The problem first noticed on 66.0.3 as that was the version that we have installed from standard repo when we set up ubuntu 18.04, then we changed to Firefox dev version in hope it would fix it no joy here. Wiping profile and reinstalling firefox did not help. Other browsers chrome, chromium work fine.

Expected results:

We do expect that firefox will download files / videos with no issues for multi user ubuntu systems like it does for the (first user with id 1001).

A similar error was reported three years ago and it seemed that /tmp/mozilla_userX directory permission where at fault. Now we do not see that as the part files are saved in /temp/mozilla_userX directories. And permission are exactly the same owner:group for the user in question.

I have also created a help thread https://support.mozilla.org/en-US/questions/1258397 but I think this is a bug / regression on multi user ubuntu systems thus the forum can not help.

Attached image photo_2019-05-05_21-51-42.jpg (deleted) —

Checking to see to see if your using the distribution's Firefox or one of our builds. Is this what you see when you check About Firefox? -- the "Mozilla Firefox for Ubuntu cannonial 1.0"

Flags: needinfo?(info)
Attached image Firefox_About.png (deleted) —
Flags: needinfo?(info)

Yes, Firefox we use for version 66.0.3 is exactly the same "Mozilla Firefox for Ubuntu canonical -1.0" and comes from official ubuntu repository. After that we added mozilla repos to our sources:

sudo add-apt-repository ppa:ubuntu-mozilla-daily/firefox-aurora
sudo add-apt-repository ppa:mozillateam/firefox-next

and downloaded the most recent version 67.0b16 in the hope it would not have that problem. But unfortunately the "all download fail" problem for the second ubuntu system user did not go away.

Additional Info. Multiple downloads at the same time are not possible. Only one file can be downloaded at the same time. Pressing retry multiple times does initiate download, but not necessarily finishes it (will very likely fail before finish). While the first download is going no other downloads can be started / retried / etc ... Hope it is a clue to a knowledgeable developer on what kind of trickery is going on here :-)

Flags: needinfo?(tgrabowski)

I am wondering whether I am the only one using firefox as second system user on ubuntu. But the download to the file system fails, that really hurts. Strangely enough if the file downloaded being directly submitted to a different program then no problem. Like downloading torrent file and handing it over to torrent client. That works fine. I can do everything in firefox, but as soon as I need to download anything to the file system directly, I need to start chrome / chromium to do so.

Please look into this issue.

Flags: needinfo?(ehumphries)
Component: Untriaged → Downloads Panel
Flags: needinfo?(ehumphries)

I think I am getting closer to the problem why ALL downloads fails on firefox (no matter which version) in Ubuntu 18.04

Looks like it has nothing to do with second system user per se, but that this specific system user in question has his /home partition encrypted with fscrypt. Depending on the kernel / fscrypt versions it seems that moving files from not-encrypted directories like /tmp/mozilla_user_0 to encrypted /home/user_0/Downloads is generating permission errors.

I replicated that two times because I set up the laptops in identical manner. :-)

PLEASE NOTE: A file can be deleted in not encrypted directory with no issues, it can also be copied from non encrypted to encrypted folders.

BUT moving it will result in errors. Thus, if Firefox internally is moving the downloaded file from /tmp/mozilla_user0 to the users encrypted directory, it gets an error message from operating system and reports download failed. And of course never deposits the file in user designated downloads directory.

I can verify that firefox does download the files correctly into temp directory. I see all the "reported failed downloads" in the /tmp/mozilla_user0 directory. I can see placeholders (files with correct name but zero size) also being created in the firefox designated final download directory.

When I attempt to move the file from temp directory (not encrypted) to final destination directory (encrypted), then I get error message and can not finish the task.

THUS, if my speculation is correct, then that means firefox is using "mv" command to move downloaded files from tmp directory to final destination directory.

In this case a robust fix would be to make firefox copy the file from tmp directory to final download destination and then just delete the file in temp directory. That will work and has no issues with fscrypt directory encryption at the moment.

Chrome / Chromium does not have this issue as it does not use /tmp directory, but possibly saves temporary files inside encrypted user directory structure. Thus, avoids moving from non encrypted to encrypted directories afterwards.

If Firefox could be changed in this regard to follow chrome / chromium that would also work. And make sure all user relevant files will stay in user home directory, instead of going through global temp directory.

Flags: needinfo?(mak77)
Depends on: 1555644

This is a problem with OS.File, on which the downloads code relies. I filed bug 1555644 to get that addressed.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(mak77)
Priority: -- → P3

I think the suggestion to store things in the user folder, instead that using /tmp could be valid anyway, provided we can identify a good destination that is not immediately visible to the user (we should not store the .part file in the final folder to avoid other problems that may block it), and that doesn't require periodic cleanup tasks from us (maybe Chrome has its own managed tmp folder, or they just do the copy/delete dance like we plan to do, this is to be verified). The additional problem is that the solution should be usable on all the distributions.

No longer depends on: 1555644

Looks like this is basically the same as bug 1521041. Let's use this to consider whether there's a better location we could use for download .part files.

In situations like this, is the entire $HOME encrypted, or only a subset?

Flags: needinfo?(info)
Summary: All downloads fail inside download manager for 2nd system user on ubuntu 18.04 (Firefox 66.03 and 67.0b16) → Download part files should be stored in some user-level folder (e.g. "local" appdata or similar folder) to avoid permission/fscrypt issues

So, I discovered https://bugzilla.mozilla.org/show_bug.cgi?id=69938 that contains a LOOOOOOOOONG discussion about storing files in /tmp and then moving them. It contains various interesting facts:

  1. the home folder may have a quota, for which it may not be the perfect default destination for files. The /tmp folder may have a quota, as well.
  2. the predownload "feature" was not just a feature, but a necessity, as Bug 55690 (containing even more details) outlines, if we'd wait for the user to pick a destination, and wait for the destination to become available, the connection to the download may be dropped. At the time it was apparently not possible to throttle the connection maintaining it open.
  3. Anyway it is necessary to ask for the file before showing the dialog because we need the headers to properly show the dialog
  4. Right click / Save As may be a workaround
  5. changing the system /tmp folder may be a workaround
  6. a lot more technical stuff is in Bug 55690, part of which today may be obsolete considered the many years passed.

Whatever solution we may think of, has likely been already reported there, and there were technical reasons for which it was not possible, most related to network and content encoding. Reopening that discussion would not be useful unless we are sure the technical limitations that existed 18 years ago today have gone. I don't have that knowledge, it's likely at the necko level.

Yes, the whole user home directory is encrypted /home/user0/

@Marco I do not think Chrome is utilizing global /tmp directory on linux. I think temp file is stored in a hidden directory inside users home like /home/user0/.cache/chromium/

Flags: needinfo?(tgrabowski)
Flags: needinfo?(info)

This was fixed as part of all the work in bug 1733587. Downloads no longer use the tmp dir.

Status: NEW → RESOLVED
Closed: 3 years ago
Depends on: 1733587
Resolution: --- → FIXED
Target Milestone: --- → 97 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: