By default geckodriver creates a temporary profile in the system tempdir. This can cause problems with container-based Firefox if the container mounts a different /tmp to the host; in that case the profile can't be read and we end up with a modal dialog displayed and the client will hang until we reach the Firefox start timeout (usually 60s).

Users are able to work around this by providing --profile-root and specifying a directory that's accessible to their Firefox instance.

The problem is that it's hard to know that this is necessary, and in particular users relying on the default Ubuntu-packaged Firefox in CI are reporting updating to 22.04 or later is breaking their test setup, because they're suddenly using a snap-packaged version and don't know to (or can't) provide a profile root directory.

One complication here is that the Ubuntu setup has a Firefox at /usr/bin/firefox that's actually a shell script that does some stuff and then runs /snap/bin/firefox. So it's not trivial to tell that we're going to end up running something under /snap/.

There are several options here:

• We could (on linux) stop using /tmp altogether and use e.g. $XDG_STATE_HOME which is default $HOME/.local/state. However users might be surprised by the location change.
• We could figure out some heuristic to determine if the binary path we have is a snap package or not.
• We could run a Firefox process (e.g. with --version) and use that to determine whether we're using a snap package. I don't know if it's possible to do this directly or if we'd need to add code directly to Firefox to support this use case (e.g. a new command line argument that dumps some relevant data).

So I tried just always using $XDG_STATE_HOME on linux, defaulting to $HOME/.local/state and it still didn't work in a way that suggests the snap can't access hidden directories. $HOME/local/state/geckodriver does work. That's unexpected because I thought the personal-files connection provided access to hidden directories, and snap connections firefox suggests that Firefox does have this.

Sorry, I don't work on snap.

Pascal: do you know who to talk to about the snap packaged Firefox?

Looping in Amin from Canonical and Johan from our releng team

Deferring to :gerard-majax who knows the internals of our Snap better than I do.

Sorry but this is lacking so much context, I can't get a grip on the problem, why we are looking into that, what problem we are trying to solve here.

What are you trying to do ?

geckodriver is a piece of software that acts as middleware for remote automation of Firefox, particuarly used for automated testing. It implements a standard HTTP protocol, and manages a Firefox instance which is being automated. That involves creating a correctly configured profile, according to the automation requirements, and then launching Firefox using that profile.

By default geckodriver will use whatever firefox instance it can find on the system (i.e. you don't need to explicitly pass in a path to a binary). For a typical linux-based CI system, that's the system one that's in /usr/bin. So before launching that Firefox instance geckodriver writes the automation-configured profile somewhere (usually in the system tempdir) and passes that in to the Firefox binary using the --profile command line flag.

That has tranditionally worked fine. But with snap-packaged Firefox it doesn't, as set out in Comment #0. The snap is unable to read the profile created by geckodriver, which itself does not run in the same snap, and so doesn't share the same system temporary directory.

This is becoming a problem because people using Ubuntu docker images are upgrading from e.g. 20.04 to 22.04 and finding that their CI jobs hang (because Firefox is displaying a modal explaining that it can't read the profile). The likely impact of this is that people updating their CI believe that Firefox is broken and consider dropping it from their CI, thus decreasing compatibility. There is a workaround; when starting geckodriver you can specify a location for the temporary profile to be written to, and so you can pick one that's visible both to the host filesystem and inside the snap container. But understanding that's the solution given the symptoms is hard, and in many cases people are using various layers of wrappers that make passing command line arguments to the binaries involved challenging.

So the question is: what can we do to ensure that users' Firefox-based automations continue to work on systems with a snap-packaged Firefox (I don't know if the same problem applies with flatpak and other similar sandboxed packages, but possibly. However the popularity of Ubuntu in CI setups makes this specific variant a important short term issue).

So the problem is just:
(1) geckodriver as a snap
(2) firefox as a snap

Need to create a profile directory from (1) that is readable by (2) ?

Geckodriver is (typically) not a snap. But yes, we need to create a profile from outside the snap-packaged Firefox that could be read inside it. And ideally it would be in a location that's widely understood to be for the purpose of temporary data, rather than e.g. directly in the users home directory.

I am confused, you mentionned geckodriver running as snap here: The snap is unable to read the profile created by geckodriver, which itself does not run in the same snap

What is the purpose of geckodriver section here then? https://github.com/canonical/firefox-snap/blob/92c47bd2556a89cc87b920d4efc85b71740ef207/snapcraft.yaml#L48-L77

By "it doesn't run in the same snap" I just meant that it's not in the Firefox one; it may also not be in one at all.

What is the purpose of geckodriver section here then? https://github.com/canonical/firefox-snap/blob/92c47bd2556a89cc87b920d4efc85b71740ef207/snapcraft.yaml#L48-L77

I had no idea that existed. So it seems like if I run /snap/bin/geckodriver I end up with something that can start the snap packaged Firefox successfully. That's good to know; we can document it for users. But may users are running setups that automatically download the latest geckodriver binary rather than using a system provided one. So it would still be good if we had a more general solution.

(In reply to James Graham [:jgraham] from comment #13)

By "it doesn't run in the same snap" I just meant that it's not in the Firefox one; it may also not be in one at all.

Sorry but the way it was articulated made it sounds like there is a geckodriver snap, and given the linked section, I was wondering.

What is the purpose of geckodriver section here then? https://github.com/canonical/firefox-snap/blob/92c47bd2556a89cc87b920d4efc85b71740ef207/snapcraft.yaml#L48-L77

I had no idea that existed. So it seems like if I run /snap/bin/geckodriver I end up with something that can start the snap packaged Firefox successfully. That's good to know; we can document it for users. But may users are running setups that automatically download the latest geckodriver binary rather than using a system provided one. So it would still be good if we had a more general solution.

This is really getting out of the scope of intersection of my knowledge and how much time I can put on it right now.

Amin will correct me, but I suspect the correct way might be(In reply to James Graham [:jgraham] from comment #10)

Geckodriver is (typically) not a snap. But yes, we need to create a profile from outside the snap-packaged Firefox that could be read inside it. And ideally it would be in a location that's widely understood to be for the purpose of temporary data, rather than e.g. directly in the users home directory.

Each snap has a temp directory, can't you just use that?

Can you write to it from outside the snap? If so, maybe, if we can work out the path we need to write to.

To be clear out situation is the following:

• geckodriver may or may not itself be in a snap (either the same one as Firefox or a different one).
• geckodriver is given (or computes) a path that can be executed to launch Firefox (this could be an actual Firefox binary, or a symlink, or a shell script that eventually launches Firefox, or something else along those lines).
• We don't have any other apriori information about the Firefox instance, so anything else we need to know we have to find out at runtime (e.g. in some situations we execute it with --version to find the Firefox version).

(In reply to James Graham [:jgraham] from comment #16)

Can you write to it from outside the snap? If so, maybe, if we can work out the path we need to write to.

I dont know, this is where my "time available to focus" statement matters, I can't verify that for now.

To be clear out situation is the following:

• geckodriver may or may not itself be in a snap (either the same one as Firefox or a different one).
• geckodriver is given (or computes) a path that can be executed to launch Firefox (this could be an actual Firefox binary, or a symlink, or a shell script that eventually launches Firefox, or something else along those lines).
• We don't have any other apriori information about the Firefox instance, so anything else we need to know we have to find out at runtime (e.g. in some situations we execute it with --version to find the Firefox version).

(In reply to James Graham [:jgraham] from comment #16)

Can you write to it from outside the snap? If so, maybe, if we can work out the path we need to write to.

snapd creates /tmp/snap-private-tmp/snap.$SNAP_NAME/tmp and bind mount it as /tmp in the snap environement

the directory is private though which means normal users don't have access, if geckodriver is expected to work unprivileged that's not going to possible to copy the profile

(In reply to seb128 from comment #18)

the directory is private though which means normal users don't have access, if geckodriver is expected to work unprivileged that's not going to possible to copy the profile

seb128, would it be possible for a tool like geckodriver (if not run via the snap package but downloaded separately) to determine if a given Firefox binary is run via snap? In our case it would be the script located at /usr/bin/firefox. We don't think that grepping this file for snap might be a good solution.

If there is nothing like that we may have to assume that if snap list has Firefox included and the /usr/bin/firefox script is selected as binary to get started by geckodriver that it is using the Firefox from the snap package and we should bail out.

Thanks!

(In reply to Henrik Skupin [:whimboo][⌚️UTC+1] from comment #19)

seb128, would it be possible for a tool like geckodriver (if not run via the snap package but downloaded separately) to determine if a given Firefox binary is run via snap? In our case it would be the script located at /usr/bin/firefox.

There is no real standard way to do that afaik. You could just check if /snap/bin/firefox is available and default to use that if it is? You might end up with users who also have a deb/upstream build in /usr/bin or /usr/local/bin but then you don't have any real way to tell which one is the preferred one anyway...

Sebastian can correct me, but if geckodriver is packaged as a snap, then we could have some way for Firefox and GeckoDriver snaps to share the private directory.

(In reply to Alexandre LISSY :gerard-majax from comment #21)

Sebastian can correct me, but if geckodriver is packaged as a snap, then we could have some way for Firefox and GeckoDriver snaps to share the private directory.

The backlog suggested it was already the case if you use the geckodriver included in the snap of firefox?

Comment 13 was mentionning some users might end up with non-snap GeckoDriver

(In reply to Alexandre LISSY :gerard-majax from comment #23)

Comment 13 was mentionning some users might end up with non-snap GeckoDriver

but your question was

if geckodriver is packaged as a snap, then we could have some way for Firefox and GeckoDriver snaps to share the private directory.

so I was replying about this case, or I didn't understand the question?

The severity field is not set for this bug.
:whimboo, could you have a look please?

For more information, please visit auto_nag documentation.

This is not a bug that we can easily fix. As such we will add a section to the known issues section within the release notes and explain that both Firefox and geckodriver need to be used from the Snap package, or not.

Given that we cannot fix that issue right now on our side we decided to add a known bug entry to the geckodriver release notes. I'm going to add a patch soon so that it will be part of the 0.33.0 release.

We still need a real fix so adding the leave-open keyword.

https://hg.mozilla.org/mozilla-central/rev/ba63e54c0ceb

I've updated the geckodriver release notes for the most recent two releases to include this known issue description.