Adding to the qf triage queue as this may have perf improvement.

I'd have thought that the slow bit would be the code that hits the disk, not the JS interpreter.

So I would argue that we should collect some evidence that OS.File is slow because it's in JS before we commit to rewriting it in native code. Is it possible to prove that by collecting profiles?

(In reply to Chris Pearce [:cpearce (GMT+13)] from comment #9)

I'd have thought that the slow bit would be the code that hits the disk, not the JS interpreter.

So I would argue that we should collect some evidence that OS.File is slow because it's in JS before we commit to rewriting it in native code. Is it possible to prove that by collecting profiles?

When we tested, OS.File was slow (at least during startup) because loading the JS code from a worker required hitting the disk while everything else in Firefox was busy doing the same thing, to load modules, UI resources, etc. This caused serious cache contention which, when we measured with Telemetry, could last more than 5 seconds for ~5 percentiles of users.

We got out of this by fast-pathing a few critical bits of OS.File into C++, which means that some of the code of OS.File is implemented twice, once in fast-path, once in slow-path. Not the nicest/most reliable of things, especially in presence of concurrency.

Depends on D76743

Depends on D76905

Turning this into a meta bug to track my work on this port.

Comment on attachment 9151573 [details]
Bug 1231711: Add WebIDL API for OS.File replacement

Revision D76743 was moved to bug 1641699. Setting attachment 9151573 [details] to obsolete.

Comment on attachment 9151857 [details]
Bug 1231711: Add C++ headers for IOUtils

Revision D76905 was moved to bug 1641699. Setting attachment 9151857 [details] to obsolete.

Comment on attachment 9152494 [details]
Bug 1231711: Stub out C++ implementation for IOUtils

Revision D77317 was moved to bug 1641699. Setting attachment 9152494 [details] to obsolete.

I'm working on an implementation of async read/write methods in bug 1642454, and so far have calls working from the main thread and (async) web workers. I'd like to keep the API as simple as possible. Will just exposing async methods will be sufficient, or is there a convincing reason to also have synchronous equivalents?

E.g. for a read method.

Promise<Uint8Array> read(DOMString path, optional long maxBytes);
UintArray readSync(DOMString pat, optional long maxBytes); // Will I need something like this?

For L10nRegistry rewrite to Rust I'd like to use this and I'll need sync I/O.

I understand it it's not the goal of this patchset and will understand if you decide it's out of scope but just providing a use case.

(In reply to Keefer Rourke from comment #18)

I'm working on an implementation of async read/write methods in bug 1642454, and so far have calls working from the main thread and (async) web workers. I'd like to keep the API as simple as possible. Will just exposing async methods will be sufficient, or is there a convincing reason to also have synchronous equivalents?

E.g. for a read method.

Promise<Uint8Array> read(DOMString path, optional long maxBytes);
UintArray readSync(DOMString pat, optional long maxBytes); // Will I need something like this?

Not having a sync version will make the workers a bit trickier to rewrite, but I think we should be able to manage on this front.

Now, as mentioned by zbraniecki, sync I/O for threads would be useful, but I think it can wait for a followup.

Hi, is it possible to append to a file with IOUtils? I'm asking because in Thunderbird, we save mail content and attachments to a tmp file before sending, we don't want to hold them all in memory, so we use OS.File.open and write out piece by piece. Since IOUtils don't expose file handler, and there isn't an appendMode in WriteAtomicOptions, I can't figure out how to do appending with IOUtils.

:rnons, not yet, but I've filed bug 1663707 to add support for this.

Can the winAttributes property be ported to IOUtils.setPermissions? Right now one can use the following to make a temporary file hidden on windows, while prefacing its filename with a dot to hide it for most linux users. OS.File.setPermissions(file, { winAttributes: { hidden: true } });

But I'm pretty sure the IOUtils version only supports the unixMode and unixHonorUmask properties. The winAttributes property is also missing from the table in the migration documentation, which makes it look like it's a 1-for-1 substitute, but I don't think it is. I don't know of any mozilla products that currently need to use this feature outside of test files, but I am using it for autoconfig scripts and enterprise deployments may also use it to set the "system" and "hidden" file attributes. So I will have to keep using OS.File for that particular method until it's ported

These are all the possible parameters afaik

aminomancer, I've filed bug 1743404 for that purpose.

Thanks! Looks good

Bug 1737308 has closed and OS.File is pending removal in bug 1776480. We can consider this complete.