Closed Bug 1577383 Opened 5 years ago Closed 2 years ago

Implement `async iterable` declaration in WebIDL

Categories

(Core :: DOM: Bindings (WebIDL), task, P3)

1.9.0 Branch
task

Tracking

()

RESOLVED FIXED
105 Branch
Tracking Status
firefox105 --- fixed

People

(Reporter: shelly, Assigned: shelly)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files, 2 obsolete files)

This bug is to discuss about implementing async iterable[1].
[1] https://heycam.github.io/webidl/#idl-async-iterable

Since DOMCursor has removed, we are looking for a replacement to implement enumerate function of DeviceStorage API[2].

[2] https://developer.mozilla.org/en-US/docs/Archive/B2G_OS/API/DeviceStorage/enumerate

How DeviceStorage gets the next File with enumerate() is handled asynchronously, caller (usually content process) asks for the next file, sends a request to parent, parent creates a blob per request, sends it back to content. The closest replacement of DOMCursor I can think of, it to define a new interface and make it async iterable, for example:

interface BlobList {
  async iterable;
};

Hello bz,
I believe Fabrice has already shared some background on this topic, we look forward to hearing your feedbacks, many thanks!

Flags: needinfo?(bzbarsky)

So in terms of implementing this... You can probably reuse a lot of the existing iterator machinery, but with the various types set to Promise<whatever> and some changes to the actual codegen for the iterator's next() method.

The contract with the C++ side will need some thought, since you can't use the "get length, get key at index, get value at index" approach sync iterators use. Worse yet is the opaque "iterator state" bit.

What will probably need to happen is that the generated iterator will have a void* member which it will initialize by calling into the backing C++ implementation (to allocate whatever), then get updated on every call into the C++ backend, but it's worth thinking a bit about whether we can do better.

Given that, what we'd probably want to do is just have the iterator pass the Promise it's about to return to the C++ impl and tell it whether we want the key, value, or key+value corresponding to the iterator state and let it decide how it wants to resolve the promise... We can either create a template helper for "resolve this promise with this array of two totally different things" to handle the key+value case, or codegen a per-binding helper using the type information from the IDL.

Flags: needinfo?(bzbarsky)
Priority: -- → P3
Severity: normal → N/A
Type: enhancement → task
Blocks: 1660328
No longer blocks: 1660328

Comment on attachment 9191830 [details]
Bug 1577383 - Implement async iterable declaration in WebIDL.

Not sure whom should I ask for patch review, can anyone give me some suggestion on the reviewer?

Assignee: nobody → shellylin

Shelly, I'd try :peterv or :edgar.

This is needed in bug 1750730, but we are adding a workaround for that.

[Exposed=(Window,Worker), SecureContext, LegacyNoInterfaceObject]
interface FileSystemDirectoryIterator {
  Promise<any> next();
};

[Exposed=(Window,Worker), SecureContext, Pref="dom.fs.enabled"]
interface FileSystemDirectoryHandle : FileSystemHandle {
  // This interface defines an async iterable, however that isn't supported yet
  // by the bindings. So for now just explicitly define what an async iterable
  // definition implies.
  //async iterable<USVString, FileSystemHandle>;
  FileSystemDirectoryIterator entries();
  FileSystemDirectoryIterator keys();
  FileSystemDirectoryIterator values();

  // ...
}

Not sure, if we can safely live with the workaround or real support for async iterable would be better.

Shelly's patch is used in the b2g fork (https://github.com/kaiostech/gecko-b2g) and works well so far.

edgar, peterv, does the patch look like something we could use?

Flags: needinfo?(peterv)
Flags: needinfo?(echen)
Blocks: OPFS

Any update on this?

Peter is reviewing the patch.

Attachment #9191830 - Attachment is obsolete: true
Blocks: 1758324

This is copy and refactoring of https://phabricator.services.mozilla.com/D142527
for the purpose of resolving conflicts in the OPFS patch stack.
It can be abandoned in favor of the original patch D142527

Depends on D143740

I have the patch updated but got the following error when using moz-phab:
I've tried moz-phab install-certificate but doesn't help, and I'm sorry for throwing such config problem here...

TypeError: sequence item 0: expected str instance, int found
Run moz-phab again with '--trace' to show debugging output
Traceback (most recent call last):
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/mozphab.py", line 70, in main
check_for_updates(with_arc=not args.no_arc)
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/updater.py", line 73, in check_for_updates
pypi_info["version"], ".".join(required_python_version)
TypeError: sequence item 0: expected str instance, int found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/shelly/.local/bin/moz-phab", line 33, in <module>
sys.exit(load_entry_point('MozPhab==1.0.0', 'console_scripts', 'moz-phab')())
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/mozphab.py", line 110, in run
main(sys.argv[1:], is_development=False)
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/mozphab.py", line 105, in main
report_to_sentry(e)
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/sentry.py", line 45, in report_to_sentry
or isinstance(e, ssl.SSLCertVerificationError)
AttributeError: module 'ssl' has no attribute 'SSLCertVerificationError'
Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit

I was told using python 3.6, which is a bit old, might be the reason for that. (Bug 1772578 is related)

(In reply to Shelly Lin [:shelly] from comment #14)

Traceback (most recent call last):
File "/home/shelly/.local/bin/moz-phab", line 33, in <module>
sys.exit(load_entry_point('MozPhab==1.0.0', 'console_scripts', 'moz-phab')())
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/mozphab.py", line 110, in run
main(sys.argv[1:], is_development=False)
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/mozphab.py", line 105, in main
report_to_sentry(e)
File "/home/shelly/.local/lib/python3.6/site-packages/mozphab/sentry.py", line 45, in report_to_sentry
or isinstance(e, ssl.SSLCertVerificationError)
AttributeError: module 'ssl' has no attribute 'SSLCertVerificationError'
Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit

I vaguely remember I also hit this with python3.6 and got fixed after upgrading to 3.8. It looks like SSLCertVerificationError isn't available in 3.6.

Flags: needinfo?(echen)

Thank you very much! I got it all setup.

Blocks: 1777142
Blocks: 1777145

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.

Assignee: nobody → shellylin
Blocks: 1781730
Pushed by rjesup@wgate.com: https://hg.mozilla.org/integration/autoland/rev/d97a870b61d7 Implement async iterable declaration in WebIDL. r=peterv
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 105 Branch

Is D147738 really "Changes Planed"?

Flags: needinfo?(jjalkanen)

That was a copy of the landed patch; we'll obsolete it soon

Flags: needinfo?(jjalkanen)
Blocks: 1782399
Blocks: 1782400
Depends on: 1788969
Blocks: 1788969
No longer depends on: 1788969
Flags: needinfo?(peterv)
Attachment #9278910 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: