Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=366522348&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OF4_BsZOTzKjhmpa-tRKmw/runs/0/artifacts/public/logs/live_backing.log

[task 2022-02-02T23:59:07.665Z] 23:59:07     INFO - PID 25298 | [Child 26578: Main Thread]: E/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:364: PeerConnectionCtx is already gone. Ignoring...
[task 2022-02-02T23:59:07.665Z] 23:59:07     INFO - PID 25298 | [Child 26578: Main Thread]: I/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:368: ~PeerConnectionImpl: PeerConnectionImpl destructor invoked for {d557baaa-74ee-4dd4-b977-dbe0bc1433da}
[task 2022-02-02T23:59:07.666Z] 23:59:07     INFO - PID 25298 | [Child 26578, Main Thread] WARNING: IPC message 'PIdleScheduler::Msg_StartedGC' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:499
[task 2022-02-02T23:59:07.667Z] 23:59:07     INFO - PID 25298 | [Child 26494, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3351
[task 2022-02-02T23:59:07.667Z] 23:59:07     INFO - PID 25298 | [Child 26494, Main Thread] WARNING: IPC message 'PIdleScheduler::Msg_StartedGC' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:499
[task 2022-02-02T23:59:07.668Z] 23:59:07     INFO - PID 25298 | [Child 26494, Main Thread] WARNING: IPC message 'PIdleScheduler::Msg_DoneGC' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:499
[task 2022-02-02T23:59:07.668Z] 23:59:07     INFO - PID 25298 | [Child 26494, Main Thread] WARNING: IPC message 'PIdleScheduler::Msg_StartedGC' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:499
[task 2022-02-02T23:59:07.669Z] 23:59:07     INFO - PID 25298 | [Child 26494, Main Thread] WARNING: IPC message 'PIdleScheduler::Msg_DoneGC' discarded: actor cannot send: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:499
[task 2022-02-02T23:59:07.670Z] 23:59:07     INFO - PID 25298 | [Child 26494, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4448
[task 2022-02-02T23:59:07.670Z] 23:59:07     INFO - PID 25298 | Assertion failure: ((bool)(__builtin_expect(!!(!NS_FAILED_impl(rv)), 1))) || !mWebRTCAlive, at /builds/worker/checkouts/gecko/dom/media/systemservices/CamerasParent.cpp:251
[task 2022-02-02T23:59:07.671Z] 23:59:07     INFO - STDOUT: Initializing stack-fixing for the first stack frame, this may take a while...
[task 2022-02-02T23:59:07.809Z] 23:59:07 CRITICAL - Loading initial page https://web-platform.test:8443/testharness_runner.html failed. Ensure that the there are no other programs bound to this port and that your firewall rules or network setup does not prevent access.
[task 2022-02-02T23:59:07.809Z] 23:59:07 CRITICAL - Traceback (most recent call last):
[task 2022-02-02T23:59:07.809Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 165, in load_runner
[task 2022-02-02T23:59:07.809Z] 23:59:07 CRITICAL -     self.dismiss_alert(lambda: self.marionette.navigate(url))
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 213, in dismiss_alert
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     f()
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 165, in <lambda>
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     self.dismiss_alert(lambda: self.marionette.navigate(url))
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/marionette.py", line 1564, in navigate
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     self._send_message("WebDriver:Navigate", {"url": url})
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/decorators.py", line 37, in _
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     m._handle_socket_failure()
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/marionette.py", line 725, in _handle_socket_failure
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     reraise(exc_cls, exc, tb)
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/third_party/six/six.py", line 703, in reraise
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     raise value
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/decorators.py", line 27, in _
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     return func(*args, **kwargs)
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/marionette.py", line 661, in _send_message
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     msg = self.client.request(name, params)
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/transport.py", line 383, in request
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     return self.receive()
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -   File "/builds/worker/workspace/build/venv/lib/python3.6/site-packages/marionette_driver/transport.py", line 230, in receive
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL -     raise socket.error("No data received over socket")
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL - OSError: No data received over socket
[task 2022-02-02T23:59:07.810Z] 23:59:07 CRITICAL - 

#01: mozilla::camera::CamerasParent::StopVideoCapture() [dom/media/systemservices/CamerasParent.cpp:251]
#02: mozilla::ipc::IProtocol::DestroySubtree(mozilla::ipc::IProtocol::ActorDestroyReason) [ipc/glue/ProtocolUtils.cpp:571]
#03: mozilla::ipc::IProtocol::DestroySubtree(mozilla::ipc::IProtocol::ActorDestroyReason) [ipc/glue/ProtocolUtils.cpp:553]
#04: mozilla::ipc::PBackgroundParent::OnChannelClose() [s3:gecko-generated-sources:b08aa34787ec758599ed2e4f02a3f1d3ca0a5ef76a807da8c233e475390b0e4b13ce8abd3ce7b7fc829b0627c92dbd3d02428e443d302c0180d02c0bc83c0bfa/ipc/ipdl/PBackgroundParent.cpp::6493]
#05: mozilla::ipc::MessageChannel::OnNotifyMaybeChannelError() [ipc/glue/MessageChannel.cpp:1932]
#06: mozilla::detail::RunnableMethodImpl<mozilla::ipc::MessageChannel*, void (mozilla::ipc::MessageChannel::*)(), false, (mozilla::RunnableKind)1, >::Run() [xpcom/threads/nsThreadUtils.h:1203]
#07: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1190]
#08: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:467]
#09: mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:301]
#10: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:331]
#11: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
#12: nsThread::ThreadFunc(void*) [xpcom/threads/nsThread.cpp:393]
#13: _pt_root [nsprpub/pr/src/pthreads/ptthread.c:204]

Before calling StopVideoCapture(), ActorDestroy() clears mChildIsAlive, which means that DispatchToVideoCaptureThread() does not wait for sVideoCaptureThread to be set.

Assumptions about safety of waiting on other threads are very risky when the
threads are shared with other consumers that may be making different
assumptions. cf. https://phabricator.services.mozilla.com/D29701#872962

This also avoids the possibility of DispatchToVideoCaptureThread() failing
from StopVideoCapture() when ActorDestroy() has cleared mChildIsAlive before
sVideoCaptureThread is set.

The thread creation hop was introduced under the assumption that creating and
joining threads from different threads would be problematic.
https://bugzilla.mozilla.org/show_bug.cgi?id=1209987#c6
No-one identified any particular OS that was problematic.

https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread
says

When a thread terminates, the thread object attains a signaled state,
satisfying any threads that were waiting on the object.

man 3p pthread_join says

The results of multiple simultaneous calls to pthread_join() specifying the
same target thread are undefined.

sMutex protects sNumOfCamerasParents and sThreadMonitor,
not mPBackgroundEventTarget nor event dispatch.

sThreadMonitor is held by the main thread while it sets sVideoCaptureThread
until after StartWithOptions() returns, at which point IsRunning() will
already return true and tasks can be queued for sVideoCaptureThread.
The queuing thread does not call PostTask() until it holds sThreadMonitor and
sVideoCaptureThread is set.

sVideoCaptureThread is cleared with sThreadMonitor held, so only one
CamerasParent will delete the thread.
StartWithOptions() is always called if sVideoCaptureThread is set and failure
is handled with a MOZ_CRASH(), so the thread always needs to be Stop()ed.

Depends on D140019

https://hg.mozilla.org/mozilla-central/rev/3d1b792d0b49
https://hg.mozilla.org/mozilla-central/rev/06e781d6a2dd
https://hg.mozilla.org/mozilla-central/rev/b79377e42171