61 hasn't branched yet, but that will be our target for the next webrtc.org update.

Mass change P1->P2 to align with new Mozilla triage process

Note: if https://bugs.chromium.org/p/webrtc/issues/detail?id=8536 isn't in webrtc.org 61, and it's applicable (I think it may well be), we'll want to cherry-pick the patch that landed upstream to fix an H264 packet-loss-recovery bug

I think we should target 64 for the next update. That entered beta on Dec. 18th.

This is the import of the third party webrtc.org code. It is too large for Phabricator (Bug 1492214) so I'm requesting review here instead.

Comment on attachment 9010747 [details] This is the import of the webrtc.org code itself. LGTM

Try run here: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=202870186&revision=288496a4f9417e6cd3cc47c16754bcb61fd7f5c5

This is a cherrypick and rebase of the build system changes in commit 9c6861308021b1f05fbf17538778ee7177e6d312. These landed on webrtc.org after 64 was branched. MozReview-Commit-ID: JQKeXWdlkq6

MozReview-Commit-ID: GBQSi6z059y Depends on D7422

MozReview-Commit-ID: HPKBZnAMKoI Depends on D7423

MozReview-Commit-ID: 2euYzBEvuNb Depends on D7424

In the past we've replaced the contents of resampler.cc with code to use Speex instead. This moves that code out into a separate class which should be easier to maintain and potentially upstream. MozReview-Commit-ID: H8ogaovdGPf Depends on D7425

MozReview-Commit-ID: 1T8mgqdkzq3 Depends on D7426

MozReview-Commit-ID: D09534DOVLj Depends on D7427

The webrtc::Trace code is removed by this update. We already had support for LOG (now RTC_LOG) in WebrtcLog.cpp. This removes the trace code from WebRtcLog.cpp and moves the aec logging code from webrtc::Trace to rtc::LogMessage. This also disables logging to stderr in rtc_base/logging.cc. We could disable it using the API, but that happens through peerconnection resulting in some logging occuring during getusermedia. The aec logs were testing with --disable-e10s. Rather than trying to work around sandboxing, I think it makes more sense to fix Bug 1404982 and store the logs in memory for retrieval from about:webrtc. Depends on D7428

Depends on D7429

Depends on D7430

Depends on D7431

Depends on D7432

Depends on D7433

This updates the copy of sigslot used by mtransport to match that used by webrtc. Depends on D7435

Comment on attachment 9013620 [details] Bug 1376873 - Fix include path in JsepSessionImpl.cpp; r=bwc Byron Campen [:bwc] has approved the revision.

Comment on attachment 9013630 [details] Bug 1376873 - Update mtransport sigslot.h; r=bwc Byron Campen [:bwc] has approved the revision.

Support for native_handle() has been removed by upstream. Depends on D7436

Depends on D7437

We'll need to queue rtp packets in both the AudioConduit and the VideoConduit. This adds a class to manage the packet queue to reduce duplicated code between the conduits. Depends on D7438

The changes to Call are required because we create the Call object on the main thread, but deliver packets and query stats from the socket thread. The changes to ChannelProxy are required because we query stats from the socket thread rather than the main thread. For RtpVideoStreamReceiver, this removes the worker_task_checker_ assertions and replaces them with a critical section. This is how the code worked prior to this update. We create the Call object (and thus eventually the RtpVideoStreamReceiver) on the main thread, but we want to deliver packets on the socket thread. To retain these assertions we'd either have to dispatch calls to deliver packets from the socket thread to the main, which seems pretty bad from a performance point of view, or we'd have to refactor the code to create the Call object on the socket thread, which seems like a major refactoring best done outside of a branch update. Going back to the previous behaviour seemed like the least bad alternative.

Depends on D7440

The only use of Mid in the current webrtc.org code is in the unit tests. RtpStreamReceiverController only allows adding sinks using SSRCs. Because of this, we'll end up dropping packets in the RtpDemuxer with the current code as none of our Mids will be recognized. Tip of webrtc.org fully supports using Mids, so we'll be able to enable this code again after the next update. Depends on D7441

We went through a lot of trouble to plumb the CPULoadState down to MediaOptimization, but the value is not actually used for anything, at least since the Branch 57 update. This removes the plumbing, since it seems we are getting along ok without it. Depends on D7442

Upstream removed the RTCPSenderInfo structure as well as code to store the packets and bytes received from the RTCP sender report. This adds code to the rtcp receiver to store those values, adds a method to the RtpRtcp module to retrieve the data and uses it to populate new fields added to the existing CallStatistics (for audio) and VideoReceiveStream::Stats (for video) structures. Depends on D7443

Depends on D7444

Depends on D7445

Upstream webrtc.org has added more stringent checks for SPS and PPS identifiers. This breaks out the SPS and PPS NALUs into separate binary blobs to ensure that the identifiers are zero. It is also necessary to set pragma pack so that the structure does not contain values which trick the parser into thinking a new NALU has begun. The ENCODED_FRAME_MAGIC is changed for a similar reason. All of the constants were determined by running mochitests with the actual h.264 encoder. For similar reasons, this also changes things so that only IFrames are sent, as the upstream code is now checking for previous IFrame identifiers and will drop frames if they are not sent. The SPS and PPS NALUs are now embedded into a single frame rather than being sent as separate frames as was previously done. This is consistent with the real H.264 plugin, and fixes a problem with intermittent failures due to occasionally bad timestamps. Depends on D7446

Depends on D7447

Depends on D7448

This is no longer necessary as we do not use the webrtc.org audio device implementations anymore. Depends on D7449

Depends on D7450

Depends on D7451

This matches the current definition for kDefaultAgcMode in voice_engine_defines.h. Depends on D7452

Depends on D7454

Depends on D7455

Depends on D7464

Depends on D7465

Depends on D7466

Depends on D7468

Depends on D7469

Depends on D7470

Depends on D7471

Depends on D7472

Upstream has added an assertion in video send stream that prevents using ReconfigureVideoEncoder if the content type has changed. This adds a mLastCodecMode member and uses it to prevent calling that method if the codec mode has changed. Depends on D7474

Depends on D7475

Windows ASAN builds complain about attempting to cast away the const qualifier in FunctionThatDoesNothing, so this adds a version specialized for const. Depends on D7476

This fixes a bug in the upstream code introduced when they removed the ConvertToI420 helper method from webrtc_libyuv.cc. The buffer size is passed into libyuv::ConvertI420 incorrectly when rotation is applied, which causes bad rendering and instabilities. Depends on D7477

Comment on attachment 9013674 [details] Bug 1376873 - Make lock_block explicit; r=bwc Byron Campen [:bwc] has approved the revision.

Comment on attachment 9013684 [details] Bug 1376873 - Setup contributing sources extension in audio_receive_stream.cc; r=ng Nico Grunbaum [:ng] has approved the revision.

The VP8SimulcastAdapter automatically handles cropping to accomodate constraints on VP8 simulcast streams that each stream have exactly half the width and height of the higher resolution stream before it. This also adjusts min_bitrate_estimate in test_peerConnection_simulcastOddResolution.html to compensate for changes in the simulcast resolutions chosen by the webrtc.org code.

Comment on attachment 9013687 [details] Bug 1376873 - Add FunctionThatDoesNothing specialized for const; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013681 [details] Bug 1376873 - Rollup of changes from cd8671232fd8 to 2a59b432d2bd; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013679 [details] Bug 1376873 - Fix GetRTPStatistics; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013672 [details] Bug 1376873 - Cherrypick rev 09133af36fba93691a22151765097c0bd581c1fa; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013671 [details] Bug 1376873 - Change OS X logging include to prevent preprocessor multiple defines; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013650 [details] Bug 1376873 - Rollup of changes from e61c4485494e to 4f12d77b4f9b; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013643 [details] Bug 1376873 - Fix GetRTCPSenderReport; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013707 [details] Bug 1376873 - Use VP8SimulcastAdapter in VideoConduit; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013620 [details] Bug 1376873 - Fix include path in JsepSessionImpl.cpp; r=bwc Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013618 [details] Bug 1376873 - Fix up logging in WebrtcLog.cpp; r=ng Nico Grunbaum [:ng] has approved the revision.

Comment on attachment 9013698 [details] Add generated moz.build files LGTM

Comment on attachment 9013697 [details] Update gn generated json files LGTM

Landry and Jan, we're planning on landing this update early in Firefox 65. Unfortunately, this means you'll need to regenerate gn json files for the BSDs based on the changed build configuration. It probably makes sense to hold off until this is reviewed, but I wanted to give you as much notice as possible. Sorry!

Thanks for the notice ! I'll have to dig out my notes from the previous upgrade, will try to do that in the coming weeks.

As error-prone as in bug 1437670 and bug 1464680. Actual bustage fixes will follow later. - DragonFly amd64 (GCC) - FreeBSD i386/amd64/aarch64/armv7 (Clang) - NetBSD i386/amd64/aarch64/armv7 (GCC) - OpenBSD i386/amd64/aarch64/armv7 (Clang)

getUserMedia audio, DataChannels, Canvas demos work. I don't have microphone or camera to test more. - event-config.h is required on everything but Windows, not sure why macOS is fine - déjà vu: bug 1326011 also had to restore changes in platform_thread.cc - alloca() is defined in stdlib.h (where Linux implicitly includes alloca.h) - BSD macro (in sys/param.h) is an archaic of the (University of California) past

Comment on attachment 9015301 [details] [diff] [review] Unbreak build on BSDs Review of attachment 9015301 [details] [diff] [review]: ----------------------------------------------------------------- ::: media/webrtc/trunk/webrtc/modules/video_capture/linux/device_info_linux.cc @@ +18,5 @@ > #include <sys/stat.h> > #include <unistd.h> > // v4l includes > +#if defined(__NetBSD__) || defined(__OpenBSD__) > +#include <sys/videoio.h> Landry, do you still need this conditional? ::: media/webrtc/trunk/webrtc/modules/video_capture/linux/video_capture_linux.cc @@ +20,5 @@ > #include <unistd.h> > > +// v4l includes > +#if defined(__NetBSD__) || defined(__OpenBSD__) > +#include <sys/videoio.h> ... and this one?

Comment on attachment 9015266 [details] BSD gn generated json/mozbuild files (compressed) https://treeherder.mozilla.org/#/jobs?repo=try&revision=30cc49dbdc0c390d0301a8ad55b18be3f22a0a23

(In reply to Jan Beich from comment #75) > Comment on attachment 9015301 [details] [diff] [review] > Unbreak build on FreeBSD > > Review of attachment 9015301 [details] [diff] [review]: > ----------------------------------------------------------------- > > ::: > media/webrtc/trunk/webrtc/modules/video_capture/linux/device_info_linux.cc > @@ +18,5 @@ > > #include <sys/stat.h> > > #include <unistd.h> > > // v4l includes > > +#if defined(__NetBSD__) || defined(__OpenBSD__) > > +#include <sys/videoio.h> > > Landry, do you still need this conditional? > > ::: > media/webrtc/trunk/webrtc/modules/video_capture/linux/video_capture_linux.cc > @@ +20,5 @@ > > #include <unistd.h> > > > > +// v4l includes > > +#if defined(__NetBSD__) || defined(__OpenBSD__) > > +#include <sys/videoio.h> > > ... and this one? videoio.h has all the V4L2 defines & ioctls, so i suppose yes ? I don't see why it should be removed, unless you have a real reason.. Is there a stash/branch somewhere with all the pending changes for this upcoming webrtc ? a list of patches ? Why is attachment 9015266 [details] a diff adding the configs as new files, while they exist in the repo ? because the diff is unreadable ?

(In reply to Landry Breuil (:gaston) from comment #77) > videoio.h has all the V4L2 defines & ioctls, so i suppose yes ? I don't see > why it should be removed, unless you have a real reason.. I wondered if OpenBSD may have added linux/ symlinks for compatibility. > Is there a stash/branch somewhere with all the pending changes for this > upcoming webrtc ? a list of patches ? Probably, branch64 on https://hg.mozilla.org/users/dminor_mozilla.com/firefox-branchupdate/ Attachment 9013698 [details] points at the commit in user repo. > Why is attachment 9015266 [details] a diff adding the configs as new files, > while they exist in the repo ? because the diff is unreadable ? Probably, removed in branch64 due to being out of date and confusing GnMozbuildWriter.

(In reply to Jan Beich from comment #78) > (In reply to Landry Breuil (:gaston) from comment #77) > > videoio.h has all the V4L2 defines & ioctls, so i suppose yes ? I don't see > > why it should be removed, unless you have a real reason.. > > I wondered if OpenBSD may have added linux/ symlinks for compatibility. yeaaah very unlikely to happen. > > Is there a stash/branch somewhere with all the pending changes for this > > upcoming webrtc ? a list of patches ? > > Probably, branch64 on > https://hg.mozilla.org/users/dminor_mozilla.com/firefox-branchupdate/ > Attachment 9013698 [details] points at the commit in user repo. Meh. Need to learn some hg-fu to point at a different branch in a different repo... sigh. > > Why is attachment 9015266 [details] a diff adding the configs as new files, > > while they exist in the repo ? because the diff is unreadable ? > > Probably, removed in branch64 due to being out of date and confusing > GnMozbuildWriter. Right, https://hg.mozilla.org/users/dminor_mozilla.com/firefox-branchupdate/rev/e66d7b9ae3fd removed them..

Comment on attachment 9015266 [details] BSD gn generated json/mozbuild files (compressed) Are there any differences between the json files included here and the ones in your other patch? If so, please update the other patch. I will apply your patches and regenerate the moz.build files prior to landing my patches, so I don't think this needs separate review.

Comment on attachment 9015301 [details] [diff] [review] Unbreak build on BSDs These changes look fine to me. To get these landed, I'd prefer one patch with the changes to non-generated code (*.gn, *.cc, etc.) and another with the gn generated .json files. If :gaston doesn't request any changes, and these .json files are the up to date versions, I can split the patch for you when I import it into my series. If you end up making changes, it would be great if you could split this up as part of that. I'll then regenerate the moz.build files as part of my patch, so we don't end up landing two separate patches for the moz.build files. Thanks! (and sorry for missing the platform_thread.cc changes)

I managed to build dminor's repo and jan's two patches on top with: $ hg update -c f6976b1207d8 $ hg qpush && hg qpush $ hg log -l3 changeset: 440484:b200ca736938 tag: bug-1376873-refix tag: qtip tag: tip user: Jan Beich <jbeich@FreeBSD.org> date: Mon Oct 08 10:21:07 2018 +0000 summary: [PATCH] Bug 1376873 - Unbreak WebRTC 64 on BSDs. r=dminor f=gaston changeset: 440483:b448bd995cca tag: bug-1376873-gnconfigs tag: qbase user: Jan Beich <jbeich@FreeBSD.org> date: Sun Mar 11 20:47:57 2018 +0000 summary: [PATCH] Bug 1376873 - Add WebRTC's gn-configs for BSDs. r=dminor f=gaston changeset: 440482:f6976b1207d8 bookmark: branch64 tag: qparent user: Dan Minor <dminor@mozilla.com> date: Thu Sep 13 07:53:01 2018 -0400 summary: Bug 1376873 - Add generated moz.build files; r=ng now to actually test the resulting build...

(In reply to Landry Breuil (:gaston) from comment #77) > Why is attachment 9015266 [details] a diff adding the configs as new files, > while they exist in the repo ? because the diff is unreadable ? I had to remove the old json configs because they messed up moz.build generation when combined with the json configs generated for the update :/

Comment on attachment 9015301 [details] [diff] [review] Unbreak build on BSDs I've been able to record a video with the webcam on https://www.webrtc-experiment.com/RecordRTC/ - recording audio didnt work but i don't think that's a regression, as i tried last week with 63.0b11 and it didn't work either, i'll have to dig further. As for the diff itself & the generated files, i glanced over it but i have to admit i wont dive into the build system madness again ( i had enough in bug #1437670), so many thanks jan for dealing with this.

Screen/window/application capture demo also works. (In reply to Dan Minor [:dminor] from comment #80) > Are there any differences between the json files included here and the ones > in your other patch? If so, please update the other patch. Only missed path to bundled libevent config, introduced in WebRTC 64 update. Initially, I didn't notice due to --with-system-libevent which mimics downstream build. (In reply to Dan Minor [:dminor] from comment #81) > These changes look fine to me. To get these landed, I'd prefer one patch > with the changes to non-generated code (*.gn, *.cc, etc.) and another with > the gn generated .json files. Sure. Generating *.json files is both error-prone (e.g., out of sync) and time-consuming (e.g., due to clean environment, proper emulation, workarounds for unrelated bustage), so I've focused on it first then tried to fix build later. Once fixed it was easier to patch *.json files than generate them again. > If :gaston doesn't request any changes, and these .json files are the up to > date versions, I can split the patch for you when I import it into my > series. Maybe squash into related commits to help other reviewers e.g., - configs for BSDs with configs for other platforms - bustage fixes for BSDs with rollup of Mozilla-specific changes

Oops, forgot to restore BSD fix to use > 1 thread by VP8/VP9 encoder. Adding "// WEBRTC_BSD" comment to prevent lossy rollup in future. https://searchfox.org/mozilla-central/rev/80ac71c1c54a/media/webrtc/trunk/webrtc/system_wrappers/source/cpu_info.cc#21-22,43-44 https://searchfox.org/mozilla-central/rev/80ac71c1c54a/media/webrtc/trunk/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc#476-479

As requested by :dminor in comment 81: - Moved *.json changes from attachment 9015301 [details] [diff] [review] - Dropped moz.build changes to avoid conflicts in future

- Switch to WEBRTC_BSD in stringutils.h like mozilla-central changeset 44fc6711973e to detect loss on WebRTC update in future - Cherry-pick https://chromium.googlesource.com/external/webrtc/+/7c4dedade158%5E!/ instead of s/GLIBCXX/GLIBC/ to fix GCC build

Comment on attachment 9016049 [details] [diff] [review] Unbreak build on BSDs, v3 Review of attachment 9016049 [details] [diff] [review]: ----------------------------------------------------------------- These changes look good to me. We'll try to get BSD support upstreamed this time so hopefully there will be less to unbreak next time, we have a bug on file: Bug 1498302.

Comment on attachment 9015932 [details] Update gn generated json files for BSDs (compressed) Merged into my local set of patches and everything seems to be working well. Thanks!

Getting ready to land this. A try job with all tests for Linux64: https://treeherder.mozilla.org/#/jobs?repo=try&revision=234c8e458dd0ad82be3ba3a904b9bb82b8c6fc0d&selectedJob=209198552 All platforms with media tests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=fdc19b32be8abe5d89bb35b969e866031f58a7c0 The only thing I find concerning is: GECKO(1063) | Assertion failure: JS::ObjectIsNotGray(target), at /builds/worker/workspace/build/src/js/src/proxy/Wrapper.cpp:353 which has happened a few times and does not seem to be an existing intermittent.

Pushed by dminor@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/f40ca921b74b Update webrtc to release 64; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/dda3c784b0c0 Update build files from tip of webrtc.org; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/2e1af73cdd2e Allow gn to have a non-default target; r=chmanchester https://hg.mozilla.org/integration/mozilla-inbound/rev/d78c5c50b6b8 Build system fixes; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/1f86d23e4806 Rollup of local modifications; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/bc92eac0522f Rollup conflict fixes for audio/video code; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/e1ea09e84feb Rollup conflict fixes for rtp_rtcp module; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/5d7a8864505d Fix up logging in WebrtcLog.cpp; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/b59e1fca5c13 Fix include path in JsepSessionImpl.cpp; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/d1f7d44e05a4 WebrtcGmpVideoCodec updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/d12ad321047d VideoConduit updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/ffdbeba93fa6 WebrtcMediaDataDecoderCodec updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/6a6771656fe5 Update mtransport sigslot.h; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/6a52b92ff037 Mediapipeline updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/19d9a30b300c Updates to dom/media/, dom/media/systemservices and dom/media/webrtc; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/909f472c2e40 Add RtpPacketQueue; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/6eca45c83131 Disable or replace thread check assertions; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/b94935661622 Use Call interface in AudioConduit; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/9fe47f8ea1b8 Disable Mid support in RtpDemuxer; r=mjf https://hg.mozilla.org/integration/mozilla-inbound/rev/2951112ade46 Remove CPULoadStateObserver; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/de3f5bd6846b Fix GetRTCPSenderReport; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/0fc8c65e163e Use audio/video sync groups; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/da8d3eb163bd Update videoconduit_unittests gtests; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/6b2919ed789b Fix fake h.264 encoder; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/79010785539c Remove Android java code from AudioConduit; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/086c722131f9 Update WebrtcMediaCodecVP8VideoCodec; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/622f78404b8a Updates to Android video capture; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/d082eb8d80c6 Set default agc mode for Android in MediaEngineWebRTCAudio; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/58e47be76761 Updates to VideoConduit.cpp for OS X; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/176b82dccf4c OS X desktop capture fixes; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/60fdf3598932 Change OS X logging include to prevent preprocessor multiple defines; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/acc06acb5342 Cherrypick rev 09133af36fba93691a22151765097c0bd581c1fa; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/381df4c6ae88 Make lock_block explicit; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/0c3c2fc4646b Allow single channel opus; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/9b82f16618b9 Fix warning in mean_variance_estimator.cc; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/5a45d2c706d6 Fix GetRTPStatistics; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/9674bbefc68e Rollup of changes from cd8671232fd8 to 2a59b432d2bd; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/3d11b041baf2 Setup contributing sources extension in audio_receive_stream.cc; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/f15c4be0f19f Keep track of active codec mode in VideoConduit; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/5d9c7a09616c Don't define _USE_MATH_DEFINES in audio_processing.h; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/23b5fb1a2577 Make FunctionThatDoesNothing use const; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/afbd3a8108de Fix up rotation in video_capture_impl.cc; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/26e09a6dee05 Use VP8SimulcastAdapter in VideoConduit; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/4efbe4d0050c Update gn generated json files; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/7317f657a9a7 Unbreak WebRTC 64 build on BSDs. r=dminor f=gaston https://hg.mozilla.org/integration/mozilla-inbound/rev/f029e9f3ceaf Add WebRTC 64 gn-configs for BSDs. r=dminor f=gaston https://hg.mozilla.org/integration/mozilla-inbound/rev/f9f879bd247c Add generated moz.build files; r=ng

RyanVM let me know that the assertion failure I saw above is Bug 1503666 and not related to the changes here.

I did some digging into the intermittents on https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&selectedJob=209457529&revision=f9f879bd247c8493c5f2236c1ddff739dc2f7c1b Some OOM on Win7 pgo, and some CreateThread errors from platform_thread.cc on Win7 opt. Perhaps related to one another. That file changed quite a bit in this merge.

Fails to build on FreeBSD. So, gn-configs have changed but no one said so. $ ./mach build [...] 1:17.10 Creating config.status 1:17.21 Reticulating splines... 1:17.70 Traceback (most recent call last): 1:17.70 File "/configure.py", line 123, in <module> 1:17.70 sys.exit(main(sys.argv)) 1:17.70 File "/configure.py", line 34, in main 1:17.70 return config_status(config) 1:17.70 File "/configure.py", line 118, in config_status 1:17.70 return config_status(args=[], **encode(sanitized_config, encoding)) 1:17.70 File "/python/mozbuild/mozbuild/config_status.py", line 143, in config_status 1:17.70 definitions = list(definitions) 1:17.70 File "/python/mozbuild/mozbuild/frontend/emitter.py", line 184, in emit 1:17.70 objs = list(emitfn(out)) 1:17.70 File "/python/mozbuild/mozbuild/frontend/emitter.py", line 1196, in emit_from_context 1:17.70 for obj in self._handle_linkables(context, passthru, generated_files): 1:17.70 File "/python/mozbuild/mozbuild/frontend/emitter.py", line 874, in _handle_linkables 1:17.70 'exist: \'%s\'' % (symbol, full_path), context) 1:17.70 mozbuild.frontend.reader.SandboxValidationError: 1:17.71 ============================== 1:17.71 FATAL ERROR PROCESSING MOZBUILD FILE 1:17.71 ============================== 1:17.71 The error occurred while processing the following file or one of the files it includes: 1:17.71 /media/webrtc/trunk/webrtc/common_audio/common_audio_gn/moz.build 1:17.71 The error occurred when validating the result of the execution. The reported error is: 1:17.71 File listed in UNIFIED_SOURCES does not exist: '/media/webrtc/trunk/webrtc/common_audio/resampler/speex_resampler.cc' 1:17.86 *** Fix above errors and then restart with\ 1:17.86 "./mach build" 1:17.86 gmake: *** [client.mk:115: configure] Error 1

Fixes comment 95.

Backout by csabou@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/63f135d2db22 Backed out 48 changesets (bug 1376873, bug 1502992) for causing windows 7 wpt crashes on several files.

Backed out for causing windows 7 pgo & opt wpt crashes on several files. Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=wpt10%2Cwindows%2C7&tochange=63f135d2db22375b51cf0ded745279afe2188022&fromchange=f9f879bd247c8493c5f2236c1ddff739dc2f7c1b&selectedJob=209497996 Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=209497996&repo=mozilla-inbound&lineNumber=28956 INFO - TEST-UNEXPECTED-CRASH | /webrtc/RTCRtpTransceiver.https.html | expected OK This initially failed on this test and after bwc and jib looked over it and commented on the bug but no fix was landed Aryx disabled the test here (https://bugzilla.mozilla.org/show_bug.cgi?id=1502992#c2) so we can get rid on the oranges in the tree. That test was disabled but then the wpts started failing on other files like: TEST-UNEXPECTED-CRASH | /webrtc/idlharness.https.window.html | expected OK - https://treeherder.mozilla.org/logviewer.html#?job_id=209560637&repo=mozilla-inbound&lineNumber=29355 TEST-UNEXPECTED-CRASH | /webrtc/RTCTrackEvent-constructor.html | expected OK - https://treeherder.mozilla.org/logviewer.html#?job_id=209568886&repo=mozilla-inbound&lineNumber=28895 TEST-UNEXPECTED-CRASH | /webrtc/no-media-call.html | expected OK - https://treeherder.mozilla.org/logviewer.html#?job_id=209569090&repo=mozilla-inbound&lineNumber=29484 as you can see on this disable push made by Aryx: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=wpt10%2Cwindows%2C7&selectedJob=209569459&revision=b9db1780bb9c56883527709e75f58c8c6a06bcb6 Decided to backout the whole patch + the disable patch in order to keep the trees green. Backout link: https://hg.mozilla.org/integration/mozilla-inbound/rev/63f135d2db22375b51cf0ded745279afe2188022

(In reply to Byron Campen [:bwc] from comment #94) > I did some digging into the intermittents on > https://treeherder.mozilla.org/#/jobs?repo=mozilla- > inbound&selectedJob=209457529&revision=f9f879bd247c8493c5f2236c1ddff739dc2f7c > 1b > > Some OOM on Win7 pgo, and some CreateThread errors from platform_thread.cc > on Win7 opt. Perhaps related to one another. That file changed quite a bit > in this merge. Thank you for having a look. I'm guessing CreateThread is also a OOM, it's probably failing to allocate the stack for the new thread. They pass in STACK_SIZE_PARAM_IS_A_RESERVATION which should mean they are not committing the full 1M; I'm not sure how much is actually committed when the thread is created. If this were a memory leak I would hope that it would be picked up by one of the ASAN jobs. It seems likely we're just close to OOM on win32 and sometimes we edge over. Perhaps with the branch update we're holding onto resources for longer that we did in the past.

this bug when landed caused an installer size regression: == Change summary for alert #17347 (as of Fri, 02 Nov 2018 19:28:53 GMT) == Regressions: 0% installer size linux32 pgo 66,977,011.42 -> 67,251,562.17 0% installer size osx-cross opt 74,041,973.92 -> 74,321,975.08 0% installer size windows2012-32 pgo 64,817,045.92 -> 65,054,935.25 0% installer size windows2012-64 pgo 67,822,630.00 -> 68,069,479.42 0% installer size linux64 pgo 67,171,045.17 -> 67,389,145.83 For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=17347

I'm cautiously optimistic that reducing the stack size here [1] from 1M to 512kb or 256kb will fix the OOM [2] [3]. For reference, we default to 256kb in threads allocated from gecko [4]. [1] https://hg.mozilla.org/integration/mozilla-inbound/file/f9f879bd247c/media/webrtc/trunk/webrtc/rtc_base/platform_thread.cc#l234 [2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=7f093f39b381836959dd1e01701424e9db84c543 [3] https://treeherder.mozilla.org/#/jobs?repo=try&revision=68910711caa132b6120fd8459cbb067fa2b59cf2 [4] https://searchfox.org/mozilla-central/rev/7c848ac7630df5baf1314b0c03e015683599efb9/xpcom/threads/nsIThreadManager.idl#52

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #100) > this bug when landed caused an installer size regression: > == Change summary for alert #17347 (as of Fri, 02 Nov 2018 19:28:53 GMT) == > > Regressions: > > 0% installer size linux32 pgo 66,977,011.42 -> 67,251,562.17 > 0% installer size osx-cross opt 74,041,973.92 -> 74,321,975.08 > 0% installer size windows2012-32 pgo 64,817,045.92 -> 65,054,935.25 > 0% installer size windows2012-64 pgo 67,822,630.00 -> 68,069,479.42 > 0% installer size linux64 pgo 67,171,045.17 -> 67,389,145.83 > > For up to date results, see: > https://treeherder.mozilla.org/perf.html#/alerts?id=17347 Hi Joel, are these increases significant enough to warrant investigation? We already modify the build configuration to avoid building portions of the webrtc.org code base we don't use, so these increases most likely reflect additional code added by upstream and/or the fact that we're now using more of the webrtc.org code itself. I'm sure there are further optimizations that can be made to the build configuration but my guess is that it would end up being very time consuming for limited improvements to installer size.

I don't know if they are concerning or not, it looks like 250K increase in size- :sylvestre- you are the contact for this data point, I will let you weigh in here.

(In reply to Jan Beich from comment #95) > Fails to build on FreeBSD. So, gn-configs have changed but no one said so. > Yes, we dropped the speex_resampler patch during review, but because review comments are no longer synced back to Bugzilla, it's not at all obvious that that happened. Thank you for your fix. I'll roll it into the existing BSD gn-config patch prior to landing.

Try job with platform_thread.cc stack size reduced for all platforms: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=210064580&revision=6e11e5491872c455ad414e0d8f224a4d548b187e

The current default stack size of 1M results in intermittent OOMs on win32 builds while running web-platform tests. The value of 256k was chosen for consistency with the default value used elsewhere in Gecko, which is defined in nsIThreadManager.idl.

(In reply to Dan Minor [:dminor] from comment #102) > > Hi Joel, are these increases significant enough to warrant investigation? I would say no here. However, I think we should spend some time on investigating the webrtc code built and shipped which is useless. The zero code coverage report shows webrtc as one of the biggest offender. https://marco-c.github.io/code-coverage-reports/#media/webrtc/trunk/webrtc 521 files in webrtc upstream have 0 code coverage which either means that we lack tests (even unit test) or that it is useless code for Firefox.

I've rebased on central as of today and things look good on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6d2b188f76f5f72f4f98ff723ee907a3ea4b3d2f. I'll be relanding this soon.

Pushed by dminor@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/ef84e45f28d9 Update webrtc to release 64; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/8c08a5bb8a99 Update build files from tip of webrtc.org; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/0e2aee1f2033 Allow gn to have a non-default target; r=chmanchester https://hg.mozilla.org/integration/mozilla-inbound/rev/4989aa9ad7e0 Build system fixes; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/28b57e3ba51d Rollup of local modifications; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/93eec571640e Rollup conflict fixes for audio/video code; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/50f89f4e45b0 Rollup conflict fixes for rtp_rtcp module; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/e84c60e2c937 Fix up logging in WebrtcLog.cpp; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/756ab3fad62b Fix include path in JsepSessionImpl.cpp; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/8b5e947a1408 WebrtcGmpVideoCodec updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/2fb84f9968af VideoConduit updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/d1535acfa6fe WebrtcMediaDataDecoderCodec updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/a6d993a9b3bc Update mtransport sigslot.h; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/3b4d25ce432d Mediapipeline updates; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/950e95f79312 Updates to dom/media/, dom/media/systemservices and dom/media/webrtc; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/941a85e51309 Add RtpPacketQueue; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/df960ea2d29e Disable or replace thread check assertions; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/5d2899adc305 Use Call interface in AudioConduit; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/b3ba8452e771 Disable Mid support in RtpDemuxer; r=mjf https://hg.mozilla.org/integration/mozilla-inbound/rev/a61420479a31 Remove CPULoadStateObserver; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/b55b0368d9f2 Fix GetRTCPSenderReport; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/551533329f3f Use audio/video sync groups; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/0cd31b95b9de Update videoconduit_unittests gtests; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/949c043fb183 Fix fake h.264 encoder; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/549eae0865b6 Remove Android java code from AudioConduit; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/334776e8e614 Update WebrtcMediaCodecVP8VideoCodec; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/1e6d72de0587 Updates to Android video capture; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/ae6e3a39494d Set default agc mode for Android in MediaEngineWebRTCAudio; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/7f5ae8174a2b Updates to VideoConduit.cpp for OS X; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/02c038eca65c OS X desktop capture fixes; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/7f5866295b84 Change OS X logging include to prevent preprocessor multiple defines; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/cca0a311c33b Cherrypick rev 09133af36fba93691a22151765097c0bd581c1fa; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/200849cab86a Make lock_block explicit; r=bwc https://hg.mozilla.org/integration/mozilla-inbound/rev/92a7c3eee9f0 Allow single channel opus; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/cb6c555654c0 Fix warning in mean_variance_estimator.cc; r=padenot https://hg.mozilla.org/integration/mozilla-inbound/rev/0c885324ba6f Fix GetRTPStatistics; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/bb450c30e765 Rollup of changes from cd8671232fd8 to 2a59b432d2bd; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/e3fdbe8c23f3 Setup contributing sources extension in audio_receive_stream.cc; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/12fbf0e80d9e Keep track of active codec mode in VideoConduit; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/846c5bd26264 Don't define _USE_MATH_DEFINES in audio_processing.h; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/a9337131657c Make FunctionThatDoesNothing use const; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/c56cb66f8651 Fix up rotation in video_capture_impl.cc; r=pehrsons https://hg.mozilla.org/integration/mozilla-inbound/rev/7e0b405f7a1d Use VP8SimulcastAdapter in VideoConduit; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/98a403f60b8c Update gn generated json files; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/a636ecdcb91a Unbreak WebRTC 64 build on BSDs. r=dminor f=gaston https://hg.mozilla.org/integration/mozilla-inbound/rev/fdadefc1af6e Add WebRTC 64 gn-configs for BSDs. r=dminor f=gaston https://hg.mozilla.org/integration/mozilla-inbound/rev/8d68d175a589 Add generated moz.build files; r=ng https://hg.mozilla.org/integration/mozilla-inbound/rev/e83c311e5293 Reduce thread stack size in platform_thread.cc; r=bwc

https://hg.mozilla.org/mozilla-central/rev/ef84e45f28d9 https://hg.mozilla.org/mozilla-central/rev/8c08a5bb8a99 https://hg.mozilla.org/mozilla-central/rev/0e2aee1f2033 https://hg.mozilla.org/mozilla-central/rev/4989aa9ad7e0 https://hg.mozilla.org/mozilla-central/rev/28b57e3ba51d https://hg.mozilla.org/mozilla-central/rev/93eec571640e https://hg.mozilla.org/mozilla-central/rev/50f89f4e45b0 https://hg.mozilla.org/mozilla-central/rev/e84c60e2c937 https://hg.mozilla.org/mozilla-central/rev/756ab3fad62b https://hg.mozilla.org/mozilla-central/rev/8b5e947a1408 https://hg.mozilla.org/mozilla-central/rev/2fb84f9968af https://hg.mozilla.org/mozilla-central/rev/d1535acfa6fe https://hg.mozilla.org/mozilla-central/rev/a6d993a9b3bc https://hg.mozilla.org/mozilla-central/rev/3b4d25ce432d https://hg.mozilla.org/mozilla-central/rev/950e95f79312 https://hg.mozilla.org/mozilla-central/rev/941a85e51309 https://hg.mozilla.org/mozilla-central/rev/df960ea2d29e https://hg.mozilla.org/mozilla-central/rev/5d2899adc305 https://hg.mozilla.org/mozilla-central/rev/b3ba8452e771 https://hg.mozilla.org/mozilla-central/rev/a61420479a31 https://hg.mozilla.org/mozilla-central/rev/b55b0368d9f2 https://hg.mozilla.org/mozilla-central/rev/551533329f3f https://hg.mozilla.org/mozilla-central/rev/0cd31b95b9de https://hg.mozilla.org/mozilla-central/rev/949c043fb183 https://hg.mozilla.org/mozilla-central/rev/549eae0865b6 https://hg.mozilla.org/mozilla-central/rev/334776e8e614 https://hg.mozilla.org/mozilla-central/rev/1e6d72de0587 https://hg.mozilla.org/mozilla-central/rev/ae6e3a39494d https://hg.mozilla.org/mozilla-central/rev/7f5ae8174a2b https://hg.mozilla.org/mozilla-central/rev/02c038eca65c https://hg.mozilla.org/mozilla-central/rev/7f5866295b84 https://hg.mozilla.org/mozilla-central/rev/cca0a311c33b https://hg.mozilla.org/mozilla-central/rev/200849cab86a https://hg.mozilla.org/mozilla-central/rev/92a7c3eee9f0 https://hg.mozilla.org/mozilla-central/rev/cb6c555654c0 https://hg.mozilla.org/mozilla-central/rev/0c885324ba6f https://hg.mozilla.org/mozilla-central/rev/bb450c30e765 https://hg.mozilla.org/mozilla-central/rev/e3fdbe8c23f3 https://hg.mozilla.org/mozilla-central/rev/12fbf0e80d9e https://hg.mozilla.org/mozilla-central/rev/846c5bd26264 https://hg.mozilla.org/mozilla-central/rev/a9337131657c https://hg.mozilla.org/mozilla-central/rev/c56cb66f8651 https://hg.mozilla.org/mozilla-central/rev/7e0b405f7a1d https://hg.mozilla.org/mozilla-central/rev/98a403f60b8c https://hg.mozilla.org/mozilla-central/rev/a636ecdcb91a https://hg.mozilla.org/mozilla-central/rev/fdadefc1af6e https://hg.mozilla.org/mozilla-central/rev/8d68d175a589 https://hg.mozilla.org/mozilla-central/rev/e83c311e5293

This broke support for opus maxplaybackrate, which I noticed while writing unit tests for AudioConduit over in Bug 1406941. I have a fix there.