I'm at TPAC this week and noticed some odd behavior related to the wifi. When the W3C wifi flakes out I will switch to the hotel wifi. Some things will start working again, but some loads of sites like gmail are still stuck. In addition, the parent process is spinning a full CPU core for very long periods. I tried taking a profile, but none of the threads measured show any activity. I tried adding worker threads, but they are also mostly idle. Are there any network related threads that I can add to the profiler? This is with nightly FF58.

You should add the socket transport thread to be sure. Did this just start within the last 24 hours or so, or has it been all week?

I was in meetings with my laptop closed a lot more Mon/Tue, so not sure. I feel like I did see it yesterday at least, but thats within your 24 hour window. At Markus's suggestion I'm trying to catch in in activity monitor sampling since I'm on mac.

Here is a mac activity monitor sample during the high CPU usage. I do see some socket server activity, but it seems mostly in select() which should not really be spinning? Anyone else see anything like a smoking gun here?

FWIW, I had to restart firefox to get out of that high CPU this last time.

Here is the socket thread from the comment 3 attachment: 2197 Thread_900202: Socket Thread + 2197 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff9c7a0351] + 2197 _pthread_start (in libsystem_pthread.dylib) + 168 [0x7fff9c7a291a] + 2197 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff9c7a299d] + 2197 _pt_root (in libnss3.dylib) + 211 [0x103b33b43] + 2197 nsThread::ThreadFunc(void*) (in XUL) + 475 [0x103e791ab] + 2197 MessageLoop::Run() (in XUL) + 73 [0x1042b8f99] + 2197 mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) (in XUL) + 171 [0x1042ef9eb] + 2197 NS_ProcessNextEvent(nsIThread*, bool) (in XUL) + 47 [0x103e8408f] + 2197 nsThread::ProcessNextEvent(bool, bool*) (in XUL) + 1651 [0x103e7ae53] + 2197 non-virtual thunk to mozilla::net::nsSocketTransportService::Run() (in XUL) + 13 [0x103f2fced] + 2175 mozilla::net::nsSocketTransportService::Run() (in XUL) + 624 [0x103f2f010] + ! 2117 mozilla::net::nsSocketTransportService::DoPollIteration(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 745 [0x103f2f799] + ! : 2103 mozilla::net::nsSocketTransportService::Poll(unsigned int*, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 271 [0x103f2de2f] + ! : | 1871 PR_Poll (in libnss3.dylib) + 1300 [0x103b2d674] + ! : | + 1862 poll (in libnss3.dylib) + 457 [0x103b1b669] + ! : | + ! 1862 __select (in libsystem_kernel.dylib) + 10,20 [0x7fff9c8ab07a,0x7fff9c8ab084] + ! : | + 9 poll (in libnss3.dylib) + 99,337,... [0x103b1b503,0x103b1b5f1,...] + ! : | 173 PR_Poll (in libnss3.dylib) + 884 [0x103b2d4d4] + ! : | + 62 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 173 [0x107152cad] + ! : | + ! 55 ssl_Poll (in libnss3.dylib) + 317 [0x103be26fd] + ! : | + ! : 30 SSL_DataPending (in libnss3.dylib) + 81 [0x103bdb5e1] + ! : | + ! : | 12 PR_ExitMonitor (in libnss3.dylib) + 162,242,... [0x103b31da2,0x103b31df2,...] + ! : | + ! : | 8 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,51,... [0x7fff9c7a04b0,0x7fff9c7a048f,...] + ! : | + ! : | 4 PR_ExitMonitor (in libnss3.dylib) + 229 [0x103b31de5] + ! : | + ! : | + 2 OSAtomicCompareAndSwapPtrBarrier$VARIANT$mp (in libsystem_platform.dylib) + 8,3 [0x7fff9a1cd9e4,0x7fff9a1cd9df] + ! : | + ! : | + 2 _pthread_cond_signal (in libsystem_pthread.dylib) + 78,243 [0x7fff9c7a1315,0x7fff9c7a13ba] + ! : | + ! : | 4 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108 [0x7fff9c7a084d] + ! : | + ! : | 1 DYLD-STUB$$pthread_cond_signal (in libnss3.dylib) + 0 [0x103be9b0c] + ! : | + ! : | 1 PR_ExitMonitor (in libnss3.dylib) + 22 [0x103b31d16] + ! : | + ! : | 1 pthread_self (in libsystem_pthread.dylib) + 0 [0x7fff9c7a186a] + ! : | + ! : 11 SSL_DataPending (in libnss3.dylib) + 47 [0x103bdb5bf] + ! : | + ! : | 5 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,7 [0x7fff9c7a04b0,0x7fff9c7a0463] + ! : | + ! : | 3 PR_EnterMonitor (in libnss3.dylib) + 146,19 [0x103b31cf2,0x103b31c73] + ! : | + ! : | 1 DYLD-STUB$$pthread_mutex_lock (in libnss3.dylib) + 0 [0x103be9b5a] + ! : | + ! : | 1 PR_EnterMonitor (in libnss3.dylib) + 19 [0x103b31c73] + ! : | + ! : | + 1 pthread_self (in libsystem_pthread.dylib) + 9 [0x7fff9c7a1873] + ! : | + ! : | 1 PR_EnterMonitor (in libnss3.dylib) + 35 [0x103b31c83] + ! : | + ! : | 1 pthread_mutex_lock (in libsystem_pthread.dylib) + 0 [0x7fff9c7a045c] + ! : | + ! : 7 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,49,... [0x7fff9c7a084d,0x7fff9c7a0812,...] + ! : | + ! : 5 SSL_DataPending (in libnss3.dylib) + 42,4,... [0x103bdb5ba,0x103bdb594,...] + ! : | + ! : 2 SSL_DataPending (in libnss3.dylib) + 12 [0x103bdb59c] + ! : | + ! : 1 ssl_FindSocket (in libnss3.dylib) + 15 [0x103bdd63f] + ! : | + ! : + 1 PR_GetIdentitiesLayer (in libnss3.dylib) + 1 [0x103b13731] + ! : | + ! : 1 ssl_FindSocket (in libnss3.dylib) + 0 [0x103bdd630] + ! : | + ! 6 ssl_Poll (in libnss3.dylib) + 281,126,... [0x103be26d9,0x103be263e,...] + ! : | + ! 1 ssl_Poll (in libnss3.dylib) + 451 [0x103be2783] + ! : | + ! 1 pt_Poll (in libnss3.dylib) + 4 [0x103b2f7b4] + ! : | + 43 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 305 [0x107152d31] + ! : | + ! 12 nsNSSShutDownList::leaveActivityState() (in XUL) + 91 [0x1071557fb] + ! : | + ! : 10 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,13,... [0x7fff9c7a04b0,0x7fff9c7a0469,...] + ! : | + ! : 2 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 5 [0x1034ef0c5] + ! : | + ! 12 nsNSSShutDownList::leaveActivityState() (in XUL) + 136 [0x107155828] + ! : | + ! : 12 mozilla::detail::ConditionVariableImpl::notify_all() (in libmozglue.dylib) + 9 [0x1034eed89] + ! : | + ! : 7 _pthread_cond_signal (in libsystem_pthread.dylib) + 183,39,... [0x7fff9c7a137e,0x7fff9c7a12ee,...] + ! : | + ! : 5 OSAtomicCompareAndSwapPtrBarrier$VARIANT$mp (in libsystem_platform.dylib) + 8 [0x7fff9a1cd9e4] + ! : | + ! 6 nsNSSShutDownList::leaveActivityState() (in XUL) + 115 [0x107155813] + ! : | + ! : 5 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,72,... [0x7fff9c7a04b0,0x7fff9c7a04a4,...] + ! : | + ! : 1 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 23 [0x1034ef0d7] + ! : | + ! 6 nsNSSShutDownList::leaveActivityState() (in XUL) + 144 [0x107155830] + ! : | + ! : 4 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,31,... [0x7fff9c7a084d,0x7fff9c7a0800,...] + ! : | + ! : 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 11 [0x1034ef11b] + ! : | + ! : | 1 pthread_mutex_unlock (in libsystem_pthread.dylib) + 0 [0x7fff9c7a07e1] + ! : | + ! : 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 0 [0x1034ef110] + ! : | + ! 3 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,68 [0x7fff9c7a084d,0x7fff9c7a0825] + ! : | + ! 2 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 21,11 [0x1034ef125,0x1034ef11b] + ! : | + ! 1 DYLD-STUB$$mozilla::detail::MutexImpl::unlock() (in XUL) + 0 [0x1081acaf8] + ! : | + ! 1 nsNSSShutDownList::leaveActivityState() (in XUL) + 107 [0x10715580b] + ! : | + 33 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 37 [0x107152c25] + ! : | + ! 12 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 101 [0x1071556b5] + ! : | + ! : 11 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,106,... [0x7fff9c7a04b0,0x7fff9c7a04c6,...] + ! : | + ! : 1 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 17 [0x1034ef0d1] + ! : | + ! 8 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,31,... [0x7fff9c7a084d,0x7fff9c7a0800,...] + ! : | + ! 7 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 222 [0x10715572e] + ! : | + ! : 6 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,80 [0x7fff9c7a084d,0x7fff9c7a0831] + ! : | + ! : 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 11 [0x1034ef11b] + ! : | + ! 4 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 129 [0x1071556d1] + ! : | + ! : 2 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 23,0 [0x1034ef0d7,0x1034ef0c0] + ! : | + ! : 2 pthread_mutex_lock (in libsystem_pthread.dylib) + 38,84 [0x7fff9c7a0482,0x7fff9c7a04b0] + ! : | + ! 1 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 106 [0x1071556ba] + ! : | + ! : 1 nsNSSShutDownList::construct(mozilla::BaseAutoLock<mozilla::StaticMutex> const&) (in XUL) + 19 [0x107155273] + ! : | + ! 1 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 314 [0x10715578a] + ! : | + 24 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 66 [0x107152c42] + ! : | + ! 20 (anonymous namespace)::getSocketInfoIfRunning(PRFileDesc*, (anonymous namespace)::Operation, nsNSSShutDownPreventionLock const&) (in XUL) + 133 [0x107161005] + ! : | + ! : 11 mozilla::psm::TransportSecurityInfo::GetErrorCode(int*) (in XUL) + 45 [0x107125ddd] + ! : | + ! : | 9 pthread_mutex_unlock (in libsystem_pthread.dylib) + 38,108,... [0x7fff9c7a0807,0x7fff9c7a084d,...] + ! : | + ! : | 2 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 11,5 [0x1034ef11b,0x1034ef115] + ! : | + ! : 8 mozilla::psm::TransportSecurityInfo::GetErrorCode(int*) (in XUL) + 28 [0x107125dcc] + ! : | + ! : | 8 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,51 [0x7fff9c7a04b0,0x7fff9c7a048f] + ! : | + ! : 1 mozilla::psm::TransportSecurityInfo::GetErrorCode(int*) (in XUL) + 0 [0x107125db0] + ! : | + ! 4 (anonymous namespace)::getSocketInfoIfRunning(PRFileDesc*, (anonymous namespace)::Operation, nsNSSShutDownPreventionLock const&) (in XUL) + 1,123,... [0x107160f81,0x107160ffb,...] + ! : | + 4 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 300,32,... [0x107152d2c,0x107152c20,...] + ! : | + 3 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 189 [0x107152cbd] + ! : | + ! 3 mozilla::LazyLogModule::operator mozilla::LogModule*() (in XUL) + 23,4 [0x103dfc937,0x103dfc924] + ! : | + 3 pt_Poll (in libnss3.dylib) + 1 [0x103b2f7b1] + ! : | + 1 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 81 [0x107152c51] + ! : | + 1 mozilla::LazyLogModule::operator mozilla::LogModule*() (in XUL) + 1 [0x103dfc921] + ! : | 41 PR_Poll (in libnss3.dylib) + 957 [0x103b2d51d] + ! : | + 13 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 37 [0x107152c25] + ! : | + ! 5 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,13 [0x7fff9c7a084d,0x7fff9c7a07ee] + ! : | + ! 3 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 101 [0x1071556b5] + ! : | + ! : 3 pthread_mutex_lock (in libsystem_pthread.dylib) + 84 [0x7fff9c7a04b0] + ! : | + ! 3 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 222 [0x10715572e] + ! : | + ! : 2 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,72 [0x7fff9c7a084d,0x7fff9c7a0829] + ! : | + ! : 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 0 [0x1034ef110] + ! : | + ! 2 nsNSSShutDownList::enterActivityState(bool&) (in XUL) + 129 [0x1071556d1] + ! : | + ! 1 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 11 [0x1034ef0cb] + ! : | + ! | 1 pthread_mutex_lock (in libsystem_pthread.dylib) + 0 [0x7fff9c7a045c] + ! : | + ! 1 pthread_mutex_lock (in libsystem_pthread.dylib) + 84 [0x7fff9c7a04b0] + ! : | + 13 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 305 [0x107152d31] + ! : | + ! 4 nsNSSShutDownList::leaveActivityState() (in XUL) + 115 [0x107155813] + ! : | + ! : 3 pthread_mutex_lock (in libsystem_pthread.dylib) + 84 [0x7fff9c7a04b0] + ! : | + ! : 1 DYLD-STUB$$pthread_mutex_lock (in libmozglue.dylib) + 0 [0x1035097ba] + ! : | + ! 3 nsNSSShutDownList::leaveActivityState() (in XUL) + 91 [0x1071557fb] + ! : | + ! : 2 pthread_mutex_lock (in libsystem_pthread.dylib) + 84 [0x7fff9c7a04b0] + ! : | + ! : 1 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 23 [0x1034ef0d7] + ! : | + ! 2 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,83 [0x7fff9c7a084d,0x7fff9c7a0834] + ! : | + ! 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 11 [0x1034ef11b] + ! : | + ! 1 nsNSSShutDownList::leaveActivityState() (in XUL) + 136 [0x107155828] + ! : | + ! : 1 mozilla::detail::ConditionVariableImpl::notify_all() (in libmozglue.dylib) + 9 [0x1034eed89] + ! : | + ! : 1 _pthread_cond_signal (in libsystem_pthread.dylib) + 642 [0x7fff9c7a1549] + ! : | + ! 1 nsNSSShutDownList::leaveActivityState() (in XUL) + 144 [0x107155830] + ! : | + ! : 1 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108 [0x7fff9c7a084d] + ! : | + ! 1 nsNSSShutDownList::leaveActivityState() (in XUL) + 107 [0x10715580b] + ! : | + 11 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 66 [0x107152c42] + ! : | + ! 10 (anonymous namespace)::getSocketInfoIfRunning(PRFileDesc*, (anonymous namespace)::Operation, nsNSSShutDownPreventionLock const&) (in XUL) + 133 [0x107161005] + ! : | + ! : 5 mozilla::psm::TransportSecurityInfo::GetErrorCode(int*) (in XUL) + 45 [0x107125ddd] + ! : | + ! : | 4 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108 [0x7fff9c7a084d] + ! : | + ! : | 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 11 [0x1034ef11b] + ! : | + ! : 4 mozilla::psm::TransportSecurityInfo::GetErrorCode(int*) (in XUL) + 28 [0x107125dcc] + ! : | + ! : | 4 pthread_mutex_lock (in libsystem_pthread.dylib) + 84,13,... [0x7fff9c7a04b0,0x7fff9c7a0469,...] + ! : | + ! : 1 mozilla::psm::TransportSecurityInfo::GetErrorCode(int*) (in XUL) + 0 [0x107125db0] + ! : | + ! 1 (anonymous namespace)::getSocketInfoIfRunning(PRFileDesc*, (anonymous namespace)::Operation, nsNSSShutDownPreventionLock const&) (in XUL) + 111 [0x107160fef] + ! : | + 2 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 173 [0x107152cad] + ! : | + ! 2 ssl_Poll (in libnss3.dylib) + 84,281 [0x103be2614,0x103be26d9] + ! : | + 1 DYLD-STUB$$mozilla::detail::MutexImpl::unlock() (in XUL) + 0 [0x1081acaf8] + ! : | + 1 nsSSLIOLayerPoll(PRFileDesc*, short, short*) (in XUL) + 81 [0x107152c51] + ! : | + 1 mozilla::LazyLogModule::operator mozilla::LogModule*() (in XUL) + 1 [0x103dfc921] + ! : | 17 PR_Poll (in libnss3.dylib) + 370,680,... [0x103b2d2d2,0x103b2d408,...] + ! : | 1 PR_Poll (in libnss3.dylib) + 335 [0x103b2d2af] + ! : | 1 PR_GetIdentitiesLayer (in libnss3.dylib) + 32 [0x103b13750] + ! : 6 mozilla::net::nsSocketTransportService::Poll(unsigned int*, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 279 [0x103f2de37] + ! : | 4 _PR_Mach_GetInterval (in libnss3.dylib) + 9 [0x103b1ab29] + ! : | + 4 mach_absolute_time (in libsystem_kernel.dylib) + 0,4,... [0x7fff9c8a4333,0x7fff9c8a4337,...] + ! : | 2 _PR_Mach_GetInterval (in libnss3.dylib) + 30,1 [0x103b1ab3e,0x103b1ab21] + ! : 4 mozilla::net::nsSocketTransportService::Poll(unsigned int*, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 305 [0x103f2de51] + ! : | 3 mozilla::TimeStamp::Now(bool) (in libmozglue.dylib) + 0 [0x1034f0fa0] + ! : | 1 mach_absolute_time (in libsystem_kernel.dylib) + 28 [0x7fff9c8a434f] + ! : 1 PR_IntervalNow (in libnss3.dylib) + 0 [0x103b24830] + ! : 1 mozilla::net::nsSocketTransportService::Poll(unsigned int*, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 56 [0x103f2dd58] + ! : | 1 nsThread::HasPendingEvents(bool*) (in XUL) + 18 [0x103e7a742] + ! : | 1 PR_GetCurrentThread (in libnss3.dylib) + 37 [0x103b32b15] + ! : | 1 pthread_getspecific (in libsystem_pthread.dylib) + 0 [0x7fff9c7a0aa2] + ! : 1 mozilla::net::nsSocketTransportService::Poll(unsigned int*, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 113 [0x103f2dd91] + ! : | 1 mozilla::net::nsSocketTransportService::PollTimeout() (in XUL) + 59 [0x103f2dc7b] + ! : 1 mozilla::net::nsSocketTransportService::Poll(unsigned int*, mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 169 [0x103f2ddc9] + ! : 1 _PR_Mach_GetInterval (in libnss3.dylib) + 30 [0x103b1ab3e] + ! 29 mozilla::net::nsSocketTransportService::DoPollIteration(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 1001 [0x103f2f899] + ! : 28 mozilla::net::nsSocketTransport::OnSocketReady(PRFileDesc*, short) (in XUL) + 236 [0x103f29c2c] + ! : | 20 mozilla::net::nsSocketOutputStream::OnSocketReady(nsresult) (in XUL) + 194 [0x103f25af2] + ! : | + 18 non-virtual thunk to mozilla::net::nsHttpConnection::OnOutputStreamReady(nsIAsyncOutputStream*) (in XUL) + 29 [0x1041f1d8d] + ! : | + ! 14 mozilla::net::nsHttpConnection::OnSocketWritable() (in XUL) + 511 [0x1041ed5ef] + ! : | + ! : 7 mozilla::net::Http2Session::ReadSegmentsAgain(mozilla::net::nsAHttpSegmentReader*, unsigned int, unsigned int*, bool*) (in XUL) + 1159 [0x10417c577] + ! : | + ! : | 3 mozilla::net::nsHttpConnection::ResumeSend() (in XUL) + 108 [0x1041eea6c] + ! : | + ! : | + 2 mozilla::net::nsSocketOutputStream::AsyncWait(nsIOutputStreamCallback*, unsigned int, unsigned int, nsIEventTarget*) (in XUL) + 1,203 [0x103f26201,0x103f262cb] + ! : | + ! : | + 1 mozilla::net::nsSocketOutputStream::AsyncWait(nsIOutputStreamCallback*, unsigned int, unsigned int, nsIEventTarget*) (in XUL) + 187 [0x103f262bb] + ! : | + ! : | + 1 pthread_mutex_unlock (in libsystem_pthread.dylib) + 83 [0x7fff9c7a0834] + ! : | + ! : | 2 mozilla::net::nsHttpConnection::ResumeSend() (in XUL) + 1,14 [0x1041eea01,0x1041eea0e] + ! : | + ! : | 1 mozilla::net::ConnectionHandle::ResumeSend() (in XUL) + 0 [0x104211ae0] + ! : | + ! : | 1 mozilla::net::nsHttpConnection::ResumeSend() (in XUL) + 26 [0x1041eea1a] + ! : | + ! : | 1 mozilla::LazyLogModule::operator mozilla::LogModule*() (in XUL) + 24 [0x103dfc938] + ! : | + ! : 4 mozilla::net::Http2Session::ReadSegmentsAgain(mozilla::net::nsAHttpSegmentReader*, unsigned int, unsigned int*, bool*) (in XUL) + 1156,643 [0x10417c574,0x10417c373] + ! : | + ! : 2 mozilla::net::Http2Session::ReadSegmentsAgain(mozilla::net::nsAHttpSegmentReader*, unsigned int, unsigned int*, bool*) (in XUL) + 264 [0x10417c1f8] + ! : | + ! : | 2 nsDeque::PopFront() (in XUL) + 10,1 [0x103e2140a,0x103e21401] + ! : | + ! : 1 mozilla::net::Http2Session::ReadSegmentsAgain(mozilla::net::nsAHttpSegmentReader*, unsigned int, unsigned int*, bool*) (in XUL) + 453 [0x10417c2b5] + ! : | + ! : 1 mozilla::net::Http2Stream::ReadSegments(mozilla::net::nsAHttpSegmentReader*, unsigned int, unsigned int*) (in XUL) + 254 [0x10418126e] + ! : | + ! : 1 mozilla::net::nsHttpTransaction::ReadSegments(mozilla::net::nsAHttpSegmentReader*, unsigned int, unsigned int*) (in XUL) + 231 [0x10420e7d7] + ! : | + ! : 1 nsStringInputStream::ReadSegments(nsresult (*)(nsIInputStream*, void*, char const*, unsigned int, unsigned int, unsigned int*), void*, unsigned int, unsigned int*) (in XUL) + 75 [0x103e559bb] + ! : | + ! : 1 mozilla::net::nsHttpTransaction::ReadRequestSegment(nsIInputStream*, void*, char const*, unsigned int, unsigned int, unsigned int*) (in XUL) + 34 [0x10420e6a2] + ! : | + ! : 1 mozilla::net::Http2Stream::OnReadSegment(char const*, unsigned int, unsigned int*) (in XUL) + 362 [0x10418271a] + ! : | + ! : 1 mozilla::net::Http2Stream::TransmitFrame(char const*, unsigned int*, bool) (in XUL) + 1137 [0x1041730b1] + ! : | + ! : 1 mozilla::net::nsHttpTransaction::OnTransportStatus(nsITransport*, nsresult, long long) (in XUL) + 1151 [0x10420e48f] + ! : | + ! : 1 nsTransportEventSinkProxy::OnTransportStatus(nsITransport*, nsresult, long long, long long) (in XUL) + 362 [0x103f441fa] + ! : | + ! : 1 nsThread::Dispatch(already_AddRefed<nsIRunnable>, unsigned int) (in XUL) + 108 [0x103e79f4c] + ! : | + ! : 1 mozilla::ThreadEventTarget::Dispatch(already_AddRefed<nsIRunnable>, unsigned int) (in XUL) + 112 [0x103e748c0] + ! : | + ! : 1 mozilla::ThreadEventQueue<mozilla::PrioritizedEventQueue<mozilla::EventQueue> >::PutEventInternal(already_AddRefed<nsIRunnable>&&, mozilla::EventPriority, mozilla::ThreadEventQueue<mozilla::PrioritizedEventQueue<mozilla::EventQueue> >::NestedSink*) (in XUL) + 256 [0x103e73b70] + ! : | + ! : 1 non-virtual thunk to nsBaseAppShell::OnDispatchedEvent() (in XUL) + 35 [0x105eac8a3] + ! : | + ! : 1 nsAppShell::ScheduleNativeEventCallback() (in XUL) + 52 [0x105f02604] + ! : | + ! : 1 CFRunLoopWakeUp (in CoreFoundation) + 240 [0x7fff96abec10] + ! : | + ! : 1 _pthread_mutex_unlock_drop (in libsystem_pthread.dylib) + 124 [0x7fff9c7a2dcc] + ! : | + ! 4 mozilla::net::nsHttpConnection::OnSocketWritable() (in XUL) + 1056,1024,... [0x1041ed810,0x1041ed7f0,...] + ! : | + 2 mozilla::net::nsHttpConnectionMgr::nsHalfOpenSocket::OnOutputStreamReady(nsIAsyncOutputStream*) (in XUL) + 985 [0x104201d69] + ! : | + 2 mozilla::net::nsHttpConnectionMgr::nsHalfOpenSocket::SetupConn(nsIAsyncOutputStream*, bool) (in XUL) + 2059 [0x10420259b] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::OnMsgReclaimConnection(int, mozilla::net::ARefBase*) (in XUL) + 660 [0x1041f70e4] + ! : | + : 1 mozilla::net::nsHttpConnection::CanReuse() (in XUL) + 264 [0x1041ef078] + ! : | + : 1 non-virtual thunk to mozilla::net::nsSocketTransport::IsAlive(bool*) (in XUL) + 13 [0x103f2ae1d] + ! : | + : 1 mozilla::net::nsSocketTransport::IsAlive(bool*) (in XUL) + 125 [0x103f2adbd] + ! : | + : 1 PSMRecv(PRFileDesc*, void*, int, int, unsigned int) (in XUL) + 111 [0x10715261f] + ! : | + : 1 ssl_Recv (in libnss3.dylib) + 119 [0x103be2497] + ! : | + : 1 ssl_SecureRecv (in libnss3.dylib) + 341 [0x103bdafd5] + ! : | + : 1 ssl_BeginClientHandshake (in libnss3.dylib) + 398 [0x103bd895e] + ! : | + : 1 ssl3_SendClientHello (in libnss3.dylib) + 1301 [0x103bc2fd5] + ! : | + : 1 tls13_SetupClientHello (in libnss3.dylib) + 121 [0x103be2c19] + ! : | + : 1 ssl_CreateECDHEphemeralKeyPair (in libnss3.dylib) + 93 [0x103bd1fdd] + ! : | + : 1 SECKEY_CreateECPrivateKey (in libnss3.dylib) + 84 [0x103b4b534] + ! : | + : 1 PK11_GenerateKeyPairWithOpFlags (in libnss3.dylib) + 2236 [0x103b6282c] + ! : | + : 1 NSC_GenerateKeyPair (in libsoftokn3.dylib) + 480 [0x1199663e0] + ! : | + : 1 sftk_DeleteAttributeType (in libsoftokn3.dylib) + 15 [0x11996ee7f] + ! : | + : 1 sftk_FindAttribute (in libsoftokn3.dylib) + 42 [0x11996df6a] + ! : | + : 1 PR_Lock (in libnss3.dylib) + 5 [0x103b31525] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::OnMsgReclaimConnection(int, mozilla::net::ARefBase*) (in XUL) + 1078 [0x1041f7286] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::OnMsgProcessPendingQ(int, mozilla::net::ARefBase*) (in XUL) + 136 [0x1041f7688] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::ProcessPendingQForEntry(mozilla::net::nsHttpConnectionMgr::nsConnectionEntry*, bool) (in XUL) + 1148 [0x1041facfc] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::DispatchPendingQ(nsTArray<RefPtr<mozilla::net::nsHttpConnectionMgr::PendingTransactionInfo> >&, mozilla::net::nsHttpConnectionMgr::nsConnectionEntry*, bool) (in XUL) + 858 [0x1041f93ea] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::TryDispatchTransaction(mozilla::net::nsHttpConnectionMgr::nsConnectionEntry*, bool, mozilla::net::nsHttpConnectionMgr::PendingTransactionInfo*) (in XUL) + 248 [0x1041f9618] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::GetSpdyActiveConn(mozilla::net::nsHttpConnectionMgr::nsConnectionEntry*) (in XUL) + 298 [0x1041fcdda] + ! : | + 1 mozilla::net::nsHttpConnectionMgr::FindCoalescableConnection(mozilla::net::nsHttpConnectionMgr::nsConnectionEntry*, bool) (in XUL) + 132 [0x1041f8014] + ! : | + 1 mozilla::net::BuildOriginFrameHashKey(nsTSubstring<char>&, mozilla::net::nsHttpConnectionInfo*, nsTSubstring<char> const&, int) (in XUL) + 195 [0x1041f8263] + ! : | + 1 mozilla::OriginAttributes::CreateSuffix(nsTSubstring<char>&) const (in XUL) + 732 [0x1048412dc] + ! : | + 1 nsTArray_Impl<mozilla::dom::URLParams::Param, nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned long, unsigned long) (in XUL) + 88 [0x10484ac38] + ! : | + 1 arena_dalloc(void*, unsigned long) (in libmozglue.dylib) + 105 [0x1034eb789] + ! : | + 1 arena_t::DallocSmall(arena_chunk_t*, void*, arena_chunk_map_t*) (in libmozglue.dylib) + 81 [0x1034ebce1] + ! : | 3 mozilla::net::nsSocketOutputStream::OnSocketReady(nsresult) (in XUL) + 135 [0x103f25ab7] + ! : | + 2 nsCOMPtr_base::assign_with_AddRef(nsISupports*) (in XUL) + 1,21 [0x103dfda11,0x103dfda25] + ! : | + 1 nsCOMPtr_base::assign_with_AddRef(nsISupports*) (in XUL) + 27 [0x103dfda2b] + ! : | + 1 non-virtual thunk to mozilla::net::nsHttpConnection::AddRef() (in XUL) + 0 [0x1041f1620] + ! : | 2 mozilla::net::nsSocketOutputStream::OnSocketReady(nsresult) (in XUL) + 99 [0x103f25a93] + ! : | + 1 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 0 [0x1034ef0c0] + ! : | + 1 pthread_mutex_lock (in libsystem_pthread.dylib) + 69 [0x7fff9c7a04a1] + ! : | 2 mozilla::net::nsSocketOutputStream::OnSocketReady(nsresult) (in XUL) + 174 [0x103f25ade] + ! : | + 2 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108,80 [0x7fff9c7a084d,0x7fff9c7a0831] + ! : | 1 mozilla::net::nsSocketOutputStream::OnSocketReady(nsresult) (in XUL) + 1 [0x103f25a31] + ! : 1 mozilla::net::nsSocketTransport::OnSocketReady(PRFileDesc*, short) (in XUL) + 42 [0x103f29b6a] + ! 17 mozilla::net::nsSocketTransportService::DoPollIteration(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 1247 [0x103f2f98f] + ! : 13 TelemetryHistogram::Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 121 [0x10726b659] + ! : | 7 (anonymous namespace)::internal_Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 203 [0x10726b79b] + ! : | + 5 base::Histogram::Accumulate(int, int, unsigned long) (in XUL) + 21,11 [0x1042b6be5,0x1042b6bdb] + ! : | + 1 base::Histogram::Add(int) (in XUL) + 39 [0x1042b6667] + ! : | + ! 1 base::Histogram::BucketIndex(int) const (in XUL) + 92 [0x1042b6afc] + ! : | + 1 base::Histogram::Add(int) (in XUL) + 1 [0x1042b6641] + ! : | 2 (anonymous namespace)::internal_Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 173 [0x10726b77d] + ! : | + 2 mozilla::Telemetry::Common::CanRecordDataset(unsigned int, bool, bool) (in XUL) + 10,1 [0x10726496a,0x107264961] + ! : | 1 (anonymous namespace)::internal_Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 40 [0x10726b6f8] + ! : | + 1 XRE_IsParentProcess() (in XUL) + 1 [0x1072e9e81] + ! : | 1 (anonymous namespace)::internal_Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 151 [0x10726b767] + ! : | 1 base::Histogram::Add(int) (in XUL) + 39 [0x1042b6667] + ! : | + 1 base::Histogram::BucketIndex(int) const (in XUL) + 63 [0x1042b6adf] + ! : | 1 base::Histogram::Add(int) (in XUL) + 26 [0x1042b665a] + ! : 3 TelemetryHistogram::Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 111 [0x10726b64f] + ! : | 2 pthread_mutex_lock (in libsystem_pthread.dylib) + 7,84 [0x7fff9c7a0463,0x7fff9c7a04b0] + ! : | 1 mozilla::detail::MutexImpl::lock() (in libmozglue.dylib) + 23 [0x1034ef0d7] + ! : 1 mozilla::detail::MutexImpl::unlock() (in libmozglue.dylib) + 11 [0x1034ef11b] + ! : 1 _pthread_mutex_unlock_drop (in libsystem_pthread.dylib) + 60 [0x7fff9c7a2d8c] + ! : 1 __psynch_mutexdrop (in libsystem_kernel.dylib) + 10 [0x7fff9c8aadce] + ! 9 mozilla::net::nsSocketTransportService::DoPollIteration(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 862,858,... [0x103f2f80e,0x103f2f80a,...] + ! 3 mozilla::net::nsSocketTransportService::DoPollIteration(mozilla::BaseTimeDuration<mozilla::TimeDurationValueCalculator>*) (in XUL) + 172 [0x103f2f55c] + ! 3 mozilla::LazyLogModule::operator mozilla::LogModule*() (in XUL) + 23,0,... [0x103dfc937,0x103dfc920,...] + 7 mozilla::net::nsSocketTransportService::Run() (in XUL) + 722 [0x103f2f072] + ! 4 TelemetryHistogram::Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 121 [0x10726b659] + ! : 2 (anonymous namespace)::internal_Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 203 [0x10726b79b] + ! : | 1 base::Histogram::Accumulate(int, int, unsigned long) (in XUL) + 11 [0x1042b6bdb] + ! : | 1 base::Histogram::Add(int) (in XUL) + 39 [0x1042b6667] + ! : | 1 base::Histogram::BucketIndex(int) const (in XUL) + 60 [0x1042b6adc] + ! : | 1 base::Histogram::ranges(unsigned long) const (in XUL) + 11 [0x1042b691b] + ! : 1 base::Histogram::Accumulate(int, int, unsigned long) (in XUL) + 11 [0x1042b6bdb] + ! : 1 base::Histogram::Add(int) (in XUL) + 39 [0x1042b6667] + ! : 1 base::Histogram::BucketIndex(int) const (in XUL) + 71 [0x1042b6ae7] + ! 2 TelemetryHistogram::Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 111 [0x10726b64f] + ! : 2 pthread_mutex_lock (in libsystem_pthread.dylib) + 84 [0x7fff9c7a04b0] + ! 1 pthread_mutex_unlock (in libsystem_pthread.dylib) + 108 [0x7fff9c7a084d] + 6 mozilla::net::nsSocketTransportService::Run() (in XUL) + 773 [0x103f2f0a5] + ! 4 TelemetryHistogram::Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 121 [0x10726b659] + ! : 2 base::Histogram::Accumulate(int, int, unsigned long) (in XUL) + 11 [0x1042b6bdb] + ! : 2 base::Histogram::Add(int) (in XUL) + 39 [0x1042b6667] + ! : 1 base::Histogram::BucketIndex(int) const (in XUL) + 60 [0x1042b6adc] + ! : + 1 base::Histogram::ranges(unsigned long) const (in XUL) + 4 [0x1042b6914] + ! : 1 base::Histogram::BucketIndex(int) const (in XUL) + 83 [0x1042b6af3] + ! 1 TelemetryHistogram::Accumulate(mozilla::Telemetry::HistogramID, unsigned int) (in XUL) + 111 [0x10726b64f] + ! : 1 pthread_mutex_lock (in libsystem_pthread.dylib) + 84 [0x7fff9c7a04b0] + ! 1 mozilla::Telemetry::AccumulateTimeDelta(mozilla::Telemetry::HistogramID, mozilla::TimeStamp, mozilla::TimeStamp) (in XUL) + 73 [0x10725e739] + 3 mozilla::net::nsSocketTransportService::Run() (in XUL) + 798 [0x103f2f0be] + ! 2 nsThread::HasPendingEvents(bool*) (in XUL) + 42 [0x103e7a75a] + ! : 2 mozilla::ThreadEventQueue<mozilla::EventQueue>::HasPendingEvent() (in XUL) + 57 [0x103e732a9] + ! : 2 mozilla::EventQueue::HasReadyEvent(mozilla::BaseAutoLock<mozilla::Mutex> const&) (in XUL) + 21,1 [0x103e66dc5,0x103e66db1] + ! 1 nsThread::HasPendingEvents(bool*) (in XUL) + 0 [0x103e7a730] + 3 mozilla::net::nsSocketTransportService::Run() (in XUL) + 791,529,... [0x103f2f0b7,0x103f2efb1,...] + 1 mozilla::net::nsSocketTransportService::Run() (in XUL) + 609 [0x103f2f001] + ! 1 mach_absolute_time (in libsystem_kernel.dylib) + 28 [0x7fff9c8a434f] + 1 mozilla::net::nsSocketTransportService::Run() (in XUL) + 757 [0x103f2f095] + ! 1 mach_absolute_time (in libsystem_kernel.dylib) + 28 [0x7fff9c8a434f] + 1 mozilla::net::nsSocketTransportService::Run() (in XUL) + 1108 [0x103f2f1f4] + 1 NS_ProcessNextEvent(nsIThread*, bool) (in XUL) + 47 [0x103e8408f] + 1 nsThread::ProcessNextEvent(bool, bool*) (in XUL) + 1651 [0x103e7ae53] + 1 nsTimerEvent::Run() (in XUL) + 213 [0x103e75995] + 1 nsTimerImpl::Fire(int) (in XUL) + 888 [0x103e85b88] + 1 mozilla::net::nsHttpConnectionMgr::Observe(nsISupports*, char const*, char16_t const*) (in XUL) + 311 [0x1041f4827] + 1 mozilla::net::nsHttpConnectionMgr::TimeoutTick() (in XUL) + 421 [0x1041f4a75] + 1 _PR_Mach_GetInterval (in libnss3.dylib) + 30 [0x103b1ab3e]

Yea, socket thread is the only one I see doing any non-trivial amount of work. Clearing flag for now.

Thanks for the data. As this looks like being in the PSM or NSS NSPR layer, moving to the right component.

OTOH, if the numbers are in ms, the sample is from a span of 2197ms, from which we sleep in OS poll (select) for 1862ms (84% of the time) on the Socket Thread. On wake, though, we are spending most of the rest of the time in PR_Poll calling on nsSSLIOLayerPoll doing not a small amount of things. Hard to say if this can lead to a 100% load of a core.

Or, is this duplicate of bug 1415387?

Ben, any chance you could capture the above as a profile? (i.e. https://perf-html.io/ )

I have tried before, but I can't seem to get the socket thread to show up there. Is it registered for the profiler sampling?

Ah, I got it to work. Let me see if I can reproduce the condition now... Unfortunately wifi seems more stable on the last day of TPAC because fewer people are here.

This did not reproduce for me any more after most of the conference attendees left. I didn't get any more data. Do we have any ability to test with artificially bad wifi, etc?

Not that I know of. Maybe try connecting to an AP at home and then just pulling the plug?

It sounds like this might be the same as bug 1415387 was (poll() eating lots of cpu) which claims to have been fixed by bug bug 1402014.

Since we don't have any way to reproduce and bug 1415387 matches the profiles I looked at, lets just dupe it. Thanks!