We've been talking about tracking build time and being able to alert on regressions etc, but that's kind of hard because of how variable the timings are. Things like having a fresh objdir vs. a clobber build can change timing dramatically. catlee suggested doing special builds that we could use for measuring timing. We floated the idea of maybe doing a clobber build since that removes the issues with dep builds (but we might also need to disable things like sccache to get honest timing). We might want to do a no-op dep build on top of that clobber to get timing for that, so we could track if someone regressed no-op time (by causing something to build all the time or something like that).

I'm not fully convinced we need this anymore. The build timings data we have on perfherder is pretty good these days.

I thought about saying that too. But your blurb about "disable things like sccache to get honest timings" and measuring things like a no-op dep build is still relevant and data worth having, IMO.

triaging gps: Don't we have builds that run w/o sccache now to get relevant timings? Can this bug be closed?

Yes, that was bug 1380893. Apparently we forgot about this bug!