Right now, tinderbox js shells are not compiled with --enable-more-deterministic. They should probably be, in order to use tinderbox js shells for fuzzing.

I don't think this is a good idea because then the behavior between the jsshell you get from try and what we actually release is different and try is supposed to reflect what we release.

(In reply to Christian Holler (:decoder) from comment #1) > I don't think this is a good idea because then the behavior between the > jsshell you get from try and what we actually release is different and try > is supposed to reflect what we release. Should tinderbox produce another binary with --enable-more-deterministic then? Will it make sense (or be practical)?

Tinderbox JS shells are just the shell produced as a byproduct of the Firefox build. If this would change the behavior of the JS engine in the browser, then we can't do that.

(In reply to Ted Mielczarek [:ted] from comment #3) > Tinderbox JS shells are just the shell produced as a byproduct of the > Firefox build. But it is used to run certain test suites, isn't it? Some of these might break with the specified flag if they rely on a non-standard JS enumeration order (although these should get fixed). Another thing is that --enable-more-deterministic produces additional output on stderr and disables the return values of gc() (not sure if any test needs that).

Yes, they'd be used to run whatever JS tests run as part of "make check". My larger point was that if this changes behavior in the JS engine, then it would also change the behavior of the JS engine in the browser builds, because it's the same code just being linked in two places.

Oooh ok, I got that now. Then it is 100% not an option because it would surely break web content (due to enumeration order problems).

Will we be able to compile and put on tinderbox a separate more-deterministic js binary (in addition to the current one we produce) then?

That's a question for releng.

catlee, decoder and I spoke on IRC and in summary: - We should have a separate spidermonkey job that builds just js with another configuration (in this case with --enable-more-deterministic), similar to https://tbpl.mozilla.org/php/getParsedLog.php?id=11457130&tree=Ionmonkey&full=1 - Name the new builds jsshell-fuzzing-* (where * refers to the different architectures). - Have the binaries uploaded to ftp tinderbox.

(In reply to Gary Kwong [:gkw, :nth10sd] from comment #9) > catlee, decoder and I spoke on IRC and in summary: > > - We should have a separate spidermonkey job that builds just js with > another configuration (in this case with --enable-more-deterministic), > similar to > https://tbpl.mozilla.org/php/getParsedLog. > php?id=11457130&tree=Ionmonkey&full=1 > > - Name the new builds jsshell-fuzzing-* (where * refers to the different > architectures). > > - Have the binaries uploaded to ftp tinderbox. Can this piggyback off the existing spidermonkey build system? The script and configs to run it in various configurations are here: https://hg.mozilla.org/build/tools/file/8a79961eb99f/scripts/spidermonkey_builds What branches would this need to run on?

> Can this piggyback off the existing spidermonkey build system? The script > and configs to run it in various configurations are here: --enable-more-deterministic cannot be a default configuration. jsshell-fuzzing-* binaries (compiled with this option enabled) should be compiled in addition to the existing jsshell-* ones. > What branches would this need to run on? At least on: ionmonkey mozilla-central mozilla-inbound mozilla-aurora mozilla-beta mozilla-release mozilla-esr10 (basically mostly where jsshell-* is created off, as well)

Is this just for fuzzing? Could we add this as another variant to our spidermonkey builds? We currently build JS only on inbound and ionmonkey with various options like nomethodjit and warnaserr. http://hg.mozilla.org/build/buildbot-configs/file/a46c106e054a/mozilla/config.py#l948

(In reply to Chris AtLee [:catlee] from comment #12) > Is this just for fuzzing? Yes. > Could we add this as another variant to our spidermonkey builds? We > currently build JS only on inbound and ionmonkey with various options like > nomethodjit and warnaserr. Yes, it should be possible (as long as there is a js shell), but please also make the jsshell-fuzzing-* shells accessible in the same directories as the jsshell-* shells, e.g. in: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012-05-30-03-05-19-mozilla-central/ so we are able to get people to bisect testcases which reproduce only using --enable-more-deterministic by having the fuzzing shells available.

We can make the builds available, but I don't think we can make them go into the same nightly directory as the firefox builds since they will be happening at a different time. Your other alternative is to modify the build system to generate two jsshells with whatever flags you want.

> We can make the builds available, but I don't think we can make them go into > the same nightly directory as the firefox builds since they will be > happening at a different time. We should be ok with them being made at a different time of the day, just need them to be made for the same date.

(In reply to Gary Kwong [:gkw, :nth10sd] from comment #15) > > We can make the builds available, but I don't think we can make them go into > > the same nightly directory as the firefox builds since they will be > > happening at a different time. > > We should be ok with them being made at a different time of the day, just > need them to be made for the same date. And these jsshell-fuzzing-* builds should also have a *.txt file listing their changeset rev as well, if they are going to be in a different folder.

> just need them to be made for the same date Do you mean the same rev?

How often do you need these produced?

> > just need them to be made for the same date > > Do you mean the same rev? erm yeah, rev. With the .txt file containing the rev as well, just to be doubly sure. > How often do you need these produced? At least daily, for a start.

It will be nice to have this happen. We can then make use of the compareJIT fuzzer on such deterministic builds. What other information needs to be provided?

Can we do this in taskcluster now?

Based on my reading of the bug, and my understanding of the current state of things based on Bug 1164656#c33 , I would lean towards this being doable in taskcluster. To figure out what exactly this'll require, I'd like to have a chat with someone over what the current expectations from this are, given that the state of things has changed since the last time this bug was looked at. Anyone who can be put on the hook as a point of contact for this?

It will be nice to have a separate additional js binary that is compiled with --enable-more-deterministic added into js configure, as a start.