We briefly touched on this subject a few months ago in npm.seamonkey. Any major build reorganization that doesn't involve rearranging the existing cvs tree is going to be fairly hard to do without requiring objdir builds. There are several directories (like appshell) which contain app specific ifdefs in source files so a separate copy of that directory would need to be compiled for each app. Simplying rearranging directories in the toplevel makefile will not do. Afaik, all of the apps currently building from mozilla/configure support objdir builds though there may be some unresolved issues with the release & installer scripts. The biggest timesink will be changing the tinderboxes and nightly builds to use objdirs if they don't already.

How about alternatives? - Get rid of ifdefs. - Move toward mozapprunner and libxul. I think dropping srcdir builds is not desirable by itself, and shouldn't be done if we can instead make headway on one or both of the above. The two are related, but if we end up having to do objdir builds of seamonkey and srcdir (or objdir, but not mandated) builds of libxul/mozapprunner, we're still better off. I would prefer to keep src/objdir independent of which app or ifdefs, though. /be

Personally, I don't understand the desire to keep a srcdir build option. However, there is another kind of solution. Right now, we pull the sources for "gecko/libxul" and any applications all in one source tree. If we were to pull the sources separately (i.e. libxul-source.tar.bz2 in one place, and firefox-source.tar.bz2 separately, with similar client.mk pull instructions, I suppose it wouldn't be that hard to support a srcdir option). The real problem we're trying to fix is building seamonkey/firefox/tbird/nvu from the same tree without building layout five times, which is pretty much required for people doing hardcore API work nowadays. Once seamonkey is on a branch and we can make libxul a reality, this becomes less of an issue.

Can you explain (preferably in concrete terms) exactly what libxul and mozapprunner are or would be? I'm going to guess that removing the ifdefs is easier said than done, since otherwise, they wouldn't be there in the first place. mailnews has quite a few MOZ_THUNDERBIRD ifdefs as well. Given past history, I'm willing to bet that those files/modules would be forked instead of made to work with all of the apps that are currently using them and I have trouble seeing that as a better solution. Right now, we'd have to compile each of those modules N times anyway so you might as well leave them unforked. And there are some general advantages for build maintainers when not having to support srcdir builds: 1) avoiding the dirty srcdir issue that constantly plagues people 2) getting people away from the 'one tree, one app' mindset since we clearly support multiple apps from our one tree. 3) avoiding VPATH discrepancies when determining the path for an automatic make variable (ie, $<) > The two are > related, but if we end up having to do objdir builds of seamonkey and srcdir (or > objdir, but not mandated) builds of libxul/mozapprunner, we're still better off. Not sure what you meant by that statement but if there's directory overlap, then you cannot do a srcdir build of X but an objdir build of Y from the same src tree.

(In reply to comment #2) > However, there is another kind of solution. Right now, we pull the sources for > "gecko/libxul" and any applications all in one source tree. If we were to pull > the sources separately (i.e. libxul-source.tar.bz2 in one place, and > firefox-source.tar.bz2 separately, with similar client.mk pull instructions, I > suppose it wouldn't be that hard to support a srcdir option). Isn't this what http://www.seawood.org/blog/archives/000031.html was all about?

my only comment here would be that our build system is pretty complex as it is, and setting up objdirs, while not rocket science, is a skill that has eluded me for the 5? years i've been on this project. I've yet to have it work a single time; every time I give up having wasted a day and go back to srcdir building. just something to keep in mind...

> Isn't this what http://www.seawood.org/blog/archives/000031.html was all about? Close but not quite. The proposal that jkeiser & I put together last year involved actually rearranging the cvs repository to achieve the app separation in a single source tree. bsmedberg is suggesting that we use completely separate source trees for each app which would contain just the pieces needed for that app (as opposed to the full seamonkey+app trees we require now). It's a trade-off. Ideally, we'd like to have it all but realistically, we only get to pick two of: a) build all apps in one tree b) keep the existing tree structure c) support srcdir builds. Last year's proposal did a&c. The proposals that bsmedberg & I discussed a few weeks ago did a&b. bsmedberg's recent proposal does b&c.

I so love being able to do make -s && make -s -C ../build

I do intend to do this, and soon.

(In reply to comment #8) > I do intend to do this, and soon. Which "this"? From cls's comment 6: a) build all apps in one tree b) keep the existing tree structure c) support srcdir builds. Are you not trying for b and c now, rather a and b? Sorry I can't keep track! /be

Well "this" being stopping supporting srcdir builds... we already have the automation in place to build XULRunner and Firefox in separate objdirs from a single mozconfig/client.mk (see bug 307311), and srcdir builds have problems all on their own with polluting the srcdir so you can't do an objdir build once you've done a srcdir build without some tricky distclean stuff that doesn't always work. I think we have fixed all the issues with camino + ffox installing requiring a srcdir build, at least I do both builds regularly with an objdir.

(In reply to comment #10) > Well "this" being stopping supporting srcdir builds... we already have the > automation in place to build XULRunner and Firefox in separate objdirs from a > single mozconfig/client.mk (see bug 307311), and srcdir builds have problems > all on their own with polluting the srcdir so you can't do an objdir build once > you've done a srcdir build without some tricky distclean stuff that doesn't > always work. Grump old man objection (I'm not the only one): I use srcdir builds all the time; I never try to switch such a built tree to objdir; I don't want you breaking my builds. Why do you want to break 'em? /be

I want to break 'em because: 1) you're going to have to use dual-objdir builds when we have split xulrunner+ffox (which is coming pretty soon) 2) One of the most common build problems is people who set their mozconfig once, build properly, then try to build again, accidentally forgetting to set their mozconfig a second time. In this case you end up with the beginnings of a srcdir build that never finished configuring (due to --enable-application=foo being missing), but they can't distclean it because there are no makefiles, and they can't build their objdir because it detects bad files in the srcdir (config.status primarily). Requiring an objdir would fix this problem. Come on, grumpy old man, how hard is it to add MOZ_OBJDIR=@TOPSRCDIR@/objdir to your mozconfig? ;-)

Are you just going to test for MOZ_OBJDIR? Would MOZ_OBJDIR=@TOPSRCDIR@ keep my existing srcdir build?

No. We should check to make sure that objdir/source.file doesn't exist. This will ensure that a whole class of bugs (generated files replacing srcdir files) don't happen.

I'm waffling on this, as there are probably a fair amount of people who download source tarballs and build from that, and I'd hate to make life harder for them. I know when I grab a source tarball of some random program, I never think about doing anything but a srcdir build. Maybe we could just prohibit doing srcdir builds if your srcdir is a Hg clone? Seems probable that if you've cloned the repo, you might want to do work in it, and junking it up with object files is a bad idea. If you just have a source tarball, you probably don't care about trying to submit patches etc.

bug 578563 makes this suck a lot less since you won't wind up with accidental srcdir builds. Otherwise, I'm inclined to WONTFIX this now, since breaking `./configure && make` seems crappy.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #16) > bug 578563 makes this suck a lot less since you won't wind up with > accidental srcdir builds. Otherwise, I'm inclined to WONTFIX this now, since > breaking `./configure && make` seems ****. Well, **** or not, it's been done. :-) See bug #797508 Thanks to :bsmedberg for the pointer here.

Good riddance.

Comment on attachment 715679 [details] [diff] [review] Don't allow srcdir builds, v1 That will break l10n builds.

(In reply to Mike Hommey [:glandium] from comment #19) > That will break l10n builds. So, do we work around l10n builds and/or file a bug to get l10n builds to not perform srcdir builds? If the former, how would we detect l10n builds from configure? (I'm not familiar with l10n builds at all outside of what the packager does).

We should do the latter whether we do the former or not. As for the former, maybe checking for the --with-l10n-base option would work. I don't think we're using this option anywhere else.

I filed bug 842760 for moving l10n builds out of source directory.

Now with an exemption for l10n builds.

Comment on attachment 715711 [details] [diff] [review] Don't allow srcdir builds, v2 Review of attachment 715711 [details] [diff] [review]: ----------------------------------------------------------------- ::: configure.in @@ +105,5 @@ > + else > + AC_MSG_ERROR([Invalid value --with-l10n-base, $L10NBASEDIR doesn't exist]) > + fi > +fi > +AC_SUBST(L10NBASEDIR) It might be better to leave the MOZ_ARG_WITH_STRING where it is now, and just do a test like if test "${with_l10n_base+set}" = set. This should actually work because arguments are handled (mostly) at the beginning of configure, and the ${with_*} variables are set there.

Glandium on code. Ted on policy shift.

Comment on attachment 715749 [details] [diff] [review] Don't allow srcdir builds, v3 Review of attachment 715749 [details] [diff] [review]: ----------------------------------------------------------------- r+ with the test change. For the error message, I think you can just skip the last two paragraphs. ::: configure.in @@ +95,5 @@ > > + > + > +dnl TODO Don't exempt L10N builds once bug 842760 is resolved. > +if test "$_topsrcdir" = "$_objdir" -a "${with_l10n_base+set}" != "set"; then I think you need "${with_l10n_base+set}" = "set" (nothing given to --with-l10n-base) @@ +106,5 @@ > + echo " * If you are building without a mozconfig file, the simplest way" > + echo " * forward is to run: \$ ./mach build" > + echo " *" > + echo " * If you are building with a mozconfig, you will need to change your" > + echo " * mozconfig to point to a different object directory." Technically, you can use a mozconfig and still use configure && make, and no matter how you change the mozconfig, it won't affect where you run configure from.

https://hg.mozilla.org/integration/mozilla-inbound/rev/e2f944980373 With minor changes to comment when things fail. I tested locally with and without --with-l10n-base and behavior is as expected.