Per thread on n.p.m.performance, we can get a 15% improvement in startup times if we package all the .xpt files into one large .xpt file. Need to modify pkgcp.pl to call xptlink after parsing the packages-* files to create this one all inclusive xpt file: ---- We found that merging the ~50 xpt files into one big file gave winnt a 13% performance boost at startup. Hence we are trying to figure out the best way to do it: 2. We can do this at package time Advantage is we can package it however we want. One uber xpt file. 3 logical xpt files... Disadvantage is developers wont run with this uber xpt file. For beta, I think going with package time or installer time is better for now since it gives us flexibility. The command to run is: xpt_link components/all.xpt components/*.xpt

I have this 95% completed. It works on linux, but win32 doesn't like system("xpt_link foo.xpt *.xpt"), it doesn't expand the glob. So I'm going to have to do manual globbing by reading all the files in the directory, then if it ends in .xpt, add it to the list, then do a system("xpt_link foo.xpt $list"). I'd probably have to do that for the mac anyway since Macperl doesn't like globbing unless it's just '*'. But this will have to wait til tomorrow or the next day. Putting PDT in the whiteboard so I'll have approval when it's ready to check in.

Putting on PDT+ for beta1. But will move to PDT- if not fixed by 03/03 or if this regresses.

When I saw the perl book with slamm, it said @files = <*.xpt> (or) @files = Glob("*.xpt") should work. It said we could use the nsFile::KGlob package If that is of any help ... I strongly suggest refering to a perl book for the correct package name.

right. but unfortunately, macperl is very very picky about what it accepts for globbing. Basically, it just doesn't work on the mac. It makes writing XP Perl and interesting challenge. 100 lines so far and counting... I've made most of the changes, now I'm testing. Should be able to hit the 3/3 PDT deadline assuming the mac testing goes well.

*all* .xpt files in one glob? So installing the browser gives you the mail .xpt files? Plus the secret netscape-only .xpt files? This really should be done at build time, and done per-"component". Or introduce extra syntax into the packager list that indicates which .xpt files for that component need to be globbed together for that component. Or maybe you're doing this globbing after the copy in the per-component staging areas. In that case never mind :-), you can combine [component]/bin/components/*.xpt into [component].xpt

I'm doing the xpt_link after pkgcp.pl has created the various components as defined in the packages-win/mac/unix files. For commercial builds, it creates a new .xpt file including the mozilla one and any new ones installed by running pkgcp.pl on the ns tree. This works on linux (although we don't really use the packages-unix files yet), and I just completed a test run on win32 (try ftp://sweetlou/products/client/seamonkey/windows/32bit/x86/2000-03-01-16-M15/Net scapeSetup.exe). The build finished, it installs, it has only 4 xpt files (browser, mail, AIMGlue, and qfaservices. It then crashes in about 10 seconds. mac doesn't like the system() call, not to mention I can't seem to find xpt_link in dist:viewer which is where it is on linux and win32. I'm pretty well stalled on this until someone can convince me that the repackaging isn't causing the win32 crash, and/or we get xpt_link for the mac.

I've pulled the xptlink stuff out of pkgcp.pl since we can't make it work on the mac and pkgcp.pl is XP perl. So I have a new perl script, xptlink.pl, which will work on win32 and unix only, and will check it in to the mozilla tree once it opens (mozilla/xpinstall/packager/xptlink) unless someone can suggest a better location.

I don't see why mozilla/xpinstall/packager/xptlink.pl wouldn't work, in the same dir as the package lists and the copying script. Either way there's a new entry in the packager directory, might as well be the file itself rather than introduce another level in the tree. Bummer about the Mac not working, it's the one that could probably have benefitted the most from this change.

xptlink.pl checked in. I'll add it to the automation and do some test builds.

Done. see the win32 bits in the url field above (or any daily bits starting 3/3) for verification. I get an error the first time I run the commercial build, but running it a second time shows no problems, and I get the same error when running this morning's non-xptlink'ed build, so I don't think it's caused by xptlink.pl. Looking at Netscape\Netscape 6\components shows only aimglue.xpt, browser.xpt, mail.xpt, qfaservices.xpt. ftp://sweetlou/products/client/seamonkey/windows/32bit/x86/2000-03-02-15-M15/Net scapeSetup.exe changes to linux and win32 build automation complete as well. jj and beard are working on the mac side of this. marking resolved/fixed, and jj or beard can open a mac-specific bug since it's going to be a rather different process from this.

Your startup problems are due to a change by dp yesterday; see thread in the .builds group between him and jband. Nothing to do with this.

so before I mark this verified, I'm a bit confused on what I should see on linux - I see about 50 xpt files, as opposed to 4 on windows - is this correct? also, has a mac-specific bug been written up?

well, in order to know what files to link together, the xpt files have to be copied into their component directories by pkgcp.pl. pkgcp.pl copies the files to /h/lithium/root/media/exposed/unix/Linux2.1x86glibc/en/5.0 and then xptlink.pl runs xptlink on each subdirectory of that directory. So if you look there, you will find only 3 xpt files: ---- autobahn.mcom.com:41% ls AIM browser install mail psm security spellchecker autobahn.mcom.com:42% pwd /.automount/lithium/root/media/exposed/unix/Linux2.1x86glibc/en/5.0 autobahn.mcom.com:43% find . -name "*.xpt" -print ./security/bin/components/security.xpt ./browser/bin/components/browser.xpt ./mail/bin/components/mail.xpt ---- The problem is that for Linux we don't have an installer yet and won't for beta1, so the actual tar.gz file people install won't have the linked xpt files. There has been talk of making xptlink.pl use a file to control what xpt files are linked together. But that's not going to happen for beta1 either, so at this point it looks like win32 is the only platform that's getting linked xpt files for beta1. jj is still working on the mac xptlink process last I heard.

mac bug is http://bugzilla.mozilla.org/show_bug.cgi?id=30151

We do NOT need a native install stub to make the .xpi files useful. The .XPI files could be produced *now* and then tested from an html page. This would catch problems with the package lists, would result in a reduction in the number of .xpt files that could then be clumped into the tar ball, and other goodness.

I'm all for that. But as far as shipping (and I think the bits that QA tests), they'll need an installer. The Linux automation already runs makeall to create psm and security xpi files. If you want to have that make all the rest of the xpi files and an update.html page, that would be great.

marking verified for windows case, separate bug on mac, linux is a different story - let me know if you need a bug for the linux case